[LU-2754] rename from MDT1 to MDT0 sometimes appears to succeed but does not

• Type: Technical task
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.4.0
• Labels: LB dne
• Environment: 2.3.60-50-g75ae281
• Assignee: Di Wang
• Reporter: John Hammond

Description

Here sys_rename is a trivial program that just calls rename(argv[1], argv[2]) and check for error.

# export MDSCOUNT=2
# llmount.sh
# cd /mnt/lustre
# find .
.
./.lustre
# lfs mkdir -i 1 m1
# echo 0 > f0
# echo 1 > m1/f1
# find .
.
./f0
./m1
./m1/f1
./.lustre
# sys_rename m1/f1 f0
# find .
.
./f0
./m1
./.lustre
# cat f0
0
# rm f0

In the same situation I can get a simple rename or rename-onto an existing directory on MDT0 to return succeess but lose the old dir.

# find .
.
./m1
./.lustre
# mkdir d0
# mkdir m1/d1
# touch m1/d1/f1
# find .
.
./d0
./m1
./m1/d1
./m1/d1/f1
./.lustre
# sys_rename m1/d1 d0
# find .
.
./d0
./m1
./.lustre
#

Rename onto may require several attempts:

# mkdir d0
# mkdir m1/d1
# touch m1/d1/f0
# find .
.
./d0
./m1
./m1/d1
./m1/d1/f0
./.lustre
# sys_rename m1/d1 d0
cannot rename `m1/d1' to `d0': Directory not empty
# sys_rename m1/d1 d0
cannot rename `m1/d1' to `d0': Directory not empty
# mkdir d2
#
# sys_rename m1/d1 d2
cannot rename `m1/d1' to `d2': Directory not empty
# sys_rename m1/d1 d2
cannot rename `m1/d1' to `d2': Directory not empty
# find .
.
./d0
./d2
./m1
./m1/d1
./m1/d1/f0
./.lustre
# mkdir d3
# sys_rename m1/d1 d3
# find .
.
./d0
./d2
./m1
./d3
./.lustre

Issue Links

• duplicates: LU-2240 implement index range lookup for osd-zfs.
• is duplicated by: LU-2773 remote directories cannot be removed by "rm -r"
• is duplicated by: LU-2755 calling rmdir on remote directory returns -ENOENT instead of -EBUSY

Comments

[John Hammond]

Fixed by Di's #5257.

[Di Wang]

yes, lfs mkdir is the alias of "lfs setdirstripe -i" in phase I. Currently lfs setdirstripe only support -i now, and in DNE phase II, lfs setstripe will support stripe_count and pool as well.

[Andreas Dilger]

I think "lfs mkdir" and "lfs setdirstripe" are aliases? I'm not sure i know what the distinction is myself.

[John Hammond]

Thanks for the clarification. Can you also explain whey we have 'lfs mkdir' and 'lfs setdirstripe'? They seem to be the same thing. Is there a plan for their semantics to diverge at some point?

[Di Wang]

Btw: I tried this with 5257. It seems the result is correct now.

[root@testnode tests]# ../utils/lfs getstripe -M /mnt/lustre/test1/f2
1
[root@testnode tests]# ../utils/lfs getstripe -M /mnt/lustre/f3
0
[root@testnode tests]# ../utils/lfs getstripe -M /mnt/lustre/test1/f2
1
[root@testnode tests]# ./multiop /mnt/lustre/f3 N /mnt/lustre/test1/f2
rename(): Invalid cross-device link

I will add more test for this. Thanks.

[Di Wang]

lfs rm_entry is used to only deleting name_entry of the remote directory, i.e. the remote directory will become orphan. Usually this will be used to delete the name entry of remote directory, when some MDT is failed and can not go back, i.e. the administrator will use this command to delete the dangling name entry.

[John Hammond]

Would someone please explain what is the use case of 'lfs rm_entry'? I can't seem to find any mention of it in the DNE design documents I've found?

[John Hammond]

Indeed, I should have chosen a better title. Otherwise valid renames from MDT0 to MDT1 all returned -EXDEV as expected. But I never got -EXDEV renaming from MDT1 to MDT0.

[Andreas Dilger]

John, note that by design, DNE Phase 1 does not support cross-MDT renames. It is supposed to return -EXDEV in this case and expects "mv" and similar tools to handle this by copying the file instead of doing a simple rename.

That said, it definitely needs to be handled correctly even at the low level, and it looks like you've found a problem here.

[Di Wang]

This needs http://review.whamcloud.com/#change,5257 as well.