[LU-4869] rmdir allowed on nonempty striped directories

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.6.0
• Fix Version/s: Lustre 2.6.0
• Type: Bug
• Priority: Blocker
• Reporter: John Hammond
• Assignee: Di Wang
• Resolution: Fixed
• Labels: dne2
• Severity: 3

Description

# lfs mkdir -c4 /mnt/lustre/d0
# touch /mnt/lustre/d0/f0
# ls -ai /mnt/lustre/d0
180144002274689036 .  180144002274689036 .   144115188193296385 ..
180144002274689036 .  144115188193296385 ..  144115188193296385 ..
180144002274689036 .  144115188193296385 ..  234187197819912199 f0
# rmdir /mnt/lustre/d0
# ls /mnt/lustre

If I hold an open file descriptor on a striped dir then I can unlink it but still create files in the non-zero stripes.

# export MDSCOUNT=4
# llmount.sh 
...
# lfs mkdir -c4 /mnt/lustre/d0
# cd /mnt/lustre/d0
# ls -ai
180144002274689025 .  180144002274689025 .  144115188193296385 ..  144115188193296385 ..
180144002274689025 .  180144002274689025 .  144115188193296385 ..  144115188193296385 ..
# rmdir /mnt/lustre/d0
# touch f0 f1 f2 f3
touch: cannot touch `f0': No such file or directory
touch: cannot touch `f1': No such file or directory
touch: cannot touch `f2': No such file or directory
touch: cannot touch `f3': No such file or directory
# cd
#
# lfs mkdir -c4 /mnt/lustre/d0
# cd /mnt/lustre/d0
# exec 3<.
# rmdir /mnt/lustre/d0
# ls -ai .
180144002274689027 .  180144002274689027 .   144115188193296385 ..
180144002274689027 .  144115188193296385 ..  144115188193296385 ..
# touch f0 f1 f2 f3
touch: cannot touch `f1': Stale file handle
# ls -ai .
180144002274689027 .  144115188193296385 ..  234187197819912194 f0
180144002274689027 .  144115188193296385 ..  198158400800948226 f2
180144002274689027 .  144115188193296385 ..  216172799310430210 f3
# exec 3<&-
# ls -ai .
ls: cannot open directory .: No such file or directory
# stat f0
  File: `f0'
  Size: 0         	Blocks: 0          IO Block: 4194304 regular empty file
Device: 2c54f966h/743766374d	Inode: 234187197819912194  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-04-08 10:04:06.000000000 -0500
Modify: 2014-04-08 10:04:06.000000000 -0500
Change: 2014-04-08 10:04:06.000000000 -0500
# stat f1
stat: cannot stat `f1': Stale file handle
# stat f2
  File: `f2'
  Size: 0         	Blocks: 0          IO Block: 4194304 regular empty file
Device: 2c54f966h/743766374d	Inode: 198158400800948226  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-04-08 10:04:06.000000000 -0500
Modify: 2014-04-08 10:04:06.000000000 -0500
Change: 2014-04-08 10:04:06.000000000 -0500
# stat f3
  File: `f3'
  Size: 0         	Blocks: 0          IO Block: 4194304 regular empty file
Device: 2c54f966h/743766374d	Inode: 216172799310430210  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-04-08 10:04:06.000000000 -0500
Modify: 2014-04-08 10:04:06.000000000 -0500
Change: 2014-04-08 10:04:06.000000000 -0500

# lfs mkdir -c4 /mnt/lustre/d0
# cd /mnt/lustre/d0
# exec 3<.
# ls -ai .
180144002274689032 .  180144002274689032 .  144115188193296385 ..  144115188193296385 ..
180144002274689032 .  180144002274689032 .  144115188193296385 ..  144115188193296385 ..
# rmdir /mnt/lustre/d0
# ls -ai .
180144002274689032 .  180144002274689032 .   144115188193296385 ..
180144002274689032 .  144115188193296385 ..  144115188193296385 ..
# touch f0 f2 f2 f3
# ls
f0  f2  f3
# touch f0 f1 f2 f3
touch: cannot touch `f1': Stale file handle
# ls
f0  f2  f3
# lfs path2fid f0 f1 f2 f3
f0: [0x340000401:0x3:0x0]
path2fid: can't get fid for f1: No such file or directory
f2: [0x2c0000401:0x3:0x0]
f3: [0x300000401:0x3:0x0]
# exec 3<&-
# cd
# stat /mnt/lustre/.lustre/fid/[0x340000401:0x3:0x0]
  File: `/mnt/lustre/.lustre/fid/[0x340000401:0x3:0x0]'
  Size: 0         	Blocks: 0          IO Block: 4194304 regular empty file
Device: 2c54f966h/743766374d	Inode: 234187197819912195  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-04-08 10:09:45.000000000 -0500
Modify: 2014-04-08 10:09:45.000000000 -0500
Change: 2014-04-08 10:09:45.000000000 -0500
# stat /mnt/lustre/.lustre/fid/[0x2c0000401:0x3:0x0]
  File: `/mnt/lustre/.lustre/fid/[0x2c0000401:0x3:0x0]'
  Size: 0         	Blocks: 0          IO Block: 4194304 regular empty file
Device: 2c54f966h/743766374d	Inode: 198158400800948227  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-04-08 10:09:45.000000000 -0500
Modify: 2014-04-08 10:09:45.000000000 -0500
Change: 2014-04-08 10:09:45.000000000 -0500

Issue Links

• is related to: LU-4603 NFS reexport leads to problems of "ls"

Comments

[Di Wang]

This probably will be fixed by patch series http://review.whamcloud.com/#/c/9699/

[Patrick Farrell (Inactive)]

Just a quick note - Without any deeper investigation, this reminds me strongly of:
https://jira.hpdd.intel.com/browse/LU-4471

[Di Wang]

If MDS unlinked an open striped directory, it needs to revalidate the layout of striped directory, so the client will be forced to revalidate the layout from the master MDT, instead of going to slave MDT directly. So it means we need LAYOUT lock for striped directory, and I will cook a patch.

[Di Wang]

http://review.whamcloud.com/#/c/10025/ I chose an easy way to fix it now. Though I had a directory layout lock patch, probably land it to 2.7.

[Di Wang]

All of fixes of readdir of stripe_dir has been merged to the fix of LU-4603
http://review.whamcloud.com/#/c/9191/

[John Hammond]

Let's mark this resolved when the fix has been merged to master. That way we don't forget about it somehow. And please don't resolve it as a duplicate of LU-4603 since that issue has a completely different description.

[Jodi Levi (Inactive)]

Patch landed to Master. Please reopen if more work for this fix is needed.