[LU-1550] open-by-fid getdents may return inconsistend data

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Affects Version/s: Lustre 2.3.0, Lustre 2.1.2
• Labels: fid readdir
• Environment: # cat /proc/fs/lustre/version ; lustre: 2.2.55; kernel: patchless_client; build: 2.2.55-gef32640-CHANGED-2.6.32-220.13.1.el6.l22.x86_64
• Severity: 3
• Assignee: WC Triage
• Reporter: John Hammond

Description

getdents() on lustre client mount returns inconsistent/bad data. According to stat(), the /mnt/lustre inode has ino 0x61ab:

[root@r62-lustre lustre]# sys_stat -x /mnt/lustre
dev              2c54f966
ino                  61ab
mode                 41ed
nlink                   3
uid                     0
gid                     0
rdev                    0
size                 1000
blksize              1000
blocks                  8
atime            4fe3593b
mtime            4fe363e7
ctime            4fe363e7

While according to getdents() it does not. Instead we get the ino based on its
FID. (Note that the output of `ls -ai /mnt/lustre' is not based
entirely on the output of getdents().)

[root@r62-lustre lustre]# sys_getdents -x /mnt/lustre
d             61ac `.lustre'
d  20000000100000d `..'
d  200000001000006 `.'

Also for a mount point shouldn't the entries for '.' and '..' have the same d_ino?

Calling path2fid on /mnt/lustre returns [0x61ab:0xc04ebc0f:0x0], and
accessing /mnt/lustre thereby seems OK. However the FID derived from
the d_ino for '.' in /mnt/lustre offers a cache-incoherent view on
that directory:

# cd /mnt/lustre
[root]# stat .
  File: `.'
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 2c54f966h/743766374d    Inode: 25003       Links: 3
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-06-21 12:26:19.000000000 -0500
Modify: 2012-06-21 13:11:51.000000000 -0500
Change: 2012-06-21 13:11:51.000000000 -0500
[root]# stat .lustre/fid/[0x200000001:0x6:0x0]
  File: `.lustre/fid/[0x200000001:0x6:0x0]'
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 2c54f966h/743766374d    Inode: 144115188092633094  Links: 3
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-06-21 13:39:08.000000000 -0500
Modify: 2012-06-21 13:39:19.000000000 -0500
Change: 2012-06-21 13:39:19.000000000 -0500

[root]# cd /mnt/lustre
[root]# ls -a
.  ..  .lustre
[root]# ls -a .lustre/fid/[0x200000001:0x6:0x0]
.  ..  .lustre

[root]# touch FOO
[root]# ls -a
.  ..  FOO  .lustre
[root]# ls -a .lustre/fid/[0x200000001:0x6:0x0]
.  ..  .lustre

[root]# touch .lustre/fid/[0x200000001:0x6:0x0]/BAR
[root]# ls -a
.  ..  FOO  .lustre
[root]# ls -a .lustre/fid/[0x200000001:0x6:0x0]
.  ..  BAR  FOO  .lustre

[root]# touch BAZ
[root]# ls -a
.  ..  BAR  BAZ  FOO  .lustre
[root]# ls -a .lustre/fid/[0x200000001:0x6:0x0]
.  ..  BAR  FOO  .lustre

Issue Links

• duplicates: LU-838 "lfs path2fid /mnt/lustre" (ROOT) returns inode number
• is related to: LU-1518 Missing/bad operations in mdd_{obf,dot_lustre}_obj_op causing LBUGs

Comments

[Ned Bass (Inactive)]

Andreas, no I don't have any first hand knowledge of this bug.

I added the "dotlustre" label to several Jira issues that discuss the ".lustre" directory, purely for my own research purposes (hope that's OK). I'm trying to piece together what this mysterious directory is for and what interfaces it supports. If it is documented somewhere I haven't been able to find it. It seems "open by fid" is the primary (only?) interface for now. It would be great if the Lustre manual contained some explanation, particularly since it shows up in the user-visible namespace, but that's an issue for LUDOC.

[Andreas Dilger]

Ned, is this still the case with a 2.4 filesystem? I'd hope that the changes under LU-838 and LFSCK would have fixed this problem.

[Andreas Dilger]

These bugs are symptoms that internal files should not be accessible through .lustre.

[Andreas Dilger]

This is the same issue as I reported in LU-838, but this one has much better detail.