[LU-11446] ldiskfs inodes nlink mismatch with DNE

Related — Thu, 4 Aug 2022 21:48:30 +0000

It is easy to break ldiskfs format in a DNE system with async updates by adding extra hard links from another MDTs:

1. start a DNE-enabled fs

[root@vm1 tests]# REFORMAT=yes MDSCOUNT=4 OSTCOUNT=4 sh llmount.sh  
...
quota/lquota options: 'hash_lqs_cur_bits=3'
Formatting mgs, mds, osts
Format mds1: /tmp/lustre-mdt1
Format mds2: /tmp/lustre-mdt2
Format mds3: /tmp/lustre-mdt3
...

2. create a file on MDT0

[root@vm1 tests]# touch /mnt/lustre/foo                                         [root@vm1 tests]#

3. create a dir on another mdt.

[root@vm1 tests]# lfs mkdir -i 1 /mnt/lustre/mdt1                               [root@vm1 tests]#

4. create 20 hard links to /mnt/lustre/foo

[root@vm1 tests]# for x in $(seq 1 20); do ln /mnt/lustre/foo /mnt/lustre/mdt1/foo-link-$x; done
[root@vm1 tests]# ls -in /mnt/lustre/foo
144115205322833921 -rw-r--r--. 21 0 0 0 Sep 15 10:06 /mnt/lustre/foo
[root@vm1 tests]#

5. shutdown the fs

[root@vm1 tests]# MDSCOUNT=4 OSTCOUNT=4 sh llmountcleanup.sh
Stopping clients: vm1.localdomain /mnt/lustre (opts:-f)
Stopping client vm1.localdomain /mnt/lustre opts:-f
Stopping clients: vm1.localdomain /mnt/lustre2 (opts:-f)

6. run e2fsck on MDT0 image.

[root@vm1 tests]# e2fsck -fnv /tmp/lustre-mdt1
e2fsck 1.42.13.wc6 (05-Feb-2017)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Inode 168 ref count is 21, should be 2.  Fix? no

Pass 5: Checking group summary information

lustre-MDT0000: ********** WARNING: Filesystem still has errors **********


         280 inodes used (0.28%, out of 100000)
           7 non-contiguous files (2.5%)
           0 non-contiguous directories (0.0%)
             # of inodes with ind/dind/tind blocks: 1/0/0
       29638 blocks used (47.42%, out of 62500)
           0 bad blocks
           1 large file

         153 regular files
         118 directories
           0 character device files
           0 block device files
           0 fifos
           1 link
           0 symbolic links (0 fast symbolic links)
           0 sockets
------------
         272 files
[root@vm1 tests]#

The inode #168 counts all links in its nlink counter, but only two links are local:

[root@vm1 tests]# debugfs -R "ncheck 168" /tmp/lustre-mdt1
debugfs 1.42.13.wc6 (05-Feb-2017)
Inode   Pathname
168     /REMOTE_PARENT_DIR/0x200000404:0x1:0x0
168     /ROOT/foo
Segmentation fault (core dumped)
[root@vm1 tests]#

If we start the fs again

[root@vm1 tests]# NOFORMAT=yes MDSCOUNT=4 OSTCOUNT=4 sh llmount.sh

all 21 links are visible through lfs fid2path output:

[root@vm1 tests]# lfs fid2path /mnt/lustre 0x200000404:0x1:0x0
/mnt/lustre/foo
/mnt/lustre/mdt1/foo-link-1
/mnt/lustre/mdt1/foo-link-2
/mnt/lustre/mdt1/foo-link-3
/mnt/lustre/mdt1/foo-link-4
/mnt/lustre/mdt1/foo-link-5
/mnt/lustre/mdt1/foo-link-6
/mnt/lustre/mdt1/foo-link-7
/mnt/lustre/mdt1/foo-link-8
/mnt/lustre/mdt1/foo-link-9
/mnt/lustre/mdt1/foo-link-10
/mnt/lustre/mdt1/foo-link-11
/mnt/lustre/mdt1/foo-link-12
/mnt/lustre/mdt1/foo-link-13
/mnt/lustre/mdt1/foo-link-14
/mnt/lustre/mdt1/foo-link-15
/mnt/lustre/mdt1/foo-link-16
/mnt/lustre/mdt1/foo-link-17
/mnt/lustre/mdt1/foo-link-18
/mnt/lustre/mdt1/foo-link-19
/mnt/lustre/mdt1/foo-link-20
[root@vm1 tests]#

Whamcloud Community JIRA

[LU-11446] ldiskfs inodes nlink mismatch with DNE