https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12026 Lustre <p>In order to make direct inode scanning on the MDT useful, in addition to storing the file size/blocks via LSOM on the MDT, we also need to store the atime/mtime/ctime on the MDT inodes when the LSOM attributes are updated.</p> <p>Currently the atime is already lazily updated on the MDS (at close time), but I'm not sure if the final mtime/ctime are sent to the MDS at close time, nor whether they are updated on the MDT inode by the MDS. If this is not being done, then any MDT-only scanning will be broken.</p> LU-12026

verify that MDS stores atime/mtime/ctime during LSOM update

Task Minor Resolved Fixed Qian Yingjin Andreas Dilger Wed, 27 Feb 2019 08:13:12 +0000 Thu, 23 Apr 2020 23:00:11 +0000 Mon, 25 Nov 2019 20:36:58 +0000 Lustre 2.12.0 Lustre 2.13.0 Lustre 2.13.0 Lustre 2.12.4 0 6 <p>It would also be useful to add a sanity test case for this, to verify that it is working properly (e.g. by checking the "<tt>lfs find</tt>" RPC count to verify that it didn't do a glimpse on the file when scanning for <tt>&#45;mtime</tt> or <tt>&#45;size</tt>.</p> <p>Yingjin, can you please at least look at the MDT code and/or run a quick manual test to verify that the mtime and ctime are updated on the MDT inode when the file is closed after a write. It would need a test that creates/writes a file, sleeps for 5s, then writes it again. Then, check the MDT with debugfs to see if the inode timestamps are updated. </p> <p>If this is not working for 2.13 then we need to make a patch and backport to 2.12/EXA5, otherwise MDT scanning will not be working properly. If this is working properly then making a test is less critical. </p> <p>Sure, I will work on it sooner. Sorry for late reply.</p> <p>Hi Andreas,</p> <p>I did a simple manual test on my local system that verifies the inode timestamps are updated:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> /dev/mapper/mds1_flakey        125368      1956    112176   2% /mnt/lustre-mds1 /dev/mapper/ost1_flakey        325368     13512    284696   5% /mnt/lustre-ost1 /dev/mapper/ost2_flakey        325368     13508    284700   5% /mnt/lustre-ost2 192.168.150.128@tcp:/lustre    650736     27020    569396   5% /mnt/lustre [root@qian tests]# dd <span class="code-keyword">if</span>=/dev/zero of=/mnt/lustre/test bs=1k count=1 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000959122 s, 1.1 MB/s [root@qian tests]# sleep 5 [root@qian tests]# stat /mnt/lustre/test   File: <span class="code-quote">'/mnt/lustre/test'</span>   Size: 1024      Blocks: 8          IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272502273  Links: 1 Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root) Context: unconfined_u:object_r:unlabeled_t:s0 Access: 2019-09-24 10:59:52.000000000 +0800 Modify: 2019-09-24 10:59:52.000000000 +0800 Change: 2019-09-24 10:59:52.000000000 +0800 Birth: - [root@qian tests]# debugfs -c -R <span class="code-quote">'stat ROOT/test'</span> /dev/mapper/mds1_flakey debugfs 1.45.2.wc1 (27-May-2019) /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps Inode: 162   Type: regular    Mode:  0644   Flags: 0x0 Generation: 667952766    Version: 0x00000001:00000001 User:     0   Group:     0   Project:     0   Size: 0 File ACL: 0 Links: 1   Blockcount: 0 Fragment:  Address: 0    <span class="code-object">Number</span>: 0    Size: 0 ctime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 atime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 mtime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 crtime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 Size of extra inode fields: 32 Extended attributes:   trusted.lma (24) = 00 00 00 00 00 00 00 00 01 04 00 00 02 00 00 00 01 00 00 00 00 00 00 00   lma: fid=[0x200000401:0x1:0x0] compat=0 incompat=0   trusted.lov (56)   security.selinux (37) = <span class="code-quote">"unconfined_u:object_r:unlabeled_t:s0\000"</span>   trusted.link (46)   trusted.som (24) = 04 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 08 00 00 00 00 00 00 00 BLOCKS: [root@qian tests]# dd <span class="code-keyword">if</span>=/dev/zero of=/mnt/lustre/test bs=1k count=1 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000835408 s, 1.2 MB/s [root@qian tests]# stat /mnt/lustre/test   File: <span class="code-quote">'/mnt/lustre/test'</span>   Size: 1024      Blocks: 1          IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272502273  Links: 1 Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root) Context: unconfined_u:object_r:unlabeled_t:s0 Access: 2019-09-24 10:59:52.000000000 +0800 Modify: 2019-09-24 11:53:23.000000000 +0800 Change: 2019-09-24 11:53:23.000000000 +0800 Birth: - [root@qian tests]# debugfs -c -R <span class="code-quote">'stat ROOT/test'</span> /dev/mapper/mds1_flakey debugfs 1.45.2.wc1 (27-May-2019) /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps Inode: 162   Type: regular    Mode:  0644   Flags: 0x0 Generation: 667952766    Version: 0x00000001:00000001 User:     0   Group:     0   Project:     0   Size: 0 File ACL: 0 Links: 1   Blockcount: 0 Fragment:  Address: 0    <span class="code-object">Number</span>: 0    Size: 0 ctime: 0x5d899333:00000000 -- Tue Sep 24 11:53:23 2019 atime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 mtime: 0x5d899333:00000000 -- Tue Sep 24 11:53:23 2019 crtime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 Size of extra inode fields: 32 Extended attributes:   trusted.lma (24) = 00 00 00 00 00 00 00 00 01 04 00 00 02 00 00 00 01 00 00 00 00 00 00 00   lma: fid=[0x200000401:0x1:0x0] compat=0 incompat=0   trusted.lov (56)   security.selinux (37) = <span class="code-quote">"unconfined_u:object_r:unlabeled_t:s0\000"</span>   trusted.link (46)   trusted.som (24) = 04 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 BLOCKS: </pre> </div></div> <p>Should I write a test script in sanity.sh to verify it later?</p> <p> </p> <p>Regards,</p> <p>Qian</p> <p>Qian, I'm happy for now that we have confirmed the timestamp attributes are being updated on the MDT inodes. It isn't really clear to me <em>where</em> this is being done? Looking at the <tt>mdt_mfd_close()</tt> code:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-comment">/* Update atime on close only. */</span> <span class="code-keyword">if</span> ((open_flags &amp; MDS_FMODE_EXEC || open_flags &amp; MDS_FMODE_READ || open_flags &amp; MDS_FMODE_WRITE) &amp;&amp; (ma-&gt;ma_valid &amp; MA_INODE) &amp;&amp; (ma-&gt;ma_attr.la_valid &amp; LA_ATIME)) { <span class="code-comment">/* Set the atime only. */</span> ma-&gt;ma_valid = MA_INODE; ma-&gt;ma_attr.la_valid = LA_ATIME; rc = mo_attr_set(info-&gt;mti_env, next, ma); } </pre> </div></div> <p>it seems that this is only updating atime but not mtime and ctime.</p> <p>For now it seems we are doing the right thing, but I'm not yet convinced that we are doing the right thing all the time.</p> <p>Yes, You're right!</p> <p> </p> <p>After further testing, I found that in some test cases, the timestamps are not being updated on MDT.</p> <p>In the previous tests, the reason that the timestamps were updated is the command "dd if=/dev/zero of=/mnt/lustre/test bs=1k count=2" truncates the file when open the file.</p> <p>After add the "conv=notrunc", the timestamps are difference.</p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> [root@qian tests]# dd <span class="code-keyword">if</span>=/dev/zero of=/mnt/lustre/test bs=1k count=2 conv=notrunc 2+0 records in 2+0 records out 2048 bytes (2.0 kB) copied, 0.00368011 s, 557 kB/s [root@qian tests]# stat /mnt/lustre/test    File: <span class="code-quote">'/mnt/lustre/test'</span>   Size: 2048      Blocks: 8          IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272502273  Links: 1 Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root) Context: unconfined_u:object_r:unlabeled_t:s0 Access: 2019-09-24 10:59:52.000000000 +0800 Modify: 2019-09-24 21:28:32.000000000 +0800 Change: 2019-09-24 21:28:32.000000000 +0800  Birth: - [root@qian tests]# debugfs -c -R <span class="code-quote">'stat ROOT/test'</span> /dev/mapper/mds1_flakey  debugfs 1.45.2.wc1 (27-May-2019) /dev/mapper/mds1_flakey: catastrophic mode - not reading inode or group bitmaps Inode: 162   Type: regular    Mode:  0644   Flags: 0x0 Generation: 667952766    Version: 0x00000001:00000001 User:     0   Group:     0   Project:     0   Size: 0 File ACL: 0 Links: 1   Blockcount: 0 Fragment:  Address: 0    <span class="code-object">Number</span>: 0    Size: 0  ctime: 0x5d8994b8:00000000 -- Tue Sep 24 11:59:52 2019  atime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019  mtime: 0x5d8994b8:00000000 -- Tue Sep 24 11:59:52 2019 crtime: 0x5d8986a8:9963a170 -- Tue Sep 24 10:59:52 2019 Size of extra inode fields: 32 Extended attributes:   trusted.lma (24) = 00 00 00 00 00 00 00 00 01 04 00 00 02 00 00 00 01 00 00 00  00 00 00 00    lma: fid=[0x200000401:0x1:0x0] compat=0 incompat=0   trusted.lov (56)   security.selinux (37) = <span class="code-quote">"unconfined_u:object_r:unlabeled_t:s0\000"</span>   trusted.link (46)   trusted.som (24) = 04 00 00 00 00 00 00 00 00 08 00 00 00 00 00 00 08 00 00 00  00 00 00 00  BLOCKS: </pre> </div></div> <p>I will patch the llite and MDT code to make it update mtime and ctime accordingly.</p> <p> </p> <p>Regards,</p> <p>Qian</p> <p>Yingjin Qian (qian@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/36286" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36286</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12026" title="verify that MDS stores atime/mtime/ctime during LSOM update" class="issue-link" data-issue-key="LU-12026"><del>LU-12026</del></a> mdt: MDS stores atime|mtime|ctime during close<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 051195a24dd851e4412710bdf8a2e6ee22e914bf</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36286/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36286/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12026" title="verify that MDS stores atime/mtime/ctime during LSOM update" class="issue-link" data-issue-key="LU-12026"><del>LU-12026</del></a> mdt: MDS stores atime|mtime|ctime during close<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: d2f7cb7934a0b38fa9503e8257f2b70ed656c11d</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36869" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36869</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12026" title="verify that MDS stores atime/mtime/ctime during LSOM update" class="issue-link" data-issue-key="LU-12026"><del>LU-12026</del></a> mdt: MDS stores atime|mtime|ctime during close<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: bc85324ec7d8793683005124fd8612d2f56d7082</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36869/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36869/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12026" title="verify that MDS stores atime/mtime/ctime during LSOM update" class="issue-link" data-issue-key="LU-12026"><del>LU-12026</del></a> mdt: MDS stores atime|mtime|ctime during close<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: b67e307f2d332c2ab1643aaa04c19c024a37b22a</p> Related LU-9538 LU-11367 LU-13374 LU-11190 Development Rank 1|i00ce7: Rank (Obsolete) 9223372036854775807