https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9663 Lustre <p>This issue was created by maloo for Saurabh Tandan &lt;saurabh.tandan@intel.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.hpdd.intel.com/test_sets/f3de6c42-4af5-11e7-b400-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f3de6c42-4af5-11e7-b400-5254006e85c2</a>.</p> <p>The sub-test test_103a failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LU-2561 test failed </pre> </div></div> <p>test_logs:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>11 commands (11 passed, 0 failed) LU-2561 newly created file is same size as directory... [3] $ mkdir -p 2561 -- ok [4] $ cd 2561 -- ok [5] $ getfacl --access . | setfacl -d -M- . -- ok [6] $ touch f1 -- ok [7] $ ls -s f1 -- failed 0 f1 ? 4 f1 [9] $ cd .. -- ok [10] $ rm -rf 2561 -- ok 7 commands (6 passed, 1 failed) sanity test_103a: @@@@@@ FAIL: LU-2561 test failed </pre> </div></div> <p>This issue is seen only during rolling upgrade/downgrade testing.</p> Servers - 2.9.58, 3591, master <br/> Client - 2.9.0, build 22, b2_9 LU-9663

sanity test_103a: LU-2561 test failed

Bug Minor Resolved Fixed Jian Yu Maloo Wed, 14 Jun 2017 01:49:03 +0000 Mon, 10 Dec 2018 23:08:00 +0000 Fri, 27 Oct 2017 00:04:22 +0000 Lustre 2.10.0 Lustre 2.11.0 Lustre 2.10.2 0 10 <p>Jian</p> <p>Can you please investigate?</p> <p>Thanks</p> <p>Peter</p> <p>Another instance for tag 2.9.59 with 1 MDT, 2 OSTs and 2 Clients for rolling upgrade testing<br/> <a href="https://testing.hpdd.intel.com/test_sets/05e12a22-56d3-11e7-8a1b-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/05e12a22-56d3-11e7-8a1b-5254006e85c2</a></p> <p>As Saurabh reported, sanity test 103a failed under the configuration of Lustre 2.9.0 client with rolling upgraded 2.9.59 servers but passed under the configuration of Lustre 2.9.0 client with freshly installed 2.9.59 servers.<br/> I did some experiments under the configuration of Lustre 2.9.0 client with freshly installed 2.9.59 servers and found the failure occurred in another way:</p> <div class="panel" style="border-width: 1px;"><div class="panelContent"> <p>&#35; pwd<br/> /mnt/lustre<br/> &#35; ls -s<br/> total 0<br/> &#35; mkdir -p 2561<br/> &#35; cd 2561<br/> &#35; ls -s<br/> total 0<br/> &#35; touch f1<br/> &#35; ls -s f1<br/> 0 f1<br/> &#35; echo a &gt; f1<br/> <font color="#FF0000">&#35; ls -s f1</font><br/> <font color="#FF0000">1 f1</font><br/> <font color="#FF0000">&#35; ls -s f1</font><br/> <font color="#FF0000">4 f1</font></p> </div></div> <p>As we can see, the results of the latest two runs of "ls -s f1" were different. So, without upgrading, the issue also exists under interop configuration. Continue investigating.</p> <p>It turns out the above "ls -s" issue also exists on the latest master branch. I filed a new ticket <a href="https://jira.whamcloud.com/browse/LU-9708" title="&quot;ls -s&quot; result changed over time" class="issue-link" data-issue-key="LU-9708"><del>LU-9708</del></a> to track that.</p> <p>For the upgrading issue in this ticket, I did more experiments under the configuration of Lustre 2.9.0 client with rolling upgraded 2.9.59 servers and found:<br/> On 2.9.0 client:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># pwd /mnt/lustre # ls -s total 0 # touch f # ls -s f 4 f # stat f File: ‘f’ Size: 0 Blocks: 8 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502275 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2017-06-23 19:39:41.000000000 +0000 Modify: 2017-06-23 19:39:41.000000000 +0000 Change: 2017-06-23 19:39:41.000000000 +0000 Birth: - </pre> </div></div> <p>The allocated size of the touched file "f" is 4 blocks instead of 0. And the "Blocks" showed by "stat f" is 8 instead of 0.</p> <p>On 2.9.59 MDS server:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># umount /dev/vda3 # mount -t ldiskfs /dev/vda3 /mnt/lustre-mds1 # cd /mnt/lustre-mds1 # stat ROOT/f File: ‘ROOT/f’ Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: fd03h/64771d Inode: 134 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2017-06-23 19:39:41.000000000 +0000 Modify: 2017-06-23 19:39:41.000000000 +0000 Change: 2017-06-23 19:39:41.000000000 +0000 Birth: - </pre> </div></div> <p>Both "Size" and "Blocks" are 0.</p> <p>After unmouting and remounting all of the 2.9.0 clients, "ls -s f" is still 4 instead of 0. Touching a new file also hit the same issue.</p> <p>The same issue also can be reproduced with clean upgrading (even only upgrading OSS from Lustre 2.9.0 to 2.9.59). The result of "ls -s" on a newly touched file is 4 instead of 0.</p> <p>I think the test here should be improved. Something like changing the "<tt>ls -s &lt;file&gt;</tt>" check of the allocated block count to "<tt>stat -c '%s' &lt;file&gt;</tt>" to check the file size would also detect this problem, and would not suffer from the change in the allocated block count.</p> <p>Jian Yu (jian.yu@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/27961" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27961</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9663" title="sanity test_103a: LU-2561 test failed" class="issue-link" data-issue-key="LU-9663"><del>LU-9663</del></a> tests: replace “ls -s” with “stat -c '%s'” to get file size<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 23131216b6407114445456b0b46a4b5e2a88557d</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/27961/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27961/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9663" title="sanity test_103a: LU-2561 test failed" class="issue-link" data-issue-key="LU-9663"><del>LU-9663</del></a> tests: replace “ls -s” with “stat -c '%s'” to get file size<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 45b8c6cf6916cf012ad0512f9cf249c152b45236</p> <p>Minh Diep (minh.diep@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29816" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29816</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9663" title="sanity test_103a: LU-2561 test failed" class="issue-link" data-issue-key="LU-9663"><del>LU-9663</del></a> tests: replace “ls -s” with “stat -c '%s'” to get file size<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 96cc82bbd18b0f5daa847a5a4027db6ea9d6855b</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/29816/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29816/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9663" title="sanity test_103a: LU-2561 test failed" class="issue-link" data-issue-key="LU-9663"><del>LU-9663</del></a> tests: replace “ls -s” with “stat -c '%s'” to get file size<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: f80d37cea5f5c4ad971afa1eac47dda6859c6a2a</p> Related LU-2561 Development Rank 1|hzzf27: Rank (Obsolete) 9223372036854775807 Severity