https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5490 Lustre <p>sanity test 133d failed as follows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== sanity test 133d: Verifying rename_stats ========================================== 01:32:10 (1408005130) CMD: onyx-47vm7 /usr/sbin/lctl list_param mdt.*.rename_stats mdt.lustre-MDT0000.rename_stats CMD: onyx-47vm7 /usr/sbin/lctl set_param mdt.*.rename_stats=clear mdt.lustre-MDT0000.rename_stats=clear total: 512 creates in 1.52 seconds: 336.70 creates/second source rename dir size: 64K target rename dir size: 8K CMD: onyx-47vm7 /usr/sbin/lctl get_param mdt.*.rename_stats mdt.lustre-MDT0000.rename_stats= rename_stats: - snapshot_time: 1408005133.29235 - same_dir 512bytes: { sample: 1, pct: 100, cum_pct: 100 } CMD: onyx-47vm7 /usr/sbin/lctl get_param mdt.*.rename_stats CMD: onyx-47vm7 /usr/sbin/lctl get_param mdt.*.rename_stats sanity test_133d: @@@@@@ FAIL: samedir_rename_size error </pre> </div></div> <p>Maloo report: <a href="https://testing.hpdd.intel.com/test_sets/4b98ba1c-23f5-11e4-b2ba-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/4b98ba1c-23f5-11e4-b2ba-5254006e85c2</a></p> Lustre build: <a href="https://build.hpdd.intel.com/job/lustre-master/2615/">https://build.hpdd.intel.com/job/lustre-master/2615/</a><br/> Distro/Arch: RHEL6.5/x86_64<br/> FSTYPE=zfs LU-5490

sanity test_133d: FAIL: samedir_rename_size error

Bug Minor Resolved Fixed Nathaniel Clark Jian Yu zfs Thu, 14 Aug 2014 22:34:09 +0000 Thu, 3 May 2018 21:11:30 +0000 Thu, 15 Mar 2018 14:13:48 +0000 Lustre 2.7.0 Lustre 2.5.3 Lustre 2.9.0 Lustre 2.11.0 Lustre 2.10.4 0 13 <p>Lustre Build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/84/" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-b2_5/84/</a><br/> Distro/Arch: RHEL6.5/x86_64<br/> FSTYPE=zfs</p> <p>The same failure occurred:<br/> <a href="https://testing.hpdd.intel.com/test_sets/880a64a2-2b23-11e4-bb80-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/880a64a2-2b23-11e4-bb80-5254006e85c2</a></p> <p>I hit this failure on master (pre-2.8.0) in review-zfs. The results are at <a href="https://testing.hpdd.intel.com/test_sessions/12f05e44-d21a-11e4-a005-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/12f05e44-d21a-11e4-a005-5254006e85c2</a></p> <p>Another failure instance on master in review-zfs: <a href="https://testing.hpdd.intel.com/test_sets/efa92684-eaf9-11e4-95aa-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/efa92684-eaf9-11e4-95aa-5254006e85c2</a></p> <p>Between this failure and <a href="https://jira.whamcloud.com/browse/LU-4203" title="Test failure sanity test_133d: crossdir_rename_size error " class="issue-link" data-issue-key="LU-4203"><del>LU-4203</del></a> there are 11 failures in the past month. I suspect there is some inconsistency with how osd-zfs is recording the directory size in the rename stats (which I think is the number of entries) vs. how the test or /proc is checking them (which is the number of bytes). I don't think we need to change osd-zfs to record the stats differently, but rather fix the test so that it handles this consistently. It isn't clear why the test only fails intermittently.</p> <p>More failures on master at:<br/> 2015-07-10 03:43:38 - <a href="https://testing.hpdd.intel.com/test_sets/840b7844-26ec-11e5-b3d7-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/840b7844-26ec-11e5-b3d7-5254006e85c2</a><br/> 2015-07-22 20:01:38 - <a href="https://testing.hpdd.intel.com/test_sets/c4fdb934-30f1-11e5-a788-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/c4fdb934-30f1-11e5-a788-5254006e85c2</a><br/> 2015-08-24 15:36:45 - <a href="https://testing.hpdd.intel.com/test_sets/fb01ec64-4aad-11e5-88e8-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/fb01ec64-4aad-11e5-88e8-5254006e85c2</a><br/> 2016-01-04 01:56:27 - <a href="https://testing.hpdd.intel.com/test_sets/fafafd4a-b2b4-11e5-8114-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/fafafd4a-b2b4-11e5-8114-5254006e85c2</a><br/> 2016-02-02 00:40:04 - <a href="https://testing.hpdd.intel.com/test_sets/95eca8d4-c968-11e5-9e6a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/95eca8d4-c968-11e5-9e6a-5254006e85c2</a><br/> 2016-02-04 07:09:54 - <a href="https://testing.hpdd.intel.com/test_sets/57eaf714-cb3b-11e5-b49e-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/57eaf714-cb3b-11e5-b49e-5254006e85c2</a><br/> 2016-02-04 09:47:15 - <a href="https://testing.hpdd.intel.com/test_sets/4b7d3746-cb4d-11e5-be8d-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/4b7d3746-cb4d-11e5-be8d-5254006e85c2</a></p> <p>Nathaniel, could you take a crack at this if you have time. Not critical, but a source of ongoing annoyance for patch landings.</p> <p>"cause" of error seems to be size of directory reported by ls/stat is one thing (i.e. "64K") and size in mdt.lustre-MDT0000.rename_stats is another (i.e. "512bytes"). Not sure why the discrepancy yet, but this is why the test is failing.</p> <p>This is still happening on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/242be33c-e0d2-11e5-97b5-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/242be33c-e0d2-11e5-97b5-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/75019ae8-dfee-11e5-9400-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/75019ae8-dfee-11e5-9400-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/29b44c56-df1d-11e5-8471-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/29b44c56-df1d-11e5-8471-5254006e85c2</a></p> <p>another on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/233fe796-0749-11e6-b5f1-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/233fe796-0749-11e6-b5f1-5254006e85c2</a></p> <p>+1 on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/1d65eb6a-975b-11e7-b761-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/1d65eb6a-975b-11e7-b761-5254006e85c2</a></p> <p>Nathaniel Clark (nathaniel.l.clark@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29053" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29053</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5490" title="sanity test_133d: FAIL: samedir_rename_size error" class="issue-link" data-issue-key="LU-5490"><del>LU-5490</del></a> tests: FORTESTONLY<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 821baea4626beb70dbca822f3109d58180db3fd2</p> <p>This test can fail for DNE as test_133d only checks 1 MDS.</p> <p>Nathaniel Clark (nathaniel.l.clark@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31585" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31585</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5490" title="sanity test_133d: FAIL: samedir_rename_size error" class="issue-link" data-issue-key="LU-5490"><del>LU-5490</del></a> tests: Sanity/133d ensure stats read is on correct MDT<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: fa38f33d74bcd91ecf8768dde3dece69d629c099</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/31585/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31585/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5490" title="sanity test_133d: FAIL: samedir_rename_size error" class="issue-link" data-issue-key="LU-5490"><del>LU-5490</del></a> tests: Sanity/133d ensure stats read is on correct MDT<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 1d80fd72bf42f653a5d6a4a31fc2c5df571f1afc</p> <p>Landed for 2.11</p> <p>+1 on b2_10 <a href="https://testing.hpdd.intel.com/test_sets/0307fc3a-4887-11e8-960d-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/0307fc3a-4887-11e8-960d-52540065bddc</a></p> <p>Minh Diep (minh.diep@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32149" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32149</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5490" title="sanity test_133d: FAIL: samedir_rename_size error" class="issue-link" data-issue-key="LU-5490"><del>LU-5490</del></a> tests: Sanity/133d ensure stats read is on correct MDT<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: dcbadac5ad9143ad19e6db8afddcf050253a84ff</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/32149/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32149/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5490" title="sanity test_133d: FAIL: samedir_rename_size error" class="issue-link" data-issue-key="LU-5490"><del>LU-5490</del></a> tests: Sanity/133d ensure stats read is on correct MDT<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: 4eb9ba01430acf7f16d9a801c8cb6b39b2c75a27</p> Related LU-8066 LU-4203 Development Rank 1|hzwtsn: Rank (Obsolete) 15318 Severity