https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16101 Lustre <p>This issue was created by maloo for jianyu &lt;yujian@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/4751c6e1-6efd-4bec-8150-035008e531bb" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/4751c6e1-6efd-4bec-8150-035008e531bb</a></p> <p>test_27J failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lov_foreign_magic: 0x0BD70BD0 lov_xattr_size: 89 lov_foreign_size: 73 lov_foreign_type: 1 lov_foreign_flags: 0x0000DA08 lfm_magic: 0x0BD70BD0 lfm_length: 73 lfm_type: 0x00000000 (none) lfm_flags: 0x0000DA08 lfm_value: '138822a8-8810-46e8-9f71-e80e38c85596@4921d343-f166-41b7-83de-3ada0c94dfbd' lfs setstripe: setstripe error for '/mnt/lustre/d27J.sanity/f27J.sanity': stripe already set lfs setstripe: setstripe error for '/mnt/lustre/d27J.sanity/f27J.sanity2': stripe already set sanity test_27J: @@@@@@ FAIL: /mnt/lustre/d27J.sanity/f27J.sanity: read should fail Trace dump: </pre> </div></div> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> sanity test_27J - /mnt/lustre/d27J.sanity/f27J.sanity: read should fail</p> SLES15 SP4 client LU-16101

sanity test_27J: read should fail

Bug Minor Resolved Fixed Andreas Dilger Maloo Tue, 23 Aug 2022 00:38:28 +0000 Thu, 8 Feb 2024 09:20:16 +0000 Mon, 17 Jul 2023 21:22:26 +0000 Lustre 2.15.2 Lustre 2.15.3 Lustre 2.15.4 Lustre 2.16.0 0 14 <p>it seems this starting to fail on master after landing <a href="https://jira.whamcloud.com/browse/LU-15959" title="support for SLES 15 SP4" class="issue-link" data-issue-key="LU-15959"><del>LU-15959</del></a> sles15sp4 kernel support</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=neilb" class="user-hover" rel="neilb">neilb</a> <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=stancheff" class="user-hover" rel="stancheff">stancheff</a> <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=simmonsja" class="user-hover" rel="simmonsja">simmonsja</a> is this issue on your radars?</p> <p>The only change was in the vvp layer due to an export issue. I doubt its due to this patch. Instead its a regression showing up on this platform.</p> <p>This test failure is due to upstream Commit 8c8387ee3f55 ("mm: stop filemap_read() from grabbing a superfluous page").</p> <p>This landed in v5.16, and SUSE has backported it to our SLE-15-SP4 kernels.</p> <p>In earlier kernels the read will fail because filemap_read() will call the -&gt;readpage method which detects the problem and reports -ENODATA.</p> <p>In later kernels filemap_read() doesn't bother calling -&gt;readpage because the size of the file is recorded in the inode as zero.  As -&gt;readpage is not called, -ENODATA is not reported - there is no error.</p> <p>To trigger a read error, we would need to make the file appear to be larger than 0.</p> <p>If the file size isn't available conveniently we might have to bypass generic_file_read_iter() - for foreign files at least.  Or maybe my i_size of foreign files MAX_INT</p> <p>similar in sanity-lfsck on 2.15.2-rc1</p> <p><a href="https://testing.whamcloud.com/test_sets/2b0561f0-3a4a-4646-81af-8f3307966170" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/2b0561f0-3a4a-4646-81af-8f3307966170</a></p> <p>Can we add this test to the always_except list for SLES15 SP4 while we are working on the proper fix? It seems to be causing quite a bit of disruption...</p> <p>"Jian Yu &lt;yujian@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49970" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49970</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: add sanity/27J to always_except for SLES15 SP4<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 49ad83985add99bb341785289819c44bc5b37a2c</p> <p>"Jian Yu &lt;yujian@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49971" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49971</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: add sanity/27J to always_except for SLES15 SP4<br/> Project: fs/lustre-release<br/> Branch: b2_15<br/> Current Patch Set: 1<br/> Commit: f82674690637802bdab8dc38a2b14240579557b5</p> <blockquote> <p>In later kernels filemap_read() doesn't bother calling -&gt;readpage because the size of the file is recorded in the inode as zero. As -&gt;readpage is not called, -ENODATA is not reported - there is no error.<br/> To trigger a read error, we would need to make the file appear to be larger than 0.</p></blockquote> <p>Jian's patch will skip this subtest for SLES15sp4 so that it doesn't always fail, but it doesn't fix the problem. Presumably there is something that needs to be done in llite to update the inode with the actual file size, instead of it being zero?</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49970/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49970/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: add sanity/27J to always_except<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 63dd644747f4eab20d640b4d87060e56c20bc37f</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49971/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49971/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: add sanity/27J to always_except<br/> Project: fs/lustre-release<br/> Branch: b2_15<br/> Current Patch Set: <br/> Commit: 1fc8fdf9070bdaf21050717124f4d511cd863d15</p> <blockquote> <p>This test failure is due to upstream Commit 8c8387ee3f55 ("mm: stop filemap_read() from grabbing a superfluous page").</p></blockquote> <p>RHEL 9.2 Beta release with kernel 5.14.0-283.el9 also has this commit:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedHeader panelHeader" style="border-bottom-width: 1px;"><b>kernel.spec</b></div><div class="preformattedContent panelContent"> <pre>* Mon Oct 24 2022 Frantisek Hrbata &lt;fhrbata@redhat.com&gt; [5.14.0-179.el9] &lt;~snip~&gt; - mm: stop filemap_read() from grabbing a superfluous page (Chris von Recklinghausen) [2120352] </pre> </div></div> <p>I was wondering if the test could be changed, rather than expect the read should return an error, it should check that the read returns 0 bytes. However, it seems reasonable that the DAOS code (and user applications) should receive an error if it reads from a layout that is not available. Otherwise, applications may assume the file is corrupted instead of just not mapped in correctly.</p> <p>Consider the behavior for files that are HSM released - we don't want the clients/applications to "successfully" return 0 bytes when reading such a file, but (normally) block until the file is restored, or in the worst case (some kind of client bug or copytool error) return an error because the file is inaccessible.</p> <p>"Andreas Dilger &lt;adilger@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51567" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51567</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: skip sanity/27J for more kernels<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 754adcb735973b71aa51c97828d79287b8624d0f</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51567/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51567/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: skip sanity/27J for more kernels<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b711af7d243f3773cec3a37f64c0e0aa8bbc363f</p> <p><del>"Patrick Farrell &lt;pfarrell@whamcloud.com&gt;" uploaded a new patch:</del> <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51696" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51696</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-16101" title="sanity test_27J: read should fail" class="issue-link" data-issue-key="LU-16101"><del>LU-16101</del></a> tests: skip 27J for 5.14 kernels</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: b445b885b490c72edde0fff4afeff6befe0fe46a</del></p> <p><del>We forgot Jian's comment here that the problematic patch is in RHEL 5.14 kernels (and I think SLES as well from earlier notes?), so this is still failing.  I've pushed a patch.</del></p> <p> </p> <p>No we didn't - sorry, sloppy reading on my part.  5.14 is included in the skip range...</p> <p>The current patch is already skipping this subtest for all kernels between 5.12.0-6.2.0, which is from when this change was first introduced until Yingjin's fix was landed, so I don't think anything else is needed here.</p> <p>&gt; sarah Sarah Liu added a comment - 21/Dec/22 7:15 PM - edited<br/> &gt; similar in sanity-lfsck on 2.15.2-rc1<br/> &gt; <a href="https://testing.whamcloud.com/test_sets/2b0561f0-3a4a-4646-81af-8f3307966170" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/2b0561f0-3a4a-4646-81af-8f3307966170</a></p> <p>right, sanity-lfsck/test_38() needs the same change for same reason.</p> <p>I have opened <a href="https://jira.whamcloud.com/browse/LU-17146" title="sanity-lfsck test_38: read should fail" class="issue-link" data-issue-key="LU-17146">LU-17146</a> to address this.</p> <p>This issue was hit in this patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49236" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49236</a><br/> The problem is that I'm doing interop testing between 2.15.4 (client) and master (server). Since the patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51567" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51567</a> is not on b2_15, the client will run the test on servers with 5.14 kernel version.</p> Related LU-17146 Development Rank 1|i02xnz: Rank (Obsolete) 9223372036854775807 Severity