https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12661 Lustre <p>This issue was created by maloo for jianyu &lt;yujian@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/60697574-bbc3-11e9-a25b-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/60697574-bbc3-11e9-a25b-52540065bddc</a></p> <p>test_817 failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== sanity test 817: nfsd won't cache write lock for exec file ======================================== 13:19:00 (1565381940) /usr/lib64/lustre/tests/sanity.sh: line 21700: /mnt/lustre/nfsexp/true: Text file busy sanity test_817: @@@@@@ FAIL: failed to execute 'true' command </pre> </div></div> <p>&lt;&lt;Please provide additional information about the failure here&gt;&gt;</p> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> sanity test_817 - failed to execute 'true' command</p> LU-12661

sanity test_817: FAIL: failed to execute 'true' command

Bug Minor Open Unresolved Dongyang Li Maloo always_except rhel8 sles12 sles15 Tue, 13 Aug 2019 02:26:37 +0000 Mon, 23 Jan 2023 18:18:25 +0000 Lustre 2.13.0 Lustre 2.12.4 Lustre 2.12.5 Lustre 2.12.6 Lustre 2.12.7 Lustre 2.12.8 Upstream 0 12 <p>Is this a ARM only bug?</p> <p>We see this with RHEL8 clients; <a href="https://testing.whamcloud.com/test_sets/fff5bbfc-d70c-11e9-9fc9-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/fff5bbfc-d70c-11e9-9fc9-52540065bddc</a></p> <p>Dongyang</p> <p>Could you please investigate?</p> <p>Thanks</p> <p>Peter</p> <p>Li Dongyang (dongyangli@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/36712" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36712</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 if kernel version &gt;= 4.14<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5a648bdc968a6bd4bcd776dcd5ad41829e71d92d</p> <p>On new kernels nfsd is not releasing the file after write, it will fail with ETXTBSY regardless of whether the NFS export is backed by a Lustre mount or not. That is because newer kernels delay fput() in the write file descriptor, holding it open with write mode and preventing it from being opened in exec mode. </p> <p>Oleg, isn't there a patch to drop the write mode lock more quickly, or similar, that would fix this?</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36712/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36712/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 if kernel version &gt;= 4.14<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 4fed33473ca2964ff19f61fdb8501b2210f923de</p> <p>The patch <a href="https://review.whamcloud.com/32265" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32265</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-4398" title="mdt_object_open_lock() may not flush conflicting handles" class="issue-link" data-issue-key="LU-4398"><del>LU-4398</del></a> llite: do not cache write open lock for exec file</tt>" was landed as 2.12.56-76-g6dd9d57bc0, which was just before the 2.12.58 build that this was first reported on, so there is likely a connection. I would have thought the 32265 patch would <b>fix</b> this problem, but maybe it only became evident for the newer kernels because sanity test_817 was re-enabled by that patch?</p> <p>I just tested 32265 on a centos8 box to be sure. the issue is still there.</p> <p>It can be reproduced on 4.14+ without lustre, so again it is a kernel problem.</p> <p>The same failure also occurred on SLES15 SP1 client (Kernel version 4.12.14-197.29-default):<br/> <a href="https://testing.whamcloud.com/test_sets/86cd8382-4a79-11ea-b58e-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/86cd8382-4a79-11ea-b58e-52540065bddc</a></p> <p>I see this with the linux lustre client as well.</p> <p>Jian Yu (yujian@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/38259" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38259</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 if kernel version &gt;= 4.14<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: a011003f4a99402c87c758558fe1f643ca2f3708</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38259/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38259/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 if kernel version &gt;= 4.14<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 42923270537d52d2b90c59a2db16d428b4f5a90c</p> <p>it also fails on SLES 12.5 (kernel version is 4.12.14-122.20-default)<br/> <a href="https://testing.whamcloud.com/test_sets/960091e2-c4eb-423f-a2cc-4c5ec02118a3" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/960091e2-c4eb-423f-a2cc-4c5ec02118a3</a> </p> <p>Dongyang, I see <a href="#comment-263035" target="_blank" rel="noopener">comment-263035</a> and <a href="#comment-276924" target="_blank" rel="noopener">comment-276924</a> report failures for SLES 4.<b>12</b>.14 kernels, but the patch skips kernels 4.<b>14</b> and later. Is that a typo in the patch? Should this test be skipped for kernels &gt;= 4.12 instead?</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/39838" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39838</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 for kernel 4.12+<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 16b41461822854124ac074aebe394602909644cc</p> <p>Andreas, the patch is only skipping for kernel 4.14 because we were only seeing failures on rhel8, I suspect we might need to skip for kernels &gt;= 4.10, we will see.</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/39838/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39838/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 for kernel 4.12+<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3e2c28437404b0ccbd7bbfb8f77788678975b63d</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/39863" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39863</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 for kernel 4.12+<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 45c8a5cda0727c8ed4aa0e3d01f1edf22ea783d0</p> <p>+1 in 2.12.8 tests: <a href="https://testing.whamcloud.com/test_sets/f95832ef-44a3-4376-8e8e-7b2bb408d560" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/f95832ef-44a3-4376-8e8e-7b2bb408d560</a></p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> == sanity test 817: nfsd won't cache write lock <span class="code-keyword">for</span> exec file ======================================== 04:32:28 (1637382748) /usr/lib64/lustre/tests/sanity.sh: line 22011: /mnt/lustre/nfsexp/<span class="code-keyword">true</span>: Text file busy sanity test_817: @@@@@@ FAIL: failed to execute <span class="code-quote">'<span class="code-keyword">true</span>'</span> command </pre> </div></div> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/39863/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39863/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> tests: skip sanity 817 for kernel 4.12+<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: dbcc2c55e1a23337d2b47ed3a9549784c47b4208</p> <p>"jsimmons &lt;jsimmons@infradead.org&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49732" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49732</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12661" title="sanity test_817: FAIL: failed to execute &#39;true&#39; command" class="issue-link" data-issue-key="LU-12661">LU-12661</a> llite: don't take open cach lock for files with exec bit set<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: e4ebba6b93ac5eb621efe9f2c3fab6025a3b24a5</p> Related LU-4398 LU-10457 LU-8585 LU-12511 Development Rank 1|i00l5b: Rank (Obsolete) 9223372036854775807 Severity