https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-171 Lustre <p>This issue was created by maloo for Prakash Surya &lt;surya1@llnl.gov&gt;</p> <p>This issue relates to the following test suite run: <a href="https://maloo.whamcloud.com/test_sets/30252906-594f-11e0-a272-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/30252906-594f-11e0-a272-52540025f9af</a>.</p> <p>The cache is not being flushed to disk before the Lustre filesystem is unmounted. After the Lustre unmount and remount, the files on the Lustre filesystem are all empty. This is what causes the file diffs to fail.</p> <p>If I add the command 'echo 3 &gt; /proc/sys/vm/drop_caches' just before the command to unmount Lustre, the tests will pass.</p> LU-171

Test failure on test suite runtests

Bug Minor Resolved Fixed Jian Yu Maloo Mon, 28 Mar 2011 08:27:49 +0000 Wed, 6 Apr 2011 11:56:08 +0000 Wed, 6 Apr 2011 11:56:08 +0000 0 5 <p>Yu Jian</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>Hi Prakash,</p> <p>Did you run 'sync' before 'echo 3 &gt; /proc/sys/vm/drop_caches'? The latter command is just to drop clean pagecache, dentries and inodes from memory. It does not free dirty objects, nor flush the data out to disk.</p> <p>In addition, I saw this message in the test output:<br/> cp: cannot create regular file `/sbin/./mount.lustre': Permission denied<br/> Did you run something special before re-mounting the Lustre filesystem?</p> <p>Yu Jian,</p> <p>As long as 'echo 3 &gt; /proc/sys/vm/drop_caches' is being called, the test will pass whether sync is called or not. I have tried both, calling sync before and not calling it at all, and it makes no difference. Also, the test with not pass without 'echo 3 &gt; /proc/sys/vm/drop_caches', even if 'sync' is still called.</p> <p>Just in case this proves useful, here are the results of the four tests:<br/> Without drop_caches or sync: <a href="https://maloo.whamcloud.com/test_sets/7ec49f8e-5c8c-11e0-a272-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/7ec49f8e-5c8c-11e0-a272-52540025f9af</a><br/> With only drop_caches (no call to sync): <a href="https://maloo.whamcloud.com/test_sets/8e1e510a-5c8c-11e0-a272-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/8e1e510a-5c8c-11e0-a272-52540025f9af</a><br/> With sync before drop_caches: <a href="https://maloo.whamcloud.com/test_sets/946183ac-5c8c-11e0-a272-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/946183ac-5c8c-11e0-a272-52540025f9af</a><br/> With only sync (no call to drop_caches): <a href="https://maloo.whamcloud.com/test_sets/9b0749a8-5c8c-11e0-a272-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/9b0749a8-5c8c-11e0-a272-52540025f9af</a></p> <p>As far as the 'cp: cannot create regular file `/sbin/./mount.lustre': Permission denied' error, I believe that is because of the way our node is configured. I am running these tests on a diskless node, and the /sbin directory is mounted read only, which causes the copy to fail. Although, the mount.lustre binary is already installed in /sbin on the diskless image (which explains the 'Permission denied' error, rather than a 'Read-only file system' error).</p> <p>Thanks Prakash for the tests.</p> <p>From the Maloo reports, I found the kernel version was 2.6.32-14chaos. And the failure reported in this ticket is very similar to the one in <a href="https://bugzilla.lustre.org/show_bug.cgi?id=23064" class="external-link" target="_blank" rel="nofollow noopener">bug 23064</a>. There are two patches for this bug:</p> <ol> <li><a href="https://bugzilla.lustre.org/attachment.cgi?id=32045" class="external-link" target="_blank" rel="nofollow noopener">attachment 32045</a></li> <li><a href="https://bugzilla.lustre.org/attachment.cgi?id=32564" class="external-link" target="_blank" rel="nofollow noopener">attachment 32564</a></li> </ol> <p>The first patch was pushed to master branch on Nov. 4, 2010. The second one was ported to master branch and landed on Mar. 24, 2011 with the other patches in <a href="http://review.whamcloud.com/307" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/307</a>.</p> <p>Could you please check whether the Lustre codes you used have the patches or not? If yes, could you please set "PTLDEBUG=-1", reproduce the issue again and upload the lctl debug log file (gathered by the test script right after the test failed) to this ticket?</p> <p>FYI, the issue could not be reproduced on RHEL6/x86_64 with kernel 2.6.32-71.18.2.el6 against the latest master codes on our test node. Here is the successful report:<br/> <a href="https://maloo.whamcloud.com/test_sets/f4ffd0c6-6031-11e0-a2b4-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/f4ffd0c6-6031-11e0-a2b4-52540025f9af</a></p> <p>Thanks for the info Yu Jian.</p> <p>I pulled down the latest master branch this morning and have not been able to reproduce the issue. Both patches you refer to in your previous comment are definitely in this tree.</p> <p>I beleive the previous tree I was working on did not have the patches from <a href="http://review.whamcloud.com/307" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/307</a>, which is likely the reason for the failed test.</p> <p>Here are the new successful test results:<br/> Run #1: <a href="https://maloo.whamcloud.com/test_sets/d39dafe6-606a-11e0-a2b4-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/d39dafe6-606a-11e0-a2b4-52540025f9af</a><br/> Run #2: <a href="https://maloo.whamcloud.com/test_sets/d7174b32-606a-11e0-a2b4-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/d7174b32-606a-11e0-a2b4-52540025f9af</a></p> <p>Thanks for the help! I imagine this ticket can be marked as resolved.</p> <p>Thanks for letting us know Prakash!</p> Development Rank 1|hzw167: Rank (Obsolete) 10271 Severity