https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5475 Lustre <p>A directory tree was copied into a subdirectory of Lustre. At least one of the subdirectories in the newly created tree in Lustre does not appear in its parent's directory listing. The directory <em>does</em> exist, and it is possible to <tt>cd</tt> into that unlisted directory.</p> <p>The same behavior is exhibited from multiple clients, so it is not a problem of just one client's cache being corrupt.</p> <p>We are seeing this with the LLNL branch of Lustre 2.4.2 (github.com/chaos/lustre).</p> <p>We could not identify any console messages associated with the problem.</p> <p>The problem was seen on the secure network, so we cannot directly provide any logs.</p> <p>This problem has suspiciously similar symptoms in common with <a href="https://jira.whamcloud.com/browse/LU-5254" title="readdir missing a directory" class="issue-link" data-issue-key="LU-5254"><del>LU-5254</del></a>.</p> LU-5475

readdir missing a directory

Bug Major Resolved Duplicate Oleg Drokin Christopher Morrone llnl Mon, 11 Aug 2014 20:14:54 +0000 Mon, 11 May 2015 21:10:20 +0000 Thu, 19 Feb 2015 15:48:31 +0000 Lustre 2.4.2 Lustre 2.7.0 Lustre 2.5.4 0 4 <p>Hm, not a lot of data in here, unfortunately.<br/> Do you have any idea if the clients that don't see the directory all had accessed the parent dir before the problem ensued (meaning the all might have a stale cache problem) or if a client not previously exposed to this directory also does not see it (a server side of some problem)?<br/> I think in 5254 new clients saw the directory so there it appeared like a cache problem on client side to me.</p> <p>It is odd that in both cases zfs is used server side. Lustre itself does not really cache anything server-side, so should there be some odd interaction within zfs (or between lustre and zfs of course) that would hide a directory from appearing in readdir output, this is exactly what would be seen.</p> <p>Is there anything else known? E.g. was there a lot of stuff in the parent dir (possibly an issue of skipping an entry between pages or something)? If you create the same parent dir with the same names inside in a different lustre place - does the problem reappear by any chance?</p> <blockquote><p>Do you have any idea if the clients that don't see the directory all had accessed the parent dir before the problem ensued (meaning the all might have a stale cache problem) or if a client not previously exposed to this directory also does not see it (a server side of some problem)?</p></blockquote> <p>It is not a stale cache problem. Nodes that have never seen the directory before (or have had their cache cleared) do not see the directory.</p> <blockquote><p>Is there anything else known?</p></blockquote> <p>Note really.</p> <blockquote><p>If you create the same parent dir with the same names inside in a different lustre place - does the problem reappear by any chance?</p></blockquote> <p>No, it is not that easily reproduced.</p> <p>I have reproduced a very similar issue where readdir is missing a file. I can reproduce this on a ZFS backed MDT with high regularity.</p> <p>Just to be clear - updates are on <a href="https://jira.whamcloud.com/browse/LU-3573" title="lustre-rsync-test test_8: @@@@@@ FAIL: Failure in replication; differences found. " class="issue-link" data-issue-key="LU-3573"><del>LU-3573</del></a></p> <p>Chris</p> <p>The patch for <a href="https://jira.whamcloud.com/browse/LU-3573" title="lustre-rsync-test test_8: @@@@@@ FAIL: Failure in replication; differences found. " class="issue-link" data-issue-key="LU-3573"><del>LU-3573</del></a> has landed to master. Do you have any known cases of affected files to verify the fix?</p> <p>Peter</p> <p>Heads up that <a href="https://jira.whamcloud.com/browse/LU-5924" title="conf-sanity test_32b: list verification failed " class="issue-link" data-issue-key="LU-5924"><del>LU-5924</del></a> seems related to the <a href="https://jira.whamcloud.com/browse/LU-3573" title="lustre-rsync-test test_8: @@@@@@ FAIL: Failure in replication; differences found. " class="issue-link" data-issue-key="LU-3573"><del>LU-3573</del></a> fix</p> <p>Just to capture that the latest version of <a href="https://jira.whamcloud.com/browse/LU-3573" title="lustre-rsync-test test_8: @@@@@@ FAIL: Failure in replication; differences found. " class="issue-link" data-issue-key="LU-3573"><del>LU-3573</del></a> has been landed to master for over a month and back ported to b2_5 also. I think that this should be safe to try out</p> <p>This was fixed by <a href="https://jira.whamcloud.com/browse/LU-3573" title="lustre-rsync-test test_8: @@@@@@ FAIL: Failure in replication; differences found. " class="issue-link" data-issue-key="LU-3573"><del>LU-3573</del></a></p> Related LU-5924 LU-3573 LU-5254 Development Rank 1|hzwten: Rank (Obsolete) 15252 Severity