https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1021 Lustre <p>When the user ran "find" or "du -sh" command on the client, MDS dumps the many call traces and the error messages and the can't response to the clients. (MDS didn't hang and down, but the response was really slow).<br/> I'm attaching the /var/log/messages and we can see many call traces on these log files. I'm not sure, all are caused by same problem, but at least the user ran several "find and "du -sh" commands on Jan 17 and Jan 18.</p> LU-1021

MDS dumps call traces after run find and du commands

Bug Minor Resolved Duplicate Johann Lombardi Shuichi Ihara Sun, 22 Jan 2012 04:47:31 +0000 Fri, 31 Aug 2012 11:16:50 +0000 Fri, 31 Aug 2012 11:16:50 +0000 Lustre 1.8.6 0 6 <p>These all look like MDS getting stuck and spending significant time in an attempt to allocate some memory and then getting stuck on resource semaphore in ldlm pool cleaning thread.</p> <p>I think there was entire class of patches accepted that would prevent lustre allocations from diving into this code.</p> <p>Would be good to get Johann take a look as he must remember that work.</p> <p>I remember such changes on the client side, but not on the MDS side.</p> <p>That said, there are many messages indicating that the threads got finally unstuck:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Jan 15 20:47:37 ALPL505 kernel: Lustre: Service thread pid 14282 completed after 1630.26s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). Jan 15 22:00:44 ALPL505 kernel: Lustre: Service thread pid 15016 completed after 343.70s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). Jan 15 23:50:51 ALPL505 kernel: Lustre: Service thread pid 14956 completed after 746.53s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). Jan 15 23:50:51 ALPL505 kernel: Lustre: Service thread pid 14308 completed after 746.52s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). ... </pre> </div></div> <p>So it seems that the system isn't really deadlocked, "just" awefully slow. Can you reproduce this issue easily?</p> <p>BTW, do you apply any patches on top of 1.8.6-wc1?</p> <p>This looks like a dupe of <a href="https://jira.whamcloud.com/browse/LU-1535" title="LustreError: 1843:0:(mds_open.c:1645:mds_close()) " class="issue-link" data-issue-key="LU-1535"><del>LU-1535</del></a>, can someone confirm?</p> <p>Lai could you please confirm whether this is a duplicate of <a href="https://jira.whamcloud.com/browse/LU-1535" title="LustreError: 1843:0:(mds_open.c:1645:mds_close()) " class="issue-link" data-issue-key="LU-1535"><del>LU-1535</del></a>?</p> <p>Yes, it is.</p> <p>Thanks Lai!</p> Development Rank 1|hzvhhb: Rank (Obsolete) 6481 Severity