https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2418 Lustre <p>Today, it is very difficult to confirm whether timeouts in Lustre are due to dropped packets in LNet. This is due to two reasons:</p> <p>1- neterrors are off by default so logging does not show dropped packets.<br/> 2- the errors counter is never incremented (see <a href="https://jira.whamcloud.com/browse/LU-2223" title="LNet counter &quot;errors&quot; never gets incremented" class="issue-link" data-issue-key="LU-2223">LU-2223</a>).</p> <p>My understanding is that neterrors are off by default because there is too much "noise" when they are on. That begs the question: how can logs which are issued that frequently be considered errors?</p> <p>I think this issue can be address in one of two ways:</p> <p>1- Clean up the neterror logs so they are not noisy and then leave neterrors on by default.<br/> 2- Add a set of new counters to LNet to count the reasons for dropped packets.</p> LU-2418

Add Way to Detect Dropped Packets on Production Systems

Improvement Major Open Unresolved WC Triage Doug Oucharek Fri, 30 Nov 2012 19:57:15 +0000 Mon, 13 Jun 2016 23:40:37 +0000 Lustre 2.4.0 0 2 <p>I remembered that they were too noisy in sites where there's always 10s of nodes down for maintenance e.g., but there's repeated attempts to communicate with them e.g. router pinger or upper layers. So if a same error happened with 50 nodes, there'd be 50 such error messages, instead of one that says this error happened with these 50 nodes.</p> <p>It became even worse at sites where console outputs of servers were gathered into one place.</p> <p><a href="https://jira.whamcloud.com/browse/LU-8223" title="De-Noise LNet neterr logs so they can be ON by default" class="issue-link" data-issue-key="LU-8223">LU-8223</a> implements one part of this solution: get neterrors on by default.</p> Duplicate LU-8223 Development Rank 1|hzvd5r: Rank (Obsolete) 5735