Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 [LU-10252] backport change LU-5718 change 12451/12 to b2_8_fe https://jira.whamcloud.com/browse/LU-10252 Lustre <p>We are encountering the issue described above on clients running lustre 2.8 with Omni-Path fabrics.  We would like the patch backported to b2_8_fe.</p> <p>Console log messages</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2017-11-12 10:20:23 [763673.420307] LNetError: 6383:0:(o2iblnd_cb.c:1105:kiblnd_init_rdma()) RDMA has too many fragments for peer 192.168.134.10@o2ib27 (256), src idx/frags: 128/256 dst idx/frags: 128/256 2017-11-12 10:20:23 [763673.438365] LNetError: 6383:0:(o2iblnd_cb.c:434:kiblnd_handle_rx()) Can't setup rdma for PUT to 192.168.134.10@o2ib27: -90 2017-11-12 10:23:00 [763830.403553] Lustre: 8245:0:(client.c:2063:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510510823/real 1510510823] req@ffff88102df6e300 x1583085670240648/t0(0) o4-&gt;lsh-OST0009-osc-ffff881035fb1000@172.19.3.26@o2ib600:6/4 lens 608/448 e 2 to 1 dl 1510510980 ref 2 fl Rpc:X/0/ffffffff rc 0/-1 2017-11-12 10:23:00 [763830.435966] Lustre: lsh-OST0009-osc-ffff881035fb1000: Connection to lsh-OST0009 (at 172.19.3.26@o2ib600) was lost; in progress operations using this service will wait for recovery to complete 2017-11-12 10:23:00 [763830.455086] Lustre: Skipped 42 previous similar messages 2017-11-12 10:23:00 [763830.488005] Lustre: lsh-OST0009-osc-ffff881035fb1000: Connection restored to 172.19.3.26@o2ib600 (at 172.19.3.26@o2ib600) </pre> </div></div> fs/lustre-release-fe LU-10252 backport change LU-5718 change 12451/12 to b2_8_fe Bug Major Resolved Fixed Sonia Sharma Olaf Faaland Thu, 16 Nov 2017 22:59:58 +0000 Wed, 20 Dec 2017 01:41:46 +0000 Thu, 7 Dec 2017 18:34:49 +0000 0 3 <p>We see this in Lustre: Build Version: 2.8.0_8.chaos.</p> <p>One of our larger clusters (~2600 clients) was moved to Lustre 2.8 recently.  We expect to see many more instances of this based on the frequency we see on a small (~128 client) test cluster.</p> <p>The first question is whether the patch mentioned, <a href="https://review.whamcloud.com/#/c/12451/12" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/12451/12</a> is the complete fix for this, or whether other patches are required.</p> <p>The second question is whether this is a low risk backport (it seems to me that it is, but you would know better).</p> <p>Thanks.</p> <p>I can't figure out how to Link this ticket to <a href="https://jira.whamcloud.com/browse/LU-5718" title="RDMA too fragmented with router" class="issue-link" data-issue-key="LU-5718"><del>LU-5718</del></a> so am noting that here.</p> <p>Hi Amir,</p> <p>Can you help with this one?</p> <p>Thanks,</p> <p>Brad</p> <p>Here is the link to the patch for -<del><a href="https://jira.whamcloud.com/browse/LU-5718" title="RDMA too fragmented with router" class="issue-link" data-issue-key="LU-5718"><del>LU-5718</del></a></del>- added to b2_8_fe. It still needs to be tested and reviewed.</p> <p><a href="https://review.whamcloud.com/#/c/30184/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/30184/</a></p> <p>Please note it needs to be accompanied by the fix from -<del><a href="https://jira.whamcloud.com/browse/LU-9420" title="Bad Check slipped into repo" class="issue-link" data-issue-key="LU-9420"><del>LU-9420</del></a></del> and <del><a href="https://jira.whamcloud.com/browse/LU-9425" title="Turn RDMA Fragment Fix on by default" class="issue-link" data-issue-key="LU-9425"><del>LU-9425</del></a></del>-. Backported patch for these -<br/> <a href="https://review.whamcloud.com/#/c/30185/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/30185/</a></p> <p><a href="https://review.whamcloud.com/#/c/30284/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/30284/</a></p> <p>Patch is back ported and appears in Sonia's comment above. Is anything else needed here?</p> Related LU-5718 Development Rank 1|hzznzb: Rank (Obsolete) 9223372036854775807 Severity