https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13638 Lustre <p>We need discard callback in some case.</p> LU-13638

lnet: discard the callback

Bug Minor Open Unresolved Yang Sheng Yang Sheng Fri, 5 Jun 2020 12:08:44 +0000 Tue, 8 Jun 2021 08:04:50 +0000 0 4 <p>Patch submit to: <a href="https://review.whamcloud.com/#/c/38845/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/38845/</a></p> <p>Yang Sheng (ys@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/40937" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40937</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13638" title="lnet: discard the callback" class="issue-link" data-issue-key="LU-13638">LU-13638</a> ptlrpc: addition change for previous commit<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8685dd43ec7fae51455a03433086265b0ccfad50</p> <p>patch <a href="https://review.whamcloud.com/#/c/38845/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/38845/</a> introcued a issue that ko2iblnd_shutdown never completed.<br/> A reproducible test case is below.</p> <ol> <li>Start Lustre with LNET-MR on the Infiniband network</li> <li>Turn off two IB ports on one of OSSs</li> <li>Umount OSTs on that particular OSS (assumed OSS failover)</li> <li>Two IB ports are back .</li> <li>OSTs are remounted on that OSS (assumed OSS failback)</li> <li>Stop all Lustre service and cleanup (lustre_rmmod) all lustre modules</li> </ol> <p>When lustre modules were unloaded on all OSSs, some of OSS's (or all of OSS) shutdown never completed due to hanging at ko2iblnd_shutdown. I also tried second patch <a href="https://review.whamcloud.com/40937" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40937</a>, but the problem was still exist.</p> <p>btw, if server applied patch <a href="https://jira.whamcloud.com/browse/LU-14499" title="o2iblnd: LU-13368 changes cause shutdown procedure to not complete" class="issue-link" data-issue-key="LU-14499">LU-14499</a> (reverted <a href="https://jira.whamcloud.com/browse/LU-13638" title="lnet: discard the callback" class="issue-link" data-issue-key="LU-13638">LU-13638</a> patch), this shutdown prolbem was gone. </p> <p>attached are debug log with enabling "net" flag. there were 4 x servers (10.0.11.22<span class="error">&#91;4-7&#93;</span>@o2ib ) for test. I've captured debug of two servers which never completed shutdown. </p> <p>Yang Sheng (ys@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/41970" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41970</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13638" title="lnet: discard the callback" class="issue-link" data-issue-key="LU-13638">LU-13638</a> o2ib: test patch<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ca80d77843ebdab6963745fc474be2e3b8985aab</p> <p>Hi, Shuichi,</p> <p>Could you test with patch: <a href="https://review.whamcloud.com/#/c/41970/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/41970/</a> please? It should stack on top of <a href="https://review.whamcloud.com/#/c/40937/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/40937/</a>. TIA.</p> <p>Thanks,<br/> YangSheng</p> <p>Hi, Shuichi,</p> <p>Do you have chance to verify the patch fixes the rmmod issue?</p> <p>Thanks,<br/> YangSheng</p> Related LU-14499 LU-13534 LU-13368 Development Rank 1|i0121z: Rank (Obsolete) 9223372036854775807 Severity