Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 [LU-15681] crash in lnet_process_id_hash() https://jira.whamcloud.com/browse/LU-15681 Lustre <p>Dear Devs,</p> <p>During heavy workload we are experiencing kernel crash caused by page fault in  lnet_process_id_hash()</p> <p>&lt;pre&gt;</p> <p>[  520.767199] BUG: unable to handle kernel paging request at 00000000deadbf1f<br/> [  520.775831] PGD 0 P4D 0 <br/> [  520.779875] Oops: 0000 <a href="#1" target="_blank" rel="noopener">1</a> SMP NOPTI<br/> [  520.785037] CPU: 10 PID: 492691 Comm: ll_ost00_016 Kdump: loaded Tainted: P           OE    --------- -  - 4.18.0-348.7.1.el8_5.x86_64 #1<br/> [  520.800422] Hardware name: HPE ProLiant DL325 Gen10 Plus/ProLiant DL325 Gen10 Plus, BIOS A43 12/03/2021<br/> [  520.812168] RIP: 0010:lnet_process_id_hash+0x5/0x50 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.820123] Code: 7e 28 39 7a 0c 75 d4 8b 7e 2c 39 7a 10 75 cc 8b 46 30 39 42 14 0f 94 c0 0f b6 c0 8d 44 40 fd c3 0f 1f 44 00 00 0f 1f 44 00 00 &lt;33&gt; 57 14 be ff ff ff ff 69 ca 47 86 c8 61 48 85 ff 74 18 0f b6 47<br/> [  520.842104] RSP: 0018:ffffaa79b40c3be0 EFLAGS: 00010202<br/> [  520.848959] RAX: ffffffffc1a36690 RBX: 5a5a5a5a5a5a5a5a RCX: 00000000deadbeef<br/> [  520.857883] RDX: 000000000cdd1d51 RSI: 0000000000000001 RDI: 00000000deadbf0b<br/> [  520.866635] RBP: ffffaa79b40c3c70 R08: ffff8ac3fecaabf8 R09: 00000000000003e8<br/> [  520.875303] R10: 0000000000000000 R11: ffff8ac3feca8ec4 R12: ffff8a4e59d5c000<br/> [  520.884015] R13: ffffffffc1baa580 R14: fffffffffffffff0 R15: 00000000deadbeef<br/> [  520.893240] FS:  0000000000000000(0000) GS:ffff8ac3fec80000(0000) knlGS:0000000000000000<br/> [  520.903294] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033<br/> [  520.911071] CR2: 00000000deadbf1f CR3: 0000000c407c6000 CR4: 0000000000350ee0<br/> [  520.920020] Call Trace:<br/> [  520.924182]  ptlrpc_connection_get+0x27f/0x920 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.931034]  target_handle_connect+0x6de/0x29d0 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.937816]  ? internal_add_timer+0x42/0x60<br/> [  520.943593]  tgt_request_handle+0x565/0x1a40 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.950382]  ? ptlrpc_nrs_req_get_nolock0+0xfb/0x1f0 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.957780]  ptlrpc_server_handle_request+0x323/0xbd0 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.965373]  ptlrpc_main+0xc06/0x1560 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.971430]  ? __schedule+0x2c5/0x760<br/> [  520.976758]  ? ptlrpc_wait_event+0x590/0x590 <span class="error">&#91;ptlrpc&#93;</span><br/> [  520.983264]  kthread+0x116/0x130<br/> [  520.987811]  ? kthread_flush_work_fn+0x10/0x10<br/> [  520.993636]  ret_from_fork+0x22/0x40</p> <p>&lt;/pre&gt;</p> <p> </p> CentOS Linux release 8.5.2111 <br/> Kernel: 4.18.0-348.7.1.el8_5.x86_64 <br/> LU-15681 crash in lnet_process_id_hash() Bug Major Resolved Duplicate WC Triage Lukasz Flis Wed, 23 Mar 2022 18:30:26 +0000 Thu, 24 Mar 2022 11:59:44 +0000 Thu, 24 Mar 2022 11:59:44 +0000 Lustre 2.15.0 0 3 <p>Hi,<br/> This is probably a duplicate of <a href="https://jira.whamcloud.com/browse/LU-15634" title="Use after free in ptlrpc" class="issue-link" data-issue-key="LU-15634"><del>LU-15634</del></a>.<br/> Maybe you can try the <a href="https://review.whamcloud.com/46763/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/46763/</a> "<a href="https://jira.whamcloud.com/browse/LU-15634" title="Use after free in ptlrpc" class="issue-link" data-issue-key="LU-15634"><del>LU-15634</del></a> ptlrpc: Use after free of 'conn' in rhashtable retry" (landed on master).</p> <p>@Etienne Aujames - thank you very much for pointing this one out. This fixed the problem in 2.15.0_RC2<br/> Tested and confirmed</p> <p>Yes, thanks <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=eaujames" class="user-hover" rel="eaujames">eaujames</a>. <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=lflis" class="user-hover" rel="lflis">lflis</a> please be sure to include the build being used in any future issues reported running pre-release code. Otherwise it is not clear which code is being run.</p> Development Rank 1|i02ljb: Rank (Obsolete) 9223372036854775807 Severity