https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1714 Lustre <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>RIP: 0010:[&lt;ffffffffa06a2807&gt;] [&lt;ffffffffa06a2807&gt;] cfs_crypto_hash_digest+0xf6/0x166 [libcfs] RSP: 0018:ffff88007c5fbda8 EFLAGS: 00010293 RAX: ffffea0001a55e00 RBX: ffff880078640000 RCX: 0000000087654321 RDX: 0000000000000000 RSI: ffffffffa06c6398 RDI: ffff880078640000 RBP: ffff88007c5fbe28 R08: 0000000000000000 R09: d84156c5635688c0 R10: d84156c5635688c0 R11: 0000000000000020 R12: 0000000000000000 R13: ffff88007c5fbe78 R14: 0000000000020000 R15: ffff88007c5fbe74 FS: 00007f2ef3eb6700(0000) GS:ffff880005000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f4777fa5000 CR3: 000000007c62c000 CR4: 00000000000406f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process insmod (pid: 8489, threadinfo ffff88007c5fa000, task ffff88007c6f00c0) Stack: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 &lt;0&gt; 0000000000000000 0000000000000000 ffff88007c321988 0000000000000000 &lt;0&gt; 00007f2ef1eee010 ffffffffa06c6390 ffff88007c5fbe28 0000000000000000 Call Trace: [&lt;ffffffffa06a29c8&gt;] cfs_crypto_register+0x151/0x35a [libcfs] [&lt;ffffffff8151cbcb&gt;] ? _spin_unlock+0x2b/0x40 [&lt;ffffffffa06b3120&gt;] ? cfs_wi_sched_create+0x3bf/0x4d4 [libcfs] [&lt;ffffffffa06a5d34&gt;] ? init_libcfs_module+0x0/0x377 [libcfs] [&lt;ffffffffa06a5d34&gt;] ? init_libcfs_module+0x0/0x377 [libcfs] [&lt;ffffffffa06a5f5f&gt;] init_libcfs_module+0x22b/0x377 [libcfs] [&lt;ffffffff8100204c&gt;] do_one_initcall+0x3c/0x1d0 [&lt;ffffffff810c1333&gt;] sys_init_module+0xe3/0x260 [&lt;ffffffff8100b0b2&gt;] system_call_fastpath+0x16/0x1b Code: c1 e8 0c 48 8d 0c c5 00 00 00 00 48 c1 e0 06 48 29 c8 48 b9 00 00 00 00 00 ea ff ff 48 8d 04 08 b9 21 43 65 87 48 39 4d 80 74 04 &lt;0f&gt; 0b eb fe f6 c2 01 74 04 0f 0b eb fe 83 e2 03 48 09 c2 48 89 RIP [&lt;ffffffffa06a2807&gt;] cfs_crypto_hash_digest+0xf6/0x166 [libcfs] RSP &lt;ffff88007c5fbda8&gt; ---[ end trace 90f8b1994767d769 ]--- Kernel panic - not syncing: Fatal exception Pid: 8489, comm: insmod Tainted: G D ---------------- 2.6.32-220.7.1.el6.lustre.2015.x86_64.debug #1 Call Trace: [&lt;ffffffff81518f20&gt;] ? panic+0x78/0x148 [&lt;ffffffff8151e564&gt;] ? oops_end+0xe4/0x100 [&lt;ffffffff8100f3bb&gt;] ? die+0x5b/0x90 [&lt;ffffffff8151dc54&gt;] ? do_trap+0xc4/0x160 [&lt;ffffffff8100d005&gt;] ? do_invalid_op+0xa5/0xb0 [&lt;ffffffff8100cff5&gt;] ? do_invalid_op+0x95/0xb0 [&lt;ffffffffa06a2807&gt;] ? cfs_crypto_hash_digest+0xf6/0x166 [libcfs] [&lt;ffffffff8151c661&gt;] ? trace_hardirqs_off_thunk+0x3a/0x3c [&lt;ffffffff8100bf9b&gt;] ? invalid_op+0x1b/0x20 [&lt;ffffffffa06a2807&gt;] ? cfs_crypto_hash_digest+0xf6/0x166 [libcfs] [&lt;ffffffffa06a27d7&gt;] ? cfs_crypto_hash_digest+0xc6/0x166 [libcfs] [&lt;ffffffffa06a29c8&gt;] ? cfs_crypto_register+0x151/0x35a [libcfs] [&lt;ffffffff8151cbcb&gt;] ? _spin_unlock+0x2b/0x40 [&lt;ffffffffa06b3120&gt;] ? cfs_wi_sched_create+0x3bf/0x4d4 [libcfs] [&lt;ffffffffa06a5d34&gt;] ? init_libcfs_module+0x0/0x377 [libcfs] [&lt;ffffffffa06a5d34&gt;] ? init_libcfs_module+0x0/0x377 [libcfs] [&lt;ffffffffa06a5f5f&gt;] ? init_libcfs_module+0x22b/0x377 [libcfs] [&lt;ffffffff8100204c&gt;] ? do_one_initcall+0x3c/0x1d0 [&lt;ffffffff810c1333&gt;] ? sys_init_module+0xe3/0x260 [&lt;ffffffff8100b0b2&gt;] ? system_call_fastpath+0x16/0x1b </pre> </div></div> debug kernel &amp; lustre with <a href="https://jira.whamcloud.com/browse/LU-1201" title="Lustre crypto hash cleanup" class="issue-link" data-issue-key="LU-1201"><strike>LU-1201</strike></a> LU-1714

crash upon loading libcfs module

Bug Minor Resolved Fixed Emoly Liu Alexander Boyko patch Tue, 7 Aug 2012 04:11:04 +0000 Wed, 4 Feb 2015 23:17:38 +0000 Wed, 20 Feb 2013 08:56:54 +0000 Lustre 2.4.0 Lustre 2.1.5 0 9 <p><a href="http://review.whamcloud.com/3552" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3552</a></p> <p>This is the same as <a href="https://jira.whamcloud.com/browse/LU-2427" title="Hit &quot;kernel BUG&quot; when running on debug kernel during recovery" class="issue-link" data-issue-key="LU-2427"><del>ORI-722</del></a>, recently found on the Orion development branch. Please see <a href="http://review.whamcloud.com/3423" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3423</a> for the current patch.</p> <p>Sure, check for sg_init_table looks the same. <a href="https://jira.whamcloud.com/browse/LU-2427" title="Hit &quot;kernel BUG&quot; when running on debug kernel during recovery" class="issue-link" data-issue-key="LU-2427"><del>ORI-722</del></a> and <a href="http://review.whamcloud.com/3423" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3423</a> relate to the lnet fix. And this issue and <a href="http://review.whamcloud.com/3552" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3552</a> - relate to libcfs crypto hash fix. Both has the similar change at autoconf. </p> <p>OK, looking more closely, this has the same cause as <a href="https://jira.whamcloud.com/browse/LU-2427" title="Hit &quot;kernel BUG&quot; when running on debug kernel during recovery" class="issue-link" data-issue-key="LU-2427"><del>ORI-722</del></a>, but affects different code.</p> <p>Alexander and Prakash, could you please get together to make a single common patch that fixes these issues, or at least have a single autoconf check. I prefer the approach in <a href="http://review.whamcloud.com/3423" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3423</a> that has a wrapper that can be used in all the code, instead of #ifdefs inline in the code as in <a href="http://review.whamcloud.com/3552" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3552</a>.</p> <p>Sure, I'm open to merging the two or landing one and reworking the other. It's probably a good idea to scan the rest of the code for areas which need similar fixes as well.</p> <p>Prakash Surya, how about merging? Or you did not do it?</p> <p>Alexander, sure I'll merge your fix into mine. I'll update this ticket when I push the new version and add you to the review list. I should be able to do that later today.</p> <p>Some other places which <em>may</em> need similar fixes:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lnet/klnds/o2iblnd/o2iblnd_cb.c:730 lnet/klnds/o2iblnd/o2iblnd_cb.c:773 lustre/obdclass/capa.c:266 lustre/obdclass/capa.c:308 lustre/obdclass/capa.c:311 lustre/obdclass/capa.c:261 lustre/obdclass/capa.c:364 </pre> </div></div> <p>Those locations all make a direct call to sg_set_page which might need to change to sg_init_one.</p> <p>Please see: <a href="http://review.whamcloud.com/3709" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3709</a></p> <p>I think it`s better to add something like this to libcfs headers.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>#ifndef HAVE_SCATTERLIST_INITTABLE #define sg_init_table(sg, nents) memset(sg, 0, sizeof(*(sg))*(nents)) #endif #ifndef HAVE_SCATTERLIST_SETPAGE sg_set_page(sg, p, len, off) sg_set_buf(sg, page_address(p) + (off &amp; ~CFS_PAGE_MASK), len) #endif </pre> </div></div> <p>Or maybe cfs_ prefix for this functions.<br/> lnet/klnds/o2iblnd/o2iblnd_cb.c already use initialized sg, and set_page is faster then set_buf, because it skip addr to page conversion.</p> <p>I decided to drop the "cfs_" prefix since there are other "sg_" calls in the code which do not use any "cfs_" wrappers. So to be consistent with those calls, I think it's best to leave them with the "sg_" prefix.</p> <p>Yujian</p> <p>Could you please help out with this one?</p> <p>Thanks</p> <p>Peter</p> <p>Prakash,<br/> there are 5 calls to sg_set_page() in lustre/obdclass/capa.c<br/> I think you should also insert a call to sg_init_table() above each of them.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> Index: b/lustre/obdclass/capa.c =================================================================== --- a/lustre/obdclass/capa.c +++ b/lustre/obdclass/capa.c @@ -265,6 +265,7 @@ int capa_hmac(__u8 *hmac, struct lustre_ } keylen = alg-&gt;ha_keylen; + sg_init_table(&amp;sl, 1); sg_set_page(&amp;sl, virt_to_page(capa), offsetof(struct lustre_capa, lc_hmac), (unsigned long)(capa) % CFS_PAGE_SIZE); @@ -306,9 +307,11 @@ int capa_encrypt_id(__u32 *d, __u32 *s, GOTO(out, rc); } + sg_init_table(&amp;sd, 1); sg_set_page(&amp;sd, virt_to_page(d), 16, (unsigned long)(d) % CFS_PAGE_SIZE); + sg_init_table(&amp;ss, 1); sg_set_page(&amp;ss, virt_to_page(s), 16, (unsigned long)(s) % CFS_PAGE_SIZE); desc.tfm = tfm; @@ -358,9 +361,11 @@ int capa_decrypt_id(__u32 *d, __u32 *s, GOTO(out, rc); } + sg_init_table(&amp;sd, 1); sg_set_page(&amp;sd, virt_to_page(d), 16, (unsigned long)(d) % CFS_PAGE_SIZE); + sg_init_table(&amp;ss, 1); sg_set_page(&amp;ss, virt_to_page(s), 16, (unsigned long)(s) % CFS_PAGE_SIZE); </pre> </div></div> <p>You have also added the following new definition for sg_set_page() in libcfs/include/libcfs/linux/libcfs.h</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> #ifndef HAVE_SCATTERLIST_SETPAGE #define sg_set_page(sg, p, len, off) \ sg_set_buf(sg, page_address(p) + ((off) &amp; ~CFS_PAGE_MASK), len) #endif </pre> </div></div> <p>This file also includes libcfs/include/libcfs/linux/kp30.h in which there was already an older definition for sg_set_page()</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> #ifndef HAVE_SCATTERLIST_SETPAGE static inline void sg_set_page(struct scatterlist *sg, struct page *page, unsigned int len, unsigned int offset) { sg-&gt;page = page; sg-&gt;offset = offset; sg-&gt;length = len; } #endif </pre> </div></div> <p>I think this older definition should be removed.</p> <p>Xyratex-bug-id: <a href="http://jira-nss.xy01.xyratex.com:8080/browse/MRP-611" class="external-link" target="_blank" rel="nofollow noopener">MRP-611</a></p> <p>b2_1 port is at <a href="http://review.whamcloud.com/5452" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5452</a></p> <p>The existing patches have landed for 2.1.5 and 2.4. Please open a new ticket to track any further changes needed in this area of code.</p> Duplicate LU-2427 Related LU-6020 Development Rank 1|hzvqan: Rank (Obsolete) 8136 Severity