https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12004 Lustre <p>I see this semi-frequently in master even after <a href="https://jira.whamcloud.com/browse/LU-11697" title="BAD WRITE CHECKSUM with t10ip4K and t10ip512 checksums" class="issue-link" data-issue-key="LU-11697"><del>LU-11697</del></a>, so this must be something else.</p> <p>This is typically only in racer and the full crash looks like this:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[ 8628.366285] Lustre: DEBUG MARKER: == racer test 1: racer on clients: centos-70.localnet DURATION=2700 ================================== 05:27:21 (1549708041) [ 8629.054425] Lustre: lfs: using old ioctl(LL_IOC_LOV_GETSTRIPE) on [0x200000402:0x4:0x0], use llapi_layout_get_by_path() [ 8630.549219] Lustre: DEBUG MARKER: racer test_1: @@@@@@ FAIL: generate lss conf (mds1) [ 8634.303466] LustreError: 14083:0:(mdt_lvb.c:430:mdt_lvbo_fill()) lustre-MDT0000: small buffer size 472 for EA 496 (max_mdsize 496): rc = -34 [ 8779.449264] BUG: unable to handle kernel paging request at ffff8800aa2dc000 [ 8779.449670] IP: [&lt;ffffffff813ee500&gt;] do_csum+0x70/0x180 [ 8779.449670] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC [ 8779.449670] CPU: 9 PID: 15375 Comm: ll_ost_io04_000 3.10.0-7.6-debug #1 [ 8779.449670] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 8779.509742] Call Trace: [ 8779.509742] [&lt;ffffffff813ee61e&gt;] ip_compute_csum+0xe/0x30 [ 8779.509742] [&lt;ffffffffa035e62e&gt;] obd_dif_ip_fn+0xe/0x10 [obdclass] [ 8779.523520] [&lt;ffffffffa035e6f9&gt;] obd_page_dif_generate_buffer+0xc9/0x190 [obdclass] [ 8779.523520] [&lt;ffffffffa05e18db&gt;] tgt_checksum_niobuf_rw+0x28b/0xea0 [ptlrpc] [ 8779.541604] [&lt;ffffffffa05e7e8d&gt;] tgt_brw_read+0xc2d/0x1e60 [ptlrpc] [ 8779.541604] [&lt;ffffffffa05e62a5&gt;] tgt_request_handle+0x915/0x1610 [ptlrpc] [ 8779.541604] [&lt;ffffffffa058b3d9&gt;] ptlrpc_server_handle_request+0x259/0xad0 [ptlrpc] [ 8779.541604] [&lt;ffffffffa058f3bc&gt;] ptlrpc_main+0xb7c/0x22c0 [ptlrpc] [ 8779.541604] [&lt;ffffffff810b4ed4&gt;] kthread+0xe4/0xf0 [ 8779.541604] [&lt;ffffffff817c4c77&gt;] ret_from_fork_nospec_begin+0x21/0x21 </pre> </div></div> <p>note that even before ti10dif was landed I still saw this, just a bit different trace.</p> <p>It seems in all cases only tgt_brw_read is hitting this</p> LU-12004

Crash in do_csum

Bug Major Open Unresolved WC Triage Oleg Drokin Mon, 25 Feb 2019 02:59:47 +0000 Mon, 8 May 2023 17:11:23 +0000 Lustre 2.13.0 0 4 <p>here's another report of this same nature, but in adler:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>BUG: unable to handle kernel paging request at ffff8802a91ff000 IP: [&lt;ffffffffa020e3f0&gt;] adler32_update+0x70/0x250 [libcfs] PGD 241b067 PUD 33ebfa067 PMD 33eab1067 PTE 80000002a91ff060 Oops: 0000 [#1] SMP DEBUG_PAGEALLOC Modules linked in: lustre(OE) ofd(OE) osp(OE) lod(OE) ost(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) lov(OE) mdc(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console i2c_piix4 pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk libata i2c_core floppy [last unloaded: libcfs] CPU: 11 PID: 22659 Comm: ll_ost_io05_006 Kdump: loaded Tainted: P OE ------------ 3.10.0-7.6-debug #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 task: ffff88029dfbe200 ti: ffff8802fa270000 task.ti: ffff8802fa270000 RIP: 0010:[&lt;ffffffffa020e3f0&gt;] [&lt;ffffffffa020e3f0&gt;] adler32_update+0x70/0x250 [libcfs] RSP: 0018:ffff8802fa273840 EFLAGS: 00010212 RAX: 0000000000001000 RBX: 0000000000001000 RCX: 0000000000000002 RDX: 0000000000001000 RSI: ffff8802a91ff000 RDI: ffff8802a91ff000 RBP: ffff8802fa2738a8 R08: 0000000000000000 R09: 0000000000001000 R10: 0000000000000001 R11: 0000000000000000 R12: ffffea00094fefca R13: 0000000000001000 R14: ffffffffa0234cd0 R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff88033dcc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff8802a91ff000 CR3: 0000000223968000 CR4: 00000000000006e0 Call Trace: [&lt;ffffffffa020d4cd&gt;] ? cfs_crypto_hash_alloc+0xcd/0x440 [libcfs] [&lt;ffffffff813910b7&gt;] crypto_shash_update+0x47/0x120 [&lt;ffffffff813913de&gt;] shash_ahash_update+0x3e/0x70 [&lt;ffffffff81391422&gt;] shash_async_update+0x12/0x20 [&lt;ffffffffa020d3b3&gt;] cfs_crypto_hash_update_page+0x93/0xc0 [libcfs] [&lt;ffffffffa061847e&gt;] tgt_checksum_niobuf_rw+0x8ce/0xea0 [ptlrpc] [&lt;ffffffffa035f0e5&gt;] ? lprocfs_stats_unlock+0x45/0x50 [obdclass] [&lt;ffffffffa0361119&gt;] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [&lt;ffffffffa1232326&gt;] ? ofd_preprw+0x5d6/0x1160 [ofd] [&lt;ffffffffa05dbcad&gt;] ? __req_capsule_get+0x15d/0x700 [ptlrpc] [&lt;ffffffffa0393e10&gt;] ? obd_dif_crc_fn+0x20/0x20 [obdclass] [&lt;ffffffffa061a41d&gt;] tgt_brw_read+0xc2d/0x1e60 [ptlrpc] [&lt;ffffffff812127f4&gt;] ? __kmalloc+0x634/0x660 [&lt;ffffffff813eca64&gt;] ? vsnprintf+0x234/0x6a0 [&lt;ffffffffa0361119&gt;] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [&lt;ffffffffa05b6fe6&gt;] ? lustre_pack_reply_v2+0x166/0x290 [ptlrpc] [&lt;ffffffffa05b717f&gt;] ? lustre_pack_reply_flags+0x6f/0x1e0 [ptlrpc] [&lt;ffffffffa05b7301&gt;] ? lustre_pack_reply+0x11/0x20 [ptlrpc] [&lt;ffffffffa061e355&gt;] tgt_request_handle+0xaf5/0x1590 [ptlrpc] [&lt;ffffffffa0211fa7&gt;] ? libcfs_debug_msg+0x57/0x80 [libcfs] [&lt;ffffffffa05c2436&gt;] ptlrpc_server_handle_request+0x256/0xad0 [ptlrpc] [&lt;ffffffffa05c6329&gt;] ptlrpc_main+0xa99/0x1f60 [ptlrpc] [&lt;ffffffff810c32ed&gt;] ? finish_task_switch+0x5d/0x1b0 [&lt;ffffffffa05c5890&gt;] ? ptlrpc_register_service+0xfb0/0xfb0 [ptlrpc] [&lt;ffffffff810b4ed4&gt;] kthread+0xe4/0xf0 [&lt;ffffffff810b4df0&gt;] ? kthread_create_on_node+0x140/0x140 [&lt;ffffffff817c4c77&gt;] ret_from_fork_nospec_begin+0x21/0x21 [&lt;ffffffff810b4df0&gt;] ? kthread_create_on_node+0x140/0x140 </pre> </div></div> <p>Oleg can you reproduce this?</p> <p>uptodate master, zfs backend:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> [ 59.782132] Lustre: DEBUG MARKER: == racer test 1: racer on clients: tmp.MXpJUErHf4 DURATION=2700 ========================================================== 12:36:17 (1667133377) .. [ 2243.840113] BUG: unable to handle kernel paging request at ffff89346d5a4000 [ 2243.840277] PGD 110e01067 P4D 110e01067 PUD 1b0979067 PMD 1b080e067 PTE 800ffffe92a5b060 [ 2243.840328] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC [ 2243.840355] CPU: 0 PID: 6959 Comm: ll_ost_io00_002 Tainted: G W O --------- - - 4.18.0 #2 [ 2243.840409] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 2243.840443] RIP: 0010:do_csum+0x6a/0x160 [ 2243.840472] Code: c7 04 0f 85 bb 00 00 00 41 89 c1 c1 e8 04 41 d1 e9 85 c0 0f 84 ff 00 00 00 83 e8 01 45 31 c0 48 83 c0 01 48 c1 e0 06 48 01 f8 &lt;48&gt; 03 17 48 13 57 08 48 13 57 10 48 13 57 18 48 13 57 20 48 13 57 [ 2243.840560] RSP: 0018:ffff89349c66fb18 EFLAGS: 00010286 [ 2243.840586] RAX: ffff89346d5a5000 RBX: 0000000000001000 RCX: 0000000000000000 [ 2243.840622] RDX: 0000000000000000 RSI: 0000000000001000 RDI: ffff89346d5a4000 [ 2243.840658] RBP: 0000000000001000 R08: 0000000000000000 R09: 0000000000000200 [ 2243.840694] R10: 0000000000000000 R11: ffff89346d5a4000 R12: ffff89346dde7002 [ 2243.840730] R13: 0000000000001000 R14: ffff89346d5a4000 R15: 0000000000000001 [ 2243.840767] FS: 0000000000000000(0000) GS:ffff8934a9c00000(0000) knlGS:0000000000000000 [ 2243.840816] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2243.840846] CR2: ffff89346d5a4000 CR3: 00000000d3b2c000 CR4: 00000000000006b0 [ 2243.840885] Call Trace: [ 2243.840912] ip_compute_csum+0x5/0x30 [ 2243.840975] obd_page_dif_generate_buffer+0xf8/0x1b0 [obdclass] [ 2243.841101] tgt_checksum_niobuf_rw+0xa2d/0x15a0 [ptlrpc] [ 2243.841200] ? obd_dif_crc_fn+0x10/0x10 [obdclass] [ 2243.841260] ? obd_dif_crc_fn+0x10/0x10 [obdclass] [ 2243.841340] tgt_brw_read+0x1752/0x2010 [ptlrpc] [ 2243.841393] ? static_obj+0x2d/0x50 [ 2243.841422] ? lockdep_init_map_waits+0x4b/0x210 [ 2243.841492] ? lustre_pack_reply_v2+0x20b/0x2b0 [ptlrpc] [ 2243.841580] ? lustre_pack_reply_flags+0x55/0x1b0 [ptlrpc] [ 2243.841671] tgt_request_handle+0x977/0x1a40 [ptlrpc] [ 2243.841756] ptlrpc_main+0x1724/0x32c0 [ptlrpc] [ 2243.841844] ? ptlrpc_wait_event+0x4b0/0x4b0 [ptlrpc] </pre> </div></div> Related Development Rank 1|i00c73: Rank (Obsolete) 9223372036854775807 Severity