[LU-18689] recovery-small test 57 crashes in osp and sptlrpc - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Unresolved
Priority: Major
Fix Version/s: Lustre 2.17.0
Affects Version/s: Lustre 2.17.0, Lustre 2.16.1
Labels:
- janitor9x

Severity:
3
Rank (Obsolete):
9223372036854775807

Description

Looks like a seemingly unrelated utils patch https://review.whamcloud.com/c/fs/lustre-release/+/55724 is exposing a number of crashes in osp and sptlrpc:

 [33152.842350] BUG: unable to handle kernel NULL pointer dereference at 0000000000000710
[33152.843008] IP: [<ffffffff817dfdf8>] down_read+0x28/0x50
[33152.843008] PGD 2c8f50067 PUD 2ced2a067 PMD 0
[33152.843008] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[33152.843008] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks floppy ata_piix virtio_blk i2c_core libata serio_raw [last unloaded: libcfs]
[33152.843008] CPU: 12 PID: 13567 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[33152.843008] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[33152.843008] task: ffff8802d789b760 ti: ffff8802f22f4000 task.ti: ffff8802f22f4000
[33152.843008] RIP: 0010:[<ffffffff817dfdf8>]  [<ffffffff817dfdf8>] down_read+0x28/0x50
[33152.843008] RSP: 0018:ffff8802f22f7da0  EFLAGS: 00010246
[33152.843008] RAX: 0000000000000710 RBX: 0000000000000710 RCX: ffff8802f22f7fd8
[33152.843008] RDX: 0000000000000000 RSI: 0000000000000015 RDI: ffffffff81a98ea7
[33152.843008] RBP: ffff8802f22f7da8 R08: ffff8803260d41b0 R09: 0000000000000000
[33152.843008] R10: 0000000000001000 R11: ffff880331406d40 R12: 0000000000000710
[33152.843008] R13: ffff88008cbed698 R14: 0000000000000001 R15: ffff8802e8c437b8
[33152.843008] FS:  00007f233e855740(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000
[33152.843008] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[33152.843008] CR2: 0000000000000710 CR3: 00000002e5c32000 CR4: 00000000000007e0
[33152.843008] Call Trace:
[33152.843008]  [<ffffffffa1535378>] active_show+0x28/0x90 [osp]
[33152.843008]  [<ffffffffa0397166>] lustre_attr_show+0x16/0x20 [obdclass]
[33152.843008]  [<ffffffff812d728c>] sysfs_kf_seq_show+0xcc/0x1e0
[33152.843008]  [<ffffffff812d58b3>] kernfs_seq_show+0x23/0x30
[33152.843008]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[33152.843008]  [<ffffffff812d643d>] kernfs_fop_read+0x11d/0x190
[33152.843008]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[33152.843008]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[33152.843008]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[33152.843008]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[33152.843008]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[33152.843008]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a

 [70958.908980] BUG: unable to handle kernel NULL pointer dereference at 0000000000000158
[70958.909948] IP: [<ffffffffa14e85b1>] osp_statfs+0x2a1/0x4e0 [osp]
[70958.909948] PGD 2b9d32067 PUD 2cd35b067 PMD 0
[70958.909948] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[70958.909948] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon i2c_piix4 pcspkr virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk serio_raw i2c_core libata [last unloaded: libcfs]
[70958.909948] CPU: 5 PID: 14375 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[70958.909948] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[70958.909948] task: ffff8800baec3760 ti: ffff8802ccf04000 task.ti: ffff8802ccf04000
[70958.909948] RIP: 0010:[<ffffffffa14e85b1>]  [<ffffffffa14e85b1>] osp_statfs+0x2a1/0x4e0 [osp]
[70958.909948] RSP: 0018:ffff8802ccf07cd8  EFLAGS: 00010246
[70958.909948] RAX: 0000000000000000 RBX: ffff8800a3b00008 RCX: 0000000000000000
[70958.909948] RDX: 0000000000000005 RSI: 0000000000000000 RDI: ffff8802d6f64d08
[70958.909948] RBP: ffff8802ccf07d28 R08: 000000000000ffff R09: 000000000000ffff
[70958.909948] R10: 20737365636f7250 R11: 0a64657265746e65 R12: 0000000000000000
[70958.909948] R13: ffff8802ccf07d38 R14: 0000000000000000 R15: ffff8802842a7188
[70958.909948] FS:  00007f8aebc05740(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000
[70958.909948] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[70958.909948] CR2: 0000000000000158 CR3: 0000000287048000 CR4: 00000000000007e0
[70958.909948] Call Trace:
[70958.909948]  [<ffffffff817ee00c>] ? system_call_fastpath+0x1f/0x24
[70958.909948]  [<ffffffffa03c53f2>] filestotal_show+0x52/0x120 [obdclass]
[70958.909948]  [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0
[70958.909948]  [<ffffffffa0378166>] lustre_attr_show+0x16/0x20 [obdclass]
[70958.909948]  [<ffffffff812d728c>] sysfs_kf_seq_show+0xcc/0x1e0
[70958.909948]  [<ffffffff812d58b3>] kernfs_seq_show+0x23/0x30
[70958.909948]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[70958.909948]  [<ffffffff812d643d>] kernfs_fop_read+0x11d/0x190
[70958.909948]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[70958.909948]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[70958.909948]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[70958.909948]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[70958.909948]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[70958.909948]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a

 [112319.862827] BUG: unable to handle kernel NULL pointer dereference at 00000000000000d0
[112319.863006] IP: [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30
[112319.863006] PGD 8d49a067 PUD 8d499067 PMD 0
[112319.863006] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[112319.863006] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_balloon pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw floppy virtio_blk i2c_core libata [last unloaded: libcfs]
[112319.863006] CPU: 4 PID: 8139 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[112319.863006] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[112319.863006] task: ffff8800a4c48010 ti: ffff8800a9f3c000 task.ti: ffff8800a9f3c000
[112319.863006] RIP: 0010:[<ffffffff81411a45>]  [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30
[112319.863006] RSP: 0018:ffff8800a9f3fdf8  EFLAGS: 00010202
[112319.863006] RAX: ffff8800a9f3ffd8 RBX: 0000000000000000 RCX: 0000000000000000
[112319.863006] RDX: fffffffffffffff4 RSI: 0000000000000001 RDI: 00000000000000c8
[112319.863006] RBP: ffff8800a9f3fe00 R08: ffff88029dd30008 R09: ffff88029dd31158
[112319.863006] R10: 0000000000000000 R11: ffff880331406d40 R12: 00000000000000c8
[112319.863006] R13: ffff8800a9f3ff18 R14: 0000000000000001 R15: ffff88009b44ab58
[112319.882548] FS:  00007fc196caf740(0000) GS:ffff880331b00000(0000) knlGS:0000000000000000
[112319.882548] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[112319.882548] CR2: 00000000000000d0 CR3: 000000009a83a000 CR4: 00000000000007e0
[112319.882548] Call Trace:
[112319.882548]  [<ffffffff817e333e>] _raw_read_lock+0x1e/0x20
[112319.882548]  [<ffffffffa0666f0e>] sptlrpc_import_sec_ref+0x1e/0x40 [ptlrpc]
[112319.882548]  [<ffffffffa06742f0>] srpc_sptlrpc_sepol_seq_show+0x30/0x130 [ptlrpc]
[112319.882548]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[112319.882548]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[112319.882548]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[112319.882548]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[112319.882548]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[112319.882548]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[112319.882548]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[112319.882548] Code: 00 eb c5 48 c7 c6 96 33 ad 81 48 89 df e8 f1 79 3c 00 eb a8 48 c7 c6 85 33 ad 81 48 89 df e8 e0 79 3c 00 eb 88 90 55 48 89 e5 53 <81> 7f 08 ed 1e af de 48 89 fb 75 11 48 89 df f0 48 ff 0f 79 05
[112319.882548] RIP  [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30

 [154763.871991] BUG: unable to handle kernel NULL pointer dereference at 0000000000000158
[154763.872624] IP: [<ffffffffa152b5b1>] osp_statfs+0x2a1/0x4e0 [osp]
[154763.872624] PGD 2de13d067 PUD 2ea003067 PMD 0
[154763.872624] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[154763.872624] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs]
[154763.872624] CPU: 10 PID: 18364 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[154763.872624] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[154763.872624] task: ffff88031d0b9280 ti: ffff8802c02b0000 task.ti: ffff8802c02b0000
[154763.872624] RIP: 0010:[<ffffffffa152b5b1>]  [<ffffffffa152b5b1>] osp_statfs+0x2a1/0x4e0 [osp]
[154763.872624] RSP: 0018:ffff8802c02b3cd8  EFLAGS: 00010246
[154763.872624] RAX: 0000000000000000 RBX: ffff8800bae1e678 RCX: 0000000000000000
[154763.872624] RDX: 000000000000000a RSI: 0000000000000000 RDI: ffff8802f6ad74c8
[154763.872624] RBP: ffff8802c02b3d28 R08: 000000000000ffff R09: 000000000000ffff
[154763.872624] R10: 20737365636f7250 R11: 0a64657265746e65 R12: 0000000000000000
[154763.872624] R13: ffff8802c02b3d38 R14: 0000000000000000 R15: ffff8802a63be318
[154763.872624] FS:  00007fab57b90740(0000) GS:ffff880331c80000(0000) knlGS:0000000000000000
[154763.872624] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[154763.872624] CR2: 0000000000000158 CR3: 00000002e5090000 CR4: 00000000000007e0
[154763.872624] Call Trace:
[154763.872624]  [<ffffffff817ee00c>] ? system_call_fastpath+0x1f/0x24
[154763.872624]  [<ffffffffa03dbdc2>] kbytesfree_show+0x52/0x140 [obdclass]
[154763.872624]  [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0
[154763.872624]  [<ffffffffa038f166>] lustre_attr_show+0x16/0x20 [obdclass]
[154763.872624]  [<ffffffff812d728c>] sysfs_kf_seq_show+0xcc/0x1e0
[154763.872624]  [<ffffffff812d58b3>] kernfs_seq_show+0x23/0x30
[154763.872624]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[154763.872624]  [<ffffffff812d643d>] kernfs_fop_read+0x11d/0x190
[154763.872624]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[154763.872624]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[154763.872624]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[154763.872624]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[154763.872624]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[154763.872624]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[154763.872624] Code: fb 03 00 90 82 55 a1 c7 05 1d fb 03 00 19 03 00 00 c7 05 17 fb 03 00 01 00 00 00 48 c7 05 14 fb 03 00 00 00 00 00 e8 7f 78 c4 fe <41> 83 be 58 01 00 00 01 0f 85 ba fd ff ff 90 f6 05 81 4f c5 fe
[154763.872624] RIP  [<ffffffffa152b5b1>] osp_statfs+0x2a1/0x4e0 [osp]

 [172834.743276] Lustre: Failing over lustre-MDT0000
[172835.207288] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[172835.208046] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks floppy serio_raw i2c_core virtio_blk libata [last unloaded: libcfs]
[172835.208253] CPU: 3 PID: 23083 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[172835.208253] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[172835.208253] task: ffff88027dcd0010 ti: ffff8802dcf00000 task.ti: ffff8802dcf00000
[172835.208253] RIP: 0010:[<ffffffffa0385646>]  [<ffffffffa0385646>] lprocfs_connect_flags_seq_show+0x36/0xc0 [obdclass]
[172835.230730] RSP: 0018:ffff8802dcf03e20  EFLAGS: 00010202
[172835.230730] RAX: 6b6b6b6b6b6b6b6b RBX: ffff8802e9b7e108 RCX: ffff8802dcf03fd8
[172835.230730] RDX: 0000000000000000 RSI: 0000000000000015 RDI: ffffffff81a98ea7
[172835.230730] RBP: ffff8802dcf03e48 R08: ffff880323f6c548 R09: ffff880323f68008
[172835.230730] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff88009962c460
[172835.230730] R13: ffff88009962cb70 R14: 0000000000000001 R15: ffff8802e9b7e108
[172835.271862] FS:  00007fd1c5063740(0000) GS:ffff880331ac0000(0000) knlGS:0000000000000000
[172835.271862] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[172835.271862] CR2: 000000000195f1b8 CR3: 0000000299d12000 CR4: 00000000000007e0
[172835.271862] Call Trace:
[172835.271862]  [<ffffffffa15132aa>] osp_connect_flags_seq_show+0x1a/0x30 [osp]
[172835.271862]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[172835.271862]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[172835.271862]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[172835.271862]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[172835.271862]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[172835.271862]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[172835.281018]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[172835.281018] Code: 41 55 4c 8d ae 10 07 00 00 41 54 49 89 f4 53 48 89 fb 4c 89 ef 48 83 ec 08 e8 97 a7 45 e1 49 8b 84 24 70 07 00 00 48 85 c0 74 79 <4c> 8b b0 f0 02 00 00 4c 8b a0 40 03 00 00 48 c7 c6 fd 30 41 a0
[172835.281018] RIP  [<ffffffffa0385646>] lprocfs_connect_flags_seq_show+0x36/0xc0 [obdclass]

 [259762.924209] BUG: unable to handle kernel NULL pointer dereference at 0000000000000158
[259762.925011] IP: [<ffffffffa14f35b1>] osp_statfs+0x2a1/0x4e0 [osp]
[259762.925011] PGD a1f4b067 PUD a02c0067 PMD 0
[259762.925011] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[259762.925011] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_console i2c_piix4 pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw virtio_blk i2c_core libata [last unloaded: libcfs]
[259762.925011] CPU: 5 PID: 15936 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[259762.925011] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[259762.925011] task: ffff880329025c40 ti: ffff8802b5864000 task.ti: ffff8802b5864000
[259762.925011] RIP: 0010:[<ffffffffa14f35b1>]  [<ffffffffa14f35b1>] osp_statfs+0x2a1/0x4e0 [osp]
[259762.925011] RSP: 0018:ffff8802b5867cd8  EFLAGS: 00010246
[259762.925011] RAX: 0000000000000000 RBX: ffff880324095d28 RCX: 0000000000000000
[259762.925011] RDX: 0000000000000005 RSI: 0000000000000000 RDI: ffff8803214e7248
[259762.925011] RBP: ffff8802b5867d28 R08: 000000000000ffff R09: 000000000000ffff
[259762.925011] R10: 20737365636f7250 R11: 0a64657265746e65 R12: 0000000000000000
[259762.925011] R13: ffff8802b5867d38 R14: 0000000000000000 R15: ffff8802ebd26f78
[259762.925011] FS:  00007f9e76ad0740(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000
[259762.925011] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[259762.925011] CR2: 0000000000000158 CR3: 0000000090eca000 CR4: 00000000000007e0
[259762.925011] Call Trace:
[259762.925011]  [<ffffffff817ee00c>] ? system_call_fastpath+0x1f/0x24
[259762.925011]  [<ffffffffa03e8512>] filesfree_show+0x52/0x120 [obdclass]
[259762.925011]  [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0
[259762.925011]  [<ffffffffa039b166>] lustre_attr_show+0x16/0x20 [obdclass]
[259762.925011]  [<ffffffff812d728c>] sysfs_kf_seq_show+0xcc/0x1e0
[259762.925011]  [<ffffffff812d58b3>] kernfs_seq_show+0x23/0x30
[259762.925011]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[259762.925011]  [<ffffffff812d643d>] kernfs_fop_read+0x11d/0x190
[259762.925011]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[259762.925011]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[259762.925011]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[259762.925011]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[259762.925011]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[259762.925011]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[259762.925011] Code: fb 03 00 90 02 52 a1 c7 05 1d fb 03 00 19 03 00 00 c7 05 17 fb 03 00 01 00 00 00 48 c7 05 14 fb 03 00 00 00 00 00 e8 7f 48 cb fe <41> 83 be 58 01 00 00 01 0f 85 ba fd ff ff 90 f6 05 81 1f cc fe
[259762.925011] RIP  [<ffffffffa14f35b1>] osp_statfs+0x2a1/0x4e0 [osp]
[259762.925011]  RSP <ffff8802b5867cd8>

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72747

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72745

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72744

Attachments

Issue Links

is related to

LU-14145 protect cli->cl_import with with_imp_locked()

Open

LU-8066 Move lustre procfs handling to sysfs and debugfs.

Open

Activity

[LU-18689] recovery-small test 57 crashes in osp and sptlrpc

Gerrit Updater added a comment - 18/Feb/25 5:04 AM

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57949/
Subject: LU-18689 osp: Unregister proc files before obd cleanup
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d9a1691ab7191125711aaadde8d043b93879733c

Gerrit Updater added a comment - 18/Feb/25 5:04 AM "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57949/ Subject: LU-18689 osp: Unregister proc files before obd cleanup Project: fs/lustre-release Branch: master Current Patch Set: Commit: d9a1691ab7191125711aaadde8d043b93879733c

James A Simmons added a comment - 05/Feb/25 5:27 PM

Sorry but Oleg's patch had two ChangeIds so git got confused. I fixed up the original patch.

James A Simmons added a comment - 05/Feb/25 5:27 PM Sorry but Oleg's patch had two ChangeIds so git got confused. I fixed up the original patch.

Gerrit Updater added a comment - 05/Feb/25 5:02 PM

"James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57973
Subject: LU-18689 ptlrpc: Cleanup targets after proc files removal
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7a9ab9a5553cd79b29b2b56528b6cf3c71cf6c9d

Gerrit Updater added a comment - 05/Feb/25 5:02 PM "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57973 Subject: LU-18689 ptlrpc: Cleanup targets after proc files removal Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 7a9ab9a5553cd79b29b2b56528b6cf3c71cf6c9d

Oleg Drokin added a comment - 03/Feb/25 3:35 PM

oh well, overnight things still come out, though at a lesser freqency I feel like and certain classe have disappeared (everythign that's inside osp_statfs in particular).

So this still looks like a beneficial set of patches, but not the silver bullet unfortunately

Oleg Drokin added a comment - 03/Feb/25 3:35 PM oh well, overnight things still come out, though at a lesser freqency I feel like and certain classe have disappeared (everythign that's inside osp_statfs in particular). So this still looks like a beneficial set of patches, but not the silver bullet unfortunately

Gerrit Updater added a comment - 03/Feb/25 7:10 AM

"Oleg Drokin <green@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57949
Subject: LU-18689 osp: Unregister proc files before obd cleanup
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ffbb9e6d1ab7d2ee466e07f22b6be7e0f60f38ff

Gerrit Updater added a comment - 03/Feb/25 7:10 AM "Oleg Drokin <green@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57949 Subject: LU-18689 osp: Unregister proc files before obd cleanup Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: ffbb9e6d1ab7d2ee466e07f22b6be7e0f60f38ff

Gerrit Updater added a comment - 03/Feb/25 7:10 AM

"Oleg Drokin <green@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57948
Subject: LU-18689 ptlrpc: Cleanup targets after proc files removal
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 94ee2810993e2894469ded2d03277330abefaeb4

Gerrit Updater added a comment - 03/Feb/25 7:10 AM "Oleg Drokin <green@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57948 Subject: LU-18689 ptlrpc: Cleanup targets after proc files removal Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 94ee2810993e2894469ded2d03277330abefaeb4

Oleg Drokin added a comment - 03/Feb/25 7:08 AM

I switched boilpot into "tests 55 56 57" in recovery-small for a day and managed to fix majority of crashes.

The remaining ones seems to be super fringe: debugfs would open a file (simple open would grab obd pointer from inode i_private) and by the time we get into seq_read parallel thread would free the whole subtree and obd. We do not pin obd when putting it into the inode (which sounds wrong ,but may be it's not the easiest to do it?)

this results in something like:

 [ 6245.884903] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 6245.885618] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks libata virtio_blk serio_raw i2c_core floppy [last unloaded: libcfs]
[ 6245.885862] CPU: 6 PID: 12955 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[ 6245.885862] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[ 6245.885862] task: ffff8802ea0b49d0 ti: ffff8800a8b18000 task.ti: ffff8800a8b18000
[ 6245.885862] RIP: 0010:[<ffffffffa048dae9>]  [<ffffffffa048dae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass]
[ 6245.885862] RSP: 0018:ffff8800a8b1bd58  EFLAGS: 00010246
[ 6245.885862] RAX: 0000000000000000 RBX: ffff8802e221c460 RCX: ffff8800a8b1bfd8
[ 6245.885862] RDX: 6b6b6b6b6b6b6b6b RSI: ffff8802e221c460 RDI: ffff8802ec514c68
[ 6245.885862] RBP: ffff8800a8b1be18 R08: ffff8802ea33e7e8 R09: ffff8802ea339158
[ 6245.885862] R10: 0000000000000000 R11: ffff880331406d40 R12: 6b6b6b6b6b6b6e5b
[ 6245.885862] R13: 6b6b6b6b6b6b6b6b R14: 0000000000000001 R15: ffff8802ec514c68
[ 6245.885862] FS:  00007f1fd19f3740(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000
[ 6245.885862] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6245.885862] CR2: 00000000025ee1e8 CR3: 00000000b9618000 CR4: 00000000000007e0
[ 6245.885862] Call Trace:
[ 6245.885862]  [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0
[ 6245.885862]  [<ffffffffa048e463>] lprocfs_import_seq_show+0x43/0xa0 [obdclass]
[ 6245.885862]  [<ffffffffa10d81ea>] osp_import_seq_show+0x1a/0x30 [osp]
[ 6245.885862]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[ 6245.885862]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[ 6245.885862]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[ 6245.885862]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[ 6245.885862]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[ 6245.885862]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[ 6245.885862]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[ 6245.885862] Code: 55 49 89 d5 41 54 4c 8d a2 f0 02 00 00 53 48 81 ec 98 00 00 00 48 89 b5 58 ff ff ff 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 <8b> 82 58 01 00 00 83 f8 0b 0f 87 04 09 00 00 48 8b 95 58 ff ff
[ 6245.885862] RIP  [<ffffffffa048dae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass]

I checked in the corefile and entire obd_dev structure is filled with the poison pattern.

curiously so far I am only seeing thise for the only two users of the LDEBUGFS_SEQ_FOPS_RW_TYPE macro and nothing else. not sure if it's a coincidence - will see what overnight results will bring me here.

Another one almost looks like a bug in the centos7 kernel:

 [11201.702850] usercopy: kernel memory exposure attempt detected from ffff8802f915bc71 (kmalloc-192) (107 bytes)
[11201.704234] ------------[ cut here ]------------
[11201.704893] kernel BUG at mm/usercopy.c:72!
[11201.705207] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[11201.705207] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic pata_acpi drm drm_panel_orientation_quirks ata_piix floppy serio_raw i2c_core virtio_blk libata [last unloaded: libcfs]
[11201.705207] CPU: 13 PID: 26666 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[11201.705207] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[11201.705207] task: ffff8802e1249280 ti: ffff8802cf2a8000 task.ti: ffff8802cf2a8000
[11201.705207] RIP: 0010:[<ffffffff81242157>]  [<ffffffff81242157>] __check_object_size+0x87/0x220
[11201.705207] RSP: 0018:ffff8802cf2abdf0  EFLAGS: 00010246
[11201.705207] RAX: 0000000000000061 RBX: ffff8802f915bc71 RCX: 0000000000000000
[11201.719242] Lustre: Unmounted lustre-client
[11201.719356] RDX: 0000000000000000 RSI: ffff880331d53a98 RDI: ffff880331d53a98
[11201.719356] RBP: ffff8802cf2abe10 R08: 0000000000000000 R09: 0000000000001c60
[11201.719356] R10: 000000000000ffa0 R11: 0000000000000198 R12: 000000000000006b
[11201.719356] R13: 0000000000000001 R14: ffff8802f915bcdc R15: ffff8802f915bc71
[11201.719356] FS:  00007f2ef0a76740(0000) GS:ffff880331d40000(0000) knlGS:0000000000000000
[11201.719356] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11201.719356] CR2: 000000000248c1a8 CR3: 0000000326584000 CR4: 00000000000007e0
[11201.719356] Call Trace:
[11201.719356]  [<ffffffff8125bae2>] filldir+0xa2/0x120
[11201.719356]  [<ffffffff8125ba40>] ? filldir64+0x130/0x130
[11201.719356]  [<ffffffff812c98b8>] proc_readdir_de+0xf8/0x200
[11201.719356]  [<ffffffff812c99e2>] proc_readdir+0x22/0x30
[11201.719356]  [<ffffffff8125b881>] iterate_dir+0x91/0x120
[11201.719356]  [<ffffffff8125be44>] SyS_getdents+0xc4/0x190
[11201.719356]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[11201.719356]  [<ffffffff8125ba40>] ? filldir64+0x130/0x130
[11201.719356]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[11201.719356]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[11201.719356]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[11201.719356] Code: 45 d1 48 c7 c6 8c e8 a9 81 48 c7 c1 1a 90 aa 81 48 0f 45 f1 49 89 c0 4d 89 e1 48 89 d9 48 c7 c7 08 57 aa 81 31 c0 e8 a3 04 59 00 <0f> 0b 0f 1f 80 00 00 00 00 48 c7 c0 00 00 00 81 4c 39 f0 73 0d
[11201.719356] RIP  [<ffffffff81242157>] __check_object_size+0x87/0x220
[11201.719356]  RSP <ffff8802cf2abdf0>

Oleg Drokin added a comment - 03/Feb/25 7:08 AM I switched boilpot into "tests 55 56 57" in recovery-small for a day and managed to fix majority of crashes. The remaining ones seems to be super fringe: debugfs would open a file (simple open would grab obd pointer from inode i_private) and by the time we get into seq_read parallel thread would free the whole subtree and obd. We do not pin obd when putting it into the inode (which sounds wrong ,but may be it's not the easiest to do it?) this results in something like: [ 6245.884903] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [ 6245.885618] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) dm_flakey dm_mod loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic drm_kms_helper pata_acpi ttm drm ata_piix drm_panel_orientation_quirks libata virtio_blk serio_raw i2c_core floppy [last unloaded: libcfs] [ 6245.885862] CPU: 6 PID: 12955 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [ 6245.885862] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [ 6245.885862] task: ffff8802ea0b49d0 ti: ffff8800a8b18000 task.ti: ffff8800a8b18000 [ 6245.885862] RIP: 0010:[<ffffffffa048dae9>] [<ffffffffa048dae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass] [ 6245.885862] RSP: 0018:ffff8800a8b1bd58 EFLAGS: 00010246 [ 6245.885862] RAX: 0000000000000000 RBX: ffff8802e221c460 RCX: ffff8800a8b1bfd8 [ 6245.885862] RDX: 6b6b6b6b6b6b6b6b RSI: ffff8802e221c460 RDI: ffff8802ec514c68 [ 6245.885862] RBP: ffff8800a8b1be18 R08: ffff8802ea33e7e8 R09: ffff8802ea339158 [ 6245.885862] R10: 0000000000000000 R11: ffff880331406d40 R12: 6b6b6b6b6b6b6e5b [ 6245.885862] R13: 6b6b6b6b6b6b6b6b R14: 0000000000000001 R15: ffff8802ec514c68 [ 6245.885862] FS: 00007f1fd19f3740(0000) GS:ffff880331b80000(0000) knlGS:0000000000000000 [ 6245.885862] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6245.885862] CR2: 00000000025ee1e8 CR3: 00000000b9618000 CR4: 00000000000007e0 [ 6245.885862] Call Trace: [ 6245.885862] [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0 [ 6245.885862] [<ffffffffa048e463>] lprocfs_import_seq_show+0x43/0xa0 [obdclass] [ 6245.885862] [<ffffffffa10d81ea>] osp_import_seq_show+0x1a/0x30 [osp] [ 6245.885862] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [ 6245.885862] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [ 6245.885862] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [ 6245.885862] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [ 6245.885862] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [ 6245.885862] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [ 6245.885862] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [ 6245.885862] Code: 55 49 89 d5 41 54 4c 8d a2 f0 02 00 00 53 48 81 ec 98 00 00 00 48 89 b5 58 ff ff ff 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 <8b> 82 58 01 00 00 83 f8 0b 0f 87 04 09 00 00 48 8b 95 58 ff ff [ 6245.885862] RIP [<ffffffffa048dae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass] I checked in the corefile and entire obd_dev structure is filled with the poison pattern. curiously so far I am only seeing thise for the only two users of the LDEBUGFS_SEQ_FOPS_RW_TYPE macro and nothing else. not sure if it's a coincidence - will see what overnight results will bring me here. Another one almost looks like a bug in the centos7 kernel: [11201.702850] usercopy: kernel memory exposure attempt detected from ffff8802f915bc71 (kmalloc-192) (107 bytes) [11201.704234] ------------[ cut here ]------------ [11201.704893] kernel BUG at mm/usercopy.c:72! [11201.705207] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC [11201.705207] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console pcspkr virtio_balloon ip_tables rpcsec_gss_krb5 drm_kms_helper ttm ata_generic pata_acpi drm drm_panel_orientation_quirks ata_piix floppy serio_raw i2c_core virtio_blk libata [last unloaded: libcfs] [11201.705207] CPU: 13 PID: 26666 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [11201.705207] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [11201.705207] task: ffff8802e1249280 ti: ffff8802cf2a8000 task.ti: ffff8802cf2a8000 [11201.705207] RIP: 0010:[<ffffffff81242157>] [<ffffffff81242157>] __check_object_size+0x87/0x220 [11201.705207] RSP: 0018:ffff8802cf2abdf0 EFLAGS: 00010246 [11201.705207] RAX: 0000000000000061 RBX: ffff8802f915bc71 RCX: 0000000000000000 [11201.719242] Lustre: Unmounted lustre-client [11201.719356] RDX: 0000000000000000 RSI: ffff880331d53a98 RDI: ffff880331d53a98 [11201.719356] RBP: ffff8802cf2abe10 R08: 0000000000000000 R09: 0000000000001c60 [11201.719356] R10: 000000000000ffa0 R11: 0000000000000198 R12: 000000000000006b [11201.719356] R13: 0000000000000001 R14: ffff8802f915bcdc R15: ffff8802f915bc71 [11201.719356] FS: 00007f2ef0a76740(0000) GS:ffff880331d40000(0000) knlGS:0000000000000000 [11201.719356] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [11201.719356] CR2: 000000000248c1a8 CR3: 0000000326584000 CR4: 00000000000007e0 [11201.719356] Call Trace: [11201.719356] [<ffffffff8125bae2>] filldir+0xa2/0x120 [11201.719356] [<ffffffff8125ba40>] ? filldir64+0x130/0x130 [11201.719356] [<ffffffff812c98b8>] proc_readdir_de+0xf8/0x200 [11201.719356] [<ffffffff812c99e2>] proc_readdir+0x22/0x30 [11201.719356] [<ffffffff8125b881>] iterate_dir+0x91/0x120 [11201.719356] [<ffffffff8125be44>] SyS_getdents+0xc4/0x190 [11201.719356] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [11201.719356] [<ffffffff8125ba40>] ? filldir64+0x130/0x130 [11201.719356] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [11201.719356] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [11201.719356] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [11201.719356] Code: 45 d1 48 c7 c6 8c e8 a9 81 48 c7 c1 1a 90 aa 81 48 0f 45 f1 49 89 c0 4d 89 e1 48 89 d9 48 c7 c7 08 57 aa 81 31 c0 e8 a3 04 59 00 <0f> 0b 0f 1f 80 00 00 00 00 48 c7 c0 00 00 00 81 4c 39 f0 73 0d [11201.719356] RIP [<ffffffff81242157>] __check_object_size+0x87/0x220 [11201.719356] RSP <ffff8802cf2abdf0>

Oleg Drokin added a comment - 03/Feb/25 4:22 AM

Looks like at least part of the crashes are at startup, so we expose proc/sysfs things before our own init is fully done:

#10 [ffff8802f3a3bcf0] async_page_fault at ffffffff817e47a8
    [exception RIP: down_read+40]
    RIP: ffffffff817dfdf8  RSP: ffff8802f3a3bda0  RFLAGS: 00010246
    RAX: 0000000000000710  RBX: 0000000000000710  RCX: ffff8802f3a3bfd8
    RDX: 0000000000000000  RSI: 0000000000000015  RDI: ffffffff81a98ea7
    RBP: ffff8802f3a3bda8   R8: ffff8802ec5189d0   R9: 0000000000000000
    R10: 0000000000001000  R11: ffff880331406d40  R12: 0000000000000710
    R13: ffff88009df333f8  R14: 0000000000000001  R15: ffff88031ea28218
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
#11 [ffff8802f3a3bdb0] active_show at ffffffffa1514378 [osp]
#12 [ffff8802f3a3bde0] lustre_attr_show at ffffffffa0390166 [obdclass]
#13 [ffff8802f3a3bdf0] sysfs_kf_seq_show at ffffffff812d728c
#14 [ffff8802f3a3be10] kernfs_seq_show at ffffffff812d58b3
#15 [ffff8802f3a3be20] seq_read at ffffffff8126f0b5
#16 [ffff8802f3a3be90] kernfs_fop_read at ffffffff812d643d
#17 [ffff8802f3a3bed0] vfs_read at ffffffff81245839
#18 [ffff8802f3a3bf08] sys_read at ffffffff8124676f
#19 [ffff8802f3a3bf50] system_call_fastpath at ffffffff817ee00c

PID: 11651    TASK: ffff8802f2dcb760  CPU: 7    COMMAND: "llog_process_th"
 #0 [ffff880331bc8e48] crash_nmi_callback at ffffffff81053d11
 #1 [ffff880331bc8e58] nmi_handle at ffffffff817e5967
 #2 [ffff880331bc8eb0] do_nmi at ffffffff817e5b5c
 #3 [ffff880331bc8ef0] end_repeat_nmi at ffffffff817e4d9c
    [exception RIP: unwind_next_frame+91]
    RIP: ffffffff8106b41b  RSP: ffff8802d2c36e90  RFLAGS: 00000012
    RAX: ffffffff81b3bff0  RBX: ffff8802d2c36ec0  RCX: 0000000000000010
    RDX: ffffffffa03e7542  RSI: ffffffffa03e7542  RDI: ffff8802d2c36ec0
    RBP: ffff8802d2c36ea0   R8: 0000000000000000   R9: 000000000000005a
    R10: 0000000000000000  R11: 0000000000000025  R12: 0000000000000000
    R13: 0000000000000000  R14: ffff8802f2dcb760  R15: ffff8803314047c0
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
--- <NMI exception stack> ---
 #4 [ffff8802d2c36e90] unwind_next_frame at ffffffff8106b41b
 #5 [ffff8802d2c36ea8] unwind_next_frame at ffffffff8106b4a6
 #6 [ffff8802d2c36eb8] __save_stack_trace at ffffffff8103dd7b
 #7 [ffff8802d2c36f40] save_stack_trace at ffffffff8103de0f
 #8 [ffff8802d2c36f50] set_track at ffffffff81219770
 #9 [ffff8802d2c36f90] free_debug_processing at ffffffff817d567f
#10 [ffff8802d2c36ff8] __slab_free at ffffffff8121e635
#11 [ffff8802d2c370a8] kfree at ffffffff8121f006
#12 [ffff8802d2c370d8] spl_kmem_free at ffffffffa0ac64ca [spl]
#13 [ffff8802d2c370e8] dbuf_hold_impl at ffffffffa0c6650b [zfs]
#14 [ffff8802d2c37130] dbuf_prefetch at ffffffffa0c6696e [zfs]
#15 [ffff8802d2c37260] dmu_zfetch at ffffffffa0c87d2d [zfs]
#16 [ffff8802d2c372f0] dbuf_read at ffffffffa0c64397 [zfs]
#17 [ffff8802d2c37390] dnode_next_offset_level at ffffffffa0c88e30 [zfs]
#18 [ffff8802d2c37400] dnode_next_offset at ffffffffa0c8b7b8 [zfs]
#19 [ffff8802d2c37458] dmu_object_next at ffffffffa0c73704 [zfs]
#20 [ffff8802d2c374e8] dmu_object_alloc_impl at ffffffffa0c73a55 [zfs]
#21 [ffff8802d2c375b8] dmu_object_alloc_dnsize at ffffffffa0c73ca9 [zfs]
#22 [ffff8802d2c375e8] __osd_object_create at ffffffffa0af2663 [osd_zfs]
#23 [ffff8802d2c37638] osd_mkreg at ffffffffa0af29a0 [osd_zfs]
#24 [ffff8802d2c37678] osd_create at ffffffffa0aedd93 [osd_zfs]
#25 [ffff8802d2c37710] local_object_create at ffffffffa03f404f [obdclass]
#26 [ffff8802d2c37770] llog_osd_create at ffffffffa03e7542 [obdclass] 
#27 [ffff8802d2c377c0] llog_create at ffffffffa0375270 [obdclass]
#28 [ffff8802d2c377f8] llog_cat_new_log at ffffffffa037bd54 [obdclass]
#29 [ffff8802d2c37868] llog_cat_add_rec at ffffffffa037cdbb [obdclass]
#30 [ffff8802d2c37900] llog_cat_add at ffffffffa037e55e [obdclass]
#31 [ffff8802d2c37948] osp_sync_llog_init at ffffffffa150c68d [osp]   
#32 [ffff8802d2c379b0] osp_sync_init at ffffffffa150f860 [osp]
#33 [ffff8802d2c37a28] osp_init0 at ffffffffa14fa0c3 [osp]
#34 [ffff8802d2c37aa0] osp_device_alloc at ffffffffa14facf9 [osp]
#35 [ffff8802d2c37ad8] obd_setup at ffffffffa03a10a3 [obdclass]
#36 [ffff8802d2c37b88] class_setup at ffffffffa03a256f [obdclass]
#37 [ffff8802d2c37be0] class_process_config at ffffffffa03a4d2b [obdclass]
#38 [ffff8802d2c37c88] class_config_llog_handler at ffffffffa03a7e1b [obdclass]
#39 [ffff8802d2c37d58] llog_process_thread at ffffffffa0377a80 [obdclass]
#40 [ffff8802d2c37e80] llog_process_thread_daemonize at ffffffffa0379652 [obdclass]
#41 [ffff8802d2c37ea8] kthread at ffffffff810ba114
#42 [ffff8802d2c37f50] ret_from_fork_nospec_begin at ffffffff817ede5d

Oleg Drokin added a comment - 03/Feb/25 4:22 AM Looks like at least part of the crashes are at startup, so we expose proc/sysfs things before our own init is fully done: #10 [ffff8802f3a3bcf0] async_page_fault at ffffffff817e47a8 [exception RIP: down_read+40] RIP: ffffffff817dfdf8 RSP: ffff8802f3a3bda0 RFLAGS: 00010246 RAX: 0000000000000710 RBX: 0000000000000710 RCX: ffff8802f3a3bfd8 RDX: 0000000000000000 RSI: 0000000000000015 RDI: ffffffff81a98ea7 RBP: ffff8802f3a3bda8 R8: ffff8802ec5189d0 R9: 0000000000000000 R10: 0000000000001000 R11: ffff880331406d40 R12: 0000000000000710 R13: ffff88009df333f8 R14: 0000000000000001 R15: ffff88031ea28218 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #11 [ffff8802f3a3bdb0] active_show at ffffffffa1514378 [osp] #12 [ffff8802f3a3bde0] lustre_attr_show at ffffffffa0390166 [obdclass] #13 [ffff8802f3a3bdf0] sysfs_kf_seq_show at ffffffff812d728c #14 [ffff8802f3a3be10] kernfs_seq_show at ffffffff812d58b3 #15 [ffff8802f3a3be20] seq_read at ffffffff8126f0b5 #16 [ffff8802f3a3be90] kernfs_fop_read at ffffffff812d643d #17 [ffff8802f3a3bed0] vfs_read at ffffffff81245839 #18 [ffff8802f3a3bf08] sys_read at ffffffff8124676f #19 [ffff8802f3a3bf50] system_call_fastpath at ffffffff817ee00c vs PID: 11651 TASK: ffff8802f2dcb760 CPU: 7 COMMAND: "llog_process_th" #0 [ffff880331bc8e48] crash_nmi_callback at ffffffff81053d11 #1 [ffff880331bc8e58] nmi_handle at ffffffff817e5967 #2 [ffff880331bc8eb0] do_nmi at ffffffff817e5b5c #3 [ffff880331bc8ef0] end_repeat_nmi at ffffffff817e4d9c [exception RIP: unwind_next_frame+91] RIP: ffffffff8106b41b RSP: ffff8802d2c36e90 RFLAGS: 00000012 RAX: ffffffff81b3bff0 RBX: ffff8802d2c36ec0 RCX: 0000000000000010 RDX: ffffffffa03e7542 RSI: ffffffffa03e7542 RDI: ffff8802d2c36ec0 RBP: ffff8802d2c36ea0 R8: 0000000000000000 R9: 000000000000005a R10: 0000000000000000 R11: 0000000000000025 R12: 0000000000000000 R13: 0000000000000000 R14: ffff8802f2dcb760 R15: ffff8803314047c0 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 --- <NMI exception stack> --- #4 [ffff8802d2c36e90] unwind_next_frame at ffffffff8106b41b #5 [ffff8802d2c36ea8] unwind_next_frame at ffffffff8106b4a6 #6 [ffff8802d2c36eb8] __save_stack_trace at ffffffff8103dd7b #7 [ffff8802d2c36f40] save_stack_trace at ffffffff8103de0f #8 [ffff8802d2c36f50] set_track at ffffffff81219770 #9 [ffff8802d2c36f90] free_debug_processing at ffffffff817d567f #10 [ffff8802d2c36ff8] __slab_free at ffffffff8121e635 #11 [ffff8802d2c370a8] kfree at ffffffff8121f006 #12 [ffff8802d2c370d8] spl_kmem_free at ffffffffa0ac64ca [spl] #13 [ffff8802d2c370e8] dbuf_hold_impl at ffffffffa0c6650b [zfs] #14 [ffff8802d2c37130] dbuf_prefetch at ffffffffa0c6696e [zfs] #15 [ffff8802d2c37260] dmu_zfetch at ffffffffa0c87d2d [zfs] #16 [ffff8802d2c372f0] dbuf_read at ffffffffa0c64397 [zfs] #17 [ffff8802d2c37390] dnode_next_offset_level at ffffffffa0c88e30 [zfs] #18 [ffff8802d2c37400] dnode_next_offset at ffffffffa0c8b7b8 [zfs] #19 [ffff8802d2c37458] dmu_object_next at ffffffffa0c73704 [zfs] #20 [ffff8802d2c374e8] dmu_object_alloc_impl at ffffffffa0c73a55 [zfs] #21 [ffff8802d2c375b8] dmu_object_alloc_dnsize at ffffffffa0c73ca9 [zfs] #22 [ffff8802d2c375e8] __osd_object_create at ffffffffa0af2663 [osd_zfs] #23 [ffff8802d2c37638] osd_mkreg at ffffffffa0af29a0 [osd_zfs] #24 [ffff8802d2c37678] osd_create at ffffffffa0aedd93 [osd_zfs] #25 [ffff8802d2c37710] local_object_create at ffffffffa03f404f [obdclass] #26 [ffff8802d2c37770] llog_osd_create at ffffffffa03e7542 [obdclass] #27 [ffff8802d2c377c0] llog_create at ffffffffa0375270 [obdclass] #28 [ffff8802d2c377f8] llog_cat_new_log at ffffffffa037bd54 [obdclass] #29 [ffff8802d2c37868] llog_cat_add_rec at ffffffffa037cdbb [obdclass] #30 [ffff8802d2c37900] llog_cat_add at ffffffffa037e55e [obdclass] #31 [ffff8802d2c37948] osp_sync_llog_init at ffffffffa150c68d [osp] #32 [ffff8802d2c379b0] osp_sync_init at ffffffffa150f860 [osp] #33 [ffff8802d2c37a28] osp_init0 at ffffffffa14fa0c3 [osp] #34 [ffff8802d2c37aa0] osp_device_alloc at ffffffffa14facf9 [osp] #35 [ffff8802d2c37ad8] obd_setup at ffffffffa03a10a3 [obdclass] #36 [ffff8802d2c37b88] class_setup at ffffffffa03a256f [obdclass] #37 [ffff8802d2c37be0] class_process_config at ffffffffa03a4d2b [obdclass] #38 [ffff8802d2c37c88] class_config_llog_handler at ffffffffa03a7e1b [obdclass] #39 [ffff8802d2c37d58] llog_process_thread at ffffffffa0377a80 [obdclass] #40 [ffff8802d2c37e80] llog_process_thread_daemonize at ffffffffa0379652 [obdclass] #41 [ffff8802d2c37ea8] kthread at ffffffff810ba114 #42 [ffff8802d2c37f50] ret_from_fork_nospec_begin at ffffffff817ede5d

Oleg Drokin added a comment - 01/Feb/25 5:32 AM

[262288.408533] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[262288.409243] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon pcspkr i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core floppy libata [last unloaded: libcfs]
[262288.409513] CPU: 7 PID: 11460 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[262288.409513] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[262288.409513] task: ffff8802e74e1280 ti: ffff8803205b4000 task.ti: ffff8803205b4000
[262288.409513] RIP: 0010:[<ffffffffa0381a7d>]  [<ffffffffa0381a7d>] lprocfs_timeouts_seq_show+0x4d/0x1d0 [obdclass]
[262288.409513] RSP: 0018:ffff8803205b7e08  EFLAGS: 00010282
[262288.409513] RAX: 000000006797b9e1 RBX: ffff880096400008 RCX: ffff8803205b7fd8
[262288.409513] RDX: 0000000000000000 RSI: 0000000000000015 RDI: ffffffff81a98ea7
[262288.409513] RBP: ffff8803205b7e48 R08: ffff8802ddf2b3f8 R09: ffff8802ddf2e7e8
[262288.409513] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff8802e4059ef8
[262288.409513] R13: ffff8803205b7f18 R14: 0000000000000001 R15: 6b6b6b6b6b6b6b6b
[262288.409513] FS:  00007fee46e86740(0000) GS:ffff880331bc0000(0000) knlGS:0000000000000000
[262288.409513] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[262288.409513] CR2: 00000000023d6058 CR3: 00000002e4628000 CR4: 00000000000007e0
[262288.409513] Call Trace:
[262288.409513]  [<ffffffffa150824a>] osp_timeouts_seq_show+0x1a/0x30 [osp]
[262288.409513]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[262288.409513]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[262288.409513]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[262288.409513]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[262288.409513]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[262288.409513]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[262288.409513]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[262288.409513] Code: 48 83 ec 18 48 89 45 c8 e8 71 e3 45 e1 4c 8b bb 70 07 00 00 4d 85 ff 0f 84 4c 01 00 00 48 85 db 0f 84 4a 01 00 00 e8 13 56 e0 ff <49> 8b 8f 78 06 00 00 49 89 c0 49 89 c5 48 c7 c2 49 f1 40 a0 31
[262288.409513] RIP  [<ffffffffa0381a7d>] lprocfs_timeouts_seq_show+0x4d/0x1d0 [obdclass]

 [67588.033592] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[67588.034299] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs]
[67588.034299] CPU: 8 PID: 3652 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[67588.041842] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[67588.041842] task: ffff8802f09ca4f0 ti: ffff88009afc4000 task.ti: ffff88009afc4000
[67588.041842] RIP: 0010:[<ffffffffa03a874d>]  [<ffffffffa03a874d>] lprocfs_server_uuid_seq_show+0x3d/0x110 [obdclass]
[67588.041842] RSP: 0018:ffff88009afc7e30  EFLAGS: 00010202
[67588.041842] RAX: ffff8802f09ca4f3 RBX: ffff880319ca0718 RCX: ffff88009afc7fd8
[67588.041842] RDX: 6b6b6b6b6b6b6b6b RSI: 0000000000000015 RDI: ffffffff81a98ea7
[67588.041842] RBP: ffff88009afc7e48 R08: ffff8802c496b3f8 R09: ffff8802c4969158
[67588.041842] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff880319ca0008
[67588.041842] R13: ffff8802e991b9c8 R14: 0000000000000001 R15: ffff8802e991b9c8
[67588.041842] FS:  00007f4a11a30740(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000
[67588.041842] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[67588.041842] CR2: 00007f6526924d70 CR3: 00000000b17e4000 CR4: 00000000000007e0
[67588.041842] Call Trace:
[67588.041842]  [<ffffffffa0e9627a>] osp_server_uuid_seq_show+0x1a/0x30 [osp]
[67588.041842]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[67588.041842]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[67588.041842]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[67588.041842]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[67588.041842]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[67588.041842]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[67588.041842]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[67588.041842] Code: f4 53 0f 84 b7 00 00 00 48 8d 9e 10 07 00 00 49 89 fd 48 89 df e8 94 76 43 e1 49 8b 94 24 70 07 00 00 48 85 d2 0f 84 8c 00 00 00 <8b> 82 58 01 00 00 83 f8 0b 77 4f f6 82 dc 02 00 00 02 48 8b 0c
[67588.041842] RIP  [<ffffffffa03a874d>] lprocfs_server_uuid_seq_show+0x3d/0x110 [obdclass]

[243523.142185] Lustre: Failing over lustre-MDT0000
[243523.481718] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[243523.482111] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata virtio_blk serio_raw floppy i2c_core [last unloaded: libcfs]
[243523.482111] CPU: 15 PID: 11636 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[243523.482111] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[243523.482111] task: ffff88031fe8a4f0 ti: ffff880084d50000 task.ti: ffff880084d50000
[243523.482111] RIP: 0010:[<ffffffffa03d6ae9>]  [<ffffffffa03d6ae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass]
[243523.482111] RSP: 0018:ffff880084d53d58  EFLAGS: 00010246
[243523.482111] RAX: 0000000000000000 RBX: ffff8802dd5f96d0 RCX: ffff880084d53fd8
[243523.482111] RDX: 6b6b6b6b6b6b6b6b RSI: ffff8802dd5f96d0 RDI: ffff8800aacb7bd8
[243523.482111] RBP: ffff880084d53e18 R08: ffff880323610008 R09: ffff8803236167e8
[243523.482111] R10: 0000000000000000 R11: ffff880331406d40 R12: 6b6b6b6b6b6b6e5b
[243523.482111] R13: 6b6b6b6b6b6b6b6b R14: 0000000000000001 R15: ffff8800aacb7bd8
[243523.482111] FS:  00007f943c61a740(0000) GS:ffff880331dc0000(0000) knlGS:0000000000000000
[243523.482111] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[243523.482111] CR2: 0000000000a68560 CR3: 00000000ad09c000 CR4: 00000000000007e0
[243523.482111] Call Trace:
[243523.482111]  [<ffffffff81406448>] ? call_rwsem_down_read_failed+0x18/0x30
[243523.482111]  [<ffffffffa03d7463>] lprocfs_import_seq_show+0x43/0xa0 [obdclass]
[243523.482111]  [<ffffffffa14e51ea>] osp_import_seq_show+0x1a/0x30 [osp]
[243523.482111]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[243523.482111]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[243523.482111]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[243523.482111]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[243523.482111]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[243523.482111]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[243523.482111]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[243523.482111] Code: 55 49 89 d5 41 54 4c 8d a2 f0 02 00 00 53 48 81 ec 98 00 00 00 48 89 b5 58 ff ff ff 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 <8b> 82 58 01 00 00 83 f8 0b 0f 87 04 09 00 00 48 8b 95 58 ff ff
[243523.482111] RIP  [<ffffffffa03d6ae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass]

 [123791.112714] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[123791.113354] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw floppy i2c_core libata virtio_blk [last unloaded: libcfs]
[123791.113683] CPU: 5 PID: 14244 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[123791.113683] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[123791.113683] task: ffff880329b8a4f0 ti: ffff8800b73d4000 task.ti: ffff8800b73d4000
[123791.113683] RIP: 0010:[<ffffffffa0691ce7>]  [<ffffffffa0691ce7>] sptlrpc_ctxs_lprocfs_seq_show+0x27/0x100 [ptlrpc]
[123791.129356] RSP: 0018:ffff8800b73d7e40  EFLAGS: 00010203
[123791.129356] RAX: 6b6b6b6b6b6b6b6b RBX: ffff88031ee68008 RCX: 0000000000000004
[123791.129356] RDX: fffffffffffffff4 RSI: 0000000000000001 RDI: ffffffffa079a5c4
[123791.129356] RBP: ffff8800b73d7e58 R08: ffff8800a9f0e7e8 R09: ffff8800a9f08008
[123791.129356] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff8800ab822528
[123791.129356] R13: ffff8800b73d7f18 R14: 0000000000000001 R15: ffff8800ab822528
[123791.129356] FS:  00007fccaebe8740(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000
[123791.129356] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[123791.129356] CR2: 0000000002671298 CR3: 00000000abdda000 CR4: 00000000000007e0
[123791.129356] Call Trace:
[123791.129356]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[123791.129356]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[123791.129356]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[123791.129356]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[123791.129356]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[123791.129356]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[123791.129356]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[123791.129356] Code: 00 00 00 00 66 66 66 66 90 55 b9 04 00 00 00 48 89 e5 41 55 41 54 49 89 fc 53 48 8b 9f 98 00 00 00 48 c7 c7 c4 a5 79 a0 48 8b 03 <4c> 8b 68 30 4c 89 ee f3 a6 75 42 48 8b bb 70 07 00 00 48 85 ff
[123791.129356] RIP  [<ffffffffa0691ce7>] sptlrpc_ctxs_lprocfs_seq_show+0x27/0x100 [ptlrpc]

 [356489.392796] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[356489.393528] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk i2c_core serio_raw libata floppy [last unloaded: libcfs]
[356489.393773] CPU: 12 PID: 17242 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[356489.393773] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[356489.402851] Lustre: Unmounted lustre-client
[356489.393773] task: ffff8802efc89280 ti: ffff8800a45f4000 task.ti: ffff8800a45f4000
[356489.393773] RIP: 0010:[<ffffffffa03c1a33>]  [<ffffffffa03c1a33>] lprocfs_stats_collect+0xa3/0x120 [obdclass]
[356489.393773] RSP: 0018:ffff8800a45f7dc8  EFLAGS: 00010202
[356489.393773] RAX: 0000000000000010 RBX: ffff8800a45f7e08 RCX: 6b6b6b6b6b6b6b6b
[356489.393773] RDX: 0000000000000008 RSI: 6b6b6b6b6b6b6b6b RDI: 0000000000000008
[356489.393773] RBP: ffff8800a45f7df0 R08: 0000000000000000 R09: 000000000000ffff
[356489.393773] R10: 0000000000000000 R11: ffff8800a45f7c46 R12: ffff88031f192e48
[356489.393773] R13: 0000000000000000 R14: ffff8803272ea2a8 R15: ffff8802eeb36b58
[356489.393773] FS:  00007faecd350740(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000
[356489.393773] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[356489.393773] CR2: 0000000001189058 CR3: 00000000a8716000 CR4: 00000000000007e0
[356489.393773] Call Trace:
[356489.393773]  [<ffffffffa03c250b>] lprocfs_stats_seq_show+0x4b/0x150 [obdclass]
[356489.393773]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[356489.393773]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[356489.393773]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[356489.393773]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[356489.393773]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[356489.393773]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[356489.393773]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[356489.393773] Code: d8 e8 c2 fe ff ff 4f 8d 44 ad 00 31 ff 31 d2 49 c1 e0 03 85 c0 74 4f 90 48 63 ca 49 83 7c cc 48 00 74 3a 4c 89 c1 49 03 4c fc 48 <48> 8b 31 48 01 33 48 8b 71 18 48 01 73 18 48 8b 71 08 48 3b 73
[356489.393773] RIP  [<ffffffffa03c1a33>] lprocfs_stats_collect+0xa3/0x120 [obdclass]

 [100489.457883] BUG: unable to handle kernel NULL pointer dereference at 0000000000000158
[100489.458535] IP: [<ffffffffa0bf65b1>] osp_statfs+0x2a1/0x4e0 [osp]
[100489.458535] PGD 31d212067 PUD 2f7de6067 PMD 0
[100489.458535] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[100489.458535] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw virtio_blk i2c_core libata [last unloaded: libcfs]
[100489.458535] CPU: 3 PID: 30430 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[100489.458535] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[100489.458535] task: ffff8802dee749d0 ti: ffff8802eef50000 task.ti: ffff8802eef50000
[100489.458535] RIP: 0010:[<ffffffffa0bf65b1>]  [<ffffffffa0bf65b1>] osp_statfs+0x2a1/0x4e0 [osp]
[100489.458535] RSP: 0018:ffff8802eef53cd8  EFLAGS: 00010246
[100489.458535] RAX: 0000000000000000 RBX: ffff8800adec0008 RCX: 0000000000000000
[100489.458535] RDX: 0000000000000003 RSI: 0000000000000000 RDI: ffff8803223b8188
[100489.458535] RBP: ffff8802eef53d28 R08: 000000000000ffff R09: 000000000000ffff
[100489.458535] R10: 20737365636f7250 R11: 0a64657265746e65 R12: 0000000000000000
[100489.458535] R13: ffff8802eef53d38 R14: 0000000000000000 R15: ffff88009c4b4218
[100489.458535] FS:  00007fabbe6c9740(0000) GS:ffff880331ac0000(0000) knlGS:0000000000000000
[100489.458535] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[100489.458535] CR2: 0000000000000158 CR3: 00000002be142000 CR4: 00000000000007e0
[100489.458535] Call Trace:
[100489.458535]  [<ffffffff817ee00c>] ? system_call_fastpath+0x1f/0x24
[100489.458535]  [<ffffffffa0410892>] blocksize_show+0x52/0x120 [obdclass]
[100489.458535]  [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0
[100489.458535]  [<ffffffffa03c3166>] lustre_attr_show+0x16/0x20 [obdclass]
[100489.458535]  [<ffffffff812d728c>] sysfs_kf_seq_show+0xcc/0x1e0
[100489.458535]  [<ffffffff812d58b3>] kernfs_seq_show+0x23/0x30
[100489.458535]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[100489.458535]  [<ffffffff812d643d>] kernfs_fop_read+0x11d/0x190
[100489.458535]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[100489.458535]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[100489.458535]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[100489.458535]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[100489.458535]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[100489.458535]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[100489.458535] Code: fb 03 00 90 32 c2 a0 c7 05 1d fb 03 00 19 03 00 00 c7 05 17 fb 03 00 01 00 00 00 48 c7 05 14 fb 03 00 00 00 00 00 e8 7f b8 5c ff <41> 83 be 58 01 00 00 01 0f 85 ba fd ff ff 90 f6 05 81 8f 5d ff
[100489.458535] RIP  [<ffffffffa0bf65b1>] osp_statfs+0x2a1/0x4e0 [osp]

[43389.554065] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[43389.554789] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk i2c_core serio_raw libata [last unloaded: libcfs]
[43389.555007] CPU: 8 PID: 15450 Comm: lctl Kdump: loaded Tainted: P           OE  ------------   3.10.0-7.9-debug #2
[43389.555007] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014
[43389.555007] task: ffff880097a01280 ti: ffff8802ed180000 task.ti: ffff8802ed180000
[43389.555007] RIP: 0010:[<ffffffff81411a45>]  [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30
[43389.555007] RSP: 0018:ffff8802ed183df8  EFLAGS: 00010202
[43389.555007] RAX: ffff8802ed183fd8 RBX: 6b6b6b6b6b6b6b6b RCX: 0000000000000000
[43389.555007] RDX: fffffffffffffff4 RSI: 0000000000000001 RDI: 6b6b6b6b6b6b6c33
[43389.555007] RBP: ffff8802ed183e00 R08: ffff8802f70d4548 R09: ffff8802f70d1158
[43389.555007] R10: 0000000000000000 R11: ffff880331406d40 R12: 6b6b6b6b6b6b6c33
[43389.555007] R13: ffff8802ed183f18 R14: 0000000000000001 R15: ffff88031ce44638
[43389.555007] FS:  00007f0ff6df2740(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000
[43389.555007] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[43389.555007] CR2: 000000000232b058 CR3: 00000002a3080000 CR4: 00000000000007e0
[43389.555007] Call Trace:
[43389.555007]  [<ffffffff817e333e>] _raw_read_lock+0x1e/0x20
[43389.555007]  [<ffffffffa0631f0e>] sptlrpc_import_sec_ref+0x1e/0x40 [ptlrpc]
[43389.555007]  [<ffffffffa063f2f0>] srpc_sptlrpc_sepol_seq_show+0x30/0x130 [ptlrpc]
[43389.555007]  [<ffffffff8126f0b5>] seq_read+0x135/0x470
[43389.555007]  [<ffffffff81245839>] vfs_read+0xb9/0x1c0
[43389.555007]  [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a
[43389.555007]  [<ffffffff8124676f>] SyS_read+0x7f/0xf0
[43389.555007]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[43389.555007]  [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24
[43389.555007]  [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a
[43389.555007] Code: 00 eb c5 48 c7 c6 96 33 ad 81 48 89 df e8 f1 79 3c 00 eb a8 48 c7 c6 85 33 ad 81 48 89 df e8 e0 79 3c 00 eb 88 90 55 48 89 e5 53 <81> 7f 08 ed 1e af de 48 89 fb 75 11 48 89 df f0 48 ff 0f 79 05
[43389.555007] RIP  [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30

coredumps available in these reports:

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72740

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72761

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72757

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72743

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72747

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72741

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72742

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72754

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72750

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72749

https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72748

Oleg Drokin added a comment - 01/Feb/25 5:32 AM [262288.408533] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [262288.409243] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon pcspkr i2c_piix4 virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw virtio_blk i2c_core floppy libata [last unloaded: libcfs] [262288.409513] CPU: 7 PID: 11460 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [262288.409513] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [262288.409513] task: ffff8802e74e1280 ti: ffff8803205b4000 task.ti: ffff8803205b4000 [262288.409513] RIP: 0010:[<ffffffffa0381a7d>] [<ffffffffa0381a7d>] lprocfs_timeouts_seq_show+0x4d/0x1d0 [obdclass] [262288.409513] RSP: 0018:ffff8803205b7e08 EFLAGS: 00010282 [262288.409513] RAX: 000000006797b9e1 RBX: ffff880096400008 RCX: ffff8803205b7fd8 [262288.409513] RDX: 0000000000000000 RSI: 0000000000000015 RDI: ffffffff81a98ea7 [262288.409513] RBP: ffff8803205b7e48 R08: ffff8802ddf2b3f8 R09: ffff8802ddf2e7e8 [262288.409513] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff8802e4059ef8 [262288.409513] R13: ffff8803205b7f18 R14: 0000000000000001 R15: 6b6b6b6b6b6b6b6b [262288.409513] FS: 00007fee46e86740(0000) GS:ffff880331bc0000(0000) knlGS:0000000000000000 [262288.409513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [262288.409513] CR2: 00000000023d6058 CR3: 00000002e4628000 CR4: 00000000000007e0 [262288.409513] Call Trace: [262288.409513] [<ffffffffa150824a>] osp_timeouts_seq_show+0x1a/0x30 [osp] [262288.409513] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [262288.409513] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [262288.409513] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [262288.409513] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [262288.409513] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [262288.409513] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [262288.409513] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [262288.409513] Code: 48 83 ec 18 48 89 45 c8 e8 71 e3 45 e1 4c 8b bb 70 07 00 00 4d 85 ff 0f 84 4c 01 00 00 48 85 db 0f 84 4a 01 00 00 e8 13 56 e0 ff <49> 8b 8f 78 06 00 00 49 89 c0 49 89 c5 48 c7 c2 49 f1 40 a0 31 [262288.409513] RIP [<ffffffffa0381a7d>] lprocfs_timeouts_seq_show+0x4d/0x1d0 [obdclass] [67588.033592] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [67588.034299] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks serio_raw virtio_blk i2c_core libata floppy [last unloaded: libcfs] [67588.034299] CPU: 8 PID: 3652 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [67588.041842] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [67588.041842] task: ffff8802f09ca4f0 ti: ffff88009afc4000 task.ti: ffff88009afc4000 [67588.041842] RIP: 0010:[<ffffffffa03a874d>] [<ffffffffa03a874d>] lprocfs_server_uuid_seq_show+0x3d/0x110 [obdclass] [67588.041842] RSP: 0018:ffff88009afc7e30 EFLAGS: 00010202 [67588.041842] RAX: ffff8802f09ca4f3 RBX: ffff880319ca0718 RCX: ffff88009afc7fd8 [67588.041842] RDX: 6b6b6b6b6b6b6b6b RSI: 0000000000000015 RDI: ffffffff81a98ea7 [67588.041842] RBP: ffff88009afc7e48 R08: ffff8802c496b3f8 R09: ffff8802c4969158 [67588.041842] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff880319ca0008 [67588.041842] R13: ffff8802e991b9c8 R14: 0000000000000001 R15: ffff8802e991b9c8 [67588.041842] FS: 00007f4a11a30740(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 [67588.041842] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [67588.041842] CR2: 00007f6526924d70 CR3: 00000000b17e4000 CR4: 00000000000007e0 [67588.041842] Call Trace: [67588.041842] [<ffffffffa0e9627a>] osp_server_uuid_seq_show+0x1a/0x30 [osp] [67588.041842] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [67588.041842] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [67588.041842] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [67588.041842] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [67588.041842] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [67588.041842] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [67588.041842] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [67588.041842] Code: f4 53 0f 84 b7 00 00 00 48 8d 9e 10 07 00 00 49 89 fd 48 89 df e8 94 76 43 e1 49 8b 94 24 70 07 00 00 48 85 d2 0f 84 8c 00 00 00 <8b> 82 58 01 00 00 83 f8 0b 77 4f f6 82 dc 02 00 00 02 48 8b 0c [67588.041842] RIP [<ffffffffa03a874d>] lprocfs_server_uuid_seq_show+0x3d/0x110 [obdclass] [243523.142185] Lustre: Failing over lustre-MDT0000 [243523.481718] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [243523.482111] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 virtio_console virtio_balloon pcspkr ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks libata virtio_blk serio_raw floppy i2c_core [last unloaded: libcfs] [243523.482111] CPU: 15 PID: 11636 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [243523.482111] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [243523.482111] task: ffff88031fe8a4f0 ti: ffff880084d50000 task.ti: ffff880084d50000 [243523.482111] RIP: 0010:[<ffffffffa03d6ae9>] [<ffffffffa03d6ae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass] [243523.482111] RSP: 0018:ffff880084d53d58 EFLAGS: 00010246 [243523.482111] RAX: 0000000000000000 RBX: ffff8802dd5f96d0 RCX: ffff880084d53fd8 [243523.482111] RDX: 6b6b6b6b6b6b6b6b RSI: ffff8802dd5f96d0 RDI: ffff8800aacb7bd8 [243523.482111] RBP: ffff880084d53e18 R08: ffff880323610008 R09: ffff8803236167e8 [243523.482111] R10: 0000000000000000 R11: ffff880331406d40 R12: 6b6b6b6b6b6b6e5b [243523.482111] R13: 6b6b6b6b6b6b6b6b R14: 0000000000000001 R15: ffff8800aacb7bd8 [243523.482111] FS: 00007f943c61a740(0000) GS:ffff880331dc0000(0000) knlGS:0000000000000000 [243523.482111] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [243523.482111] CR2: 0000000000a68560 CR3: 00000000ad09c000 CR4: 00000000000007e0 [243523.482111] Call Trace: [243523.482111] [<ffffffff81406448>] ? call_rwsem_down_read_failed+0x18/0x30 [243523.482111] [<ffffffffa03d7463>] lprocfs_import_seq_show+0x43/0xa0 [obdclass] [243523.482111] [<ffffffffa14e51ea>] osp_import_seq_show+0x1a/0x30 [osp] [243523.482111] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [243523.482111] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [243523.482111] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [243523.482111] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [243523.482111] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [243523.482111] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [243523.482111] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [243523.482111] Code: 55 49 89 d5 41 54 4c 8d a2 f0 02 00 00 53 48 81 ec 98 00 00 00 48 89 b5 58 ff ff ff 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 <8b> 82 58 01 00 00 83 f8 0b 0f 87 04 09 00 00 48 8b 95 58 ff ff [243523.482111] RIP [<ffffffffa03d6ae9>] lprocfs_import_seq_show_locked+0x39/0x970 [obdclass] [123791.112714] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [123791.113354] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common i2c_piix4 pcspkr virtio_balloon virtio_console ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix serio_raw floppy i2c_core libata virtio_blk [last unloaded: libcfs] [123791.113683] CPU: 5 PID: 14244 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [123791.113683] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [123791.113683] task: ffff880329b8a4f0 ti: ffff8800b73d4000 task.ti: ffff8800b73d4000 [123791.113683] RIP: 0010:[<ffffffffa0691ce7>] [<ffffffffa0691ce7>] sptlrpc_ctxs_lprocfs_seq_show+0x27/0x100 [ptlrpc] [123791.129356] RSP: 0018:ffff8800b73d7e40 EFLAGS: 00010203 [123791.129356] RAX: 6b6b6b6b6b6b6b6b RBX: ffff88031ee68008 RCX: 0000000000000004 [123791.129356] RDX: fffffffffffffff4 RSI: 0000000000000001 RDI: ffffffffa079a5c4 [123791.129356] RBP: ffff8800b73d7e58 R08: ffff8800a9f0e7e8 R09: ffff8800a9f08008 [123791.129356] R10: 0000000000000000 R11: ffff880331406d40 R12: ffff8800ab822528 [123791.129356] R13: ffff8800b73d7f18 R14: 0000000000000001 R15: ffff8800ab822528 [123791.129356] FS: 00007fccaebe8740(0000) GS:ffff880331b40000(0000) knlGS:0000000000000000 [123791.129356] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [123791.129356] CR2: 0000000002671298 CR3: 00000000abdda000 CR4: 00000000000007e0 [123791.129356] Call Trace: [123791.129356] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [123791.129356] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [123791.129356] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [123791.129356] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [123791.129356] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [123791.129356] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [123791.129356] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [123791.129356] Code: 00 00 00 00 66 66 66 66 90 55 b9 04 00 00 00 48 89 e5 41 55 41 54 49 89 fc 53 48 8b 9f 98 00 00 00 48 c7 c7 c4 a5 79 a0 48 8b 03 <4c> 8b 68 30 4c 89 ee f3 a6 75 42 48 8b bb 70 07 00 00 48 85 ff [123791.129356] RIP [<ffffffffa0691ce7>] sptlrpc_ctxs_lprocfs_seq_show+0x27/0x100 [ptlrpc] [356489.392796] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [356489.393528] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common virtio_balloon virtio_console pcspkr i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks virtio_blk i2c_core serio_raw libata floppy [last unloaded: libcfs] [356489.393773] CPU: 12 PID: 17242 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [356489.393773] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [356489.402851] Lustre: Unmounted lustre-client [356489.393773] task: ffff8802efc89280 ti: ffff8800a45f4000 task.ti: ffff8800a45f4000 [356489.393773] RIP: 0010:[<ffffffffa03c1a33>] [<ffffffffa03c1a33>] lprocfs_stats_collect+0xa3/0x120 [obdclass] [356489.393773] RSP: 0018:ffff8800a45f7dc8 EFLAGS: 00010202 [356489.393773] RAX: 0000000000000010 RBX: ffff8800a45f7e08 RCX: 6b6b6b6b6b6b6b6b [356489.393773] RDX: 0000000000000008 RSI: 6b6b6b6b6b6b6b6b RDI: 0000000000000008 [356489.393773] RBP: ffff8800a45f7df0 R08: 0000000000000000 R09: 000000000000ffff [356489.393773] R10: 0000000000000000 R11: ffff8800a45f7c46 R12: ffff88031f192e48 [356489.393773] R13: 0000000000000000 R14: ffff8803272ea2a8 R15: ffff8802eeb36b58 [356489.393773] FS: 00007faecd350740(0000) GS:ffff880331d00000(0000) knlGS:0000000000000000 [356489.393773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [356489.393773] CR2: 0000000001189058 CR3: 00000000a8716000 CR4: 00000000000007e0 [356489.393773] Call Trace: [356489.393773] [<ffffffffa03c250b>] lprocfs_stats_seq_show+0x4b/0x150 [obdclass] [356489.393773] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [356489.393773] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [356489.393773] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [356489.393773] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [356489.393773] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [356489.393773] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [356489.393773] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [356489.393773] Code: d8 e8 c2 fe ff ff 4f 8d 44 ad 00 31 ff 31 d2 49 c1 e0 03 85 c0 74 4f 90 48 63 ca 49 83 7c cc 48 00 74 3a 4c 89 c1 49 03 4c fc 48 <48> 8b 31 48 01 33 48 8b 71 18 48 01 73 18 48 8b 71 08 48 3b 73 [356489.393773] RIP [<ffffffffa03c1a33>] lprocfs_stats_collect+0xa3/0x120 [obdclass] [100489.457883] BUG: unable to handle kernel NULL pointer dereference at 0000000000000158 [100489.458535] IP: [<ffffffffa0bf65b1>] osp_statfs+0x2a1/0x4e0 [osp] [100489.458535] PGD 31d212067 PUD 2f7de6067 PMD 0 [100489.458535] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC [100489.458535] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) dm_flakey dm_mod libcfs(OE) loop zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) jbd2 mbcache crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm drm_panel_orientation_quirks ata_piix floppy serio_raw virtio_blk i2c_core libata [last unloaded: libcfs] [100489.458535] CPU: 3 PID: 30430 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [100489.458535] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [100489.458535] task: ffff8802dee749d0 ti: ffff8802eef50000 task.ti: ffff8802eef50000 [100489.458535] RIP: 0010:[<ffffffffa0bf65b1>] [<ffffffffa0bf65b1>] osp_statfs+0x2a1/0x4e0 [osp] [100489.458535] RSP: 0018:ffff8802eef53cd8 EFLAGS: 00010246 [100489.458535] RAX: 0000000000000000 RBX: ffff8800adec0008 RCX: 0000000000000000 [100489.458535] RDX: 0000000000000003 RSI: 0000000000000000 RDI: ffff8803223b8188 [100489.458535] RBP: ffff8802eef53d28 R08: 000000000000ffff R09: 000000000000ffff [100489.458535] R10: 20737365636f7250 R11: 0a64657265746e65 R12: 0000000000000000 [100489.458535] R13: ffff8802eef53d38 R14: 0000000000000000 R15: ffff88009c4b4218 [100489.458535] FS: 00007fabbe6c9740(0000) GS:ffff880331ac0000(0000) knlGS:0000000000000000 [100489.458535] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [100489.458535] CR2: 0000000000000158 CR3: 00000002be142000 CR4: 00000000000007e0 [100489.458535] Call Trace: [100489.458535] [<ffffffff817ee00c>] ? system_call_fastpath+0x1f/0x24 [100489.458535] [<ffffffffa0410892>] blocksize_show+0x52/0x120 [obdclass] [100489.458535] [<ffffffff812222d2>] ? __kmalloc_node+0x1f2/0x3e0 [100489.458535] [<ffffffffa03c3166>] lustre_attr_show+0x16/0x20 [obdclass] [100489.458535] [<ffffffff812d728c>] sysfs_kf_seq_show+0xcc/0x1e0 [100489.458535] [<ffffffff812d58b3>] kernfs_seq_show+0x23/0x30 [100489.458535] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [100489.458535] [<ffffffff812d643d>] kernfs_fop_read+0x11d/0x190 [100489.458535] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [100489.458535] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [100489.458535] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [100489.458535] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [100489.458535] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [100489.458535] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [100489.458535] Code: fb 03 00 90 32 c2 a0 c7 05 1d fb 03 00 19 03 00 00 c7 05 17 fb 03 00 01 00 00 00 48 c7 05 14 fb 03 00 00 00 00 00 e8 7f b8 5c ff <41> 83 be 58 01 00 00 01 0f 85 ba fd ff ff 90 f6 05 81 8f 5d ff [100489.458535] RIP [<ffffffffa0bf65b1>] osp_statfs+0x2a1/0x4e0 [osp] [43389.554065] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC [43389.554789] Modules linked in: lustre(OE) osp(OE) ofd(OE) lod(OE) mdt(OE) mdd(OE) mgs(OE) osd_zfs(OE) lquota(OE) lfsck(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) crc32_generic crc_t10dif crct10dif_generic crct10dif_common pcspkr virtio_balloon virtio_console i2c_piix4 ip_tables rpcsec_gss_krb5 ata_generic pata_acpi drm_kms_helper ttm drm ata_piix drm_panel_orientation_quirks floppy virtio_blk i2c_core serio_raw libata [last unloaded: libcfs] [43389.555007] CPU: 8 PID: 15450 Comm: lctl Kdump: loaded Tainted: P OE ------------ 3.10.0-7.9-debug #2 [43389.555007] Hardware name: Red Hat KVM, BIOS 1.16.0-3.module_el8.7.0+1218+f626c2ff 04/01/2014 [43389.555007] task: ffff880097a01280 ti: ffff8802ed180000 task.ti: ffff8802ed180000 [43389.555007] RIP: 0010:[<ffffffff81411a45>] [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30 [43389.555007] RSP: 0018:ffff8802ed183df8 EFLAGS: 00010202 [43389.555007] RAX: ffff8802ed183fd8 RBX: 6b6b6b6b6b6b6b6b RCX: 0000000000000000 [43389.555007] RDX: fffffffffffffff4 RSI: 0000000000000001 RDI: 6b6b6b6b6b6b6c33 [43389.555007] RBP: ffff8802ed183e00 R08: ffff8802f70d4548 R09: ffff8802f70d1158 [43389.555007] R10: 0000000000000000 R11: ffff880331406d40 R12: 6b6b6b6b6b6b6c33 [43389.555007] R13: ffff8802ed183f18 R14: 0000000000000001 R15: ffff88031ce44638 [43389.555007] FS: 00007f0ff6df2740(0000) GS:ffff880331c00000(0000) knlGS:0000000000000000 [43389.555007] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [43389.555007] CR2: 000000000232b058 CR3: 00000002a3080000 CR4: 00000000000007e0 [43389.555007] Call Trace: [43389.555007] [<ffffffff817e333e>] _raw_read_lock+0x1e/0x20 [43389.555007] [<ffffffffa0631f0e>] sptlrpc_import_sec_ref+0x1e/0x40 [ptlrpc] [43389.555007] [<ffffffffa063f2f0>] srpc_sptlrpc_sepol_seq_show+0x30/0x130 [ptlrpc] [43389.555007] [<ffffffff8126f0b5>] seq_read+0x135/0x470 [43389.555007] [<ffffffff81245839>] vfs_read+0xb9/0x1c0 [43389.555007] [<ffffffff817edf49>] ? system_call_after_swapgs+0x96/0x13a [43389.555007] [<ffffffff8124676f>] SyS_read+0x7f/0xf0 [43389.555007] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [43389.555007] [<ffffffff817ee00c>] system_call_fastpath+0x1f/0x24 [43389.555007] [<ffffffff817edf55>] ? system_call_after_swapgs+0xa2/0x13a [43389.555007] Code: 00 eb c5 48 c7 c6 96 33 ad 81 48 89 df e8 f1 79 3c 00 eb a8 48 c7 c6 85 33 ad 81 48 89 df e8 e0 79 3c 00 eb 88 90 55 48 89 e5 53 <81> 7f 08 ed 1e af de 48 89 fb 75 11 48 89 df f0 48 ff 0f 79 05 [43389.555007] RIP [<ffffffff81411a45>] do_raw_read_lock+0x5/0x30 coredumps available in these reports: https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72740 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72761 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72757 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72743 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72747 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72741 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72742 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72754 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72750 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72749 https://knox.linuxhacker.ru/crashdb_ui_external.py.cgi?newid=72748

People

Assignee:: Oleg Drokin

Reporter:: Oleg Drokin

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 01/Feb/25 5:32 AM

Updated:: 20/Feb/25 8:19 PM