[LU-14763] lu_object.c:2491:lu_object_assign_fid()) ASSERTION( rc == 0 ) failed

Related — Tue, 15 Jun 2021 00:45:10 +0000
Writing more than 100M files into across 8 x MDT with DNE2 (no DoM enabled), it hit the following LBUG.
[ 3804.396449] LustreError: 5252:0:(lu_object.c:2491:lu_object_assign_fid()) ASSERTION( rc == 0 ) failed: failed hashtable insertion: rc = -16
[ 3804.398795] LustreError: 5252:0:(lu_object.c:2491:lu_object_assign_fid()) LBUG
[ 3804.400126] Pid: 5252, comm: mdt00_008 3.10.0-1160.21.1.el7_lustre.ddn15.x86_64 #1 SMP Fri May 7 21:36:39 UTC 2021
[ 3804.400128] Call Trace:
[ 3804.400225] [<0>] libcfs_call_trace+0x90/0xf0 [libcfs]
[ 3804.400233] [<0>] lbug_with_loc+0x4c/0xa0 [libcfs]
[ 3804.400275] [<0>] lu_object_assign_fid+0x37b/0x3a0 [obdclass]
[ 3804.400285] [<0>] osp_object_assign_fid+0x79/0x150 [osp]
[ 3804.400295] [<0>] osp_create+0x5e8/0xbf0 [osp]
[ 3804.400323] [<0>] lod_sub_create+0x1f5/0x480 [lod]
[ 3804.400337] [<0>] lod_striped_create+0x20c/0x570 [lod]
[ 3804.400349] [<0>] lod_create+0x230/0x2d0 [lod]
[ 3804.400365] [<0>] mdd_create_object_internal+0xae/0x2f0 [mdd]
[ 3804.400375] [<0>] mdd_create_object+0x83/0x940 [mdd]
[ 3804.400384] [<0>] mdd_create+0xf90/0x1800 [mdd]
[ 3804.400435] [<0>] mdt_reint_open+0x2422/0x3800 [mdt]
[ 3804.400455] [<0>] mdt_reint_rec+0x83/0x210 [mdt]
[ 3804.400474] [<0>] mdt_reint_internal+0x730/0xb00 [mdt]
[ 3804.400497] [<0>] mdt_intent_open+0x93/0x430 [mdt]
[ 3804.400514] [<0>] mdt_intent_opc+0x1ba/0xb40 [mdt]
[ 3804.400531] [<0>] mdt_intent_policy+0x1a1/0x360 [mdt]
[ 3804.400589] [<0>] ldlm_lock_enqueue+0x3c5/0xa90 [ptlrpc]
[ 3804.400649] [<0>] ldlm_handle_enqueue0+0xa86/0x1630 [ptlrpc]
[ 3804.400708] [<0>] tgt_enqueue+0x62/0x210 [ptlrpc]
[ 3804.400762] [<0>] tgt_request_handle+0x7f3/0x1760 [ptlrpc]
[ 3804.400814] [<0>] ptlrpc_server_handle_request+0x253/0xb00 [ptlrpc]
[ 3804.400862] [<0>] ptlrpc_main+0xb3c/0x14d0 [ptlrpc]
[ 3804.400868] [<0>] kthread+0xd1/0xe0
[ 3804.400874] [<0>] ret_from_fork_nospec_begin+0x7/0x21
[ 3804.400984] [<0>] 0xfffffffffffffffe
[ 3804.400988] Kernel panic - not syncing: LBUG
[ 3804.401763] CPU: 1 PID: 5252 Comm: mdt00_008 Kdump: loaded Tainted: G OE ------------ T 3.10.0-1160.21.1.el7_lustre.ddn15.x86_64 #1
[ 3804.403957] Hardware name: DDN SFA400NVXE, BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
[ 3804.405562] Call Trace:
[ 3804.406007] [<ffffffffad98305a>] dump_stack+0x19/0x1b
[ 3804.406913] [<ffffffffad97c5b2>] panic+0xe8/0x21f
[ 3804.407790] [<ffffffffc09712cb>] lbug_with_loc+0x9b/0xa0 [libcfs]
[ 3804.408914] [<ffffffffc0ab5a1b>] lu_object_assign_fid+0x37b/0x3a0 [obdclass]
[ 3804.410145] [<ffffffffc1664c1f>] ? osp_precreate_get_fid+0x21f/0x530 [osp]
[ 3804.411392] [<ffffffffc16580b9>] osp_object_assign_fid+0x79/0x150 [osp]
[ 3804.412558] [<ffffffffc1659fb8>] osp_create+0x5e8/0xbf0 [osp]
[ 3804.413541] [<ffffffffc1597525>] lod_sub_create+0x1f5/0x480 [lod]
[ 3804.414587] [<ffffffffc1584bdc>] lod_striped_create+0x20c/0x570 [lod]
[ 3804.415683] [<ffffffffc15853f0>] lod_create+0x230/0x2d0 [lod]
[ 3804.416677] [<ffffffffc160961e>] mdd_create_object_internal+0xae/0x2f0 [mdd]
[ 3804.417932] [<ffffffffc15f46c3>] mdd_create_object+0x83/0x940 [mdd]
[ 3804.419042] [<ffffffffc15f9eb0>] mdd_create+0xf90/0x1800 [mdd]
[ 3804.420071] [<ffffffffc147fac2>] mdt_reint_open+0x2422/0x3800 [mdt]
[ 3804.421174] [<ffffffffc0ad1f59>] ? check_unlink_entry+0x19/0xe0 [obdclass]
[ 3804.422373] [<ffffffffc0ad2d91>] ? upcall_cache_get_entry+0x3f1/0x8c0 [obdclass]
[ 3804.423648] [<ffffffffc145c2e2>] ? ucred_set_audit_enabled.isra.13+0x22/0x60 [mdt]
[ 3804.424956] [<ffffffffc14719b3>] mdt_reint_rec+0x83/0x210 [mdt]
[ 3804.425970] [<ffffffffc1449d60>] mdt_reint_internal+0x730/0xb00 [mdt]
[ 3804.427103] [<ffffffffc1457279>] ? mdt_intent_fixup_resent+0x39/0x240 [mdt]
[ 3804.428341] [<ffffffffc1457513>] mdt_intent_open+0x93/0x430 [mdt]
[ 3804.429444] [<ffffffffc144d74a>] mdt_intent_opc+0x1ba/0xb40 [mdt]
[ 3804.430506] [<ffffffffc1457480>] ? mdt_intent_fixup_resent+0x240/0x240 [mdt]
[ 3804.431737] [<ffffffffc1455701>] mdt_intent_policy+0x1a1/0x360 [mdt]
[ 3804.432921] [<ffffffffc0d585b5>] ldlm_lock_enqueue+0x3c5/0xa90 [ptlrpc]
[ 3804.434060] [<ffffffffc0978a27>] ? cfs_hash_bd_add_locked+0x67/0x90 [libcfs]
[ 3804.435300] [<ffffffffc097c0fe>] ? cfs_hash_add+0xbe/0x1a0 [libcfs]
[ 3804.436452] [<ffffffffc0d80dc6>] ldlm_handle_enqueue0+0xa86/0x1630 [ptlrpc]
[ 3804.437685] [<ffffffffc0da4200>] ? lustre_msg_buf_v2+0x110/0x1e0 [ptlrpc]
[ 3804.438892] [<ffffffffc0dfeed2>] tgt_enqueue+0x62/0x210 [ptlrpc]
[ 3804.440027] [<ffffffffc0e078a3>] tgt_request_handle+0x7f3/0x1760 [ptlrpc]
[ 3804.442414] [<ffffffffc0de97e1>] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc]
[ 3804.444916] [<ffffffffc096e02e>] ? ktime_get_real_seconds+0xe/0x10 [libcfs]
[ 3804.447268] [<ffffffffc0db66e3>] ptlrpc_server_handle_request+0x253/0xb00 [ptlrpc]
[ 3804.449700] [<ffffffffad2c6e90>] ? wake_up_atomic_t+0x30/0x30
[ 3804.451787] [<ffffffffc0db816c>] ptlrpc_main+0xb3c/0x14d0 [ptlrpc]
[ 3804.453866] [<ffffffffad2d4c1e>] ? finish_task_switch+0x4e/0x1c0
[ 3804.456005] [<ffffffffc0db7630>] ? ptlrpc_wait_event+0x5c0/0x5c0 [ptlrpc]
[ 3804.458139] [<ffffffffad2c5da1>] kthread+0xd1/0xe0
[ 3804.460008] [<ffffffffad2c5cd0>] ? insert_kthread_work+0x40/0x40
[ 3804.462034] [<ffffffffad995ddd>] ret_from_fork_nospec_begin+0x7/0x21
[ 3804.464094] [<ffffffffad2c5cd0>] ? insert_kthread_work+0x40/0x40
Whamcloud Community JIRA

[LU-14763] lu_object.c:2491:lu_object_assign_fid()) ASSERTION( rc == 0 ) failed
