[LU-16596] Lustre client crashed with ASSERTION( ldlm_is_granted(lock) ) failed

• Type: Bug
• Resolution: Duplicate
• Priority: Major
• Environment: Rocky Linux 8.7, Kernel 4.18.0-425.10.1.el8_7.x86_64, Lustre Client 2.15.2 (compiled from b2_15 branch).
• Severity: 3
• Assignee: WC Triage
• Reporter: Dominika Wanat

Description

Yesterday one of our Lustre clients built on top of the 2.15.2 release (from the b2_15 branch) crashed with the following LBUG:

[1361556.476660] LustreError: 882654:0:(ldlm_lock.c:1097:ldlm_grant_lock_with_skiplist()) ASSERTION( ldlm_is_granted(lock) ) failed: 
[1361556.478018] LustreError: 882654:0:(ldlm_lock.c:1097:ldlm_grant_lock_with_skiplist()) LBUG
[1361556.478681] Pid: 882654, comm: ldlm_bl_47 4.18.0-425.10.1.el8_7.x86_64 #1 SMP Thu Jan 12 16:32:13 UTC 2023
[1361556.479401] Call Trace TBD:
[1361556.479664] [<0>] libcfs_call_trace+0x6f/0xa0 [libcfs]
[1361556.480109] [<0>] lbug_with_loc+0x3f/0x70 [libcfs]
[1361556.480552] [<0>] ldlm_grant_lock_with_skiplist+0x642/0x780 [ptlrpc]
[1361556.481236] [<0>] ldlm_inodebits_drop+0xba/0x160 [ptlrpc]
[1361556.481706] [<0>] ldlm_cli_inodebits_convert+0x426/0x6c0 [ptlrpc]
[1361556.482242] [<0>] ldlm_cli_convert+0x68/0x2a0 [ptlrpc]
[1361556.482673] [<0>] ll_md_blocking_ast+0x131/0x2f0 [lustre]
[1361556.483137] [<0>] ldlm_handle_bl_callback+0xbc/0x3f0 [ptlrpc]
[1361556.483646] [<0>] ldlm_bl_thread_main+0x633/0x930 [ptlrpc]
[1361556.484103] [<0>] kthread+0x10b/0x130
[1361556.484440] [<0>] ret_from_fork+0x1f/0x40
[1361556.484767] Kernel panic - not syncing: LBUG
[1361556.485112] CPU: 9 PID: 882654 Comm: ldlm_bl_47 Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-425.10.1.el8_7.x86_64 #1
[1361556.486068] Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module_el8.3.0+555+a55c8938 04/01/2014
[1361556.486785] Call Trace:
[1361556.487006]  dump_stack+0x41/0x60
[1361556.487297]  panic+0xe7/0x2ac
[1361556.487542]  ? ret_from_fork+0x1f/0x40
[1361556.487849]  lbug_with_loc.cold.8+0x18/0x18 [libcfs]
[1361556.488254]  ldlm_grant_lock_with_skiplist+0x642/0x780 [ptlrpc]
[1361556.488751]  ldlm_inodebits_drop+0xba/0x160 [ptlrpc]
[1361556.489183]  ldlm_cli_inodebits_convert+0x426/0x6c0 [ptlrpc]
[1361556.489679]  ? ll_have_md_lock+0x169/0x3f0 [lustre]
[1361556.490087]  ldlm_cli_convert+0x68/0x2a0 [ptlrpc]
[1361556.490492]  ll_md_blocking_ast+0x131/0x2f0 [lustre]
[1361556.490906]  ? obd_stale_export_get+0x75/0x190 [obdclass]
[1361556.491438]  ldlm_handle_bl_callback+0xbc/0x3f0 [ptlrpc]
[1361556.491950]  ldlm_bl_thread_main+0x633/0x930 [ptlrpc]
[1361556.492385]  ? finish_wait+0x80/0x80
[1361556.492675]  ? ldlm_handle_bl_callback+0x3f0/0x3f0 [ptlrpc]
[1361556.493150]  kthread+0x10b/0x130
[1361556.493413]  ? set_kthread_struct+0x50/0x50
[1361556.493803]  ret_from_fork+0x1f/0x40

I found a similar LU: https://jira.whamcloud.com/browse/LU-13927 but with the Lustre 2.12.5 on MDS, patches linked in this LU are already there.

Issue Links

• is related to: LU-17278 ldlm_cli_inodebits_convert() should not grant failed locks