[LU-5579] MDS crashed by "mdt_check_resent_lock()) ASSERTION( lock != NULL ) failed"

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.7.0, Lustre 2.8.0, Lustre 2.5.4
• Labels: HB llnl patch
• Severity: 3
• Assignee: Liang Zhen (Inactive)
• Reporter: Liang Zhen (Inactive)

Description

After I enabled message delay on routers, MDS crashed quite soon...

<0>LustreError: 13914:0:(mdt_handler.c:2333:mdt_check_resent_lock()) ASSERTION( lock != NULL ) failed: Invalid lock handle 0x80235817e79ffcd
<0>LustreError: 13914:0:(mdt_handler.c:2333:mdt_check_resent_lock()) LBUG
<4>Pid: 13914, comm: mdt00_009
<4>
<4>Call Trace:
<4> [<ffffffffa0706895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
<4> [<ffffffffa0706e97>] lbug_with_loc+0x47/0xb0 [libcfs]
<4> [<ffffffffa1107111>] mdt_check_resent_lock+0x1b1/0x1f0 [mdt]
<4> [<ffffffffa111c22d>] mdt_getattr_name_lock+0x51d/0x1a50 [mdt]
<4> [<ffffffffa111dc82>] mdt_intent_getattr+0x292/0x470 [mdt]
<4> [<ffffffffa110b879>] mdt_intent_policy+0x499/0xca0 [mdt]
<4> [<ffffffffa0a64549>] ldlm_lock_enqueue+0x129/0x9d0 [ptlrpc]
<4> [<ffffffffa0a9048b>] ldlm_handle_enqueue0+0x51b/0x13a0 [ptlrpc]
<4> [<ffffffffa07074ce>] ? cfs_timer_arm+0xe/0x10 [libcfs]
<4> [<ffffffffa0b11d12>] tgt_enqueue+0x62/0x1d0 [ptlrpc]
<4> [<ffffffffa0b1259e>] tgt_request_handle+0x71e/0xb10 [ptlrpc]
<4> [<ffffffffa0ac15c4>] ptlrpc_main+0xe64/0x1990 [ptlrpc]
<4> [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
<4> [<ffffffff810623a9>] ? find_busiest_queue+0x69/0x150
<4> [<ffffffff815294ce>] ? thread_return+0x4e/0x760
<4> [<ffffffffa0ac0760>] ? ptlrpc_main+0x0/0x1990 [ptlrpc]
<4> [<ffffffff8109abf6>] kthread+0x96/0xa0
<4> [<ffffffff8100c20a>] child_rip+0xa/0x20
<4> [<ffffffff8109ab60>] ? kthread+0x0/0xa0
<4> [<ffffffff8100c200>] ? child_rip+0x0/0x20
<4>
<0>Kernel panic - not syncing: LBUG
<4>Pid: 13914, comm: mdt00_009 Tainted: P           ---------------    2.6.32-431.23.3.el6_lustre.x86_64 #1
<4>Call Trace:
<4> [<ffffffff81528dbc>] ? panic+0xa7/0x16f
<4> [<ffffffffa0706eeb>] ? lbug_with_loc+0x9b/0xb0 [libcfs]
<4> [<ffffffffa1107111>] ? mdt_check_resent_lock+0x1b1/0x1f0 [mdt]
<4> [<ffffffffa111c22d>] ? mdt_getattr_name_lock+0x51d/0x1a50 [mdt]
<4> [<ffffffffa111dc82>] ? mdt_intent_getattr+0x292/0x470 [mdt]
<4> [<ffffffffa110b879>] ? mdt_intent_policy+0x499/0xca0 [mdt]
<4> [<ffffffffa0a64549>] ? ldlm_lock_enqueue+0x129/0x9d0 [ptlrpc]
<4> [<ffffffffa0a9048b>] ? ldlm_handle_enqueue0+0x51b/0x13a0 [ptlrpc]
<4> [<ffffffffa07074ce>] ? cfs_timer_arm+0xe/0x10 [libcfs]
<4> [<ffffffffa0b11d12>] ? tgt_enqueue+0x62/0x1d0 [ptlrpc]
<4> [<ffffffffa0b1259e>] ? tgt_request_handle+0x71e/0xb10 [ptlrpc]
<4> [<ffffffffa0ac15c4>] ? ptlrpc_main+0xe64/0x1990 [ptlrpc]
<4> [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
<4> [<ffffffff810623a9>] ? find_busiest_queue+0x69/0x150
<4> [<ffffffff815294ce>] ? thread_return+0x4e/0x760
<4> [<ffffffffa0ac0760>] ? ptlrpc_main+0x0/0x1990 [ptlrpc]
<4> [<ffffffff8109abf6>] ? kthread+0x96/0xa0
<4> [<ffffffff8100c20a>] ? child_rip+0xa/0x20
<4> [<ffffffff8109ab60>] ? kthread+0x0/0xa0
<4> [<ffffffff8100c200>] ? child_rip+0x0/0x20

Issue Links

• is blocked by: LU-5604 Lots of FAIL_ID checking are lost
• is related to: LU-5630 mdt_getattr_name_lock()) ASSERTION( lock != NULL )