[LU-5196] HSM: client task stuck waiting for mutex in ll_layout_refresh

• Type: Bug
• Resolution: Duplicate
• Priority: Critical
• Affects Version/s: Lustre 2.5.1
• Labels: hsm
• Severity: 3
• Assignee: WC Triage
• Reporter: Patrick Farrell

Description

Internal Cray bug report (from Frank Zago):
After trying to migrate a file back on Lustre, from the TAS backend, I noticed nothing was happening. I ctrl-C the TAS copytool (CMM), and it doesn't exit.

The kernel has the following trace:

Lustre: Layout lock feature supported.
Lustre: Mounted tas01-client
INFO: task tas_cmm:18139 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
tas_cmm D 0000000000000000 0 18139 14601 0x00000080
ffff8801052c5a58 0000000000000082 0000000000000000 0000000000000000
ffff880106ef8580 0000000000000000 ffff8801313c70b8 ffff8801389e8800
ffff88007e070638 ffff8801052c5fd8 000000000000fbc8 ffff88007e070638
Call Trace:
[<ffffffff8152935e>] __mutex_lock_slowpath+0x13e/0x180
[<ffffffff815291fb>] mutex_lock+0x2b/0x50
[<ffffffffa09dc04c>] ll_layout_refresh+0x25c/0xfe0 [lustre]
[<ffffffffa067bb1a>] ? ldlm_lock_add_to_lru_nolock+0x4a/0x110 [ptlrpc]
[<ffffffffa03830d7>] ? cfs_hash_bd_lookup_intent+0x37/0x130 [libcfs]
[<ffffffffa0a01040>] ? ll_md_blocking_ast+0x0/0x7d0 [lustre]
[<ffffffffa0694550>] ? ldlm_completion_ast+0x0/0x920 [ptlrpc]
[<ffffffffa04db721>] ? cl_io_slice_add+0xc1/0x190 [obdclass]
[<ffffffffa0a28240>] vvp_io_init+0x340/0x490 [lustre]
[<ffffffffa04dab68>] cl_io_init0+0x98/0x160 [obdclass]
[<ffffffffa04ce995>] ? cl_env_get+0x195/0x350 [obdclass]
[<ffffffffa04dd794>] cl_io_init+0x64/0xe0 [obdclass]
[<ffffffffa0a1f751>] cl_glimpse_size0+0x91/0x1d0 [lustre]
[<ffffffffa09d0855>] ll_inode_revalidate_it+0x1a5/0x1d0 [lustre]
[<ffffffff81196666>] ? final_putname+0x26/0x50
[<ffffffffa09d08c9>] ll_getattr_it+0x49/0x170 [lustre]
[<ffffffffa09d0a27>] ll_getattr+0x37/0x40 [lustre]
[<ffffffff81227163>] ? security_inode_getattr+0x23/0x30
[<ffffffff8118e631>] vfs_getattr+0x51/0x80
[<ffffffff8118e6c4>] vfs_fstatat+0x64/0xa0
[<ffffffff8118e82b>] vfs_stat+0x1b/0x20
[<ffffffff8118e854>] sys_newstat+0x24/0x50
[<ffffffff810e1cc7>] ? audit_syscall_entry+0x1d7/0x200
[<ffffffff810e1abe>] ? __audit_syscall_exit+0x25e/0x290
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b

Issue Links

• duplicates: LU-4727 Lhsmtool_posix process stuck in ll_layout_refresh() when restoring