[LU-1704] Test failure on test suite sanity-quota test_98: MDS hung at upcall_cache_get_entry()

Mon, 29 May 2017 04:53:53 +0000

This issue was created by maloo for nasf <yong.fan@whamcloud.com>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/18ffa926-dcb2-11e1-853a-52540035b04c.

MDS log show that:

04:38:17:Lustre: Service thread pid 5092 was inactive for 40.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:
04:38:17:Pid: 5092, comm: ll_mdt_01
04:38:17:
04:38:17:Call Trace:
04:38:17: [<00000000c06245ad>] wait_for_completion+0x6b/0x8f
04:38:17: [<00000000c041f867>] default_wake_function+0x0/0xc
04:38:17: [<00000000c0433be1>] call_usermodehelper_keys+0xc3/0xcf
04:38:17: [<00000000c0433bed>] __call_usermodehelper+0x0/0x43
04:38:17: [<00000000f8e2d2de>] upcall_cache_get_entry+0x2de/0x1270 [lvfs]
04:38:17: [<00000000c041c52f>] pvclock_clocksource_read+0x101/0x117
04:38:17: [<00000000c04f4df5>] vsnprintf+0x49d/0x4db
04:38:17: [<00000000f933d7bd>] mds_init_ucred+0xbd/0x110 [mds]
04:38:17: [<00000000f92e5d67>] mds_handle+0x3387/0xa150 [mds]
04:38:17: [<00000000f9179c15>] ptlrpc_server_log_handling_request+0x135/0x1a0 [ptlrpc]
04:38:17: [<00000000f9164a3c>] lustre_msg_get_opc+0x10c/0x1f0 [ptlrpc]
04:38:17: [<00000000f917ea09>] ptlrpc_main+0x1529/0x2230 [ptlrpc]
04:38:17: [<00000000c044d834>] audit_syscall_exit+0x2d4/0x2ea
04:38:17: [<00000000f917d4e0>] ptlrpc_main+0x0/0x2230 [ptlrpc]
04:38:17: [<00000000c0405c87>] kernel_thread_helper+0x7/0x10
04:38:17: <IRQ>
04:41:26:INFO: task ll_mdt_01:5092 blocked for more than 120 seconds.
04:41:26:"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
04:41:26:ll_mdt_01 D 00001A8D 6780 5092 1 5093 5091 (L-TLB)
04:41:26: c296fd38 00000046 39e41c12 00001a8d e8e16aa0 39e2ed5a 00001a8d 00000009
04:41:26: dbaf6aa0 39e41e90 00001a8d 0000027e 00000000 dbaf6bac c200c500 c2315580
04:41:26: 00000003 c20f33d8 c200c544 c20f33d8 00000000 00000001 c296fe58 c296fdbc
04:41:27:Call Trace:
04:41:27: [<c06245ad>] wait_for_completion+0x6b/0x8f
04:41:27: [<c041f867>] default_wake_function+0x0/0xc
04:41:27: [<c0433be1>] call_usermodehelper_keys+0xc3/0xcf
04:41:27: [<c0433bed>] __call_usermodehelper+0x0/0x43
04:41:27: [<f8e2d2de>] upcall_cache_get_entry+0x2de/0x1270 [lvfs]
04:41:27: [<c041c52f>] pvclock_clocksource_read+0x101/0x117
04:41:27: [<c04f4df5>] vsnprintf+0x49d/0x4db
04:41:27: [<f933d7bd>] mds_init_ucred+0xbd/0x110 [mds]
04:41:28: [<f92e5d67>] mds_handle+0x3387/0xa150 [mds]
04:41:28: [<f9179c15>] ptlrpc_server_log_handling_request+0x135/0x1a0 [ptlrpc]
04:41:28: [<f9164a3c>] lustre_msg_get_opc+0x10c/0x1f0 [ptlrpc]
04:41:29: [<f917ea09>] ptlrpc_main+0x1529/0x2230 [ptlrpc]
04:41:29: [<c044d834>] audit_syscall_exit+0x2d4/0x2ea
04:41:29: [<f917d4e0>] ptlrpc_main+0x0/0x2230 [ptlrpc]
04:41:29: [<c0405c87>] kernel_thread_helper+0x7/0x10
04:41:29: =======================

Whamcloud Community JIRA

[LU-1704] Test failure on test suite sanity-quota test_98: MDS hung at upcall_cache_get_entry()