Details
-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
None
-
None
-
3
-
9223372036854775807
Description
This issue was created by maloo for wangshilong <wshilong@ddn.com>
This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/30cc74a4-5923-11e9-8e92-52540065bddc
test_90c failed with the following error:
Timeout occurred after 379 mins, last suite running was conf-sanity, restarting cluster to continue tests
Looks ZFS transaction blocked which hang on OST umount
19270.120645] Pid: 1384, comm: ll_ost00_006 3.10.0-957.1.3.el7_lustre.x86_64 #1 SMP Mon Mar 25 21:14:10 UTC 2019 [19270.122330] Call Trace: [19270.122837] [<ffffffffc042f2d5>] cv_wait_common+0x125/0x150 [spl] [19270.123988] [<ffffffffc042f315>] __cv_wait+0x15/0x20 [spl] [19270.124988] [<ffffffffc05a62bf>] txg_wait_synced+0xef/0x140 [zfs] [19270.126335] [<ffffffffc11d39db>] osd_trans_stop+0x53b/0x5e0 [osd_zfs] [19270.127563] [<ffffffffc132ed45>] ofd_trans_stop+0x25/0x60 [ofd] [19270.128692] [<ffffffffc1333335>] ofd_destroy+0x2c5/0x960 [ofd] [19270.129759] [<ffffffffc132b5a4>] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [19270.130919] [<ffffffffc1321677>] ofd_destroy_hdl+0x267/0x970 [ofd] [19270.132027] [<ffffffffc0fed06a>] tgt_request_handle+0x91a/0x15c0 [ptlrpc] [19270.133698] [<ffffffffc0f906ae>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [19270.135099] [<ffffffffc0f941ac>] ptlrpc_main+0xbbc/0x2090 [ptlrpc] [19270.136246] [<ffffffff9d4c1c31>] kthread+0xd1/0xe0 [19270.137169] [<ffffffff9db74c37>] ret_from_fork_nospec_end+0x0/0x39 [19270.138304] [<ffffffffffffffff>] 0xffffffffffffffff [19270.139282] Pid: 1381, comm: ll_ost00_005 3.10.0-957.1.3.el7_lustre.x86_64 #1 SMP Mon Mar 25 21:14:10 UTC 2019 [19270.141084] Call Trace: [19270.141612] [<ffffffffc042f2d5>] cv_wait_common+0x125/0x150 [spl] [19270.143092] [<ffffffffc042f315>] __cv_wait+0x15/0x20 [spl] [19270.144111] [<ffffffffc05a62bf>] txg_wait_synced+0xef/0x140 [zfs] [19270.145275] [<ffffffffc11d39db>] osd_trans_stop+0x53b/0x5e0 [osd_zfs] [19270.146454] [<ffffffffc132ed45>] ofd_trans_stop+0x25/0x60 [ofd] [19270.147545] [<ffffffffc1333335>] ofd_destroy+0x2c5/0x960 [ofd] [19270.148616] [<ffffffffc132b5a4>] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [19270.149771] [<ffffffffc1321677>] ofd_destroy_hdl+0x267/0x970 [ofd] [19270.150910] [<ffffffffc0fed06a>] tgt_request_handle+0x91a/0x15c0 [ptlrpc] [19270.152179] [<ffffffffc0f906ae>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [19270.153593] [<ffffffffc0f941ac>] ptlrpc_main+0xbbc/0x2090 [ptlrpc] [19270.154744] [<ffffffff9d4c1c31>] kthread+0xd1/0xe0 [19270.155633] [<ffffffff9db74c37>] ret_from_fork_nospec_end+0x0/0x39 [19270.156767] [<ffffffffffffffff>] 0xffffffffffffffff [19270.157685] Pid: 1393, comm: ll_ost00_009 3.10.0-957.1.3.el7_lustre.x86_64 #1 SMP Mon Mar 25 21:14:10 UTC 2019 [19270.159379] Call Trace: [19270.159829] [<ffffffffc042f2d5>] cv_wait_common+0x125/0x150 [spl] [19270.160932] [<ffffffffc042f315>] __cv_wait+0x15/0x20 [spl] [19270.161933] [<ffffffffc05a62bf>] txg_wait_synced+0xef/0x140 [zfs] [19270.163056] [<ffffffffc11d39db>] osd_trans_stop+0x53b/0x5e0 [osd_zfs] [19270.164205] [<ffffffffc132ed45>] ofd_trans_stop+0x25/0x60 [ofd] [19270.165278] [<ffffffffc1333335>] ofd_destroy+0x2c5/0x960 [ofd] [19270.166331] [<ffffffffc132b5a4>] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [19270.167492] [<ffffffffc1321677>] ofd_destroy_hdl+0x267/0x970 [ofd] [19270.168600] [<ffffffffc0fed06a>] tgt_request_handle+0x91a/0x15c0 [ptlrpc] [19270.169846] [<ffffffffc0f906ae>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [19270.171223] [<ffffffffc0f941ac>] ptlrpc_main+0xbbc/0x2090 [ptlrpc] [19270.172365] [<ffffffff9d4c1c31>] kthread+0xd1/0xe0 [19270.173243] [<ffffffff9db74c37>] ret_from_fork_nospec_end+0x0/0x39 [19270.174349] [<ffffffffffffffff>] 0xffffffffffffffff ....
VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
conf-sanity test_90c - Timeout occurred after 379 mins, last suite running was conf-sanity, restarting cluster to continue tests