[LU-4950] sanity-benchmark test fsx hung: txg_sync was stuck on OSS

• Type: Bug
• Resolution: Duplicate
• Priority: Critical
• Affects Version/s: Lustre 2.6.0, Lustre 2.5.1, Lustre 2.7.0, Lustre 2.4.3, Lustre 2.5.3, Lustre 2.8.0, Lustre 2.10.0
• Labels: zfs
• Environment: Lustre build: http://build.whamcloud.com/job/lustre-b2_5/47/; FSTYPE=zfs
• Severity: 3
• Assignee: Alex Zhuravlev
• Reporter: Jian Yu

Description

While running sanity-benchmark test fsx with the following configuration, it timed out in 3600s.

MDSCOUNT=1
MDSSIZE=2097152
OSTCOUNT=2
OSTSIZE=8388608
FSTYPE=zfs
SLOW=yes

txg_sync was stuck on OSS node:

txg_sync      D 0000000000000001     0  2665      2 0x00000080
 ffff88006f4cbba0 0000000000000046 00000000ffffffff 000037007418a66d
 ffff88006f4cbb10 ffff88007cba1a60 0000000000bba084 ffffffffaca9e91e
 ffff880070bc05f8 ffff88006f4cbfd8 000000000000fbc8 ffff880070bc05f8
Call Trace:
 [<ffffffff810a7091>] ? ktime_get_ts+0xb1/0xf0
 [<ffffffff81528823>] io_schedule+0x73/0xc0
 [<ffffffffa0142e7c>] cv_wait_common+0x8c/0x100 [spl]
 [<ffffffff8109b290>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa0142f08>] __cv_wait_io+0x18/0x20 [spl]
 [<ffffffffa02864ab>] zio_wait+0xfb/0x1b0 [zfs]
 [<ffffffffa021e5a5>] dsl_pool_sync+0x2f5/0x540 [zfs]
 [<ffffffffa023725e>] spa_sync+0x40e/0xa00 [zfs]
 [<ffffffff8103ea79>] ? kvm_clock_get_cycles+0x9/0x10
 [<ffffffffa024253a>] txg_sync_thread+0x27a/0x4b0 [zfs]
 [<ffffffff81059329>] ? set_user_nice+0xc9/0x130
 [<ffffffffa02422c0>] ? txg_sync_thread+0x0/0x4b0 [zfs]
 [<ffffffffa013e9bf>] thread_generic_wrapper+0x5f/0x70 [spl]
 [<ffffffffa013e960>] ? thread_generic_wrapper+0x0/0x70 [spl]
 [<ffffffff8109aee6>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8109ae50>] ? kthread+0x0/0xa0
 [<ffffffff8100c200>] ? child_rip+0x0/0x20

Maloo report: https://maloo.whamcloud.com/test_sets/7ec85f26-cb6e-11e3-95c9-52540035b04c

Issue Links

• duplicates: LU-5575 Failure on test suite replay-ost-single test_5
• is duplicated by: LU-6812 sanity-quota test_7a: test failed to respond and timed out
• is duplicated by: LU-7374 obdfilter-survey test_1a: test failed to respond and timed out
• is related to: LU-12234 sanity-benchmark test iozone hangs in txg_sync
• is related to: LU-9247 replay-ost-single test_5: test failed to respond and timed out
• is related to: LU-5575 Failure on test suite replay-ost-single test_5
• is related to: LU-5242 Test hang sanity test_132, test_133: umount ost
• is related to: LU-2887 sanity-quota test_12a: slow due to ZFS VMs sharing single disk
• is related to: LU-5214 Failure on test suite replay-ost-single test_5
• is related to: LU-6105 Update ZFS/SPL version to 0.6.3-1.2
• links to: ZFS Issue 1640