[LU-505] system hang when running sanity-quota on RHEL5-x86_64-OFED

Thu, 18 Aug 2011 18:42:54 +0000

It looks like ~~LU-337~~.

Lustre: DEBUG MARKER: == sanity-quota test 1: Block hard limit (normal use and out of quota) =============================== 20:12:17 (1310785937)
Lustre: DEBUG MARKER: User quota (limit: 76296 kbytes)
Lustre: DEBUG MARKER: Write ...
Lustre: DEBUG MARKER: Done
Lustre: DEBUG MARKER: Write out of block quota ...
LustreError: 11-0: an error occurred while communicating with 192.168.4.131@o2ib. The ost_write operation failed with -122
LustreError: 13718:0:(vvp_io.c:990:vvp_io_commit_write()) Write page 17172 of inode ffff8101be922610 failed -122
Lustre: DEBUG MARKER: cancel_lru_locks osc start
Lustre: DEBUG MARKER: cancel_lru_locks osc stop
LustreError: 11-0: an error occurred while communicating with 192.168.4.131@o2ib. The ost_write operation failed with -122
LustreError: 13735:0:(vvp_io.c:990:vvp_io_commit_write()) Write page 19074 of inode ffff8101be922610 failed -122
Lustre: DEBUG MARKER: --------------------------------------
Lustre: DEBUG MARKER: Group quota (limit: 76296 kbytes)
LustreError: 13963:0:(quota_ctl.c:328:client_quota_ctl()) ptlrpc_queue_wait failed, rc: -5
Lustre: DEBUG MARKER: Write ...
INFO: task pdflush:396 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
pdflush D ffff810001004420 0 396 123 397 395 (L-TLB)
ffff81033fed1b30 0000000000000046 ffff8101bddf8c40 000000000277d917
ffff81033e9c4048 000000000000000a ffff81033fc5a0c0 ffffffff80311b60
000019742ba1db4e 0000000000001bbf ffff81033fc5a2a8 000000000277d8d8
Call Trace:
[<ffffffff8006ec4e>] do_gettimeofday+0x40/0x90
[<ffffffff80028c84>] sync_page+0x0/0x43
[<ffffffff800637ca>] io_schedule+0x3f/0x67
[<ffffffff80028cc2>] sync_page+0x3e/0x43
[<ffffffff8006390e>] __wait_on_bit_lock+0x36/0x66
[<ffffffff8003ff51>] __lock_page+0x5e/0x64
[<ffffffff800a291a>] wake_bit_function+0x0/0x23
[<ffffffff800481c7>] pagevec_lookup_tag+0x1a/0x21
[<ffffffff8001d11f>] mpage_writepages+0x14f/0x37d
[<ffffffff88bf39e0>] :lustre:ll_writepage+0x0/0x460
[<ffffffff80062ff0>] thread_return+0x62/0xfe
[<ffffffff801538ac>] __next_cpu+0x19/0x28
[<ffffffff8008cf85>] find_busiest_group+0x20d/0x621
[<ffffffff8005aeb3>] do_writepages+0x20/0x2f
[<ffffffff8002fe38>] __writeback_single_inode+0x19e/0x318
[<ffffffff80021104>] sync_sb_inodes+0x1b5/0x26f
[<ffffffff800a26d4>] keventd_create_kthread+0x0/0xc4
[<ffffffff80051332>] writeback_inodes+0x82/0xd8
[<ffffffff800cbbce>] wb_kupdate+0xd4/0x14e
[<ffffffff800568c8>] pdflush+0x0/0x1fb
[<ffffffff80056a19>] pdflush+0x151/0x1fb
[<ffffffff800cbafa>] wb_kupdate+0x0/0x14e
[<ffffffff80032b26>] kthread+0xfe/0x132
[<ffffffff8009f2bb>] request_module+0x0/0x14d
[<ffffffff8005dfb1>] child_rip+0xa/0x11
[<ffffffff800a26d4>] keventd_create_kthread+0x0/0xc4
[<ffffffff80032a28>] kthread+0x0/0x132
[<ffffffff8005dfa7>] child_rip+0x0/0x11

INFO: task pdflush:396 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
pdflush D ffff810001004420 0 396 123 397 395 (L-TLB)
ffff81033fed1b30 0000000000000046 ffff8101bddf8c40 000000000277d917
ffff81033e9c4048 000000000000000a ffff81033fc5a0c0 ffffffff80311b60
000019742ba1db4e 0000000000001bbf ffff81033fc5a2a8 000000000277d8d8
Call Trace:
[<ffffffff8006ec4e>] do_gettimeofday+0x40/0x90
[<ffffffff80028c84>] sync_page+0x0/0x43
[<ffffffff800637ca>] io_schedule+0x3f/0x67
[<ffffffff80028cc2>] sync_page+0x3e/0x43
[<ffffffff8006390e>] __wait_on_bit_lock+0x36/0x66
[<ffffffff8003ff51>] __lock_page+0x5e/0x64
[<ffffffff800a291a>] wake_bit_function+0x0/0x23
[<ffffffff800481c7>] pagevec_lookup_tag+0x1a/0x21
[<ffffffff8001d11f>] mpage_writepages+0x14f/0x37d
[<ffffffff88bf39e0>] :lustre:ll_writepage+0x0/0x460
[<ffffffff80062ff0>] thread_return+0x62/0xfe
[<ffffffff801538ac>] __next_cpu+0x19/0x28
[<ffffffff8008cf85>] find_busiest_group+0x20d/0x621
[<ffffffff8005aeb3>] do_writepages+0x20/0x2f
[<ffffffff8002fe38>] __writeback_single_inode+0x19e/0x318
[<ffffffff80021104>] sync_sb_inodes+0x1b5/0x26f
[<ffffffff800a26d4>] keventd_create_kthread+0x0/0xc4
[<ffffffff80051332>] writeback_inodes+0x82/0xd8
[<ffffffff800cbbce>] wb_kupdate+0xd4/0x14e
[<ffffffff800568c8>] pdflush+0x0/0x1fb
[<ffffffff80056a19>] pdflush+0x151/0x1fb
[<ffffffff800cbafa>] wb_kupdate+0x0/0x14e
[<ffffffff80032b26>] kthread+0xfe/0x132
[<ffffffff8009f2bb>] request_module+0x0/0x14d
[<ffffffff8005dfb1>] child_rip+0xa/0x11
[<ffffffff800a26d4>] keventd_create_kthread+0x0/0xc4
[<ffffffff80032a28>] kthread+0x0/0x132
[<ffffffff8005dfa7>] child_rip+0x0/0x11

Whamcloud Community JIRA

[LU-505] system hang when running sanity-quota on RHEL5-x86_64-OFED