https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7602 Lustre <p>This bug created to track activity from <a href="http://review.whamcloud.com/17712" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/17712</a><br/> <a href="https://jira.whamcloud.com/browse/LU-7602" title="Repeated timeouts with ZFS 0.6.5.2" class="issue-link" data-issue-key="LU-7602"><del>LU-7602</del></a> zfs: reset ZFS baseline to 0.6.4.2</p> <p>ZFS 0.6.5.2 is known to introduce I/O problems<br/> Typical timeout - slightly different from the stack traces in the Gerrit ticket</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">Dec 23 11:47:33 iws2 kernel: LNet: Service thread pid 30734 was inactive <span class="code-keyword">for</span> 200.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace <span class="code-keyword">for</span> debugging purposes: Dec 23 11:47:33 iws2 kernel: Pid: 30734, comm: ll_ost00_000 Dec 23 11:47:33 iws2 kernel: Dec 23 11:47:33 iws2 kernel: Call Trace: Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa06cb330&gt;] ? vdev_mirror_child_done+0x0/0x30 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffff815395d3&gt;] io_schedule+0x73/0xc0 Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa05a3eaf&gt;] cv_wait_common+0xaf/0x130 [spl] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffff810a1460&gt;] ? autoremove_wake_function+0x0/0x40 Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa05a3f48&gt;] __cv_wait_io+0x18/0x20 [spl] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa070c29b&gt;] zio_wait+0x10b/0x1e0 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa06638a9&gt;] dbuf_read+0x439/0x850 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa066c168&gt;] dmu_buf_hold+0x68/0x90 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa0661fa8&gt;] ? dbuf_rele_and_unlock+0x268/0x390 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa06d5e0a&gt;] zap_lockdir+0x5a/0x770 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa06d797a&gt;] zap_lookup_norm+0x4a/0x190 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa06d7b53&gt;] zap_lookup+0x33/0x40 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa067bbe6&gt;] dmu_tx_hold_zap+0x146/0x210 [zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa107b3b5&gt;] osd_declare_object_create+0x2d5/0x440 [osd_zfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa11bba24&gt;] ofd_precreate_objects+0x4e4/0x19d0 [ofd] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa04bc6c1&gt;] ? libcfs_debug_msg+0x41/0x50 [libcfs] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa11c8bdb&gt;] ? ofd_grant_create+0x23b/0x3e0 [ofd] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa11ab83e&gt;] ofd_create_hdl+0x56e/0x2640 [ofd] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa0bbefe0&gt;] ? lustre_pack_reply_v2+0x220/0x280 [ptlrpc] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa0c294cc&gt;] tgt_request_handle+0x8ec/0x1470 [ptlrpc] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa0bd0b41&gt;] ptlrpc_main+0xe41/0x1910 [ptlrpc] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffffa0bcfd00&gt;] ? ptlrpc_main+0x0/0x1910 [ptlrpc] Dec 23 11:47:33 iws2 kernel: [&lt;ffffffff810a0fce&gt;] kthread+0x9e/0xc0 Dec 23 11:47:33 iws2 kernel: [&lt;ffffffff8100c28a&gt;] child_rip+0xa/0x20 Dec 23 11:47:33 iws2 kernel: [&lt;ffffffff810a0f30&gt;] ? kthread+0x0/0xc0 Dec 23 11:47:33 iws2 kernel: [&lt;ffffffff8100c280&gt;] ? child_rip+0x0/0x20 Dec 23 11:47:33 iws2 kernel: </pre> </div></div> Hyperion/SWL - LU-7602

Repeated timeouts with ZFS 0.6.5.2

Bug Blocker Resolved Duplicate Jian Yu Cliff White Wed, 23 Dec 2015 19:54:55 +0000 Wed, 23 Dec 2015 21:30:26 +0000 Wed, 23 Dec 2015 21:28:27 +0000 Lustre 2.8.0 0 6 <p>Hi Cliff,</p> <p>Patch <a href="http://review.whamcloud.com/17712" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/17712</a> hit build failure on sles11sp2 server. I created TEI-4369 to disable the build.</p> <p>In the meantime, since builds on other distros passed, could you please verify whether the timeout issue is resolved or not after resetting ZFS baseline to 0.6.4.2? Thank you.</p> <p>Cliff, do you have the stack traces for all the threads on the OSS? It seems this <tt>ll_ost00_000</tt> thread is waiting for the ZFS TXG to commit, but it would be useful to know what the other threads are doing in the meantime.</p> <p>Closing this as a duplicate of <a href="https://jira.whamcloud.com/browse/LU-7404" title="ZFS OSS - Numerous timeouts - SWL" class="issue-link" data-issue-key="LU-7404"><del>LU-7404</del></a> since that already has more information in it. The stack trace shown here is from the OSS, which is blocked on the OST object precreate. The stack trace shown in the 17712 ticket is the timeout on the MDS caused by waiting for new OST object precreation to complete, which is only a symptom of the actual deadlock problem on the OSS. </p> <p>I dumped the stacks on iws2. It's a while since the error, this file includes all the timeout stacks</p> Duplicate LU-7404 Development Rank 1|hzxwpr: Rank (Obsolete) 9223372036854775807 Severity