https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10525 Lustre <p>racer test_1 - test_1 failed with 2<br/> ^^^^^^^^^^^^^ DO NOT REMOVE LINE ABOVE ^^^^^^^^^^^^^</p> <p>This issue was created by maloo for sarah_lw &lt;wei3.liu@intel.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.hpdd.intel.com/test_sets/60989662-f935-11e7-a7cd-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/60989662-f935-11e7-a7cd-52540065bddc</a></p> <p>test_1 failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>test_1 failed with 2 </pre> </div></div> <p>server: 2.10.3 RC1 EL7<br/> client: 2.10.3 RC1 EL6.9</p> <p>test log</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000-mdc-ffff880041847c00: Connection restored to 10.9.5.195@tcp (at 10.9.5.195@tcp) Lustre: Skipped 3 previous similar messages LustreError: 16503:0:(lcommon_cl.c:181:cl_file_inode_init()) Failure to initialize cl object [0x200000401:0x2366:0x0]: -16 LustreError: 32509:0:(lcommon_cl.c:181:cl_file_inode_init()) Failure to initialize cl object [0x200000401:0x2486:0x0]: -16 INFO: task dir_create.sh:12824 blocked for more than 120 seconds. Tainted: G W -- ------------ 2.6.32-696.18.7.el6.x86_64 #1 "echo 0 &gt; /proc/sys/kernel/hung_task_timeout_secs" disables this message. dir_create.sh D 0000000000000000 0 12824 12691 0x00000080 ffff88007a03fb68 0000000000000086 ffff88007a03fba8 0000000000000080 0000000000000000 ffff880059833149 0000004b00000000 ffffffffa103b393 0000000000000098 0020000000000080 ffff88002c1a1068 ffff88007a03ffd8 Call Trace: [&lt;ffffffff8154d146&gt;] __mutex_lock_slowpath+0x96/0x210 [&lt;ffffffff811b5f68&gt;] ? __d_lookup+0xd8/0x150 [&lt;ffffffff8154cc6b&gt;] mutex_lock+0x2b/0x50 [&lt;ffffffff811aa63b&gt;] do_lookup+0x11b/0x230 [&lt;ffffffff811aace0&gt;] __link_path_walk+0x200/0x1060 [&lt;ffffffff81342f5c&gt;] ? memory_open+0x3c/0xa0 [&lt;ffffffff811abdfa&gt;] path_walk+0x6a/0xe0 [&lt;ffffffff811ad5da&gt;] do_filp_open+0x1fa/0xd20 [&lt;ffffffff8115aeca&gt;] ? handle_mm_fault+0x2aa/0x3f0 [&lt;ffffffff812a97fa&gt;] ? strncpy_from_user+0x4a/0x90 [&lt;ffffffff811bac52&gt;] ? alloc_fd+0x92/0x160 [&lt;ffffffff81197507&gt;] do_sys_open+0x67/0x130 [&lt;ffffffff8155660b&gt;] ? system_call_after_swapgs+0x16b/0x220 [&lt;ffffffff81556604&gt;] ? system_call_after_swapgs+0x164/0x220 [&lt;ffffffff815565fd&gt;] ? system_call_after_swapgs+0x15d/0x220 [&lt;ffffffff81197610&gt;] sys_open+0x20/0x30 [&lt;ffffffff815566d6&gt;] system_call_fastpath+0x16/0x1b [&lt;ffffffff8155656a&gt;] ? system_call_after_swapgs+0xca/0x220 INFO: task ls:18508 blocked for more than 120 seconds. </pre> </div></div> LU-10525

racer test_1: Failure to initialize cl object

Bug Minor Open Unresolved WC Triage Maloo Tue, 16 Jan 2018 21:41:52 +0000 Tue, 14 Aug 2018 21:49:08 +0000 Lustre 2.10.3 Lustre 2.10.4 Lustre 2.10.5 0 6 <p>the message quoted here is just a symptom on the client that something is hogging MDS resources so the client cannot send more than one request and that one request is stuck in processing on mds.</p> <p>I looked at hte logs on mds and it seems to be somewhat confirmed there with all the "cannot add more time, not sending early replies" messages, but no definite point at what is stuck where and since it was not a crash, we did not collect any crashdumps either, I guess. As such we don't really know all that much about what was going on here that lead to MDS being stuck.</p> <p>+1 on b2_10 <a href="https://testing.hpdd.intel.com/test_sets/03bf84be-5c0e-11e8-b9d3-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/03bf84be-5c0e-11e8-b9d3-52540065bddc</a></p> <p>racer test 1 times out with dir_create, dd, mv, lfs, etc. hung is a D state for servers and client el7 with logs at <a href="https://testing.whamcloud.com/test_sets/a3be8a94-9c02-11e8-a9f7-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/a3be8a94-9c02-11e8-a9f7-52540065bddc</a>. We don't see the 'initialize cl object' error in this hang.</p> Related LU-6758 Development Rank 1|hzzr7r: Rank (Obsolete) 9223372036854775807 Severity