https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5785 Lustre <p>This issue was created by maloo for sarah &lt;sarah@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.hpdd.intel.com/test_sets/548b249c-565a-11e4-97f1-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/548b249c-565a-11e4-97f1-5254006e85c2</a>.</p> <p>The sub-test test_failover_ost failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>test_failover_ost returned 1 </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Lustre: DEBUG MARKER: ==== Checking the clients loads AFTER failover -- failure NOT OK Lustre: DEBUG MARKER: /usr/sbin/lctl mark ost5 has failed over 1 times, and counting... Lustre: DEBUG MARKER: ost5 has failed over 1 times, and counting... Lustre: lustre-OST0006: Will be in recovery for at least 1:00, or until 3 clients reconnect LustreError: 360:0:(ldlm_lib.c:1730:check_for_next_transno()) lustre-OST0006: waking for gap in transno, VBR is OFF (skip: 4295345303, ql: 2, comp: 1, conn: 3, next: 4295345312, last_committed: 4295345240) Lustre: lustre-OST0006: Recovery over after 0:22, of 3 clients 3 recovered and 0 were evicted. LustreError: 30883:0:(ldlm_lockd.c:1288:ldlm_handle_enqueue0()) ### delayed lvb init failed (rc -2) ns: filter-lustre-OST0006_UUID lock: ffff88006744d080/0x8abfa2edda4a4ccc lrc: 2/0,0 mode: --/PR res: [0xde56:0x0:0x0].0 rrc: 1 type: EXT [0-&gt;0] (req 0-&gt;0) flags: 0x40000000000000 nid: local remote: 0x7a6f60280c307e9a expref: -99 pid: 30883 timeout: 0 lvb_type: 0 LustreError: 30888:0:(ofd_io.c:600:ofd_preprw_write()) lustre-OST0006: BRW to missing obj 0x0:56912 LustreError: 30888:0:(ofd_io.c:600:ofd_preprw_write()) Skipped 1 previous similar message Lustre: DEBUG MARKER: /usr/sbin/lctl mark Duration: 86400 </pre> </div></div> client and server: lustre-master build # 2690<br/> client is SLES11 SP3 LU-5785

recovery-mds-scale test_failover_ost: test_failover_ost returned 1

Bug Minor Resolved Duplicate Mikhail Pershin Maloo Wed, 22 Oct 2014 01:53:25 +0000 Mon, 27 Apr 2015 22:07:25 +0000 Mon, 27 Apr 2015 22:07:24 +0000 Lustre 2.7.0 0 6 <p>Mike,<br/> Could you please have a look at this one and comment?<br/> Thank you!</p> <p>The console logs reports:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>23:57:55:Lustre: lustre-OST0006: deleting orphan objects from 0x0:56878 to 0x0:56897 00:19:16:Lustre: lustre-OST0006: deleting orphan objects from 0x0:57116 to 0x0:57142 </pre> </div></div> <p>so it doesn't look like orphan cleanup caused the object to be removed. Maybe there is something in the debug logs?</p> <p>I wonder if it makes sense to dump the debug log to shared storage just before the reboot to try and capture more useful information for debugging?</p> <p>It looks like this test is run several times a day, but almost all of them are failing for one reason or another. It makes sense to investigate and fix some of these failures.</p> <p>On Client 3 (onyx-45vm6):</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> tar: etc/sysconfig/network/ifroute-lo: Cannot write: No such file or directory tar: etc/sysconfig/network/routes: Cannot write: No such file or directory tar: etc/sysconfig/network/if-down.d/ndp-proxy: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors </pre> </div></div> <p>It looks like this is a duplicate of <a href="https://jira.whamcloud.com/browse/LU-4621" title="recovery-mds-scale: test_failover_ost" class="issue-link" data-issue-key="LU-4621"><del>LU-4621</del></a>.</p> <p>Hi Hongchao,<br/> Could you please confirm this? </p> Duplicate LU-4621 LU-5485 Related LU-5782 Development Rank 1|hzwz7r: Rank (Obsolete) 16237 Severity