Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 [LU-7098] sanity test_17m: test failed to respond and timed out https://jira.whamcloud.com/browse/LU-7098 Lustre <p>This issue was created by maloo for Bob Glossman &lt;bob.glossman@intel.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.hpdd.intel.com/test_sets/1d53141c-519f-11e5-84c4-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/1d53141c-519f-11e5-84c4-5254006e85c2</a>.</p> <p>The sub-test test_17m failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>test failed to respond and timed out </pre> </div></div> <p>seen in test of sles12 client/server on master</p> <p>Info required for matching: sanity 17m</p> LU-7098 sanity test_17m: test failed to respond and timed out Bug Minor Resolved Fixed Yang Sheng Maloo Thu, 3 Sep 2015 14:39:31 +0000 Fri, 8 Jul 2016 22:56:22 +0000 Mon, 30 Nov 2015 17:41:36 +0000 Lustre 2.8.0 0 4 <p>another seen on master, also sles12 client/server:<br/> <a href="https://testing.hpdd.intel.com/test_sets/b3dea53e-5a5c-11e5-825b-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/b3dea53e-5a5c-11e5-825b-5254006e85c2</a></p> <p>this may be a blocker for landing sles12 on master. whatever the problem is it seems to be master onlly, sles12 test on other branches have passed.</p> <p>another instance seen in sles12 client/server test on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/878101f8-62bd-11e5-a45a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/878101f8-62bd-11e5-a45a-5254006e85c2</a></p> <p>from console log for mds2</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>03:57:07:onyx-44vm7 login: [ 4264.945248] BUG: unable to handle kernel paging request at ffffc9800335e000 03:57:07:[ 4264.947111] IP: [&lt;ffffffff8151853a&gt;] _raw_spin_lock+0xa/0x30 03:57:07:[ 4264.948315] PGD 0 03:57:07:[ 4264.948761] Oops: 0002 [#1] SMP 03:57:08:[ 4264.949076] Modules linked in: osp(OEN) mdd(OEN) lod(OEN) mdt(OEN) lfsck(OEN) mgc(OEN) osd_ldiskfs(OEN) lquota(OEN) fid(OEN) fld(OEN) ksocklnd(OEN) ptlrpc(OEN) obdclass(OEN) lnet(OEN) sha512_generic(E) crypto_null(E) libcfs(OEN) ldiskfs(OEN) rpcsec_gss_krb5(E) auth_rpcgss(E) nfsv4(E) dns_resolver(E) nfs(E) lockd(E) sunrpc(E) fscache(E) iscsi_boot_sysfs(E) ib_ipoib(E) rdma_ucm(E) ib_ucm(E) ib_uverbs(E) ib_umad(E) af_packet(E) rdma_cm(E) ib_cm(E) iw_cm(E) ib_sa(E) ib_mad(E) ib_core(E) ib_addr(E) ppdev(E) parport_pc(E) pvpanic(E) serio_raw(E) parport(E) pcspkr(E) virtio_balloon(E) 8139too(E) 8139cp(E) mii(E) button(E) processor(E) i2c_piix4(E) dm_mod(E) ext4(E) crc16(E) mbcache(E) jbd2(E) ata_generic(E) ata_piix(E) ahci(E) libahci(E) virtio_blk(E) floppy(E) uhci_hcd(E) ehci_hcd(E) cirrus(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) drm_kms_helper(E) usbcore(E) usb_common(E) ttm(E) drm(E) virtio_pci(E) virtio_ring(E) virtio(E) libata(E) sg(E) scsi_mod(E) autofs4(E) 03:57:08:[ 4264.949076] Supported: No, Unsupported modules are loaded 03:57:08:[ 4264.949076] CPU: 1 PID: 2567 Comm: mdt00_002 Tainted: G OEN 3.12.44-52.10_lustre.gb2a3954-default #1 03:57:08:[ 4264.949076] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007 03:57:08:[ 4264.949076] task: ffff88007acaa040 ti: ffff88007acb8000 task.ti: ffff88007acb8000 03:57:09:[ 4264.949076] RIP: 0010:[&lt;ffffffff8151853a&gt;] [&lt;ffffffff8151853a&gt;] _raw_spin_lock+0xa/0x30 03:57:09:[ 4264.949076] RSP: 0018:ffff88007acb9980 EFLAGS: 00010246 03:57:09:[ 4264.949076] RAX: 0000000000010000 RBX: ffff88006bc77400 RCX: 0000000000000007 03:57:09:[ 4264.949076] RDX: ffffc9800335e000 RSI: 0000000000000000 RDI: ffffc9800335e000 03:57:09:[ 4264.949076] RBP: ffff88007b7f3b40 R08: 00000000000000ec R09: 00000000000000ec 03:57:09:[ 4264.949076] R10: 0000000000000025 R11: 000000000000000e R12: ffff88007b08b140 03:57:09:[ 4264.949076] R13: 0000000000000000 R14: 000000000000000d R15: 0000000000000001 03:57:09:[ 4264.949076] FS: 0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000 03:57:09:[ 4264.949076] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b 03:57:09:[ 4264.949076] CR2: ffffc9800335e000 CR3: 0000000077c3d000 CR4: 00000000000006e0 03:57:09:[ 4264.949076] Stack: 03:57:09:[ 4264.949076] ffffffffa09e45e7 ffff880036e4d490 ffffc9800335e000 ffffffff00001886 03:57:09:[ 4264.949076] ffff88007b7f3b40 ffff88007acb9a48 ffff88006bc77400 0000000000000000 03:57:10:[ 4264.949076] 000000000000000d ffff88007b7f3b40 ffffffffa09dd170 0000000000000000 03:57:10:[ 4264.949076] Call Trace: 03:57:10:[ 4264.949076] [&lt;ffffffffa09e45e7&gt;] ldlm_resource_get+0x67/0xa30 [ptlrpc] 03:57:10:[ 4264.949076] [&lt;ffffffffa09dd170&gt;] ldlm_lock_create+0x60/0xb30 [ptlrpc] 03:57:10:[ 4264.949076] [&lt;ffffffffa09f9f2e&gt;] ldlm_cli_enqueue_local+0xce/0x950 [ptlrpc] 03:57:10:[ 4264.949076] [&lt;ffffffffa0df972a&gt;] mdt_object_local_lock+0x1ea/0xad0 [mdt] 03:57:10:[ 4264.949076] [&lt;ffffffffa0dfacb1&gt;] mdt_getattr_name_lock+0x9f1/0x18a0 [mdt] 03:57:10:[ 4264.949076] [&lt;ffffffffa0dfbdef&gt;] mdt_intent_getattr+0x28f/0x440 [mdt] 03:57:11:[ 4264.949076] [&lt;ffffffffa0dfef2c&gt;] mdt_intent_policy+0x59c/0xb50 [mdt] 03:57:11:[ 4264.949076] [&lt;ffffffffa09ddf63&gt;] ldlm_lock_enqueue+0x323/0x890 [ptlrpc] 03:57:11:[ 4264.949076] [&lt;ffffffffa0a06361&gt;] ldlm_handle_enqueue0+0x741/0x1870 [ptlrpc] 03:57:11:[ 4264.949076] [&lt;ffffffffa0a897fd&gt;] tgt_enqueue+0x5d/0x210 [ptlrpc] 03:57:11:[ 4264.949076] [&lt;ffffffffa0a8dd33&gt;] tgt_request_handle+0x7e3/0x1190 [ptlrpc] 03:58:35:[ 4264.949076] [&lt;ffffffffa0a37aa9&gt;] ptlrpc_server_handle_request+0x209/0xa70 [ptlrpc] 03:58:36:[ 4264.949076] [&lt;ffffffffa0a3b1ba&gt;] ptlrpc_main+0xb2a/0x1ea0 [ptlrpc] 03:58:37:[ 4264.949076] [&lt;ffffffff810770f4&gt;] kthread+0xb4/0xc0 03:58:37:[ 4264.949076] [&lt;ffffffff81520618&gt;] ret_from_fork+0x58/0x90 03:58:37:[ 4264.949076] Code: fa 66 0f 1f 44 00 00 48 83 c7 04 f0 ff 0f 74 05 e8 fc 28 d9 ff 48 89 d0 c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 00 00 01 00 &lt;f0&gt; 0f c1 07 89 c2 c1 ea 10 66 39 c2 89 d1 75 01 c3 0f b7 07 66 03:58:38:[ 4264.949076] RIP [&lt;ffffffff8151853a&gt;] _raw_spin_lock+0xa/0x30 03:58:38:[ 4264.949076] RSP &lt;ffff88007acb9980&gt; 03:58:38:[ 4264.949076] CR2: ffffc9800335e000 03:58:38:[ 0.004005] Failed to access perfctr msr (MSR c1 is 0) 03:58:38:[ 1.334430] systemd[1]: /usr/lib/systemd/system-generators/kdump-device-timeout-generator exited with exit status 2. 03:58:39:[ 4.691663] irq 11: nobody cared (try booting with the "irqpoll" option) 03:58:39:[ 4.692007] handlers: 03:58:39:[ 4.692007] [&lt;ffffffffa00caf80&gt;] usb_hcd_irq [usbcore] 03:58:39:[ 4.692007] Disabling IRQ #11 03:58:39:Unable to ioctl(KDSETLED) -- are you not on the console? (Inappropriate ioctl for device) 03:58:39:Deletion of old dump only on local disk. 03:58:39:Extracting dmesg 03:58:40:------------------------------------------------------------------------------- 03:58:40: 03:58:41:The dmesg log is saved to /mnt/2015-09-23-20:57/dmesg.txt. 03:58:42: 03:58:42:makedumpfile Completed. 03:58:42:------------------------------------------------------------------------------- 03:58:42:Saving dump using makedumpfile 03:58:42:------------------------------------------------------------------------------- 03:58:42: Excluding unnecessary pages : [ 0.0 %] / Excluding unnecessary pages : [100.0 %] | Excluding unnecessary pages : [100.0 %] \ Excluding unnecessary pages : [ 0.0 %] - Excluding unnecessary pages : [100.0 %] /[ 7.347488] Out of memory: Kill process 77 (haveged) score 34 or sacrifice child 03:58:42:[ 7.348136] Killed process 77 (haveged) total-vm:12032kB, anon-rss:3124kB, file-rss:652kB 05:17:27:********** Timeout by autotest system ********** </pre> </div></div> <p>Yang Sheng (yang.sheng@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/16804" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/16804</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7098" title="sanity test_17m: test failed to respond and timed out" class="issue-link" data-issue-key="LU-7098"><del>LU-7098</del></a> osd-ldiskfs: don't alloc inode directly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 79a01fd4c0308fc3e242bed7e0d4efe647cd523e</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/16804/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/16804/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7098" title="sanity test_17m: test failed to respond and timed out" class="issue-link" data-issue-key="LU-7098"><del>LU-7098</del></a> osd-ldiskfs: don't alloc inode directly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 323293bab2c8d65c6f1c0b3c04671ed073719cbe</p> <p>Landed for 2.8</p> Duplicate Related LU-7305 Development Rank 1|hzxmdz: Rank (Obsolete) 9223372036854775807 Severity