https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5507 Lustre <p>While running sanity-quota test 18, one of the client nodes hit the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[60756.462327] BUG: unable to handle kernel NULL pointer dereference at 0000000000000007^M [60756.465418] IP: [&lt;ffffffffa088a9d1&gt;] lustre_msg_get_opc+0x1/0x100 [ptlrpc]^M [60756.466234] PGD 0 ^M [60756.466234] Oops: 0000 [#1] SMP ^M [60756.466234] CPU 0 ^M [60756.466234] Modules linked in: lustre(EN) obdecho(EN) mgc(EN) lov(EN) osc(EN) mdc(EN) lmv(EN) fid(EN) fld(EN) ptlrpc(EN) obdclass(EN) lvfs(EN) ksocklnd(EN) lnet(EN) libcfs(EN) ext2 sha512_generic sha1_generic md5 crc32c nfs lockd fscache auth_rpcgss nfs_acl sunrpc rdma_ucm rdma_cm iw_cm ib_addr ib_srp scsi_transport_srp scsi_tgt ib_ipoib ib_cm ib_uverbs ib_umad iw_cxgb3 cxgb3 mdio mlx4_en mlx4_ib ib_sa mlx4_core ib_mthca ib_mad ib_core mperf loop dm_mod floppy 8139too ipv6 ipv6_lib rtc_cmos pcspkr virtio_balloon i2c_piix4 8139cp mii button ttm drm_kms_helper drm i2c_core sysimgblt sysfillrect syscopyarea uhci_hcd ehci_hcd usbcore usb_common intel_agp intel_gtt scsi_dh_emc scsi_dh_rdac scsi_dh_alua scsi_dh_hp_sw scsi_dh virtio_pci ata_generic virtio_blk virtio virtio_ring ata_piix edd ext3 mbcache jbd fan processor ahci libahci libata scsi_mod thermal thermal_sys hwmon [last unloaded: libcfs]^M [60756.466234] Supported: No, Unsupported modules are loaded^M [60756.466234] ^M [60756.466234] Pid: 12735, comm: ptlrpcd_rcv Tainted: G EN 3.0.101-0.35-default #1 Red Hat KVM^M [60756.466234] RIP: 0010:[&lt;ffffffffa088a9d1&gt;] [&lt;ffffffffa088a9d1&gt;] lustre_msg_get_opc+0x1/0x100 [ptlrpc]^M [60756.466234] RSP: 0018:ffff880078f3dcb0 EFLAGS: 00010286^M [60756.466234] RAX: ffff8800201efa08 RBX: 0000000000000000 RCX: 0000000000000002^M [60756.466234] RDX: 0000000000000002 RSI: 0000000000000000 RDI: ffffffffffffffff^M [60756.466234] RBP: ffff88006a655500 R08: ffff8800201efa08 R09: 00000000000000d8^M [60756.466234] R10: 000000000000000a R11: 0000000000000000 R12: ffff88006295c800^M [60756.466234] R13: ffff8800201efa08 R14: ffff880079dcbee0 R15: ffff88006e9838f0^M [60756.466234] FS: 0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000^M [60756.494980] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b^M [60756.494980] CR2: 0000000000000007 CR3: 000000007ae8a000 CR4: 00000000000006f0^M [60756.494980] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000^M [60756.494980] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400^M [60756.494980] Process ptlrpcd_rcv (pid: 12735, threadinfo ffff880078f3c000, task ffff880017dde540)^M [60756.494980] Stack:^M [60756.494980] 0000000000000000 ffffffffa099d04b ffff880079dcbc00 000000c10002ee7e^M [60756.494980] ffff880079dcbc00 000000c10002ee7e ffff880079dcbc00 ffff8800290c0a88^M [60756.494980] ffff8800290c0800 ffffffffa087eb5a 00000000ebc0de01 ffff880079dcbc00^M [60756.494980] Call Trace:^M [60756.494980] [&lt;ffffffffa099d04b&gt;] mdc_replay_open+0xab/0x430 [mdc]^M [60756.494980] [&lt;ffffffffa087eb5a&gt;] ptlrpc_replay_interpret+0x14a/0x740 [ptlrpc]^M [60756.494980] [&lt;ffffffffa0880452&gt;] ptlrpc_check_set+0x532/0x1b30 [ptlrpc]^M [60756.494980] [&lt;ffffffffa08abdcb&gt;] ptlrpcd_check+0x52b/0x550 [ptlrpc]^M [60756.494980] [&lt;ffffffffa08ac32b&gt;] ptlrpcd+0x24b/0x3b0 [ptlrpc]^M [60756.494980] [&lt;ffffffff810829a6&gt;] kthread+0x96/0xa0^M [60756.494980] [&lt;ffffffff8146b164&gt;] kernel_thread_helper+0x4/0x10^M [60756.494980] Code: 89 44 24 48 48 83 c4 58 4c 89 e0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 45 31 ed e9 fb fe ff ff 66 66 66 2e 0f 1f 84 00 00 00 00 00 53 ^M [60756.494980] 7f 08 d3 0b d0 0b 48 89 fb 74 73 c7 05 49 0[ 0.000000] Initializing cgroup subsys cpuset^M [ 0.000000] Initializing cgroup subsys cpu^M </pre> </div></div> <p>Maloo report: <a href="https://testing.hpdd.intel.com/test_sets/4f4c437a-268b-11e4-84f2-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/4f4c437a-268b-11e4-84f2-5254006e85c2</a></p> Lustre Build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/80/">https://build.hpdd.intel.com/job/lustre-b2_5/80/</a><br/> Distro/Arch: SLES11SP3/x86_64 (client), RHEL6.5/x86_64 (server) LU-5507

sanity-quota test_18: Oops: IP: lustre_msg_get_opc+0xe/0x110 [ptlrpc]

Bug Blocker Resolved Fixed Niu Yawei Jian Yu Wed, 20 Aug 2014 06:38:28 +0000 Tue, 9 Jun 2015 17:17:24 +0000 Mon, 5 Jan 2015 03:50:23 +0000 Lustre 2.5.3 Lustre 2.7.0 0 7 <p>Lustre client build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/80/" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-b2_5/80/</a><br/> Lustre server build: <a href="https://build.hpdd.intel.com/job/lustre-b2_4/73/" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-b2_4/73/</a> (2.4.3)<br/> Distro/Arch: RHEL6.5/x86_64</p> <p>The same failure occurred: <a href="https://testing.hpdd.intel.com/test_sets/ea35137e-266f-11e4-8ee8-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/ea35137e-266f-11e4-8ee8-5254006e85c2</a></p> <p>So far, the failure has not occurred in Lustre b2_5 build #82 and #83.</p> <p>Lustre Build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/86/" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-b2_5/86/</a> (2.5.3 RC1)</p> <p>The same failure occurred: <a href="https://testing.hpdd.intel.com/test_sets/651d9592-30da-11e4-b503-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/651d9592-30da-11e4-b503-5254006e85c2</a></p> <p>This seems to occur sometimes. Any idea why?</p> <p>Seems it's a race of close vs. open replay, that's introduced in the fix of <a href="https://jira.whamcloud.com/browse/LU-2613" title="opening and closing file can generate &#39;unreclaimable slab&#39; space" class="issue-link" data-issue-key="LU-2613"><del>LU-2613</del></a> (4322e0f9): To free the queued open &amp; close requests promptly, we free them on file close, however, replay open may jump in at this time to fix the stale open handle on the open &amp; close requests. I'm going to post a patch soon.</p> <p><a href="http://review.whamcloud.com/12667" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/12667</a></p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/12667/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/12667/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5507" title="sanity-quota test_18: Oops: IP: lustre_msg_get_opc+0xe/0x110 [ptlrpc]" class="issue-link" data-issue-key="LU-5507"><del>LU-5507</del></a> recovery: don't replay closed open<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: cfbfcc6ad9ebb5893be2d1e85fc959794fd914ed</p> <p>patch landed on master.</p> Related LU-5169 Development Rank 1|hzwu2v: Rank (Obsolete) 15364 Severity