Whamcloud Community JIRA

Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us

9.4.14

940014

05-12-2023

[LU-4134] obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). https://jira.whamcloud.com/browse/LU-4134 Lustre <p>using ioctl api - isn't safe as we lack a protect to use name2obd / uuid2obd / num2obd calls result during shutdown.</p> <p>Xyratex bugs: MRP-509, MRP-1396</p> lustre 2.1/2.5 on any OS. LU-4134

obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property).

Bug Critical Resolved Fixed Yang Sheng Alexey Lyashkov patch Tue, 22 Oct 2013 18:05:45 +0000 Thu, 4 Jan 2018 18:26:13 +0000 Mon, 11 Dec 2017 08:39:46 +0000 Lustre 2.1.0 Lustre 2.5.0 Lustre 2.11.0 Lustre 2.10.3 0 10 <p>patch tries to make shutdown / setup obd device make lots clear to fix various bugs in lustre code.</p> <p><a href="http://review.whamcloud.com/8045" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8045</a></p> <p>Alexey, the patch needs a rebase, are you able to re-submit? Please see the Gerrit comments</p> <p>Cliff, </p> <p>I was busy with <a href="https://jira.whamcloud.com/browse/LU-4495" title="client evicted on parallel append write to the shared file." class="issue-link" data-issue-key="LU-4495"><del>LU-4495</del></a>, and will refresh it quickly. Xyratex inspection found a few bugs which fixed in new version also.</p> <p>Alexey, there are a few issues with the patch in our last tests, would a refresh be possible? Please see the comments on macros in Gerrit. </p> <p>Needs rebasing to make any progress.</p> <p>need fix a gerrit to ability to login via google.</p> <p>Alexey I rebased your patch against the latest master. Please let me know if it is correct.</p> <p>P.S<br/> Not the greatest thing but I moved to yahoo mail to continue my gerrit work.</p> <p>I updated the patch, all tests are passed.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/8045/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/8045/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: obd_device improvement<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 45900a7777ac02130d8bf65724c4b6cffca9d546</p> <p>Landed for 2.11</p> <p>Minh Diep (minh.diep@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29740" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29740</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: obd_device improvement<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: c82f9686fbe5d5a9998e2738efdc5fc4761eed7c</p> <p>This patch will cause a double free in failure path as below:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[23427.124766] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == conf-sanity test 41c: concurrent mounts of MDT\/OST should all fail but one ======================== 16:20:02 \(1509664802\) [23427.175328] Lustre: DEBUG MARKER: == conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one ======================== 16:20:02 (1509664802) [23427.320400] Lustre: DEBUG MARKER: grep -c /mnt/lustre-ost1' ' /proc/mounts [23427.356134] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' [23433.423356] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm7: executing set_default_debug -1 all 4 [23433.459190] Lustre: DEBUG MARKER: trevis-33vm7: executing set_default_debug -1 all 4 [23434.318418] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm3: executing set_default_debug -1 all 4 [23434.356057] Lustre: DEBUG MARKER: trevis-33vm3: executing set_default_debug -1 all 4 [23435.195522] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm7: executing set_default_debug -1 all 4 [23435.238307] Lustre: DEBUG MARKER: trevis-33vm7: executing set_default_debug -1 all 4 [23436.043367] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm3: executing set_default_debug -1 all 4 [23436.078104] Lustre: DEBUG MARKER: trevis-33vm3: executing set_default_debug -1 all 4 [23436.305850] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost1 [23436.354028] Lustre: DEBUG MARKER: test -b /dev/lvm-Role_OSS/P1 [23436.392706] Lustre: DEBUG MARKER: e2label /dev/lvm-Role_OSS/P1 [23436.427055] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre-ost1; mount -t lustre /dev/lvm-Role_OSS/P1 /mnt/lustre-ost1 [23436.493237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro,no_mbcache,nodelalloc [23436.602541] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n health_check [23436.637692] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests//usr/lib64/lustre/tests:/usr/lib64/lustre/tests:/usr/lib64/lustre/tests/../utils:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lust [23436.877956] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm8: executing set_default_debug -1 all 4 [23436.915153] Lustre: DEBUG MARKER: trevis-33vm8: executing set_default_debug -1 all 4 [23436.944609] Lustre: DEBUG MARKER: e2label /dev/lvm-Role_OSS/P1 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' [23436.978188] Lustre: DEBUG MARKER: e2label /dev/lvm-Role_OSS/P1 2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}' [23437.019478] Lustre: DEBUG MARKER: e2label /dev/lvm-Role_OSS/P1 2>/dev/null [23439.851173] Lustre: lustre-OST0000: deleting orphan objects from 0x0:3 to 0x0:33 [23443.391660] Lustre: DEBUG MARKER: grep -c /mnt/lustre-ost1' ' /proc/mounts [23443.424213] Lustre: DEBUG MARKER: umount -f /mnt/lustre-ost1 [23449.552602] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' [23473.647115] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests//usr/lib64/lustre/tests:/usr/lib64/lustre/tests:/usr/lib64/lustre/tests/../utils:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lust [23473.914217] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm2: executing load_modules_local [23473.920620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm7: executing load_modules_local [23473.927710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm8: executing load_modules_local [23473.936093] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm3: executing load_modules_local [23473.966215] Lustre: DEBUG MARKER: trevis-33vm2: executing load_modules_local [23473.973080] Lustre: DEBUG MARKER: trevis-33vm7: executing load_modules_local [23473.996666] Lustre: DEBUG MARKER: trevis-33vm8: executing load_modules_local [23474.007128] Lustre: DEBUG MARKER: trevis-33vm3: executing load_modules_local [23475.560023] Lustre: 9927:0:(gss_svc_upcall.c:1186:gss_init_svc_upcall()) Init channel is not opened by lsvcgssd, following request might be dropped until lsvcgssd is active [23475.560034] Lustre: 9927:0:(gss_mech_switch.c:71:lgss_mech_register()) Register gssnull mechanism [23475.560039] Key type lgssc registered [23475.624355] Lustre: Echo OBD driver; http://www.lustre.org/ [23476.635995] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm3: executing set_default_debug -1 all 4 [23476.669582] Lustre: DEBUG MARKER: trevis-33vm3: executing set_default_debug -1 all 4 [23477.478297] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm7: executing set_default_debug -1 all 4 [23477.513178] Lustre: DEBUG MARKER: trevis-33vm7: executing set_default_debug -1 all 4 [23478.326013] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm3: executing set_default_debug -1 all 4 [23478.362859] Lustre: DEBUG MARKER: trevis-33vm3: executing set_default_debug -1 all 4 [23478.582138] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests//usr/lib64/lustre/tests:/usr/lib64/lustre/tests:/usr/lib64/lustre/tests/../utils:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lust [23478.862393] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-33vm8: executing lsmod [23478.898933] Lustre: DEBUG MARKER: trevis-33vm8: executing lsmod [23478.942337] Lustre: DEBUG MARKER: test -b /dev/lvm-Role_OSS/P1 [23478.974285] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param fail_loc=0x80000716 [23479.008341] Lustre: DEBUG MARKER: mount -t lustre /dev/lvm-Role_OSS/P1 /mnt/lustre-ost1 [23479.010416] Lustre: DEBUG MARKER: mount -t lustre /dev/lvm-Role_OSS/P1 /mnt/lustre-ost1 [23479.061423] LustreError: 10335:0:(libcfs_fail.h:165:cfs_race()) cfs_race id 716 sleeping [23479.063837] LustreError: 10338:0:(libcfs_fail.h:170:cfs_race()) cfs_fail_race id 716 waking [23479.063860] LustreError: 10335:0:(libcfs_fail.h:168:cfs_race()) cfs_fail_race id 716 awake, rc=0 [23479.063923] LustreError: 10335:0:(genops.c:489:class_register_device()) lustre-OST0000-osd: already exists, won't add [23479.063932] LustreError: 10335:0:(genops.c:415:class_free_dev()) ASSERTION( obd->obd_magic == OBD_DEVICE_MAGIC ) failed: ffff8800793ae9c8 obd_magic 5a5a5a5a != ab5cd6ef [23479.075186] LustreError: 10335:0:(genops.c:415:class_free_dev()) LBUG [23479.075187] Pid: 10335, comm: mount.lustre [23479.075187] [23479.075187][ 0.075856] ioremap error for 0x7ffff000-0x80000000, requested 0x2, got 0x0 [ 0.076057] dmi: Firmware registration failed. </pre> </div></div> <p>Yang Sheng (yang.sheng@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29967" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29967</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: fix double free in failure path<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 4fbe59cd035b7656645a9bd2a98a8b215390dd8f</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/29967/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29967/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: fix double free in failure path<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 77d2e604b6bc3c319adefca069b5182c325a43b0</p> <p>Patch landed to 2.11.0. Close this ticket.</p> <p>Minh Diep (minh.diep@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30487" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30487</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: fix double free in failure path<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 30db03bdae598353ca803b8f0342258167ebc9b7</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/29740/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29740/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: obd_device improvement<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: 09debb54bcf6d04ca9ff257fa721260f80ce8138</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/30487/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30487/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4134" title="obdfilter-suvery bugs and panics (ioctl API isn't protected over shutdown/setup property). " class="issue-link" data-issue-key="LU-4134"><del>LU-4134</del></a> obdclass: fix double free in failure path<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: 2c2f42ccd55601b410e35f76d89b2691f11250b1</p> Duplicate Related LU-10209 Development Rank 1|hzw6lr: Rank (Obsolete) 11207 Severity