https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12253 Lustre <p>current master has this problem in sanity-pfl (sanity-pfl test 6: Migrate composite file)</p> <p>I suspect one of the recent lfs landings is the culprit?</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>run_suite sanity-pfl /home/green/git/lustre-release/lustre/tests/sanity-pfl.sh *** Error in `/home/green/git/lustre-release/lustre/utils/lfs': double free or corruption (!prev): 0x0000000001b21990 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x81489)[0x7f19b4458489] /home/green/git/lustre-release/lustre/utils/lfs[0x42003a] /home/green/git/lustre-release/lustre/utils/lfs[0x419af7] /home/green/git/lustre-release/lustre/utils/lfs[0x41a2d6] /home/green/git/lustre-release/lustre/utils/lfs[0x40f033] /home/green/git/lustre-release/lustre/utils/lfs[0x429e91] /home/green/git/lustre-release/lustre/utils/lfs[0x4031fe] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f19b43f93d5] /home/green/git/lustre-release/lustre/utils/lfs[0x4032fa] ======= Memory map: ======== 00400000-00442000 r-xp 00000000 fd:00 1786 /home/green/git/lustre-release/lustre/utils/lfs 00641000-00642000 r--p 00041000 fd:00 1786 /home/green/git/lustre-release/lustre/utils/lfs 00642000-00643000 rw-p 00042000 fd:00 1786 /home/green/git/lustre-release/lustre/utils/lfs 00643000-0064c000 rw-p 00000000 00:00 0 01b07000-01b28000 rw-p 00000000 00:00 0 [heap] 7f19afdea000-7f19afdff000 r-xp 00000000 00:25 1049992 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f19afdff000-7f19afffe000 ---p 00015000 00:25 1049992 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f19afffe000-7f19affff000 r--p 00014000 00:25 1049992 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f19affff000-7f19b0000000 rw-p 00015000 00:25 1049992 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f19b0000000-7f19b0021000 rw-p 00000000 00:00 0 7f19b0021000-7f19b4000000 ---p 00000000 00:00 0 7f19b41d3000-7f19b41d5000 r-xp 00000000 00:25 1051404 /usr/lib64/libdl-2.17.so 7f19b41d5000-7f19b43d5000 ---p 00002000 00:25 1051404 /usr/lib64/libdl-2.17.so 7f19b43d5000-7f19b43d6000 r--p 00002000 00:25 1051404 /usr/lib64/libdl-2.17.so 7f19b43d6000-7f19b43d7000 rw-p 00003000 00:25 1051404 /usr/lib64/libdl-2.17.so 7f19b43d7000-7f19b4599000 r-xp 00000000 00:25 1051741 /usr/lib64/libc-2.17.so 7f19b4599000-7f19b4799000 ---p 001c2000 00:25 1051741 /usr/lib64/libc-2.17.so 7f19b4799000-7f19b479d000 r--p 001c2000 00:25 1051741 /usr/lib64/libc-2.17.so 7f19b479d000-7f19b479f000 rw-p 001c6000 00:25 1051741 /usr/lib64/libc-2.17.so 7f19b479f000-7f19b47a4000 rw-p 00000000 00:00 0 7f19b47a4000-7f19b47a7000 r-xp 00000000 00:25 1050461 /usr/lib64/libkeyutils.so.1.5 7f19b47a7000-7f19b49a6000 ---p 00003000 00:25 1050461 /usr/lib64/libkeyutils.so.1.5 7f19b49a6000-7f19b49a7000 r--p 00002000 00:25 1050461 /usr/lib64/libkeyutils.so.1.5 7f19b49a7000-7f19b49a8000 rw-p 00003000 00:25 1050461 /usr/lib64/libkeyutils.so.1.5 7f19b49a8000-7f19b4aa9000 r-xp 00000000 00:25 1054169 /usr/lib64/libm-2.17.so 7f19b4aa9000-7f19b4ca8000 ---p 00101000 00:25 1054169 /usr/lib64/libm-2.17.so 7f19b4ca8000-7f19b4ca9000 r--p 00100000 00:25 1054169 /usr/lib64/libm-2.17.so 7f19b4ca9000-7f19b4caa000 rw-p 00101000 00:25 1054169 /usr/lib64/libm-2.17.so 7f19b4caa000-7f19b4cc9000 r-xp 00000000 00:25 1050150 /usr/lib64/libyaml-0.so.2.0.4 7f19b4cc9000-7f19b4ec8000 ---p 0001f000 00:25 1050150 /usr/lib64/libyaml-0.so.2.0.4 7f19b4ec8000-7f19b4ec9000 r--p 0001e000 00:25 1050150 /usr/lib64/libyaml-0.so.2.0.4 7f19b4ec9000-7f19b4eca000 rw-p 0001f000 00:25 1050150 /usr/lib64/libyaml-0.so.2.0.4 7f19b4eca000-7f19b4edf000 r-xp 00000000 00:25 1050148 /usr/lib64/libz.so.1.2.7 7f19b4edf000-7f19b50de000 ---p 00015000 00:25 1050148 /usr/lib64/libz.so.1.2.7 7f19b50de000-7f19b50df000 r--p 00014000 00:25 1050148 /usr/lib64/libz.so.1.2.7 7f19b50df000-7f19b50e0000 rw-p 00015000 00:25 1050148 /usr/lib64/libz.so.1.2.7 7f19b50e0000-7f19b5102000 r-xp 00000000 00:25 1053941 /usr/lib64/ld-2.17.so 7f19b52f1000-7f19b52f6000 rw-p 00000000 00:00 0 7f19b52fe000-7f19b5301000 rw-p 00000000 00:00 0 7f19b5301000-7f19b5302000 r--p 00021000 00:25 1053941 /usr/lib64/ld-2.17.so 7f19b5302000-7f19b5303000 rw-p 00022000 00:25 1053941 /usr/lib64/ld-2.17.so 7f19b5303000-7f19b5304000 rw-p 00000000 00:00 0 7ffcf6288000-7ffcf62aa000 rw-p 00000000 00:00 0 [stack] 7ffcf6346000-7ffcf6348000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] </pre> </div></div> LU-12253

lfs: double free or corruption (!prev)

Bug Blocker Resolved Fixed WC Triage Oleg Drokin Tue, 30 Apr 2019 16:56:30 +0000 Mon, 6 May 2019 18:11:53 +0000 Tue, 30 Apr 2019 18:01:32 +0000 Lustre 2.13.0 Lustre 2.13.0 0 4 <p>I see this issue in sanity-pfl test 6, but it doesn't cause the test to fail. Looking at the suite_log at <a href="https://testing.whamcloud.com/test_sets/aff2a216-6b3f-11e9-bd0e-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/aff2a216-6b3f-11e9-bd0e-52540065bddc</a> , </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== sanity-pfl test 6: Migrate composite file ========================================================= 09:14:18 (1556615658) striped dir -i2 -c2 /mnt/lustre/d6.sanity-pfl *** Error in `/usr/bin/lfs': double free or corruption (!prev): 0x0000000001548990 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x81489)[0x7faf4e47a489] /lib64/liblustreapi.so.1(+0xff2a)[0x7faf4f57af2a] /lib64/liblustreapi.so.1(+0x99e7)[0x7faf4f5749e7] /lib64/liblustreapi.so.1(+0xa1c6)[0x7faf4f5751c6] /usr/bin/lfs[0x40ffb3] /lib64/liblustreapi.so.1(Parser_execarg+0x51)[0x7faf4f585961] /usr/bin/lfs[0x4041fe] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7faf4e41b3d5] /usr/bin/lfs[0x40426c] ======= Memory map: ======== 00400000-00425000 r-xp 00000000 fd:01 145944 /usr/bin/lfs 00624000-00625000 r--p 00024000 fd:01 145944 /usr/bin/lfs 00625000-00626000 rw-p 00025000 fd:01 145944 /usr/bin/lfs 0152e000-0154f000 rw-p 00000000 00:00 0 [heap] 7faf48000000-7faf48021000 rw-p 00000000 00:00 0 7faf48021000-7faf4c000000 ---p 00000000 00:00 0 7faf4ddb5000-7faf4ddca000 r-xp 00000000 fd:01 131092 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7faf4ddca000-7faf4dfc9000 ---p 00015000 fd:01 131092 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7faf4dfc9000-7faf4dfca000 r--p 00014000 fd:01 131092 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7faf4dfca000-7faf4dfcb000 rw-p 00015000 fd:01 131092 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7faf4dfcb000-7faf4dfcd000 r-xp 00000000 fd:01 136292 /usr/lib64/libdl-2.17.so 7faf4dfcd000-7faf4e1cd000 ---p 00002000 fd:01 136292 /usr/lib64/libdl-2.17.so 7faf4e1cd000-7faf4e1ce000 r--p 00002000 fd:01 136292 /usr/lib64/libdl-2.17.so 7faf4e1ce000-7faf4e1cf000 rw-p 00003000 fd:01 136292 /usr/lib64/libdl-2.17.so 7faf4e1cf000-7faf4e1f4000 r-xp 00000000 fd:01 136651 /usr/lib64/libtinfo.so.5.9 7faf4e1f4000-7faf4e3f4000 ---p 00025000 fd:01 136651 /usr/lib64/libtinfo.so.5.9 7faf4e3f4000-7faf4e3f8000 r--p 00025000 fd:01 136651 /usr/lib64/libtinfo.so.5.9 7faf4e3f8000-7faf4e3f9000 rw-p 00029000 fd:01 136651 /usr/lib64/libtinfo.so.5.9 7faf4e3f9000-7faf4e5bb000 r-xp 00000000 fd:01 136286 /usr/lib64/libc-2.17.so 7faf4e5bb000-7faf4e7bb000 ---p 001c2000 fd:01 136286 /usr/lib64/libc-2.17.so 7faf4e7bb000-7faf4e7bf000 r--p 001c2000 fd:01 136286 /usr/lib64/libc-2.17.so 7faf4e7bf000-7faf4e7c1000 rw-p 001c6000 fd:01 136286 /usr/lib64/libc-2.17.so 7faf4e7c1000-7faf4e7c6000 rw-p 00000000 00:00 0 7faf4e7c6000-7faf4e7c9000 r-xp 00000000 fd:01 136945 /usr/lib64/libkeyutils.so.1.5 7faf4e7c9000-7faf4e9c8000 ---p 00003000 fd:01 136945 /usr/lib64/libkeyutils.so.1.5 7faf4e9c8000-7faf4e9c9000 r--p 00002000 fd:01 136945 /usr/lib64/libkeyutils.so.1.5 7faf4e9c9000-7faf4e9ca000 rw-p 00003000 fd:01 136945 /usr/lib64/libkeyutils.so.1.5 7faf4e9ca000-7faf4ea06000 r-xp 00000000 fd:01 137326 /usr/lib64/libreadline.so.6.2 7faf4ea06000-7faf4ec06000 ---p 0003c000 fd:01 137326 /usr/lib64/libreadline.so.6.2 7faf4ec06000-7faf4ec08000 r--p 0003c000 fd:01 137326 /usr/lib64/libreadline.so.6.2 7faf4ec08000-7faf4ec0e000 rw-p 0003e000 fd:01 137326 /usr/lib64/libreadline.so.6.2 7faf4ec0e000-7faf4ec10000 rw-p 00000000 00:00 0 7faf4ec10000-7faf4ed11000 r-xp 00000000 fd:01 136294 /usr/lib64/libm-2.17.so 7faf4ed11000-7faf4ef10000 ---p 00101000 fd:01 136294 /usr/lib64/libm-2.17.so 7faf4ef10000-7faf4ef11000 r--p 00100000 fd:01 136294 /usr/lib64/libm-2.17.so 7faf4ef11000-7faf4ef12000 rw-p 00101000 fd:01 136294 /usr/lib64/libm-2.17.so 7faf4ef12000-7faf4ef31000 r-xp 00000000 fd:01 141229 /usr/lib64/libyaml-0.so.2.0.4 7faf4ef31000-7faf4f130000 ---p 0001f000 fd:01 141229 /usr/lib64/libyaml-0.so.2.0.4 7faf4f130000-7faf4f131000 r--p 0001e000 fd:01 141229 /usr/lib64/libyaml-0.so.2.0.4 7faf4f131000-7faf4f132000 rw-p 0001f000 fd:01 141229 /usr/lib64/libyaml-0.so.2.0.4 7faf4f132000-7faf4f14b000 r-xp 00000000 fd:01 145979 /usr/lib64/liblnetconfig.so.4.0.0 7faf4f14b000-7faf4f34b000 ---p 00019000 fd:01 145979 /usr/lib64/liblnetconfig.so.4.0.0 7faf4f34b000-7faf4f34c000 r--p 00019000 fd:01 145979 /usr/lib64/liblnetconfig.so.4.0.0 7faf4f34c000-7faf4f34d000 rw-p 0001a000 fd:01 145979 /usr/lib64/liblnetconfig.so.4.0.0 7faf4f34d000-7faf4f355000 rw-p 00000000 00:00 0 7faf4f355000-7faf4f36a000 r-xp 00000000 fd:01 136604 /usr/lib64/libz.so.1.2.7 7faf4f36a000-7faf4f569000 ---p 00015000 fd:01 136604 /usr/lib64/libz.so.1.2.7 7faf4f569000-7faf4f56a000 r--p 00014000 fd:01 136604 /usr/lib64/libz.so.1.2.7 7faf4f56a000-7faf4f56b000 rw-p 00015000 fd:01 136604 /usr/lib64/libz.so.1.2.7 7faf4f56b000-7faf4f58e000 r-xp 00000000 fd:01 145983 /usr/lib64/liblustreapi.so.1.0.0 7faf4f58e000-7faf4f78e000 ---p 00023000 fd:01 145983 /usr/lib64/liblustreapi.so.1.0.0 7faf4f78e000-7faf4f78f000 r--p 00023000 fd:01 145983 /usr/lib64/liblustreapi.so.1.0.0 7faf4f78f000-7faf4f790000 rw-p 00024000 fd:01 145983 /usr/lib64/liblustreapi.so.1.0.0 7faf4f790000-7faf4f799000 rw-p 00000000 00:00 0 7faf4f799000-7faf4f7bb000 r-xp 00000000 fd:01 136279 /usr/lib64/ld-2.17.so 7faf4f9a8000-7faf4f9af000 rw-p 00000000 00:00 0 7faf4f9b7000-7faf4f9ba000 rw-p 00000000 00:00 0 7faf4f9ba000-7faf4f9bb000 r--p 00021000 fd:01 136279 /usr/lib64/ld-2.17.so 7faf4f9bb000-7faf4f9bc000 rw-p 00022000 fd:01 136279 /usr/lib64/ld-2.17.so 7faf4f9bc000-7faf4f9bd000 rw-p 00000000 00:00 0 7fffce35b000-7fffce37e000 rw-p 00000000 00:00 0 [stack] 7fffce3e3000-7fffce3e5000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 5+0 records in </pre> </div></div> <p>yes, it does not cause the test to fail but use after free like that in the tools is still a problem.</p> <p>I seem to get it go away by reverting <a href="https://jira.whamcloud.com/browse/LU-11367" title="integrate LSOM with lfs find" class="issue-link" data-issue-key="LU-11367"><del>LU-11367</del></a></p> <p>Reverted <a href="https://jira.whamcloud.com/browse/LU-11367" title="integrate LSOM with lfs find" class="issue-link" data-issue-key="LU-11367"><del>LU-11367</del></a> from master to make this go away for now.</p> Related LU-11367 Development Rank 1|i00fnr: Rank (Obsolete) 9223372036854775807 Severity