https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16954 Lustre <p>This issue was created by maloo for emoly &lt;emoly@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/6fe16807-4d7e-4087-9ad5-7a68e355230f" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/6fe16807-4d7e-4087-9ad5-7a68e355230f</a></p> <p>test_64f failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[ 3230.932454] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock onyx-53vm4@tcp:/lustre /mnt/lustre [ 3230.946766] sysfs: cannot create duplicate filename '/devices/virtual/bdi/lustre-ffff8dd549f3d000' [ 3230.949612] CPU: 0 PID: 251880 Comm: mount.lustre Kdump: loaded Tainted: G OE 5.15.0-52-generic #58~20.04.1-Ubuntu [ 3230.952966] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 3230.954694] Call Trace: [ 3230.955792] &lt;TASK&gt; [ 3230.956840] dump_stack_lvl+0x4a/0x63 [ 3230.958180] dump_stack+0x10/0x16 [ 3230.959417] sysfs_warn_dup.cold+0x17/0x2b [ 3230.960781] sysfs_create_dir_ns+0xbc/0xd0 [ 3230.962169] kobject_add_internal+0xba/0x2c0 [ 3230.963546] kobject_add+0x7e/0xb0 [ 3230.964737] ? mutex_lock+0x13/0x40 [ 3230.965939] device_add+0x125/0x8f0 [ 3230.967123] device_create_groups_vargs+0xd8/0x100 [ 3230.968520] device_create+0x49/0x70 [ 3230.969687] bdi_register_va.part.0+0x3f/0x210 [ 3230.971062] bdi_register_va+0x1f/0x30 [ 3230.972250] super_setup_bdi_name+0x79/0xe0 [ 3230.973549] ll_fill_super+0xc85/0x1b30 [lustre] [ 3230.975069] ? lustre_start_mgc+0xf56/0x1a10 [obdclass] [ 3230.976781] ? lustre_start_mgc+0xf56/0x1a10 [obdclass] [ 3230.978273] lustre_fill_super+0xeb/0x490 [lustre] [ 3230.979672] ? ll_alloc_inode+0x180/0x180 [lustre] [ 3230.981068] mount_nodev+0x49/0xa0 [ 3230.982172] lustre_mount+0x18/0x20 [lustre] [ 3230.983474] legacy_get_tree+0x2b/0x50 [ 3230.984653] vfs_get_tree+0x2a/0xc0 [ 3230.985759] ? capable+0x19/0x20 [ 3230.986837] path_mount+0x461/0xa70 [ 3230.987916] ? putname+0x57/0x70 [ 3230.988920] do_mount+0x80/0xa0 [ 3230.989889] __x64_sys_mount+0x8b/0xe0 [ 3230.990958] do_syscall_64+0x5c/0xc0 [ 3230.991983] ? syscall_exit_to_user_mode+0x27/0x50 [ 3230.993258] ? __x64_sys_read+0x1a/0x20 [ 3230.994348] ? do_syscall_64+0x69/0xc0 [ 3230.995445] entry_SYSCALL_64_after_hwframe+0x61/0xcb [ 3230.996774] RIP: 0033:0x7fd233293eae [ 3230.997849] Code: 48 8b 0d 85 1f 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 &lt;48&gt; 3d 01 f0 ff ff 73 01 c3 48 8b 0d 52 1f 0f 00 f7 d8 64 89 01 48 [ 3231.002521] RSP: 002b:00007fffe15baf68 EFLAGS: 00000286 ORIG_RAX: 00000000000000a5 [ 3231.004799] RAX: ffffffffffffffda RBX: 00007fffe15be348 RCX: 00007fd233293eae [ 3231.006635] RDX: 000056144e7a10b2 RSI: 00007fffe15bafc8 RDI: 00005614504bf500 [ 3231.008459] RBP: 00005614504c0ad0 R08: 00005614504c0ad0 R09: 0000000000000000 [ 3231.010300] R10: 0000000001000000 R11: 0000000000000286 R12: 00000000fffffff5 [ 3231.012165] R13: 00007fffe15bafc8 R14: 0000000000000000 R15: 000056144e7a10b2 [ 3231.014018] &lt;/TASK&gt; [ 3231.014949] kobject_add_internal failed for lustre-ffff8dd549f3d000 with -EEXIST, don't try to register things with the same name in the same directory. [ 3231.020139] LustreError: 251880:0:(super25.c:187:lustre_fill_super()) llite: Unable to mount &lt;unknown&gt;: rc = -17 [ 3231.353241] Lustre: DEBUG MARKER: /usr/sbin/lctl mark sanity test_64f: @@@@@@ FAIL: mount failed [ 3231.627375] Lustre: DEBUG MARKER: sanity test_64f: @@@@@@ FAIL: mount failed [ 3231.947200] Lustre: DEBUG MARKER: /usr/sbin/lctl dk &gt; /autotest/autotest-1/2023-07-10/e2fsprogs-reviews_review-e2fsprogs-part-2_1303_8_8e1e4e70-b5a1-4584-bdbe-d637b9866068//sanity.test_64f.debug_log.$(hostname -s).1688990136.log; </pre> </div></div> <p>Test session details:<br/> clients: <a href="https://build.whamcloud.com/job/lustre-master/4442" class="external-link" target="_blank" rel="nofollow noopener">https://build.whamcloud.com/job/lustre-master/4442</a> - 5.15.0-52-generic<br/> servers: <a href="https://build.whamcloud.com/job/lustre-master/4442" class="external-link" target="_blank" rel="nofollow noopener">https://build.whamcloud.com/job/lustre-master/4442</a> - 4.18.0-425.10.1.el8_lustre.x86_64</p> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> sanity test_64f - mount failed</p> LU-16954

mount failed: File exists(cannot create duplicate filename '/devices/virtual/bdi/lustre-ffffxxx')

Bug Minor Resolved Fixed Qian Yingjin Maloo Tue, 11 Jul 2023 01:49:18 +0000 Mon, 2 Oct 2023 01:01:44 +0000 Thu, 28 Sep 2023 14:55:12 +0000 Lustre 2.16.0 0 9 <p>+1 <a href="https://testing.whamcloud.com/test_logs/2a6125b0-ef39-4420-9986-68406193286e/show_text" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_logs/2a6125b0-ef39-4420-9986-68406193286e/show_text</a></p> <p><del>"Emoly Liu &lt;emoly@whamcloud.com&gt;" uploaded a new patch:</del> <a href="https://review.whamcloud.com/c/tools/e2fsprogs/+/51627" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/tools/e2fsprogs/+/51627</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> test: fortestonly</del><br/> <del>Project: tools/e2fsprogs</del><br/> <del>Branch: master-lustre</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: 2ccd59ad60c8dc4e1c037201dca25a4d6b10d9c8</del></p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=dongyang" class="user-hover" rel="dongyang">dongyang</a> , could you please have a look at this issue? My test above showed this issue was caused by  the (latest) e2fsprogs master-lustre branch. </p> <p><del>"Emoly Liu &lt;emoly@whamcloud.com&gt;" uploaded a new patch:</del> <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51693" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51693</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> test: fortestonly to verify commit f5a75ea</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: 9998b59ba73195d3b6fc7146807216c9a38b77bc</del></p> <p>"Li Dongyang &lt;dongyangli@ddn.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51701" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51701</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> Revert "<a href="https://jira.whamcloud.com/browse/LU-16697" title="Lustre should set appropriate BDI_CAP_* and s_iflags for writeback and cgroup wb" class="issue-link" data-issue-key="LU-16697"><del>LU-16697</del></a> llite: Set BDI_CAP_* flags for lustre"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 77d98384ac0bb5d9b5d1076c62c13069c648ecfd</p> <p>The Maloo testing results above showed that this issue was caused by the patch of <a href="https://jira.whamcloud.com/browse/LU-16697" title="Lustre should set appropriate BDI_CAP_* and s_iflags for writeback and cgroup wb" class="issue-link" data-issue-key="LU-16697"><del>LU-16697</del></a> at <a href="https://review.whamcloud.com/#/c/fs/lustre-release/+/50497/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/fs/lustre-release/+/50497/</a> (commit f5a75ea44db32ac27ada327b4752c3bc611cf9df).</p> <p>I can not reproduce this bug locally...</p> <p><del>"Qian Yingjin &lt;qian@ddn.com&gt;" uploaded a new patch:</del> <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51752" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51752</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> llite: debug patch for duplicate filename</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: 360264582ae7c06aad7ac8e19fcffbf6c698c68c</del></p> <p><del>"Qian Yingjin &lt;qian@ddn.com&gt;" uploaded a new patch:</del> <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51755" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51755</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> test: fortestonly for duplicate filename failure</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: 4280e036f9edadfbbe411b3903e6449d609ef59c</del></p> <p>It seems to be related to the kernel patch: fs: explicitly unregister per-superblock BDIs.<br/> See <a href="https://lore.kernel.org/all/20211021124441.668816-4-hch@lst.de/T/#u" class="external-link" target="_blank" rel="nofollow noopener">https://lore.kernel.org/all/20211021124441.668816-4-hch@lst.de/T/#u</a> for more details: <a href="https://lkml.kernel.org/lkml/20220601111059.v4.1.I0e579520b03aa244906b8fe2ef1ec63f2ab7eecf@changeid/" class="external-link" target="_blank" rel="nofollow noopener">https://lkml.kernel.org/lkml/20220601111059.v4.1.I0e579520b03aa244906b8fe2ef1ec63f2ab7eecf@changeid/</a></p> <p>Could Maloo testing system change to use the newer kenrel such as linux-image-unsigned-5.17.0-1003-oem/jammy 5.17.0-1003.3 amd64?<br/> This kernel contains the patch. We'd better to try whether it fixes this failure.</p> <p>Regards,<br/> Qian</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=mdiep" class="user-hover" rel="mdiep">mdiep</a> Any possibility we can update the ubuntu 2204 kernel to &gt; v5.17? Or can we specify any Test-Parameters to do that?</p> <p>A ticket already created: DCO-9417</p> <p>According to the testing output (dmesg) in <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51755" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51755</a>,<br/> Maloo testing on Ubuntu2204: ll_put_super():<br/> <a href="https://testing.whamcloud.com/test_logs/0c171cbd-c4d4-4721-ba61-3d79b542b0a7/show_text" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_logs/0c171cbd-c4d4-4721-ba61-3d79b542b0a7/show_text</a></p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> PUT SB - force: 0 unstable_nr: 0 ref: 2 wb_list: 0 </pre> </div></div> <p>local test on CentOS 8: ll_put_super():</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> PUT SB - force: 0 unstable_nr: 0 ref: 1 wb_list: 0 sb@00000000c38268c1 cfg_instance lustre-client-ffff935c2b64d800 </pre> </div></div> <p>On Ubuntu 2204, the bdi refcnt is 2, 1 more larger than on CentOS8.</p> <p>This results the sysfs not unregister.<br/> So I think the kernel patch "kernel patch: fs: explicitly unregister per-superblock BDIs" should help to solve this problem.</p> <p>However, I still have no idea why bdi refcnt on Unbunt2204 is 2 during Lustre umount in ll_put_super...</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51701/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51701/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> llite: do not set SB_I_CGROUPWB on super block<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: d604d68c3f8ab8fbb52f5dd19651a11ac3dc0374</p> <p>The Maloo testing results verifies that this bug was fixed in the newer Ubuntun 2204 kernel &gt;= v5.17:<br/> <a href="https://testing.whamcloud.com/test_sessions/related?jobs=lustre-reviews&amp;builds=96950#redirect" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sessions/related?jobs=lustre-reviews&amp;builds=96950#redirect</a></p> <p>The patch kernel patch: fs: explicitly unregister per-superblock BDIs (See <a href="https://lore.kernel.org/all/20211021124441.668816-4-hch@lst.de/T/#u" class="external-link" target="_blank" rel="nofollow noopener">https://lore.kernel.org/all/20211021124441.668816-4-hch@lst.de/T/#u</a> for more details) fixed the duplicate filename failure during Lustre mount.</p> <p>It fixed by calling @bdi_unregister() directly to unregister the sysfs BDI interface.<br/> In the newer kernel &gt;= v5.17, @bdi_unregister is a kernel export symbol.<br/> However, for Ubuntu 2204 v5.15, the function @bdi_unregister is not exported...</p> <p>Thus, there is no easy way to fix this problem in the kernel &lt;= v5.17.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> void bdi_unregister(struct backing_dev_info *bdi) { ... <span class="code-keyword">if</span> (bdi-&gt;dev) { bdi_debug_unregister(bdi); device_unregister(bdi-&gt;dev); bdi-&gt;dev = NULL; } ... </pre> </div></div> <p>The solution maybe directly call @bdi_debug_unregister(bdi) and device_unregister(bdi-&gt;dev) explicitly in -&gt;put_super().</p> <p>I will make a patch to see whether it will cause any problem by this way.</p> <p>"Qian Yingjin &lt;qian@ddn.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51955" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51955</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> llite: add SB_I_CGROUPWB on super block<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 24263dc9ae44ad02639178f5b167dcd0f1ccc68f</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51955/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51955/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16954" title="mount failed: File exists(cannot create duplicate filename &#39;/devices/virtual/bdi/lustre-ffffxxx&#39;)" class="issue-link" data-issue-key="LU-16954"><del>LU-16954</del></a> llite: add SB_I_CGROUPWB on super block for cgroup<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: dcc1dd39a67f15de9174e7acdda599e3c54c1421</p> <p>It looks like this work is now complete for 2.16. The only remaining patches are fortestonly (so presumably will be abandoned)</p> Related LU-16697 LU-16970 LU-17012 Development Rank 1|i03q4v: Rank (Obsolete) 9223372036854775807 Severity