https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13085 Lustre <p>I attempted to list the contents of /p/lustre3/faaland1/ using tab complete to determine the "faaland1" part of the path. There was a delay of several seconds, then the ls command returned with appropriate output. On the console of the client node, I saw the following:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[Tue Dec 17 18:07:31 2019] LustreError: 24055:0:(namei.c:87:ll_set_inode()) Can not initialize inode [0x2400013a0:0x6b6:0x0] without object type: valid = 0x100000001 [Tue Dec 17 18:07:31 2019] LustreError: 24055:0:(namei.c:87:ll_set_inode()) Skipped 6 previous similar messages [Tue Dec 17 18:07:31 2019] LustreError: 24055:0:(llite_lib.c:2426:ll_prep_inode()) new_inode -fatal: rc -12 [Tue Dec 17 18:07:31 2019] LustreError: 24055:0:(llite_lib.c:2426:ll_prep_inode()) Skipped 6 previous similar messages </pre> </div></div> <p>See <a href="https://github.com/LLNL/lustre" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/LLNL/lustre</a> for the patch stacks.</p> corona82 login node <br/> RHEL 7.7 derivative <br/> kernel-3.10.0-1062.7.1.1chaos.ch6.x86_64 <br/> lustre-2.12.3_2.chaos-1.4mofed.ch6.x86_64 <br/> filesystem running lustre-2.10.8_5.chaos LU-13085

(namei.c:87:ll_set_inode()) Can not initialize inode [0x540028b1f:0x2:0x0] without object type: valid = 0x100000001

Bug Minor Resolved Duplicate Lai Siyao Olaf Faaland llnl Wed, 18 Dec 2019 02:31:13 +0000 Sat, 1 Feb 2020 16:16:35 +0000 Wed, 22 Jan 2020 15:42:27 +0000 Lustre 2.10.8 0 3 <p>There are 2 MDTs on this file system.  The listed fid's info and my directory's info:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@corona82:~]# lfs fid2path /p/lustre3/ [0x2400013a0:0x6b6:0x0] /p/lustre3/bennion1 [root@corona82:~]# lfs getdirstripe /p/lustre3/bennion1 lmv_stripe_count: 0 lmv_stripe_offset: 1 lmv_hash_type: none [root@corona82:~]# lfs getdirstripe /p/lustre3/faaland1 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none [root@corona82:~]# lfs path2fid /p/lustre3/faaland1 [0x200000bd0:0x745:0x0]</pre> </div></div> <p>The two MDS nodes have nothing in dmesg from the time when the error was reported on the client:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>---------------- eporter81 ---------------- [Tue Dec 17 16:07:18 2019] Lustre: Skipped 3 previous similar messages [Tue Dec 17 16:12:11 2019] Lustre: MGS: Connection restored to ae14aba1-e9ca-73c5-d91e-d5258431f3c5 (at 192.168.128.137@o2ib20) [Tue Dec 17 16:12:11 2019] Lustre: Skipped 1 previous similar message [Tue Dec 17 16:12:23 2019] Lustre: MGS: Connection restored to 57067c2f-6884-8432-75c3-5d29b7b44621 (at 192.168.128.138@o2ib20) [Tue Dec 17 16:12:23 2019] Lustre: Skipped 1 previous similar message ---------------- eporter82 ---------------- [Tue Dec 17 15:23:34 2019] Lustre: lustre3-MDT0001: Connection restored to 6e206333-0ad1-f33f-5455-92e3e1592cf5 (at 192.168.128.140@o2ib20) [Tue Dec 17 16:07:39 2019] Lustre: lustre3-MDT0001: haven't heard from client a8de14f3-4714-ff41-67ca-ffcfd5d3ce43 (at 192.168.128.138@o2ib20) in 227 seconds. I think it's dead, and I am evicting it. exp ffff99e50e087800, cur 1576627639 expire 1576627489 last 1576627412 [Tue Dec 17 16:07:39 2019] Lustre: Skipped 1 previous similar message [Tue Dec 17 16:12:33 2019] Lustre: lustre3-MDT0001: Connection restored to ae14aba1-e9ca-73c5-d91e-d5258431f3c5 (at 192.168.128.137@o2ib20) [Tue Dec 17 16:12:45 2019] Lustre: lustre3-MDT0001: Connection restored to a8de14f3-4714-ff41-67ca-ffcfd5d3ce43 (at 192.168.128.138@o2ib20)</pre> </div></div> <p>I saw the same error reported by the client earlier today, but I've not yet been able to reproduce it on demand.</p> <p>In the earlier case, the FID reported is on a different file system, lustre2.  That other file system is running a slightly different tag, lustre-2.10.8_4.chaos.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@corona82:~]# dmesg -T | grep ll_prep_inode -C2 [Mon Dec 16 12:54:27 2019] LNetError: 45583:0:(peer.c:3451:lnet_peer_ni_add_to_recoveryq_locked()) Skipped 14 previous similar messages [Mon Dec 16 13:02:43 2019] LustreError: 118530:0:(namei.c:87:ll_set_inode()) Can not initialize inode [0x580023454:0x2:0x0] without object type: valid = 0x100000001 [Mon Dec 16 13:02:43 2019] LustreError: 118530:0:(llite_lib.c:2426:ll_prep_inode()) new_inode -fatal: rc -12 [Mon Dec 16 13:04:37 2019] LNetError: 45583:0:(peer.c:3451:lnet_peer_ni_add_to_recoveryq_locked()) lpni 192.168.128.86@o2ib36 added to recovery queue. Health = 900 [Mon Dec 16 13:04:37 2019] LNetError: 45583:0:(peer.c:3451:lnet_peer_ni_add_to_recoveryq_locked()) Skipped 11 previous similar messages [root@corona82:~]# lfs fid2path /p/lustre2/ [0x580023454:0x2:0x0] /p/lustre2/faaland1/make-busy/mdt14</pre> </div></div> <p>Let me know what information you'd like me to try to gather for the next occurrence.</p> <p>I gathered debug logs from the client and attached them. </p> <ul> <li>first instance: dk.corona82.1576534399.gz</li> <li>second instance with more debug: dk.corona82.1576635179.gz</li> </ul> <p>Also the debug logs from the porter MDS nodes (/p/lustre3 file system):</p> <ul> <li>dk.porter8x.1576637982</li> </ul> <p>Lai</p> <p>Could you please investigate?</p> <p>Thanks</p> <p>Peter</p> <p>This is a duplicate of <a href="https://jira.whamcloud.com/browse/LU-13099" title="ll_set_inode()) Can not initialize inode warning" class="issue-link" data-issue-key="LU-13099"><del>LU-13099</del></a>, and the fix is on <a href="https://review.whamcloud.com/#/c/37089/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/37089/</a>.</p> Related LU-13099 Development Rank 1|i00r5r: Rank (Obsolete) 9223372036854775807 Severity