https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1531 Lustre <p><span class="error">&#91;sanity@r62-lustre lustre&#93;</span>$ cat .lustre/fid/<span class="error">&#91;0x1:0x0:0x0&#93;</span></p> <p>LustreError: 2316:0:(osd_compat.c:383:osd_compat_objid_lookup()) ASSERTION( map ) failed: <br/> LustreError: 2316:0:(osd_compat.c:383:osd_compat_objid_lookup()) LBUG<br/> Pid: 2316, comm: mdt_00</p> <p>Call Trace:<br/> <span class="error">&#91;&lt;ffffffffa02c2905&gt;&#93;</span> libcfs_debug_dumpstack+0x55/0x80 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa02c2f17&gt;&#93;</span> lbug_with_loc+0x47/0xb0 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0b7f889&gt;&#93;</span> osd_compat_objid_lookup+0x479/0x520 <span class="error">&#91;osd_ldiskfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0b74165&gt;&#93;</span> osd_oi_lookup+0x55/0xb0 <span class="error">&#91;osd_ldiskfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0b6ea90&gt;&#93;</span> osd_object_init+0x260/0x6a0 <span class="error">&#91;osd_ldiskfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa048ab0c&gt;&#93;</span> ? lu_object_add+0x2c/0x30 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa048c5d5&gt;&#93;</span> lu_object_alloc+0xd5/0x310 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa048cba8&gt;&#93;</span> ? htable_lookup+0x108/0x1c0 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa048ce61&gt;&#93;</span> lu_object_find_at+0x201/0x450 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffff81273c47&gt;&#93;</span> ? vsscanf+0x617/0x7c0<br/> <span class="error">&#91;&lt;ffffffffa048d0ef&gt;&#93;</span> lu_object_find_slice+0x1f/0x80 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a4f160&gt;&#93;</span> mdd_object_find+0x10/0x70 <span class="error">&#91;mdd&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a79006&gt;&#93;</span> obf_lookup+0xd6/0x270 <span class="error">&#91;mdd&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0b3f136&gt;&#93;</span> cml_lookup+0x66/0x1b0 <span class="error">&#91;cmm&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ad50e7&gt;&#93;</span> ? mdt_version_get_check+0x47/0xe0 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ae9c37&gt;&#93;</span> mdt_reint_open+0x6f7/0x18b0 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a7782e&gt;&#93;</span> ? md_ucred+0x1e/0x60 <span class="error">&#91;mdd&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ab81a5&gt;&#93;</span> ? mdt_ucred+0x15/0x20 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0acf21c&gt;&#93;</span> ? mdt_root_squash+0x2c/0x3e0 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ad3b21&gt;&#93;</span> mdt_reint_rec+0x41/0xe0 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0acd37a&gt;&#93;</span> mdt_reint_internal+0x50a/0x810 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0acd94d&gt;&#93;</span> mdt_intent_reint+0x1ed/0x500 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ac9c91&gt;&#93;</span> mdt_intent_policy+0x371/0x6a0 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa05ae831&gt;&#93;</span> ldlm_lock_enqueue+0x361/0x8f0 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa05d611a&gt;&#93;</span> ldlm_handle_enqueue0+0x48a/0xf40 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ac9836&gt;&#93;</span> mdt_enqueue+0x46/0x130 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0abf2a2&gt;&#93;</span> mdt_handle_common+0x922/0x1740 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0ac0195&gt;&#93;</span> mdt_regular_handle+0x15/0x20 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0604782&gt;&#93;</span> ptlrpc_server_handle_request+0x412/0xeb0 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa02c365e&gt;&#93;</span> ? cfs_timer_arm+0xe/0x10 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa02d3d9f&gt;&#93;</span> ? lc_watchdog_touch+0x6f/0x180 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa05fd5e2&gt;&#93;</span> ? ptlrpc_wait_event+0xb2/0x2c0 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffff81051ab3&gt;&#93;</span> ? __wake_up+0x53/0x70<br/> <span class="error">&#91;&lt;ffffffffa06059f7&gt;&#93;</span> ptlrpc_main+0x7d7/0x1610 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0605220&gt;&#93;</span> ? ptlrpc_main+0x0/0x1610 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffff8100c14a&gt;&#93;</span> child_rip+0xa/0x20<br/> <span class="error">&#91;&lt;ffffffffa0605220&gt;&#93;</span> ? ptlrpc_main+0x0/0x1610 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0605220&gt;&#93;</span> ? ptlrpc_main+0x0/0x1610 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffff8100c140&gt;&#93;</span> ? child_rip+0x0/0x20</p> <p>LustreError: dumping log to /tmp/lustre-log.1339790039.2316</p> # uname -r<br/> 2.6.32-220.13.1.el6.l22.x86_64<br/> # cat /proc/fs/lustre/version <br/> lustre: 2.2.55<br/> kernel: patchless_client<br/> build: 2.2.55--CHANGED-2.6.32-220.13.1.el6.l22.x86_64 LU-1531

Accessing .lustre/fid/[0x1:0x0:0x0] triggers LBUG in osd_compat_objid_lookup()

Bug Minor Closed Fixed Richard Henwood John Hammond Fri, 15 Jun 2012 15:56:06 +0000 Tue, 6 Nov 2012 21:56:40 +0000 Tue, 6 Nov 2012 21:56:34 +0000 Lustre 2.3.0 Lustre 2.4.0 0 3 <p>This is code that was landed to master from orion in commit 4980567857699c7f902ebda336ea98fdc4b83100.</p> <p>It definitely shouldn't be possible for regular users to trigger an LASSERT(), or otherwise access invalid FIDs via .lustre, so the MDS needs to validate the FID is sane and allowed to be accessed before passing it down to the lower layers.</p> <p>You can also get here using lfs path2fid, in fact by using the sample FID provided.</p> <p><span class="error">&#91;root&#93;</span># lfs fid2path /mnt/lustre PANTS<br/> bad FID format <span class="error">&#91;PANTS&#93;</span>, should be <span class="error">&#91;0x1:0x2:0x0&#93;</span></p> <p>fid2path error: Invalid argument<br/> <span class="error">&#91;root&#93;</span># lfs fid2path /mnt/lustre <span class="error">&#91;0x1:0x2:0x0&#93;</span></p> <p>LustreError: 2446:0:(osd_compat.c:381:osd_compat_objid_lookup()) ASSERTION( map ) failed: <br/> LustreError: 2446:0:(osd_compat.c:381:osd_compat_objid_lookup()) LBUG<br/> Pid: 2446, comm: mdt00_002</p> <p>Call Trace:<br/> <span class="error">&#91;&lt;ffffffffa0308905&gt;&#93;</span> libcfs_debug_dumpstack+0x55/0x80 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0308f17&gt;&#93;</span> lbug_with_loc+0x47/0xb0 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0b73c59&gt;&#93;</span> osd_compat_objid_lookup+0x479/0x520 <span class="error">&#91;osd_ldiskfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffff81275d46&gt;&#93;</span> ? vsnprintf+0x2b6/0x5f0<br/> <span class="error">&#91;&lt;ffffffffa0b68565&gt;&#93;</span> osd_oi_lookup+0x55/0xc0 <span class="error">&#91;osd_ldiskfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0b5dbf8&gt;&#93;</span> osd_object_init+0x338/0xd80 <span class="error">&#91;osd_ldiskfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa048a22e&gt;&#93;</span> ? dt_object_init+0xe/0x10 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0486f65&gt;&#93;</span> lu_object_alloc+0xd5/0x310 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0487538&gt;&#93;</span> ? htable_lookup+0x108/0x1c0 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa04877f1&gt;&#93;</span> lu_object_find_at+0x201/0x450 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa05f4271&gt;&#93;</span> ? lustre_pack_reply_v2+0x1e1/0x280 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa05f330d&gt;&#93;</span> ? lustre_msg_buf+0x5d/0x60 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0487a56&gt;&#93;</span> lu_object_find+0x16/0x20 <span class="error">&#91;obdclass&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a71da6&gt;&#93;</span> mdt_object_find+0x56/0x170 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a75fda&gt;&#93;</span> mdt_get_info+0x22a/0xa90 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a71f0d&gt;&#93;</span> ? mdt_unpack_req_pack_rep+0x4d/0x4d0 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a7a922&gt;&#93;</span> mdt_handle_common+0x922/0x1740 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0a7b815&gt;&#93;</span> mdt_regular_handle+0x15/0x20 <span class="error">&#91;mdt&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa060457d&gt;&#93;</span> ptlrpc_server_handle_request+0x40d/0xea0 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa030965e&gt;&#93;</span> ? cfs_timer_arm+0xe/0x10 <span class="error">&#91;libcfs&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa05fba37&gt;&#93;</span> ? ptlrpc_wait_event+0xa7/0x2a0 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffff81051ba3&gt;&#93;</span> ? __wake_up+0x53/0x70<br/> <span class="error">&#91;&lt;ffffffffa0605b79&gt;&#93;</span> ptlrpc_main+0xb69/0x1870 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0605010&gt;&#93;</span> ? ptlrpc_main+0x0/0x1870 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffff8100c14a&gt;&#93;</span> child_rip+0xa/0x20<br/> <span class="error">&#91;&lt;ffffffffa0605010&gt;&#93;</span> ? ptlrpc_main+0x0/0x1870 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffffa0605010&gt;&#93;</span> ? ptlrpc_main+0x0/0x1870 <span class="error">&#91;ptlrpc&#93;</span><br/> <span class="error">&#91;&lt;ffffffff8100c140&gt;&#93;</span> ? child_rip+0x0/0x20</p> <p>I couldn't reproduce this issue with a recent Master build (924) and I can't reproduce.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># cat .lustre/fid/[0x1:0x0:0x0] cat: .lustre/fid/[0x1:0x0:0x0]: Invalid argument </pre> </div></div> <p>Seems like this issue has been fixed by <a href="https://jira.whamcloud.com/browse/LU-1518" title="Missing/bad operations in mdd_{obf,dot_lustre}_obj_op causing LBUGs" class="issue-link" data-issue-key="LU-1518"><del>LU-1518</del></a>.</p> <p>Richard, it would probably be mired appropriate to have closed this as "Fixed" rather than "Cannot Reproduce", since it was a real bug that was fixed with a code change. The "Cannot Reproduce" label indicates that a reported bug was not seen in later testing for whatever reason, and no change was made to the code to resolve the issue. </p> <p>In any case, I'm glad this problem is gone. </p> <p>Correcting resolution message. This issue was a real problem that was /fixed/.</p> <p>double checking if this is actually fixed on Lustre 2.3 ...</p> <p>Given Andreas's comment that this issue was introduced by an Orion commit, 2.3 should not be affected. I've checked it anyway, and I couldn't reproduce.</p> <p>However, this issue is fixed.</p> <p>This issue is present on Master.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lfs fid2path /mnt/lustre/ [0x1:0x2:0x0] </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LustreError: 31926:0:(mdt_handler.c:2447:mdt_obj()) ASSERTION( lu_device_is_mdt(o-&gt;lo_dev) ) failed: LustreError: 31926:0:(mdt_handler.c:2447:mdt_obj()) LBUG Pid: 31926, comm: mdt00_000 Call Trace: [&lt;ffffffffa03d3905&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] [&lt;ffffffffa03d3f17&gt;] lbug_with_loc+0x47/0xb0 [libcfs] [&lt;ffffffffa0b851cf&gt;] mdt_obj+0x5f/0x80 [mdt] [&lt;ffffffffa0b892e6&gt;] mdt_object_find+0x66/0x170 [mdt] [&lt;ffffffffa0b8dcaa&gt;] mdt_get_info+0x22a/0xa90 [mdt] [&lt;ffffffffa0b8943d&gt;] ? mdt_unpack_req_pack_rep+0x4d/0x4c0 [mdt] [&lt;ffffffffa0b91322&gt;] mdt_handle_common+0x932/0x1740 [mdt] [&lt;ffffffffa0b92205&gt;] mdt_regular_handle+0x15/0x20 [mdt] [&lt;ffffffffa06ed7ac&gt;] ptlrpc_server_handle_request+0x41c/0xe00 [ptlrpc] [&lt;ffffffffa03d465e&gt;] ? cfs_timer_arm+0xe/0x10 [libcfs] [&lt;ffffffffa06e4b87&gt;] ? ptlrpc_wait_event+0xa7/0x2a0 [ptlrpc] [&lt;ffffffff810533f3&gt;] ? __wake_up+0x53/0x70 [&lt;ffffffffa06eed81&gt;] ptlrpc_main+0xbf1/0x19e0 [ptlrpc] [&lt;ffffffffa06ee190&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] [&lt;ffffffff8100c14a&gt;] child_rip+0xa/0x20 [&lt;ffffffffa06ee190&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] [&lt;ffffffffa06ee190&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] [&lt;ffffffff8100c140&gt;] ? child_rip+0x0/0x20 Kernel panic - not syncing: LBUG Pid: 31926, comm: mdt00_000 Not tainted 2.6.32-279.5.1.el6_lustre.x86_64 #1 Call Trace: [&lt;ffffffff814fd58a&gt;] ? panic+0xa0/0x168 [&lt;ffffffffa03d3f6b&gt;] ? lbug_with_loc+0x9b/0xb0 [libcfs] [&lt;ffffffffa0b851cf&gt;] ? mdt_obj+0x5f/0x80 [mdt] [&lt;ffffffffa0b892e6&gt;] ? mdt_object_find+0x66/0x170 [mdt] [&lt;ffffffffa0b8dcaa&gt;] ? mdt_get_info+0x22a/0xa90 [mdt] [&lt;ffffffffa0b8943d&gt;] ? mdt_unpack_req_pack_rep+0x4d/0x4c0 [mdt] [&lt;ffffffffa0b91322&gt;] ? mdt_handle_common+0x932/0x1740 [mdt] [&lt;ffffffffa0b92205&gt;] ? mdt_regular_handle+0x15/0x20 [mdt] [&lt;ffffffffa06ed7ac&gt;] ? ptlrpc_server_handle_request+0x41c/0xe00 [ptlrpc] [&lt;ffffffffa03d465e&gt;] ? cfs_timer_arm+0xe/0x10 [libcfs] [&lt;ffffffffa06e4b87&gt;] ? ptlrpc_wait_event+0xa7/0x2a0 [ptlrpc] [&lt;ffffffff810533f3&gt;] ? __wake_up+0x53/0x70 [&lt;ffffffffa06eed81&gt;] ? ptlrpc_main+0xbf1/0x19e0 [ptlrpc] [&lt;ffffffffa06ee190&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] [&lt;ffffffff8100c14a&gt;] ? child_rip+0xa/0x20 [&lt;ffffffffa06ee190&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] [&lt;ffffffffa06ee190&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] [&lt;ffffffff8100c140&gt;] ? child_rip+0x0/0x20 </pre> </div></div> <p>a fix has landed in master:</p> <p><a href="http://review.whamcloud.com/#change,4255" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4255</a></p> Related LU-1518 Development Rank 1|hzvguv: Rank (Obsolete) 6380 Severity