https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4675 Lustre <p>If an orphan object with stripe_index != 0 is linked to a recreated MDS inode in <a href="http://review.whamcloud.com/7810" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7810</a>, but not all of the objects are present (e.g. some of the stripes of that file were lost, but a non-zero stripe_index orphan remained) the client will crash if the file is accessed (e.g. "ls -l"):</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LustreError: 19393:0:(ldlm_resource.c:1077:ldlm_resource_get()) ASSERTION( name-&gt;name[0] != 0 ) failed: LustreError: 19393:0:(ldlm_resource.c:1077:ldlm_resource_get()) LBUG Pid: 19393, comm: ls Call Trace: [&lt;ffffffffa0ef9895&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] [&lt;ffffffffa0ef9e97&gt;] lbug_with_loc+0x47/0xb0 [libcfs] [&lt;ffffffffa07c4f20&gt;] ldlm_resource_get+0x700/0x900 [ptlrpc] [&lt;ffffffffa07bf1b9&gt;] ldlm_lock_create+0x59/0xcc0 [ptlrpc] [&lt;ffffffffa07d8314&gt;] ldlm_cli_enqueue+0xa4/0x790 [ptlrpc] [&lt;ffffffffa09ebd44&gt;] osc_enqueue_base+0x1e4/0x5b0 [osc] [&lt;ffffffffa0a082fd&gt;] osc_lock_enqueue+0x1ed/0x8c0 [osc] [&lt;ffffffffa105be7c&gt;] cl_enqueue_try+0xfc/0x300 [obdclass] [&lt;ffffffffa0a5d42a&gt;] lov_lock_enqueue+0x22a/0x850 [lov] [&lt;ffffffffa105be7c&gt;] cl_enqueue_try+0xfc/0x300 [obdclass] [&lt;ffffffffa105d0cf&gt;] cl_enqueue_locked+0x6f/0x1f0 [obdclass] [&lt;ffffffffa105dd1e&gt;] cl_lock_request+0x7e/0x270 [obdclass] [&lt;ffffffffa123dba0&gt;] cl_glimpse_lock+0x180/0x490 [lustre] [&lt;ffffffffa123e415&gt;] cl_glimpse_size0+0x1a5/0x1d0 [lustre] [&lt;ffffffffa11eb55d&gt;] ll_inode_revalidate_it+0x1cd/0x660 [lustre] [&lt;ffffffffa11eba3a&gt;] ll_getattr_it+0x4a/0x1b0 [lustre] [&lt;ffffffffa11ebbd7&gt;] ll_getattr+0x37/0x40 [lustre] [&lt;ffffffff81186db1&gt;] vfs_getattr+0x51/0x80 [&lt;ffffffff81186e40&gt;] vfs_fstatat+0x60/0x80 [&lt;ffffffff81186ece&gt;] vfs_lstat+0x1e/0x20 [&lt;ffffffff81186ef4&gt;] sys_newlstat+0x24/0x50 </pre> </div></div> <p>I'm not sure what the right way to handle this is, since this would affect all old clients trying to access files in <tt>.lustre/lost+found</tt> so fixing just the 2.6 client is not enough. Either we need to backport the fix to 2.5.2 and 2.4.3 and 2.1.7 clients (not very good, since we aren't sure if the client has the fix), or use some other lmm_magic or lmm_pattern to ensure that unpatched clients will not understand it.</p> <p>In the second case (using a different lmm_magic or lmm_pattern, maybe LOV_PATTERN_F_SPARSE?) the lfsck_layout_extend_lovea() code would need to decide as stripes are added if the layout is sparse (set the flag, old clients cannot access) or if it is full (clear the flag, old clients can access).</p> LU-4675

LFSCK should mark files with holes so clients do not access them

Technical task LU-4701 Blocker Resolved Fixed nasf Andreas Dilger Wed, 26 Feb 2014 23:45:08 +0000 Thu, 29 May 2014 03:41:59 +0000 Thu, 29 May 2014 03:41:59 +0000 Lustre 2.6.0 Lustre 2.6.0 0 3 <p>To be clear, I don't think that the LASSERT() should be removed, since object </p> {0,0} <p> should never be accessed. Rather, the LOV code should just skip such objects entirely, and return -EIO in such a case. Please include a test case that creates such a file, and runs a number of different operations on it (stat, read, write, touch, chown, unlink) to make sure the different paths are covered.</p> <p>Here is the patch:</p> <p><a href="http://review.whamcloud.com/10042" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10042</a></p> <p>When the layout LFSCK repairs orphan OST-object, if the parent<br/> MDT-object was lost, then it will re-create the MDT-object and<br/> regenerate the LOV EA and fill the target LOV EA slot with the<br/> orphan information, and fill other slots with zero (LOV hole);<br/> if related LOV EA slot is invalid or hole, then it will refill<br/> the target LOV EA slot; if the target slot exceeds current LOV<br/> EA tail, then extend the LOV EA, and fill the gaps as zero.</p> <p>Some of the LOV EA holes may cannot be re-filled finally becuase<br/> of lost some OST-objects. And even if they can be re-filled, but<br/> there are still some possible race accessings from client before<br/> the re-filling. If the client access the LOV EA with hole(s), it<br/> may cause some strange behaviour, such as trigger LBUG()/LASSERT()<br/> on the client.</p> <p>So we will make the client to be aware of the LOV EA is incomplete.<br/> We introduce a new LOV EA pattern flag LOV_PATTERN_F_HOLE for that:<br/> any time when the LFSCK repairs the LOV EA with hole(s), the LOV EA<br/> will be marked as LOV_PATTERN_F_HOLE; when all the holes in the LOV<br/> EA are refilled, the LOV_PATTERN_F_HOLE will be dropped.</p> <p>For a new client, it recongizes the pattern flag LOV_PATTERN_F_HOLE,<br/> then it can permit/forbid some opertions on the file with LOV holes:</p> <p>1) getattr/getxattr opertions are permitted, such as stat/ls -l, and<br/> so on. The file size is the sum on the known stripes. So it gives<br/> the administrator chance to know how much data has been recovered.</p> <p>2) Normal read the file with LOV EA hole is not permitted to avoid<br/> the LOV EA holes cases to be hidden. Instead, the administrator<br/> can dump the data via new "lfs dump" tool.</p> <p>3) If the modification only changes MDS-side metadata, such as chmod,<br/> then it is permitted.</p> <p>4) unlink/rm the file which has LOV EA holes is permitted.</p> <p>5) For other modifications, if the modification will change something<br/> on OST side, such as write/touch/chown, then it will be denied.</p> <p>For a old client, since it will not recognize the new pattern flag<br/> LOV_PATTERN_F_HOLE. So the LOV EA with holes will be dicarded with<br/> failure, but it will not cause the client to be crashed.</p> <p>I've encountered this client crash while trying to track down what files were causing LFSCK to fail during phase 1 of the scan.</p> <p>To be clear, while <a href="http://review.whamcloud.com/10042" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10042</a> is fixing the problem of LFSCK creating layouts with holes in them, there is still the separate bug (<a href="https://jira.whamcloud.com/browse/LU-4958" title="do not crash accessing LOV object with FID {0, 0}" class="issue-link" data-issue-key="LU-4958"><del>LU-4958</del></a>) for clients crashing because of bad layout data from the network, regardless of whether that was caused by LFSCK or not. It still makes sense for the client to validate the layout (magic, pattern, objects) and return an error of they are bad. </p> <p>The master patch has been landed. The patches for old client will be done under <a href="https://jira.whamcloud.com/browse/LU-4958" title="do not crash accessing LOV object with FID {0, 0}" class="issue-link" data-issue-key="LU-4958"><del>LU-4958</del></a></p> Related LU-4958 Development Rank 1|hzwg1j: Rank (Obsolete) 12832