https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2449 Lustre <p>tags 2.3.56 and 2.3.57 builds are broken with:<br/> /home/green/git/lustre-release/lustre/osd-zfs/osd_index.c:540:2: error: #warning "fix '.' and '..' handling"</p> <p>This must be addressed before the next tag</p> LU-2449

ZFS osd to fix . and .. handling

Bug Minor Resolved Fixed Alex Zhuravlev Oleg Drokin zfs Sat, 8 Dec 2012 22:27:15 +0000 Wed, 26 Jan 2022 23:35:08 +0000 Thu, 14 Mar 2013 00:53:03 +0000 Lustre 2.4.0 Lustre 2.4.0 0 9 <p>Bobijam</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>Added Alex and Andreas to the watcher list.</p> <p>here is a short description of the problem: ZFS does not store "." and ".." on a disk, instead it stores parent dnode in znode and generate dots in zfs_readdir():</p> <p> while (!done) {<br/> uint64_t objnum;<br/> /*</p> <ul> <li>Special case `.', `..', and `.zfs'.<br/> */<br/> if (*pos == 0) { (void) strcpy(zap.za_name, "."); zap.za_normalization_conflict = 0; objnum = zp-&gt;z_id; } <p> else if (*pos == 1) </p> { (void) strcpy(zap.za_name, ".."); zap.za_normalization_conflict = 0; objnum = parent; } <p> else if (*pos == 2 &amp;&amp; zfs_show_ctldir(zp)) {<br/> (void) strcpy(zap.za_name, ZFS_CTLDIR_NAME);<br/> zap.za_normalization_conflict = 0;<br/> objnum = ZFSCTL_INO_ROOT;</p></li> </ul> <p>to comply with ZFS on-disk format we should:<br/> 1) initialize parent dnode attribute upon insert("..")<br/> 2) do nothing upon insert(".")<br/> 3) implement logic similar to zfs_readdir() in iterator methods: osd_zap_it_get(), osd_zap_it_key(), etc</p> <p>one issue is that parent SA can store upto 8 bytes:</p> <p> SA_ADD_BULK_ATTR(bulk, cnt, SA_ZPL_PARENT(uos), NULL, &amp;parent, 8);</p> <p>making it impossible to put dnode and fid in. there are two options here:<br/> 1) fetch parent dnode, then fetch LMA from that parent dnode &#8211; might be expensive, but usually should be cached<br/> 2) store parent fid as an extra EA</p> <p>also, remember insert("..") can be called few times (in different transactions) as a result of rename.</p> <p>We already store the parent FID in the linkEA on the object. That allows finding the FID of ".." without searching. FID of "." can be generated from LMA. </p> <p>LinkEA is being manipulated by MDD and it's not considered as mandatory, AFAIU. for example, we ignore errors related to LinkEA.</p> <p>We only ignore such errors when there are many links on a regular file. There should always be enough space for a single linkEA on a directory to point to the parent. All ZFS filesystems are 2.x, so they should also have linkEA entries already, unlike upgraded 1.8 filesystems.</p> <p>I agree MDD creating linkEA might be a bit bothersome, but it is only being read by the OSD code, never modified. If it isn't available, it is always possible to do OI lookup for parent dnode number to find FID?</p> <p>yes, of course. a minor correction: should be possible to load LMA of specific dnode to find its FID.</p> <p><a href="http://review.whamcloud.com/4990" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4990</a></p> <p>not the final version though.. some compatibility with the existing setup is required.</p> <p>the patch has been updated with the compatibility and additional infrastructure to simulate old setup (where ./.. are stored on a disk).</p> <p>hmm, this patch provides forward compatibility, but not backward: it does remove ./.. if they are stored on a disk,<br/> but do not add new one (they are generated on demand like ZPL does). with the current master code that will result<br/> in non-existing ./.. and number of consequences.</p> <p>Christopher, what would be your opinion on this? would you be OK to switch to a new tag and do not go back?</p> <p>Alex, I think this is the reasonable path forward. The only other option is to start filling in and using the "zp_parent" field, but not delete the "." and ".." entries yet, and that can be done in a separate patch. That would allow migrating to the new code, and only disabling storing of "." and ".." once the first patch is known to be working.</p> <p>To comment further - I don't think it is necessary to make the extra effort to remove the "." and ".." entries from the on-disk ZAPs, so long as they are filtered out at the readdir level. The legacy filesystems/directories holding "." and ".." will disappear eventually.</p> <p>not that I'm against this <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> just got to think a potential ZPL user can be confused finding two ./.. </p> <p>If thats the worst that we'll see (two sets of . or ..), then I think that is reasonable, and we don't need to jump through hoops to eliminate it. A tool external to lustre to clean it up might be nice. The Sequoia filesystem could live for another 5 years.</p> <p>Brian, any chance you will have time to inspect the patch in <a href="http://review.whamcloud.com/4990?" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4990?</a></p> <p>Brian, would you have time to inspect this patch this week? Thank you!</p> <p>Landed for 2.4</p> <p>Still a few fixes for this.</p> <p>Still <a href="http://review.whamcloud.com/5413" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5413</a> to land.</p> <p>Landed for 2.4</p> <p><a href="http://review.whamcloud.com/5629" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5629</a> &#8211; this patch implements fallback path for lookup(..) when LinkEA is not available.</p> <p>the only remaining bit to close the ticket is to update parent dnode for regular files on object creation.</p> <p><a href="http://review.whamcloud.com/#change,5642" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5642</a> &#8211; with this patch all the objects should be getting parent dnode attribute initialized properly.</p> <p>Landed for 2.4</p> <p>Lowering priority now that first patch has landed.</p> Duplicate LU-2231 Related LU-2890 Development Rank 1|hzvdgn: Rank (Obsolete) 5787 Severity