Minor Description
The Lustre Manual should be updated to inform users about how to enable "dirdata" on an upgraded 1.8->2.x MDT, once they are sure that they will not be downgrading the MDS to 1.8 again. This will minimize performance impact on newly created files.
The command should be `tune2fs -O dirdata /dev/
{mdtdev}`, and I've verified that newly created dir will store FID in it.The need to be able to downgrade from 2.x to 1.8 is only in the case of "simple" upgrade to 2.x that has hit problems and needs to be able to downgrade. If the upgrade has been successful, and then the admin (separately) enables the "dir_data" feature using tune2fs on the filesystem, this should be enough to allow storing FIDs in the directory entries. After that point, the filesystem should not be downgraded to 1.8 anymore.
What definitely should be avoided is any automatic enabling of the "dir_data" feature on the filesystem when it is first mounted, since this will cause problems if there are FIDs stored in the directory entries, then the filesystem is downgraded to 1.8, the FID-in-LMA is deleted upon access (reverting to IGIF for that inode), and then the filesystem is upgraded again. That would cause the FID-in-dirent to contain invalid data that OI scrub and e2fsck will not fix yet.
So, my understanding is that if you are sure there is no need to downgrade to 1.8, it should be possible with 2.1+ to use "tune2fs -O dir_data /dev/{mdtdev}
" to enable this feature, and then newly-created files/links will store the FID in the directory. I don't know if we have tested this process or not.
Assuming this is OK, it would then be possible in that case to "refresh" the directory with a script to re-link filenames that are expected to live for a long time, assuming they are not in use, something like:
lfs find /mnt/lustre/some/dir -type f | while read F; do FTMP="$F.XXXXXX.$$.$RANDOM" mv "$F" "$FTMP" && mv "$FTMP" "$F" done
In a later phase of LFSCK, the FID-in-dirent data will be verified and refreshed if missing, but this is not part of the Phase I deliverable.
This existing behaviour is not a net performance loss in many use cases, since it is prefetching the inode into MDS memory for use as soon as e.g. "ls" does a stat() on the file. There would only be a visible slowdown in the case of e.g. "find" that is not accessing any of the file attributes, and only generating pathnames.
Attachments
Issue Links
- is related to
-
LUDOC-145 LFSCK Phase 1.5 - Enable dirdata before lfsck -t namespace when upgrading from 1.8->2.x
-
- Resolved
-
Activity
Changes pushed to gerrit and ready for review. http://review.whamcloud.com/#/c/7844/
Andreas suggests subject experts: Di for FIDs, Fan Yong for dirdata
Cliff suggests covering this in a separate section in Ch 14 "Lustre Maintenance" (so it appears in the TOC) and then including a xref to this section from Ch 16 "Upgrading Lustre". It can be done at the time of the upgrade or at a later time.
My comment was true for all the 2.X before 2.4. But yes, we hope things are better with 2.4.
Chris, I don't think that that is right. LFSCK 1.5 should be able to bring the working fid2path functionality once a full scrub is complete (or, presumably, the full path has been scrubbed). It's landed for 2.4 so it isn't a long way off, it is eminent! Your fid2path use case ought to be a part of the test demonstration. I think it will work.
We further learned today that the dirdata option (note that there is no underscore in dirdata, as far as I know) doesn't actually make fid2path work for new files (we at least knew that it wouldn't work for existing files). In order for fid2path to work, both dirdata needs to be set and the entire path to the file in lustre needs to be newly created.
In other words, there is really no sane way to update an old filesystem to fully work with fids right now, even for new files. Xyratex's solution is probably the only way to do it right now. Some day LFSCK is planned to do it as well, but that is still a long way off.
But right now you're basically out of luck. This all needs to be explained in the manual.
From email sent to Peter Jones from a contact at LLNL...
"We just got burned by lack of fid documentation. LUDOC-68 would be
excellent. We probably need a section that explains FIDs independant of
the changelog section.
Cliff,
This is also part of the Lustre Manual improvement project.
Changes have been merged. Resolving ticket.