Details
-
Technical task
-
Resolution: Fixed
-
Critical
-
Lustre 2.4.0
-
None
-
6666
Description
It would be very useful to have an OBD_COMPAT_NOIGIF flag stored in the last_rcvd file after the first OI Scrub has completed and stored the IGIF FID-in-LMA and in the OI, and after this it should refuse access to IGIF FIDs that are not in the OI.
The OBD_COMPAT_NOIGIF flag should also be set for newly formatted filesystems, since they should never have IGIF inodes from the beginning.
This prevents users from trying to access e.g. the OSD root inode "[2:0:0]" even without knowing the generation number, since OSD_OII_NOGEN == 0, and this allows accessing a file just by guessing.
Attachments
Issue Links
- is related to
-
LU-2780 Use real inode for .lustre/fid
-
- Resolved
-
Activity
Initial OI scrub will auto start every time when the MDT mount, and scan very small files set which needed by the mount processing, during such scan, the SF_UPGRADE may be set, depends on whether it is upgrade from lustre-1.8 or not.
The background OI scrub is conditional triggered when MDT mount, if the administrator does not specify "-o noscrub" when mount, and:
1) former OI scrub crashed, or
2) former OI scrub paused, or
3) detect file-level backup/restore, or
4) detect upgrading from lustre-1.8
Then the background OI scrub will be triggered. And for the last case (4), the "SF_UPGRADE" flag will be set.
Will scrub automatically start and set the SF_UPGRADE flag at MDT mount time?
OI scrub will auto detect the flag "SF_UPGRADE" in the file "OI_scrub" to decide whether allow IGIF lookup directly or not.
The flag "SF_UPGRADE" will be set when OI scrub auto detect the upgrading (can by initial OI scrub or by normal background OI scrub), and will be cleared when the (background) OI scrub finished the upgrading process (including IGIF-in-LMA, IGIF-in-OI, IGIF-in-dirent (optional)); if the upgrading is paused or stopped, then the flag "SF_UPGRADE" will NOT be cleared.
For new formatted MDT with Lustre-2.x, OI scrub cannot find the upgrading, so will NOT set the flag "SF_UPGRADE".
Does it answer your concerned issues?
I'm going to reopen this until I understand the situation a bit better.
I'm not sure I trust that the ability to allow/deny IGIF FIDs can or should be decided dynamically at each startup. How does it know each time whether the upgrade and LFSCK has been completed? How does it know that LFSCK was never run, but the filesystem was formatted with 2.x or later?
I did not introduce new flags "OBD_COMPAT_NOIGIF", instead, just process the IGIF lookup inside OSD as following:
For the MDT upgraded from 1.8 device, OI scrub will immobilize the IGIF and insert the mapping for "IGIF <=> ino#" into the OI file. There are three cases to be processed:
1) After the upgrading, all IGIFs have been processed and inserted into OI files. Then any IGIF object lookup should be processed as normal FID does, means via OI. If no entry in the OI file, then the IGIF object does not exist.
2) During the upgrading, some IGIFs may be already in OI files, others may be not yet. Under such case, lookup an IGIF in OI files may return -ENOENT, but it does not means the IGIF object does not exist. Since the new immobilize IGIF is the same as the original one before backup/restore, OSD will generate local identifier from IGIF directly as old 2.x did.
3) The upgrading is paused, and backup/restore. Means upgrading from 1.8 may be partly processed, but some clients may hold some immobilized IGIFs, and use them to access related objects. Under such case, OSD does not know whether an given IGIF has been processed or to be processed, and it also cannot generate local ino#/gen# directly from the immobilized IGIF because of the backup/restore. Then force OSD to lookup the given IGIF in OI files, and if no entry, then ask the client to retry after upgrading completed. No better choice.
This is the patch:
http://review.whamcloud.com/#change,6254
Having this depend on initial OI scrub to check each time whether IGIF FIDs are allowed isn't as robust as I would have liked - the SF_UPGRADE step would only be relevant the first time after an upgrade and not after a later backup/restore. This opens the possibility of IGIF lookups even after the first OI scrub is finished. However, it is at least preventing this access of the time, and much better than we have today.