Details

    • Technical task
    • Resolution: Fixed
    • Critical
    • Lustre 2.4.0
    • Lustre 2.4.0
    • None
    • 6666

    Description

      It would be very useful to have an OBD_COMPAT_NOIGIF flag stored in the last_rcvd file after the first OI Scrub has completed and stored the IGIF FID-in-LMA and in the OI, and after this it should refuse access to IGIF FIDs that are not in the OI.

      The OBD_COMPAT_NOIGIF flag should also be set for newly formatted filesystems, since they should never have IGIF inodes from the beginning.

      This prevents users from trying to access e.g. the OSD root inode "[2:0:0]" even without knowing the generation number, since OSD_OII_NOGEN == 0, and this allows accessing a file just by guessing.

      Attachments

        Issue Links

          Activity

            [LU-2742] add OBD_COMPAT_NOIGIF to block client IGIF lookups not in OI

            Having this depend on initial OI scrub to check each time whether IGIF FIDs are allowed isn't as robust as I would have liked - the SF_UPGRADE step would only be relevant the first time after an upgrade and not after a later backup/restore. This opens the possibility of IGIF lookups even after the first OI scrub is finished. However, it is at least preventing this access of the time, and much better than we have today.

            adilger Andreas Dilger added a comment - Having this depend on initial OI scrub to check each time whether IGIF FIDs are allowed isn't as robust as I would have liked - the SF_UPGRADE step would only be relevant the first time after an upgrade and not after a later backup/restore. This opens the possibility of IGIF lookups even after the first OI scrub is finished. However, it is at least preventing this access of the time, and much better than we have today.

            Hi Andreas, can we mark it as resolved?

            yong.fan nasf (Inactive) added a comment - Hi Andreas, can we mark it as resolved?

            Initial OI scrub will auto start every time when the MDT mount, and scan very small files set which needed by the mount processing, during such scan, the SF_UPGRADE may be set, depends on whether it is upgrade from lustre-1.8 or not.

            The background OI scrub is conditional triggered when MDT mount, if the administrator does not specify "-o noscrub" when mount, and:
            1) former OI scrub crashed, or
            2) former OI scrub paused, or
            3) detect file-level backup/restore, or
            4) detect upgrading from lustre-1.8

            Then the background OI scrub will be triggered. And for the last case (4), the "SF_UPGRADE" flag will be set.

            yong.fan nasf (Inactive) added a comment - Initial OI scrub will auto start every time when the MDT mount, and scan very small files set which needed by the mount processing, during such scan, the SF_UPGRADE may be set, depends on whether it is upgrade from lustre-1.8 or not. The background OI scrub is conditional triggered when MDT mount, if the administrator does not specify "-o noscrub" when mount, and: 1) former OI scrub crashed, or 2) former OI scrub paused, or 3) detect file-level backup/restore, or 4) detect upgrading from lustre-1.8 Then the background OI scrub will be triggered. And for the last case (4), the "SF_UPGRADE" flag will be set.

            Will scrub automatically start and set the SF_UPGRADE flag at MDT mount time?

            adilger Andreas Dilger added a comment - Will scrub automatically start and set the SF_UPGRADE flag at MDT mount time?

            OI scrub will auto detect the flag "SF_UPGRADE" in the file "OI_scrub" to decide whether allow IGIF lookup directly or not.

            The flag "SF_UPGRADE" will be set when OI scrub auto detect the upgrading (can by initial OI scrub or by normal background OI scrub), and will be cleared when the (background) OI scrub finished the upgrading process (including IGIF-in-LMA, IGIF-in-OI, IGIF-in-dirent (optional)); if the upgrading is paused or stopped, then the flag "SF_UPGRADE" will NOT be cleared.

            For new formatted MDT with Lustre-2.x, OI scrub cannot find the upgrading, so will NOT set the flag "SF_UPGRADE".

            Does it answer your concerned issues?

            yong.fan nasf (Inactive) added a comment - OI scrub will auto detect the flag "SF_UPGRADE" in the file "OI_scrub" to decide whether allow IGIF lookup directly or not. The flag "SF_UPGRADE" will be set when OI scrub auto detect the upgrading (can by initial OI scrub or by normal background OI scrub), and will be cleared when the (background) OI scrub finished the upgrading process (including IGIF-in-LMA, IGIF-in-OI, IGIF-in-dirent (optional)); if the upgrading is paused or stopped, then the flag "SF_UPGRADE" will NOT be cleared. For new formatted MDT with Lustre-2.x, OI scrub cannot find the upgrading, so will NOT set the flag "SF_UPGRADE". Does it answer your concerned issues?

            I'm going to reopen this until I understand the situation a bit better.

            I'm not sure I trust that the ability to allow/deny IGIF FIDs can or should be decided dynamically at each startup. How does it know each time whether the upgrade and LFSCK has been completed? How does it know that LFSCK was never run, but the filesystem was formatted with 2.x or later?

            adilger Andreas Dilger added a comment - I'm going to reopen this until I understand the situation a bit better. I'm not sure I trust that the ability to allow/deny IGIF FIDs can or should be decided dynamically at each startup. How does it know each time whether the upgrade and LFSCK has been completed? How does it know that LFSCK was never run, but the filesystem was formatted with 2.x or later?
            yong.fan nasf (Inactive) added a comment - - edited

            I did not introduce new flags "OBD_COMPAT_NOIGIF", instead, just process the IGIF lookup inside OSD as following:

            For the MDT upgraded from 1.8 device, OI scrub will immobilize the IGIF and insert the mapping for "IGIF <=> ino#" into the OI file. There are three cases to be processed:

            1) After the upgrading, all IGIFs have been processed and inserted into OI files. Then any IGIF object lookup should be processed as normal FID does, means via OI. If no entry in the OI file, then the IGIF object does not exist.

            2) During the upgrading, some IGIFs may be already in OI files, others may be not yet. Under such case, lookup an IGIF in OI files may return -ENOENT, but it does not means the IGIF object does not exist. Since the new immobilize IGIF is the same as the original one before backup/restore, OSD will generate local identifier from IGIF directly as old 2.x did.

            3) The upgrading is paused, and backup/restore. Means upgrading from 1.8 may be partly processed, but some clients may hold some immobilized IGIFs, and use them to access related objects. Under such case, OSD does not know whether an given IGIF has been processed or to be processed, and it also cannot generate local ino#/gen# directly from the immobilized IGIF because of the backup/restore. Then force OSD to lookup the given IGIF in OI files, and if no entry, then ask the client to retry after upgrading completed. No better choice.

            This is the patch:
            http://review.whamcloud.com/#change,6254

            yong.fan nasf (Inactive) added a comment - - edited I did not introduce new flags "OBD_COMPAT_NOIGIF", instead, just process the IGIF lookup inside OSD as following: For the MDT upgraded from 1.8 device, OI scrub will immobilize the IGIF and insert the mapping for "IGIF <=> ino#" into the OI file. There are three cases to be processed: 1) After the upgrading, all IGIFs have been processed and inserted into OI files. Then any IGIF object lookup should be processed as normal FID does, means via OI. If no entry in the OI file, then the IGIF object does not exist. 2) During the upgrading, some IGIFs may be already in OI files, others may be not yet. Under such case, lookup an IGIF in OI files may return -ENOENT, but it does not means the IGIF object does not exist. Since the new immobilize IGIF is the same as the original one before backup/restore, OSD will generate local identifier from IGIF directly as old 2.x did. 3) The upgrading is paused, and backup/restore. Means upgrading from 1.8 may be partly processed, but some clients may hold some immobilized IGIFs, and use them to access related objects. Under such case, OSD does not know whether an given IGIF has been processed or to be processed, and it also cannot generate local ino#/gen# directly from the immobilized IGIF because of the backup/restore. Then force OSD to lookup the given IGIF in OI files, and if no entry, then ask the client to retry after upgrading completed. No better choice. This is the patch: http://review.whamcloud.com/#change,6254

            People

              yong.fan nasf (Inactive)
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: