Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-3336

LFSCK II: MDT-OST OST orphan handling

Details

    • New Feature
    • Resolution: Fixed
    • Critical
    • Lustre 2.6.0
    • Lustre 2.6.0
    • 89
    • 8252

    Description

      This ticket relates to handling of OST orphan cleanup. This includes:

      • create an in-memory bitmap for OST FIDs that have been referenced during MDS Pass 1 checking
      • after the OST Pass 2 scanning is complete, any OST objects that have not been referenced need to be verified with the MDT, and added to lost+found or unlinked (depending on administrator preference)

      Attachments

        Issue Links

          is blocked by

          Technical task - A technical task. LU-3469 OSP dt_sync() operation should flush pending destroys and other updates

          • Blocker - Blocks development and/or testing work, production could not run.
          • Resolved
          is blocking

          New Feature - A new feature of the product, which has yet to be developed. LU-1267 LFSCK II: MDT-OST consistency check/repair

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-4670 suite sanity test_222a test223: f223.sanity1: No such file or directory

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-11302 schedule while atomic with lfsck code

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            [LU-3336] LFSCK II: MDT-OST OST orphan handling
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-11302 [ LU-11302 ]
            yong.fan nasf (Inactive) made changes -
            Resolution New: Fixed [ 1 ]
            Status Original: In Progress [ 3 ] New: Resolved [ 5 ]
            yong.fan nasf (Inactive) made changes -
            Status Original: Open [ 1 ] New: In Progress [ 3 ]
            yong.fan nasf (Inactive) made changes -
            Status Original: In Progress [ 3 ] New: Open [ 1 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-4670 [ LU-4670 ]
            yong.fan nasf (Inactive) made changes -
            Status Original: Open [ 1 ] New: In Progress [ 3 ]
            adilger Andreas Dilger made changes -
            Fix Version/s New: Lustre 2.6.0 [ 10595 ]
            adilger Andreas Dilger made changes -
            Affects Version/s New: Lustre 2.6.0 [ 10595 ]
            Affects Version/s Original: Lustre 2.5.0 [ 10295 ]
            rhenwood Richard Henwood (Inactive) made changes -
            Link New: This issue is blocked by LU-3469 [ LU-3469 ]
            adilger Andreas Dilger made changes -
            Description Original: In Lustre, for striped (non-zero striped) file, the layout information for each OST-object is recorded as extended attributes (XATTR_NAME_LOV) in its MDT-object on MDT. Such EA contains the OST index, OID or FID of the OST-object, and so on. On OST-side, each OST-object records the information (MDT-object FID) that indicates which file the OST object belongs to. Over the lifetime of an active filesystem, the layout information in MDT-object's EA may be inconsistent with the information on OST. There are several inconsistent cases as following:

            1. OST-object1 is marked as part of file1 on MDT, but OST-object1 is unassigned or uninitialized on OST.
            2. OST-object1 is marked as part of file1 on OST, but there is no record for OST-object1 on MDT.
            3. OST-object1 is marked as part of file1 on OST, but MDT records that it belongs to file2.
            4. OST-object1 is marked as part of file1 on OST, but both file1 and file2 on MDT claim that it owns OST-object1.

            These inconsistent cases will misguide client/MDS when access related OST-objects, waste space, lose data, even destroy all the system. In LFSCK phase II, we will implement an online tool to check and repair the file layout inconsistency. Such tool will use the inode iterator implemented in LFSCK phase I to scan the whole system, and can be driven together with other LFSCK components.

            On the other hand, the owner information, UID and GID of OST-objects will also be verified to match that of the MDT-object, the inconsistent ones will be fixed, to ensure correct quota allocation.             		New: This ticket relates to handling of OST orphan cleanup. This includes:

            - create an in-memory bitmap for OST FIDs that have been referenced during MDS Pass 1 checking
            - after the OST Pass 2 scanning is complete, any OST objects that have not been referenced need to be verified with the MDT, and added to {{lost+found}} or unlinked (depending on administrator preference)

            People

              yong.fan nasf (Inactive)
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: