Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-8465

parallel e2fsck performance at scale

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Major
    • Lustre 2.14.0
    • None
    • None
    • 9223372036854775807

    Description

      Mentioned in LU-7592 by Andreas Dilger

      e2fsck performance will become an issue at this scale, and it would likely need to be parallelized to be able to complete in a reasonable time. It could reasonably expect multiple disks at this scale, so having larger numbers of IOs in flight would help, as would an event-driven model with aio that generates lists of blocks to check (itable blocks first), submits them to disk, and then processes them as they are read, generating more blocks to read (more itable blocks, indirect/index/xattr/directory blocks, etc), repeat.

      Attachments

        1. vault20-pFSCK-domingo.pdf
          616 kB
        2. test_max_less_equal.c
          8 kB
        3. pfsck-test.xlsx
          18 kB
        4. parallel_e2fsck.pdf
          1.01 MB
        5. 128_threads.txt
          42 kB

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-5949 Verify e2fsck inode badness is working for duplicate blocks

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-14126 parallel e2fsck does not work well with MMP

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-14894 Parallel pass2 support for e2fsck

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-14213 enable parallel e2fsck by default

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-14679 parallel e2fsck for pass2 directory scanning

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-11882 OST recreated objects gets badness mark from e2fsck

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. LU-12051 ldiskfs directory shrink

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-7370 improve e2fsck lost+found insert performance

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-7592 Сhange force_over_128tb lustre mount option to force_over_256b for ldiskfs

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-8974 Сhange force_over_256tb lustre mount option to force_over_512tb for ldiskfs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              wshilong Wang Shilong (Inactive)
              artem_blagodarenko Artem Blagodarenko (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: