Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14958

configurable hash table size for jbd2

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • Lustre 2.16.0, Lustre 2.15.3
    • None
    • None
    • 9223372036854775807

    Description

      revoke hash table can be enourmous with multi-GB journal. this may result in millions of revoke records which are loaded and inserted into specific hashtable during journal replay.
      currently the revoke hashtable's size is hard-coded as 256, thus every slot may get too many records.

      a simple benchmark of that code:
      1048576 - 95 seconds
      2097152 - 580 seconds
      in the fields it can be upto 30M records to find/insert.

      with 8192 buckets in the hash table:
      4194304 - 59 seconds
      8388608 - 247 seconds

      Attachments

        1. dumpe2fs.txt
          2 kB
        2. jbd2_debugfs.gz
          106.24 MB

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-18161 add/fix e2fsck journal hash table for revoke records

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-17117 debugfs should be able to disable MMP directly

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-14958] configurable hash table size for jbd2
            adilger Andreas Dilger made changes -
            Link New: This issue is related to EX-11412 [ EX-11412 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to EX-10081 [ EX-10081 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-18161 [ LU-18161 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-17117 [ LU-17117 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to DDN-4247 [ DDN-4247 ]
            pjones Peter Jones made changes -
            Resolution New: Fixed [ 1 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            pjones Peter Jones made changes -
            Fix Version/s New: Lustre 2.15.3 [ 15998 ]
            pjones Peter Jones made changes -
            Fix Version/s New: Lustre 2.16.0 [ 15190 ]
            pjones Peter Jones made changes -
            Link New: This issue is related to EX-5786 [ EX-5786 ]
            eaujames Etienne Aujames made changes -
            Attachment New: dumpe2fs.txt [ 41381 ]

            People

              bzzz Alex Zhuravlev
              bzzz Alex Zhuravlev
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: