Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16972

optimize e2fsck ea_refcount processing

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • None
    • None
    • Exascaler version: 5.2.5
    • 3
    • 9223372036854775807

    Description

      Filesystems with a large number of external xattr blocks, such as MDTs with large PFL layouts that overflow the in-inode xattr space, cause a significant slowdown during pass1 inode processing because of inefficient management of the refcount_extra list that stores the refcounts for shared external xattr blocks.

      The refcount_extra structure is implemented as a linear array of ea_refcount_el elements:

      struct ea_refcount_el { 
              /* ea_key could either be an inode number or block number. */
              ea_key_t        ea_key;
              ea_value_t      ea_value;
      }; 
      

      During element insertion, a binary search is used in the array to find the element, and if not found, the higher elements in the array are memmove()'d upward to make room for the new element in the list. If the currently allocated array is full, then it is realloc()'d with 100 more elements, and the whole array is copied to the new memory.

      If there are a large number of shared xattr blocks then the array can grow very large and the processing of the linear array will be very inefficient and CPU intensive due to frequent large memory allocations and copies.

      Attachments

        Activity

          People

            dongyang Dongyang Li
            adilger Andreas Dilger
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: