Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10329

DNE3: REMOTE_PARENT_DIR scalability

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Lustre 2.14.0
    • Labels:
    • Rank (Obsolete):
      9223372036854775807

      Description

      For DNE filesystems where there are large numbers of remote entries, for example once LU-4684 is landed to restripe directories, or DNE2 striped directories with many renames, the size of the REMOTE_PARENT_DIR may become very large.

      In order to limit contention and scaling issues in REMOTE_PARENT_DIR it makes sense to have multiple such directories. As a starting point, one REMOTE_PARENT_DIR_MDTxxxx for each remote MDT would be useful, but it may be necessary to have a tree of directories similar to the O/<seq>/dN object directories. Having a separate REMOTE_PARENT_DIR_MDTxxxx per MDT would also allow LFSCK to efficiently scan remote entries for a given MDT, if there was a problem (e.g. MDT was marked offline and returned into the namespace later).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                laisiyao Lai Siyao
                Reporter:
                adilger Andreas Dilger
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: