Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14146 Massive directory metadata operation performance decrease
  3. LU-15529

optimize directory migration parent locking

Details

    • Technical task
    • Resolution: Fixed
    • Minor
    • Lustre 2.16.0
    • None
    • None
    • 9223372036854775807

    Description

      currently directory migration locks all stripes of parent directory, it can be changed to lock source and target parent stripe only, and with the first change in LU-15526, the lock conflicts can be largely reduced, this can improve directory migration/restripe/auto split performance.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-15977 sanityn test_80b: migration stopped 2

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Technical task - A technical task. LU-17821 LMR1c: Improve DNE Distributed Transaction Performance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            [LU-15529] optimize directory migration parent locking
            bzzz Alex Zhuravlev added a comment -

            the good thing is that sanityn/80b seem to be working well now.

            bzzz Alex Zhuravlev added a comment - the good thing is that sanityn/80b seem to be working well now.
            laisiyao Lai Siyao added a comment -

            I'm looking into it.

            laisiyao Lai Siyao added a comment - I'm looking into it.
            bzzz Alex Zhuravlev added a comment - - edited

            started to hit an error during sanityn using clean master branch:

            == sanityn test complete, duration 197 sec =============== 19:33:43 (1684870423)
rm: cannot remove '/mnt/lustre/d80b.sanityn/migrate_dir': Directory not empty

            bisection:

            COMMIT		TESTED	PASSED	FAILED		COMMIT DESCRIPTION
1d6b96a1cf      1       0       1       BAD     LU-15529 mdt: optimize dir migration locking
c2e5181fe1      10      10      0       GOOD    LU-16221 build: modify kmodtool for rhel9
            bzzz Alex Zhuravlev added a comment - - edited started to hit an error during sanityn using clean master branch: == sanityn test complete, duration 197 sec =============== 19:33:43 (1684870423) rm: cannot remove '/mnt/lustre/d80b.sanityn/migrate_dir' : Directory not empty bisection: COMMIT TESTED PASSED FAILED COMMIT DESCRIPTION 1d6b96a1cf 1 0 1 BAD LU-15529 mdt: optimize dir migration locking c2e5181fe1 10 10 0 GOOD LU-16221 build: modify kmodtool for rhel9
            pjones Peter Jones added a comment -

            Landed for 2.16

            pjones Peter Jones added a comment - Landed for 2.16
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40891/
            Subject: LU-15529 mdt: optimize dir migration locking
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 1d6b96a1cf0468bc81949960aa649cde8f927008

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40891/ Subject: LU-15529 mdt: optimize dir migration locking Project: fs/lustre-release Branch: master Current Patch Set: Commit: 1d6b96a1cf0468bc81949960aa649cde8f927008
            laisiyao Lai Siyao added a comment -

            patch is on https://review.whamcloud.com/#/c/40891/

            laisiyao Lai Siyao added a comment - patch is on https://review.whamcloud.com/#/c/40891/

            People

              laisiyao Lai Siyao
              laisiyao Lai Siyao
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: