Massive directory metadata operation performance decrease (LU-14146)

[LU-15529] optimize directory migration parent locking Created: 07/Feb/22  Updated: 24/May/23  Resolved: 12/May/23

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.16.0

Type: Technical task Priority: Minor
Reporter: Lai Siyao Assignee: Lai Siyao
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Related
Rank (Obsolete): 9223372036854775807

 Description   

currently directory migration locks all stripes of parent directory, it can be changed to lock source and target parent stripe only, and with the first change in LU-15526, the lock conflicts can be largely reduced, this can improve directory migration/restripe/auto split performance.



 Comments   
Comment by Lai Siyao [ 19/Feb/22 ]

patch is on https://review.whamcloud.com/#/c/40891/

Comment by Gerrit Updater [ 10/May/23 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40891/
Subject: LU-15529 mdt: optimize dir migration locking
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1d6b96a1cf0468bc81949960aa649cde8f927008

Comment by Peter Jones [ 12/May/23 ]

Landed for 2.16

Comment by Alex Zhuravlev [ 23/May/23 ]

started to hit an error during sanityn using clean master branch:

== sanityn test complete, duration 197 sec =============== 19:33:43 (1684870423)
rm: cannot remove '/mnt/lustre/d80b.sanityn/migrate_dir': Directory not empty

bisection:

COMMIT		TESTED	PASSED	FAILED		COMMIT DESCRIPTION
1d6b96a1cf      1       0       1       BAD     LU-15529 mdt: optimize dir migration locking
c2e5181fe1      10      10      0       GOOD    LU-16221 build: modify kmodtool for rhel9
Comment by Lai Siyao [ 24/May/23 ]

I'm looking into it.

Comment by Alex Zhuravlev [ 24/May/23 ]

the good thing is that sanityn/80b seem to be working well now.

Generated at Sat Feb 10 03:19:05 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.