[LU-17426] parallel cross-directory rename of regular files on single MDT Created: 16/Jan/24  Updated: 04/Feb/24

Status: Open
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.14.0, Lustre 2.16.0
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Andreas Dilger Assignee: Lai Siyao
Resolution: Unresolved Votes: 0
Labels: performance

Issue Links:
Related
is related to LU-17427 reduce hold time for BFL rename lock Open
is related to LU-12125 Allow parallel rename of regular files Resolved
is related to LU-17446 BL AST should stop resending if lock ... Open
is related to LU-15913 rename stress test leads to REMOTE_PA... Reopened
is related to LU-17434 DNE3: add exclude list for remote sub... Open
is related to LU-17441 move rename RPC handling to MDS_IO_PO... Open
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

It should be possible to allow cross-directory rename of regular files (strictly, any non-directory) on the same MDT without holding the BigFilesystemLock (BFL), as file renames cannot change the directory hierarchy.

That should improve the performance for these rename operations, and reduce contention between local MDT file renames in different parts of the directory tree.

That would leave directory and cross-MDT renames needing the BFL, which should be an increasingly small fraction of all rename operations.

 Comments   
Comment by Oleg Drokin [ 16/Jan/24 ]

the case of renaming a file into itself just because the source and target directory are the same due to a race condition need to be considered I imagine.

Comment by Gerrit Updater [ 18/Jan/24 ]

"Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/53726
Subject: LU-17426 mdt: relax same MDT file rename lock
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8fafececea102c547e59b5ba029c6c88f2bb4490

Comment by Gerrit Updater [ 19/Jan/24 ]

"Andreas Dilger <adilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/53738
Subject: LU-17426 tests: add crossdir parallel rename test
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d27a62f1b4105d5d0d41cd840734d9b7ef29074a

Comment by Gerrit Updater [ 04/Feb/24 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/53726/
Subject: LU-17426 mdt: relax same MDT file rename lock
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e66dd92775a547d755846349bfed97d831194f70

Generated at Sat Feb 10 03:35:21 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.