[LU-10948] client cache open lock after N opens

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.9.0
• Environment: cent server/sles client

Epic/Theme

Performance

• Assignee: Oleg Drokin
• Reporter: Bob Ciotti (Inactive)

Description

listed as minor but when a user does this, we start to get phone calls form other users and then page POC to identify code/user. Workaround is to terminate user job(s).

Oleg has said that Lustre has an existing feature for a client to acquire open lock but off by default. This to mimic NFS behavior.

Ideal change would be that we can specify a number of times that a file is opened on single client at which time lock is acquired. (e.g. 10th time)

Use case is naive user who loop in this way on like 5000+ threads in java:

do until till the sun turns black()

{ fd = open(*my_thread_ID, O_APPEND) calculate_something_small_but_useful() write(fd, *fortytwo, 42) close(fd }

Users often don't have complete control over the code they run and as a result may not be able to quickly make even simple changes.

Issue Links

• is duplicated by: LU-5426 Add more controls for open file cache
• is related to: LU-12262 Improve sbi_flags checking
• is related to: LU-14694 racer timeouts with LU-10948
• is related to: LU-14743 Interop: sanity test 429 fails with 'multiop failed'
• is related to: LU-16463 llapi_open_by_fid open should not be cached systematically
• is related to: LU-10957 Return more lock bits from MDS for open-create
• is related to: LU-11623 Allow caching of open-created dentries
• is related to: LU-7915 investigate heuristics for SPARK client getting MDS openlock
• is related to: LU-4398 mdt_object_open_lock() may not flush conflicting handles
• is related to: LU-10269 Fixes for selective trybits
• is related to: LU-11199 mdsrate open() performance degradation
• is related to: LU-8585 All Lustre test suites should pass with subdirectory mount
• is related to: LU-10955 Cancel open lock as part of close request
• is related to: LU-7915 investigate heuristics for SPARK client getting MDS openlock

Comments

[Andreas Dilger]

Patch https://review.whamcloud.com/32156 "LU-10948 mdt: Always return lookup lock on opens" is not landed yet.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43907/
Subject: LU-10948 mdt: New connect flag for non-open-by-fid lock request
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 72c9a6e5fb6e11fca1b1438ac18f58ff7849ed7d

[Gerrit Updater]

Oleg Drokin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43907
Subject: LU-10948 mdt: New connect flag for non-open-by-fid lock request
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f6acd02aa06420264b83e4ef2d4bf2f0f95794c7

[Gerrit Updater]

Oleg Drokin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43903
Subject: LU-10948 mdt: New connect flag for non-open-by-fid lock request
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 14edc80c617871e550d41a0f2e0b0f1491e02f37

[Jay Lan (Inactive)]

Can you port #32158 to b2_12? Thanks!

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32158/
Subject: LU-10948 llite: Introduce inode open heat counter
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 41d99c4902836b7265db946dfa49cf99381f0db4

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/41091/
Subject: LU-10948 llite: Revalidate dentries in ll_intent_file_open
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 6ad2679352ee9a14eceb8c70b37df084bc07a6f7