[LU-2919] Exclusive open implementation

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.5.0
• Fix Version/s: Lustre 2.5.0
• Type: Technical task
• Priority: Critical
• Reporter: Jinshan Xiong (Inactive)
• Assignee: Jinshan Xiong (Inactive)
• Resolution: Fixed
• Labels: HSM

Description

Proposed way to do exclusive open:

0. First of all, we have to perform most of the work in kernel space;
1. Client exclusively opens the file with IT_RELEASE_OPEN.
1.1 exclusive open means the open will fail if the file is being opened by somebody else;

2. the MDT will handle IT_RELEASE as follows:
2.1 Revoke OPEN_LOCK on this file, just request EX mode of MDS_INODELOCK_OPEN lock and release it;
2.2 Acquire a rwsem, say open_rwsem, with write mode before trying to the file; for the normal open, it should acquire read mode of open_rwsem;
2.3 Check if the file is already being opened by others, if not return with -EBUSY;
2.4 Check if the file can be released;
2.5 Set a special flag in struct mdt_file_data to mark this open is exclusive;
2.5 Release open rwsem.

From now on, if the file is opened by others, it will mark mdt_file_data that the exclusive open is broken.

3. Client: if IT_RELEASE_OPEN is finished successfully, and do followings:
3.1 Acquire full PW or EX extent lock to flush dirty cache;
3.2 Pack the handle of layout lock, data version and other attars;
3.3 Close the file with IT_RELEASE_CLOSE.

4. Back to MDT to handle IT_RELEASE CLOSE:
4.1 Grab the open_rwsem
4.2 Check if the exclusive open has ever been broken, in that case, the RELEASE process will fail;
4.3 Verify the data version matches archive;
4.4 Grab EX layout lock
4.5 Swap the layout
4.6 Release EX layout lock
4.7 Close the exclusive open

Basically we avoid granting EX layout lock back to client and introduce exclusive open so that we know it if the file has ever being accessed. I hope this can simplify things a little bit. Also, exclusive open can be used to implement file lease.

And a bugfix for recovery:

I'd like to update the special handling of replaying exclusive open request.

There is a bug in current mode as follows:
1. Client 1 does exclusive open
2. Client 2 does open and close, so it will break the exclusive open above
3. MDT crashes and restarts
4. Client 1 replays exclusive open request and will lose the information that it has already been broken

The root cause of this bug is that we don't store the exclusive open bit in persistent storage so we can't get it back over recovery.

The fix is to check on the MDT side whenever the exclusive open is being replayed, it will be marked as broken all the time.

Issue Links

• is related to: LU-3608 HSM Master Ticket for 2.5 Landings

Comments

[Aurelien Degremont (Inactive)]

As Johann suggested, I've push a prototype of this patch, extract from my work on Release patch.
We know this patch is not perfect, but Johann thought it could useful for Jinshan.

http://review.whamcloud.com/5956

[Jinshan Xiong (Inactive)]

Another implementation is at http://review.whamcloud.com/6730 which is framework of lustre file lease but right now only excl open is supported.