Details
-
Improvement
-
Resolution: Duplicate
-
Minor
-
None
-
None
-
None
-
9223372036854775807
Description
Originally SOM (Size On MDT) feature wants to achieve a strict size management for Lustre via the concept like I/O epoch. It means that when the file is opened for writing (I/O epoch starts), the size on MDT is not strict correct in the write I/O epoch and strict size should be obtained from Lustre OSTs. The size will change into a strict correct version when the last writer close the file (I/O epoch ends).
However, evict or crash of the client in a cluster might cause incomplete process of closing a file, but the data stripping objects on OST have been updated, thus result in inaccurate SOM.
Correct the size on last close is too complex to implement due to the problem of how to solve the size consistency in the failure and recovery case. Thus, we remove the unfinished work for SOM from Lustre. And implement the LSOM with loose consistency.
LSOM can be converted into a strict correct version especially when there is no any I/O activity on the file. Thus it would better to transmit LSOM with loose consistency into a strict strong consistency version.
Originally combination with LFSCK to correct file size stored on LSOM EA is proposed, but LFSCK can not use the CLIO engine and there is not very easy to obtain the file size from MDT.
A new suggestion is proposed as follows:
- Each time open the file with O_RDWR or O_WRONLY mode for write, MDT adds a changelog record for the writer.
- There is a dedicated Lustre client which is used to scan and consume this kind of Lustre changelog records. For each such record, the client opens the file with lease intent lock. A successful lease open ensures that it is unique opener for this file in the cluster wide. And then the client do stat on the file, update the LSOM and change it into a strong consistency version during the lease close.
- When MDT receives an open request for write, if the LSOM is a LAZY version, do nothing; if the LSOM data is marked with STRICT correct (strong consistency), the MDT will change it into a loose consistent version. Or a better implementation can delay this change until the first write to this file object just like FLR transmits from read-only state to write pending state.
- When a client do stat() on the file, if LSOM data is STRICT correct, can be directly using without any glimpse lock RPC to Lustre OSTs.
Any suggestion is welcome!
