[LU-10661] Persistent open counter

Wed, 16 May 2018 02:35:27 +0000

In current FLR implementation, in order to resync a stale component, client has to find and copy all data covering by the extent of this component. If the mirrored file is large and stale component has very wide extent, it would cause significant amount of data move even though only a tiny region was written.

It would be good if clients could report the region they have written in the last close RPC. The written extents wouldn't be accurate, instead they just give a rough idea about which regions of the file have been changed. The client should be able to merge and alter writing extent so that the extents sent to the MDT would be limited to a small number, for example, the MDT could allow up to 4 items of extent to be sent by close RPC.

There is a problem for evicted clients because they could no longer send writing extents. The MDT needs to know this situation and then safely fall back to full component resync. Consistent open counter is proposed to solve this problem.

With consistent open counters, the MDT will write the files that have opened into a persistent storage. If the file with persistent open counter is closed successfully, the counters should reach zero at last. If there exists evicted clients opening this file, then the MDT will know it because there is no open handlers for this file in memory, but persistent counter is greater than zero.

Maintaining open counters has huge overhead because now open will always produce a sync transaction. This can be optimized. When the MDT receives the first write intent RPC to modify a mirrored file, it will enable persistent open counter for this particular file. From that time on, all openers to this file will have to bear with a sync trans.

Whamcloud Community JIRA

[LU-10661] Persistent open counter