Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20852

FLR-ECRO: immediate close resync

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • Lustre 2.18.0

      One of the core functionalities that is not provided with the initial FLR-ECRO delayed parity implementation is the parity resync engine. The expectation is that there is some external policy engine or scripting that is monitoring the Changelog and calling "lfs mirror resync FILE" for each file that is modified in the changelog (e.g. CLOSE records).

      However, another option has been proposed, which is for the MDS to return a flag to the client in the MDS_CLOSE reply, that indicates to the client that it is the last client to close the file, and no other client has the file open. The client could then call an upcall that executes "lfs mirror resync FILE" on the file (or some smarter proxy script that results in parity resync by pathname or FID) to regenerate the parity "immediately" after the file is closed.

      There are several benefits to doing the parity reconstruction on the client that just wrote the file:

      • it is more likely that the file data is still resident on the client after the write and no additional read is needed to generate the parity (though subject to the file and memory size and memory pressure on that node)
      • the parity could be generated quickly after the file is closed, instead of potentially minutes later when an asynchronous changelog watcher processed the file
      • the parity reconstruction is distributed across all clients writing the files rather than being done on a limited set of nodes
        There are also some potential drawbacks of running the resync on the clients:
      • for very large files the data may no longer be in cache and drive a significant amount of data through the node
      • the resync process may interfere with other application IO or computation (though the parity calculations themselves are relatively fast)
      • files may be opened and closed repeatedly and interfere with parity reconstruction, or make the parity stale before it can be written

            wc-triage WC Triage
            adilger Andreas Dilger
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: