[LU-15414] FLDB mirroring

Related — Sun, 12 Mar 2023 10:17:13 +0000

For reliability, it would be desirable to replicate the FLDB file across multiple MDTs, in case the FLDB file on MDT0000 is lost or corrupted. Since the FLDB itself is changing very rarely (only when new MDTs or OSTs are added to the filesystem, or 4B SEQ numbers have been allocated by one target), there should not be any noticeable performance overhad from having multiple mirrors.

Since the FLDB will almost always be in sync across MDTs, it would be possible for the clients/servers to contact any MDT with an FLDB replica, and only query the MDT0000 FLDB copy if the requested SEQ number could not be located on the other MDT.

When there are many MDTs, it may be impractical to have an FLDB copy on every MDT in the filesystem, so it makes sense to (deterministically) have FLDB copies only on a subseet of MDTs, such as having backups on MDT0001, MDT0003, MDT0005, MDT0007, MDT0009, MDTxxxx where x=3 ⁿ , 5 ⁿ , 7 ⁿ (like ext4 superblock copies). This would provide one backup for 2 MDTs, 2 backups for 4 MDTs, 3 for 8 MDTs, ..., up to 23 replicas with 65536 MDTs. One drawback of this mechanism is that the replicas may not be available if the MDTs are not densely numbered, but that is a very uncommon configuration, and almost certainly MDT0000 and MDT0001 should be available.

Whamcloud Community JIRA

[LU-15414] FLDB mirroring