Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20893

FTM: MGS followers on nodes serving targets of multiple filesystems

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • 3
    • 9223372036854775807

      An MGS follower (patch https://review.whamcloud.com/64202 ("LU-19916 mgs: start MGS follower on MDTs/OSTs")) only has replicas of the config logs that the MGC feeding it copies into the local target's CONFIGS/, i.e. those of the filesystem(s) with a target on that node. It accepts any client MGC that connects with OBD_CONNECT_FTM, regardless of which filesystem the client mounts.

      When one MGS0 serves several filesystems, and a server node hosts targets of only some of them, a client of another filesystem whose MGC fails over to that node's NID is accepted by the follower:

      • a new mount fails when mgs_llog_open() of <fsname>-client returns -ENOENT, where before the follower existed the connect was refused and the MGC moved on to the next NID;
      • an already-mounted client is granted the config lock with no log behind it, and stays on the follower after MGS0 returns (MGC imports do not fail back), so it misses later config changes for that filesystem.

      The follower should refuse service for a filesystem it has no replica of. This cannot simply be done at connect time: a client's MGC is named by the MGS NID (MGC<nid>) and is shared by every filesystem the client mounts from that MGS, and the MGS connect carries no fsname. The refusal has to happen per config lock enqueue or llog request, with an error that makes the client MGC reconnect to another MGS NID, and has to cope with one MGC serving both a filesystem the follower has and one it does not.

      Related design questions for MGS followers on nodes serving targets of multiple filesystems:

      • only one MGS (MGS0 or a follower) runs per node, so whether a follower started on an FS1 target should also hold log replicas for FS2;
      • whether the follower needs to access multiple local targets at once (one per fsname) to serve each filesystem's logs;
      • how config lock revocation and IR entries for each filesystem reach the follower's clients.

      Hosting several separate filesystems on one server is uncommon, and MGS followers are off by default (mount option "mgs_follower", patch https://review.whamcloud.com/68474 ("LU-19916 mgs: do not offer a target with no MGS as an MGS")), so this is not on the critical path for LU-19916.

            wc-triage WC Triage
            adilger Andreas Dilger
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: