Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16722

MGS config log restructuring

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • Lustre 2.17.0
    • Lustre 2.16.0
    • 9223372036854775807

    Description

      Restructure the MGS config management system to better handle modern environments.

      Allow the MGS configuration to be handled by some external database (e.g. CTDB or ETCD).

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-16475 Reusing OST indexes after lctl del_ost

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-16738 Improve mount.lustre with many MGS NIDs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-17691 add interface to deregister file system/targets on MGS

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-18164 MGS should dynamically track server NIDs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. LU-10360 use Imperative Recovery logs for client->MDT/OST connections

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Technical task - A technical task. LU-17819 LMR1a: Replicate MGS Services to Multiple MDS

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Technical task - A technical task. LU-18823 LMR1b: Replicate MDT0000 Services to Other MDS

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-13306 allow clients to accept mgs_nidtbl_entry with IPv6 NIDs

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-13308 update changelog_ext_nid to handle IPv6 addresses

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-16722] MGS config log restructuring
            adilger Andreas Dilger added a comment - - edited

            As yet, no investigation has been done in this area. Open questions for discussion and resoultion include:

            • do clients only connect to a single MGS, but perform some kind of round-robin selection of the target NID to use at mount (maybe using their own NID to seed the starting offset), to distribute load among the MGS instances?
            • to servers connect to all MGS NIDs so that the target NID list is kept updated?
            • how are config logs replicated between MGT instances?
              • Using FLR layouts for the files on the MGT would re-use existing infrastructure, and give a clear indication of which is the primary mirror and which mirrors might be "stale" and need to be resync'd from the primary. On the flip side, this might complicate the replication.
              • Using a llog consumer on the backup MGTs to read the config logs and store locally (as the MDT and OST llog copies are handled) would give more independence between the MGTs
            • are the "backup" MGTs read-only, or could one of them take over in case the primary MGT fails?  Or should there be a manual resync process from any of the backup MGTs if the primary is corrupted?  There are not very many files on the MGT, so doing a full reformat/resync is relatively simple.
                 14  100644 (1)      0      0   12288 25-Nov-2023 00:50 mountdata
     82  100644 (1)      0      0    8192 31-Dec-1969 17:00 nodemap
     85  100644 (1)      0      0   11032 31-Dec-1969 17:00 params
     86  100644 (1)      0      0   22808 31-Dec-1969 17:00 testfs-client
     87  100644 (1)      0      0   28552 31-Dec-1969 17:00 testfs-MDT0000
    154  100644 (1)      0      0   28016 31-Dec-1969 17:00 testfs-MDT0001
    156  100644 (1)      0      0   27480 31-Dec-1969 17:00 testfs-MDT0002
    158  100644 (1)      0      0   26944 31-Dec-1969 17:00 testfs-MDT0003
    160  100644 (1)      0      0   17992 31-Dec-1969 17:00 testfs-OST0000
    166  100644 (1)      0      0   17680 31-Dec-1969 17:00 testfs-OST0001
    167  100644 (1)      0      0   17144 31-Dec-1969 17:00 testfs-OST0002
    168  100644 (1)      0      0   16608 31-Dec-1969 17:00 testfs-OST0003
            adilger Andreas Dilger added a comment - - edited As yet, no investigation has been done in this area. Open questions for discussion and resoultion include: do clients only connect to a single MGS, but perform some kind of round-robin selection of the target NID to use at mount (maybe using their own NID to seed the starting offset), to distribute load among the MGS instances? to servers connect to all MGS NIDs so that the target NID list is kept updated? how are config logs replicated between MGT instances? Using FLR layouts for the files on the MGT would re-use existing infrastructure, and give a clear indication of which is the primary mirror and which mirrors might be "stale" and need to be resync'd from the primary. On the flip side, this might complicate the replication. Using a llog consumer on the backup MGTs to read the config logs and store locally (as the MDT and OST llog copies are handled) would give more independence between the MGTs are the "backup" MGTs read-only, or could one of them take over in case the primary MGT fails?  Or should there be a manual resync process from any of the backup MGTs if the primary is corrupted?  There are not very many files on the MGT, so doing a full reformat/resync is relatively simple.      14  100644 (1)      0      0   12288 25-Nov-2023 00:50 mountdata      82  100644 (1)      0      0    8192 31-Dec-1969 17:00 nodemap      85  100644 (1)      0      0   11032 31-Dec-1969 17:00 params      86  100644 (1)      0      0   22808 31-Dec-1969 17:00 testfs-client      87  100644 (1)      0      0   28552 31-Dec-1969 17:00 testfs-MDT0000     154  100644 (1)      0      0   28016 31-Dec-1969 17:00 testfs-MDT0001     156  100644 (1)      0      0   27480 31-Dec-1969 17:00 testfs-MDT0002     158  100644 (1)      0      0   26944 31-Dec-1969 17:00 testfs-MDT0003     160  100644 (1)      0      0   17992 31-Dec-1969 17:00 testfs-OST0000     166  100644 (1)      0      0   17680 31-Dec-1969 17:00 testfs-OST0001     167  100644 (1)      0      0   17144 31-Dec-1969 17:00 testfs-OST0002     168  100644 (1)      0      0   16608 31-Dec-1969 17:00 testfs-OST0003

            People

              adilger Andreas Dilger
              cfaber Colin Faber
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: