Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-11510

preserve PFL/FLR/DoM layout with lfs_migrate

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.14.0, Lustre 2.12.5
    • Lustre 2.11.0, Lustre 2.12.0
    • None
    • 3
    • 9223372036854775807

    Description

      The lfs_migrate tool does not currently preserve the PFL/FLR/DoM components of the layout when a file is migrated. By default, it will get the stripe_count and stripe_size of the file (maybe pool and mirror_count if https://review.whamcloud.com/32977 is updated to do so), but it does not preserve more complex layout details.

      Dropping the initial components of a PFL file is probably desirable, since it is unlikely that a file will become smaller, and it makes sense to keep the stripe_count of a file at the widest needed.

      We do need to keep the number of mirrors across migration, and it would probably be good to keep any OST pools per mirror, if they exist.

      Separately, it would be useful to have some way to migrate only the mirror(s) that exist on particular OSTs if we are trying to empty particular OSTs.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-13703 fix lfs_migrate to work with PFL arguments

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-9629 lfs migrate does not work as a non-root user

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-10801 argument parsing in lfs_migrate is broken

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-9392 lfs migrate -o and lfs setstripe -o should pick OST from ost_list

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-9982 Clients striping from mapped FID in nodemap

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            [LU-11510] preserve PFL/FLR/DoM layout with lfs_migrate
            gerrit Gerrit Updater added a comment -

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/37814/
            Subject: LU-11510 lfs: migrate a composite layout file correctly
            Project: fs/lustre-release
            Branch: b2_12
            Current Patch Set:
            Commit: dc88cb0843e57be3930558e4e1b5c9b5d3af3311

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/37814/ Subject: LU-11510 lfs: migrate a composite layout file correctly Project: fs/lustre-release Branch: b2_12 Current Patch Set: Commit: dc88cb0843e57be3930558e4e1b5c9b5d3af3311
            gerrit Gerrit Updater added a comment -

            Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37814
            Subject: LU-11510 lfs: migrate a composite layout file correctly
            Project: fs/lustre-release
            Branch: b2_12
            Current Patch Set: 1
            Commit: fdb231d505dda018515f754475527041e67af7c8

            gerrit Gerrit Updater added a comment - Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37814 Subject: LU-11510 lfs: migrate a composite layout file correctly Project: fs/lustre-release Branch: b2_12 Current Patch Set: 1 Commit: fdb231d505dda018515f754475527041e67af7c8
            pjones Peter Jones added a comment -

            Landed for 2.14

            pjones Peter Jones added a comment - Landed for 2.14
            gerrit Gerrit Updater added a comment -

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36082/
            Subject: LU-11510 lfs: migrate a composite layout file correctly
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 8bedfa377fbd0c9f1b6ea2c40d36fdcaa52137df

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36082/ Subject: LU-11510 lfs: migrate a composite layout file correctly Project: fs/lustre-release Branch: master Current Patch Set: Commit: 8bedfa377fbd0c9f1b6ea2c40d36fdcaa52137df
            emoly.liu Emoly Liu added a comment -

            implementing the LU-9392 functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins

            This is close to my idea.

            emoly.liu Emoly Liu added a comment - implementing the LU-9392 functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins This is close to my idea.
            adilger Andreas Dilger added a comment -

            the next step is to move the file to the new OSTs by adding an option to lfs_migrate tool to let the customer specify the OST list.

            Normally when doing large-scale migration, the OST selection would be handled by the MDS by disabling object creation on some OSTs (via "lctl set_param osp.<fsname>-OST[xxxx-yyyy].max_create_count=0") and running "lfs find -ost xxxx,...,yyyy" to list all of the files on those OSTs to migrate. I think the common case is finding files on OSTs that shouldn't be used, and the MDS QOS policy will already bias file allocation onto OSTs with more free space.

            I think would be useful for making easier file layout policies for users are:

            • using "lfs setstripe -copy XXX" to select a template layout from an existing file/directory (this should already work with the new patch), but allowing an "alias" like -copy=@mypattern that looks for the file "$HOME/.lfs/mypattern" or similar
            • finishing patch https://review.whamcloud.com/28972 "LU-9982 lustre: Clients striping from mapped FID in nodemap" so that it is possible to set a per-fileset default layout
            • implementing the LU-9392 functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins
            adilger Andreas Dilger added a comment - the next step is to move the file to the new OSTs by adding an option to lfs_migrate tool to let the customer specify the OST list. Normally when doing large-scale migration, the OST selection would be handled by the MDS by disabling object creation on some OSTs (via "lctl set_param osp.<fsname>-OST [xxxx-yyyy] .max_create_count=0") and running "lfs find -ost xxxx,...,yyyy" to list all of the files on those OSTs to migrate. I think the common case is finding files on OSTs that shouldn't be used, and the MDS QOS policy will already bias file allocation onto OSTs with more free space. I think would be useful for making easier file layout policies for users are: using " lfs setstripe - copy XXX " to select a template layout from an existing file/directory (this should already work with the new patch), but allowing an "alias" like -copy=@mypattern that looks for the file " $HOME/.lfs/mypattern " or similar finishing patch https://review.whamcloud.com/28972 " LU-9982 lustre: Clients striping from mapped FID in nodemap " so that it is possible to set a per-fileset default layout implementing the LU-9392 functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins
            gerrit Gerrit Updater added a comment - - edited

            Emoly Liu (emoly@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37401
            Subject: LU-11510 lfs: create a comp file correctly when migrating
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: a12d86989152bc6fbb9ad3e6566c24c29b077f76

            gerrit Gerrit Updater added a comment - - edited Emoly Liu (emoly@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37401 Subject: LU-11510 lfs: create a comp file correctly when migrating Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: a12d86989152bc6fbb9ad3e6566c24c29b077f76
            dstewart David Stewart added a comment -

            Hi Emoly/Andreas, do we have any updates on this?

            dstewart David Stewart added a comment - Hi Emoly/Andreas, do we have any updates on this?

            People

              emoly.liu Emoly Liu
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: