Details
-
Bug
-
Resolution: Fixed
-
Minor
-
Lustre 2.11.0, Lustre 2.12.0
-
None
-
3
-
9223372036854775807
Description
The lfs_migrate tool does not currently preserve the PFL/FLR/DoM components of the layout when a file is migrated. By default, it will get the stripe_count and stripe_size of the file (maybe pool and mirror_count if https://review.whamcloud.com/32977 is updated to do so), but it does not preserve more complex layout details.
Dropping the initial components of a PFL file is probably desirable, since it is unlikely that a file will become smaller, and it makes sense to keep the stripe_count of a file at the widest needed.
We do need to keep the number of mirrors across migration, and it would probably be good to keep any OST pools per mirror, if they exist.
Separately, it would be useful to have some way to migrate only the mirror(s) that exist on particular OSTs if we are trying to empty particular OSTs.
Attachments
Issue Links
Activity
Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37814
Subject: LU-11510 lfs: migrate a composite layout file correctly
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: fdb231d505dda018515f754475527041e67af7c8
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36082/
Subject: LU-11510 lfs: migrate a composite layout file correctly
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8bedfa377fbd0c9f1b6ea2c40d36fdcaa52137df
implementing the LU-9392 functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins
This is close to my idea.
the next step is to move the file to the new OSTs by adding an option to lfs_migrate tool to let the customer specify the OST list.
Normally when doing large-scale migration, the OST selection would be handled by the MDS by disabling object creation on some OSTs (via "lctl set_param osp.<fsname>-OST[xxxx-yyyy].max_create_count=0") and running "lfs find -ost xxxx,...,yyyy" to list all of the files on those OSTs to migrate. I think the common case is finding files on OSTs that shouldn't be used, and the MDS QOS policy will already bias file allocation onto OSTs with more free space.
I think would be useful for making easier file layout policies for users are:
- using "lfs setstripe -
copy XXX" to select a template layout from an existing file/directory (this should already work with the new patch), but allowing an "alias" like -copy=@mypattern that looks for the file "$HOME/.lfs/mypattern" or similar - finishing patch https://review.whamcloud.com/28972 "LU-9982 lustre: Clients striping from mapped FID in nodemap" so that it is possible to set a per-fileset default layout
- implementing the LU-9392 functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins
Emoly Liu (emoly@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/37401
Subject: LU-11510 lfs: create a comp file correctly when migrating
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a12d86989152bc6fbb9ad3e6566c24c29b077f76
It seems like the core problem is that "lfs migrate" doesn't handle PFL files very well. Starting with a 2-component stripe_count=<1, 4> file (with the 4-stripe component initialized):
# lfs getstripe !$
lfs getstripe /myth/tmp/pfl/64M
/myth/tmp/pfl/64M
lcm_layout_gen: 3
lcm_entry_count: 2
lcme_id: 1
lcme_flags: init
lcme_extent.e_start: 0
lcme_extent.e_end: 33554432
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
lmm_pool: audio
lmm_objects:
- 0: { l_ost_idx: 2, l_fid: [0x100020000:0x26cdc7:0x0] }
lcme_id: 2
lcme_flags: init
lcme_extent.e_start: 33554432
lcme_extent.e_end: EOF
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
lmm_objects:
- 0: { l_ost_idx: 4, l_fid: [0x100040000:0x311e78:0x0] }
- 1: { l_ost_idx: 0, l_fid: [0x100000000:0x22ca87:0x0] }
- 2: { l_ost_idx: 1, l_fid: [0x100010000:0x1b1931:0x0] }
- 3: { l_ost_idx: 3, l_fid: [0x100030000:0x22233d:0x0] }
Using "lfs getstripe -c /myth/tmp/pfl/64M" returns a stripe_count of 4 from the last instantiated component as one would expect. Running "lfs migrate" without any arguments doesn't preserve the PFL layout AND it doesn't even keep the stripe count from the last initialized component of the source file. At a minimum, "lfs migrate /myth/tmp/pfl/64M" should use stripe_count=4 to create the new file, or copy the whole PFL layout but instead it is using the filesystem-wide default stripe_count=1:
# lfs getstripe !$ lfs getstripe /myth/tmp/pfl/64M /myth/tmp/pfl/64M lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: 1 lmm_layout_gen: 4 lmm_stripe_offset: 3 obdidx objid objid group 3 2237247 0x22233f 0
I think it makes sense to use the "lfs migrate --copy" option internally to copy the layout from the source file, or from the parent directory for "lfs_migrate -R". The "--yaml" option still can specify an arbitrary layout. That can potentially simplify migration
Emoly Liu (emoly@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36082
Subject: LU-11510 scripts: add --copy and --yaml to lfs_migarate
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 16fb4f3eb169c05809784d14f54cf3fd4f67fd36
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/37814/
Subject:
LU-11510lfs: migrate a composite layout file correctlyProject: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: dc88cb0843e57be3930558e4e1b5c9b5d3af3311