[LU-6051] "lfs_migrate" improvements

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.11.0, Lustre 2.10.4
• Type: Improvement
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: Steve Guminski (Inactive)
• Resolution: Fixed
• Labels: easy
• Severity: 3

Description

There are a couple of improvements to make to the lfs_migrate script:

• pass all of the "lfs setstripe" options through to "lfs migrate" when calling "lfs_migrate" (they do not overlap), so it would be possible to just strip out existing lfs_migrate options and pass the rest to "lfs migrate"
• the man page should be updated to say it is safe to use on in-use files for 2.5+ MDTs (when the lfs migrate command was in place and working correctly)
• the man page should be split into a separate lfs-migrate.1 man page with more description and examples (per LU-4315)
• the hard-link migration doesn't need to do anything special if "lfs migrate" works, and if it fails then "lfs path2fid" and "lfs fid2path" (or "lfs path2fid --parents") can be used to generate the list of links to a file and then hard-link

Issue Links

• is related to: LU-8235 Add pool awareness to lfs_migrate
• is related to: LU-4315 split lfs.1 and lctl.8 man pages into...
• is related to: LU-8207 Add auto-stripe option to lfs_migrate
• is related to: LU-9629 lfs migrate does not work as a non-ro...
• is related to: LU-11621 Add copy_file_range() API and use it ...

Comments

[Evan Felix]

I've attached a version of lfs_migrate that deals pretty well with hard links if you specify -l on the command line. it has one bug I decided not to fix: if there are two slashes in a path passed in it does not work well since the paths back from fid2path do not have these in them.

[Andreas Dilger]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/25851
Subject: LU-6051 utils: allow lfs_migrate to handle hard links
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a8083fd4cdcc723b376d4b46370cf019469710ae

[Andreas Dilger]

Hi Evan, great to hear from you again. I pushed your patch to Gerrit for review.

In the future it would be great if you submitted the patch directly to Gerrit instead of attaching the whole file to the Jira ticket. Please see the https://wiki.hpdd.intel.com/display/PUB/Submitting+Changes and https://wiki.hpdd.intel.com/display/PUB/Using+Gerrit wiki pages for details. Also, if you have a Gerrit account you can be added to the patch for review comments.

I made a few cleanups to the patch beore submitting it to gerrit. The one item that still needs to be done is to add a regression test to sanity.sh to test for hard link migration, along with the other lfs_migrate tests. Please see lustre/tests/sanity.sh. .

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/25878
Subject: LU-6051 utils: improve efficiency of lfs_migrate cmp
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 6b556f4ac78cd31b7e2fb38c019ca38542f5bc40

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/25878/
Subject: LU-6051 utils: improve efficiency of lfs_migrate cmp
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d32895446d4f74b6e49513109007736a4fe48df2

[Andreas Dilger]

Once you are finished with this issue, you may want to look into LU-9629 as well.

[Steve Guminski (Inactive)]

I've assigned the new LU-9629 ticket to myself.

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/27553
Subject: LU-6051 utils: allow lfs_migrate to migrate links
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b2ec1c1eb062c053e689136fc5bb2e6f79210128

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/27553/
Subject: LU-6051 utils: allow lfs_migrate to migrate links
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6086d0d69c84ce57c4daff18595fa7b620ed2b04

[Gerrit Updater]

Steve Guminski ([e-mail]) uploaded a new patch: https://review.whamcloud.com/28597
Subject: LU-6051 utils: Fix incorrect request for getstripe help
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 6886fc5d73240e008396c525a626b545b9cd8d2f

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/28597/
Subject: LU-6051 utils: Remove incorrect request for getstripe help
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ad45173d94e0be94de90368019be52a905bcb993

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/25851/
Subject: LU-6051 utils: allow lfs_migrate to handle hard links
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 80a2ff7137d3504e5672c6a68561d4ae8d5a28e3

[Peter Jones]

All the patches seem to have landed - are more still to come?

[Andreas Dilger]

The other lfs migrate patches are tracked by their own tickets.

[Gerrit Updater]

Steve Guminski ([e-mail]) uploaded a new patch: https://review.whamcloud.com/29950
Subject: LU-6051 lfs: Update lfs_migrate man page for in-use files
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2c47428fbf9d6624b6b652d239730bf0e06cf7a4

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30799
Subject: LU-6051 utils: allow lfs_migrate to migrate links
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: 263465020d55735841f89fe325168da7c8ee982a

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30800
Subject: LU-6051 utils: Remove incorrect request for getstripe help
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: 7e23e96e3cc8eec0013c1c1b908b15c08fed4a4a

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30801
Subject: LU-6051 utils: allow lfs_migrate to handle hard links
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: 9f0a1fda800cf58ed278ed86f8338ba90545dccc

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/29950/
Subject: LU-6051 lfs: Update lfs_migrate man page for in-use files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6b097499d9ce191b49a4e9c4b590ca2b7e89700f

[Gerrit Updater]

John L. Hammond ([e-mail]) merged in patch https://review.whamcloud.com/30799/
Subject: LU-6051 utils: allow lfs_migrate to migrate links
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: 84e0507e79b2baa5b011aed17989395d79ec7f8a

[Gerrit Updater]

John L. Hammond ([e-mail]) merged in patch https://review.whamcloud.com/30800/
Subject: LU-6051 utils: Remove incorrect request for getstripe help
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: 44a4b7931f5dbaf0a07c41aaedf6ad93d671dca9

[Gerrit Updater]

John L. Hammond ([e-mail]) merged in patch https://review.whamcloud.com/30801/
Subject: LU-6051 utils: allow lfs_migrate to handle hard links
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: 19fb581aa291f5d192a2e192e5079ae300b5e7aa