[LU-5896] lfs: can't release a migrated file

• Type: Bug
• Resolution: Unresolved
• Priority: Major
• Affects Version/s: Upstream
• Labels: CEA
• Severity: 3
• Assignee: Artem Blagodarenko
• Reporter: Frank Zago (Inactive)

Description

After a file has been archived and migrated, it's not possible to release it.

# dd if=/dev/urandom of=rand1 bs=4096 count=1024
...
# lfs hsm_archive rand1
# lfs migrate -o 0 -S 65536 rand1
# lfs hsm_release rand1
Cannot send HSM request (use of rand1): Device or resource busy

The file content is still accessible, and its hsm state is correct.

Issue Links

• is related to: LU-13048 Non-blocking migrate: Unable to release file from disk (Cannot send HSM request)
• is related to: LU-5895 lfs: data version changed during migration

Comments

[Guillaume Courrier]

Yes, this patch is still useful. I didn't take the time to make the tests pass so it was missing reviews. I'll update the patch with your remarks and make the tests pass.

[Andreas Dilger]

eaujames, courrier,
there is still patch https://review.whamcloud.com/35022 "LU-5896 hsm: can't remove a migrated file" open on this ticket. Is that patch still useful and should be refreshed, or should it be abandoned?

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54473/
Subject: LU-5896 wirecheck: add swap layout structs to wirecheck
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8edc3e7876e36e0895706f7ededd4dd71b66c71a

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57791
Subject: LU-5896 wirecheck: add swap layout structs to wirecheck
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 84e14674ab2e256d824ffbdb7befa5a520aefc1d

[Gerrit Updater]

"Guillaume Courrier <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/54473
Subject: LU-5896 wirecheck: add swap layout structs to wirecheck
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a2a4f464b1122002944d92acbeb41a729bf80041

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/53296
Subject: LU-5896 mdt: allow release after blocking migrations
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 2efedbd708390c745ef0a61dee8943a293fde429

[Gerrit Updater]

"Guillaume Courrier <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49377
Subject: LU-5896: allow release after blocking migrations
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 041adca0986b27cb8ce905343f9bf95a068ddbd0

[Etienne Aujames]

Hi,

Is there anyone maintaining the https://review.whamcloud.com/36549 or actively developing something for this issue (or for the non-blocking case LU-13048) ?
If not, the CEA, would like to continue the development for the blocking and the non-blocking version.

This become a blocking issue for the CEA. Without a workaround, this forces to re-archive every migrated files (not acceptable on filesystem mixing SSD and rotational pools with HSM).

[Quentin Bouget (Inactive)]

But I am worried that an extra flag SWAP_LAYOUTS_DATA_MIGRATE will be used maliciously by a user...

It cannot really be that malicious. To be able to do a swap_layout, one has to have write-access to the file. At that point, you can just truncate the file to 0 or overwrite it.

We can not simply swap the HSM xattrs, as the HSM copy is naming and locating according to FID. (...)

The archived copy does not need to be named using the file's FID in Lustre. That is just what the lustre copytool does.
I also feel like you think migrating a file changes its FID. It does not.

[Ben Evans (Inactive)]

HSM solves this by calling ll_data_version to get the current data_version of the fid (see ll_ioc_copy_start)

There's no reason we couldn't do a similar thing in the kernel layer for migrate.  Get the currrent dv, compare it to the version stored in the HSM xattr.  If they're the same, perform the layout swap and update the xattr.  If they're different, perform the layout swap, and make sure to mark the xattr with the DIRTY flag, if it's not already.