[LU-4640] Last unlink should trigger HSM remove by default

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.10.0
• Affects Version/s: Lustre 2.5.0
• Labels: HSM patch
• Severity: 3
• Assignee: Bruno Faccini (Inactive)
• Reporter: Aurelien Degremont (Inactive)

Description

In current code, when an archived file is removed from the file system, no HSM request is triggered. We rely on Changelogs and RobinHood reading them to have it sending corresponding hsm_remove requests to clean those orphans in HSM backend. This is done in purpose, to provide a way to implement "soft unlink". RobinHood will remove the file in the backend after a grace time. During this time, Admins could restore the file from the HSM if they want to (using import feature).

Requiring a RobinHood setup to handle this cleaning is a too big limitation. We should consider modifying this behaviour.

By default, Lustre should automatically add a HSM_REMOVE request for any last unlink. This way, no file will be leaked in the archive.
A tunable should be added to disable this behaviour (should we add this to hsm_policy?) and go back to a mode where an external component is responsible for tracking UNLINK changelogs and add hsm_remove requests when needed (Robinhood) (current behaviour).

Issue Links

• is related to: LU-9654 fix problem of Remove Archive on Last Unlink policy
• is related to: LU-7881 sanity-hsm test_26b: @@@@@@ FAIL: Copytool should have stopped

Comments

[Peter Jones]

Landed for 2.10

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/18946/
Subject: LU-4640 mdt: implement Remove Archive on Last Unlink policy
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: dd4b034540d7dda499ebbb8c465d3435ad46b82a

[Li Xi (Inactive)]

The patch #26980 was on the top of version 6 of path #18946. And version 8 of patch #18946 looks better than #26980 now.

[Li Xi (Inactive)]

I could have pushed a new patch using the the same change ID of #18946. But I am still not sure whether my thought is correct or not

[Li Xi (Inactive)]

Hi Bruno,

Thanks for helping. Yeah, I create a new patch based on your latest patch.

I first improved the test 26c and 26d. And after that, the test results immediately shows that the mdt_attr_get_complex returns -2 because the object is deleted. That means, the MA_HSM and MA_INODE needs to be read before unlink happens. That is why I split mdt_handle_last_unlink to mdt_handle_last_unlink_prepare and mdt_handle_last_unlink_commit.

Let's see what is the test result thi time.