Details
-
Bug
-
Resolution: Fixed
-
Major
-
None
-
3
-
9223372036854775807
Description
Migrating a file between MDTs changes the FID of the file. The file FID is the used to identify the file in the HSM archive (unfortunately). So, for example, if a released file is migrated from one MDT to another then it cannot be restored.
Attachments
Issue Links
Activity
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/17511/
Subject: LU-6866 hsm: prevent migration of HSM archived files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: fdddeb25a54324cbcc2c99daf38b9f778fdbbec3
John L. Hammond (john.hammond@intel.com) uploaded a new patch: http://review.whamcloud.com/17511
Subject: LU-6866 hsm: prevent migration of HSM archived files
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3f194332d27db47f0ff1ec6ef947454f8b51c61d
Reopening since lhsmtool_posix does not implement the discussed functionality.
Thanks Robert. Then I will close this one, and push the migration record patch to LU-6868
Please see LU-7207 for the proposed format for the external ID EA and including it in the delete changelog.
As long is there is no unlink changelog generated, then that's fine. A migration changelog is useful for the PE to know that file has changed MDTs, but this doesn't really impact the archive. I'll open a new ticket for adding UUID to delete changelog, which is much more important for HSM.
I believe currently MDT migration triggers unlink and create records in the source MDT and target MDT changelogs respectively, so one option is to just add a MIGRATE flag to those records so we can differentiate the migrate unlink from a real unlink. Or is do yo have a better solution for this already?
The whole migration is implemented in MDD layer, where changelogs are generated, and there are no changelog created for migration. So I can just add one for this ticket. I will push the patch to LU-6868.
I believe currently MDT migration triggers unlink and create records in the source MDT and target MDT changelogs respectively, so one option is to just add a MIGRATE flag to those records so we can differentiate the migrate unlink from a real unlink. Or is do yo have a better solution for this already?
Would it be possible to standardize the EA field we use for the UUID instead of creating a hsm layout? As Ihara mentioned, we eed to support the ability to assign UUIDs to a file, so it is useful to use the normal EA interface for this instead of adding a new API. Also, some backends might use different kinds of identifiers, so it might not always be an actual UUID. If we have a standardize this, then we can add this extra data to the delete changelog record, and that solves the main problem we have today with using external IDs.
If my undestanding is correct, archive UUID means we can define unique ID that is stored into EA?
If so, that would be really helpful. Robert and Andreas mentioned above, one of big limitation with WOS and Lustre today, that is delete operation. we are storing OID (WOS object ID) into EA after migration, but if file is removed, it's hard to search objects in WOS becouse OIDs is gone when file removed.
So, we need extra database to map FID and OID for now. If the changelog keeps UUID ratehr than FID, that would be easy to do asynchronous remove objects in WOS without external database after unlink files on the Lustre.
Landed for 2.8