Details
-
Improvement
-
Resolution: Fixed
-
Minor
-
None
-
9223372036854775807
Description
It would be useful to store the JobID that created a file in an xattr on the MDT inode. This would allow tracking the source of files after the fact, and can help users with the provenance of their file data (e.g. which run the files came from, which can then be used to determine runtime parameters used, whether the run was successful or not, etc.).
It isn't clear whether this would be needed on the OST inodes as well, but I think initially the MDT inode would be enough, as this would be the only xattr visible to users. The OST xattr would only be accessible for low-level scanning tools.
Attachments
Issue Links
- duplicates
-
-
- Resolved
-
- has to be done before
-
-
- Closed
-
- is related to
-
LU-18798 user.job xattr is storing quotes around job name
-
- Open
-
-
-
- Open
-
-
-
- Resolved
-
- is related to
-
-
- Open
-
-
-
- Open
-
-
-
- Open
-
- mentioned in
-
Page Loading...
Activity
Hi Andreas,
I'd be willing to work on adding that in. I'm pretty busy at the moment so I likely won't be able to start working on it for 2-3 weeks, but if it can wait that long I'm happy to give it a shot.
Thomas, I just realized that this is storing the xattr on the MDT inode, but it would also be useful if the same xattr was also stored on the OST objects for debugging/scanning/recovery purposes. Is this something that you would be interested to work on?
"Xing Huang <hxing@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52332
Subject: LU-13031 tests: avoid new user xattr in interop
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: ed56abd1070e9812e55728b14c760580b03074e0
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/52095/
Subject: LU-13031 tests: skip sanity/test_205h,205i in interop
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a6df532162556028c2ab9b974989fc0cca68d4fe
"Thomas Bertschinger <bertschinger@lanl.gov>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52095
Subject: LU-13031 tests: skip sanity/test_205h,205i in interop
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1532029b43cfdc41b6747ea2d4076f12d28e8be0
The version_code helper supports 4-component version numbers, so since this is being done after the landing it is helpful to include the full version from "git describe" of the commit hash of the patch itself. If it were before landing it could use the 4-component version of the patch before landing, even though it is slightly inaccurate..
It is also helpful to include a brief description of why the test is being skipped. Please see some examples in patch https://review.whamcloud.com/52009 "LU-16097 tests: skip quota subtests in interop".
Yes, you've got this exactly right. That's what interop is and that check is what's needed.
Generally you'd just use 2.15.57, the tag that was live when the patch was merged. We don't do interop testing between different development versions, just between dev versions and older full versions. It's also fine if in some narrow scenario we skip a test that would pass.
So that MDS version check you suggested for 2.15.57 is what's needed. Oops on the reviewers (ie, including me) for not noting that at the time.
Does interop refer to some testing that mixes different client and server versions? Is the failure occurring when the server version is older than the client and the MDS doesn't have this new change?
If it just needs a version check like:
(( $MDS1_VERSION >= $(version_code 2.15.??) )) || skip ...
that should be simple enough, I can submit a fix for that.
The version I get when I check out this commit and run `lctl get_param version` is 2.15.57. Is that the correct code to put in the version check? Or do I need one higher than that (since I guess you could check out a commit newer than the 2.15.57 tag but older than this change and its version would pass...). If I need one higher than 2.15.57, do I use 2.15.58, 2.16.0, or something else?
No rush. For some reason I thought this was part of the original implementation, but upon looking at the patch again I saw it was only on the MDT inode.
I think it makes sense to put this in ofd_attr_handle_id() where it is initializing the object ownership the first time the OST object is written to (in the "if (mask != 0)" block if it cleared the SUID/SGID bits). That allows a "one shot" setting of the user.job xattr without having to check each time whether it exists or not. The parameter obdfilter.*.jobid_xattr should be used like mdt.*.jobid_xattr to control the xattr name.