Medium Description
Admins or users would want to pin files to a given pool, as a way of staging files before use, or to prevent any mirroring or purging to/from a given pool. One way to accomplish this would be to designate an xattr that external tools would check for and honor.
A pin xattr, lustre.pin, has already been introduced for PCC purpose. It currently supports PCC info, and is only used by PCC for now, but the spec already plans to include other information such as pool names in this xattr. The content of this xattr is in JSON format, and is like:
lustre.pin=[{"hsm":2},{"hsm":3},{"pool":"ssd"}]
If a pool is specified (multiple are possible), we would consider the file is pinned to this pool, and a mirror stored on this pool cannot be deleted. We could also introduce the "only_pool" property, so that the file is pinned exclusively to this pool. This means no mirror to a different pool would be created in that case.
That would give us the following possible values as examples:
lustre.pin=[{"pool":"ssd"},{"pool":"hdd"}]
or
lustre.pin=[{"only_pool":"ssd"}]
"pool" and "only_pool" would be mutually exclusive given their meaning.
Using this already existing xattr lustre.pin provides the ability to control which GIDs can pin/unpin files, as this feature was already added for PCC by patch "LU-10499 pcc: tunable GID to pin/unpin files". This patch adds a tunable "mdt.*.enable_pin_gid" that controls the GID to pin/unpin files. Default is gid=0, so it is limited to root, gid=-1 means any user can pin/unpin, and gid=<gid> is used to allow a specific GID (e.g. wheel).
To support the new pool pin/unpin capability on Lustre, we would need to extend the llapi with dedicated functions, e.g:
int llapi_pool_pin_fd(int fd, __u32 id); int llapi_pool_pin_file(const char *path, __u32 id); int llapi_pool_pin_fid(const char *lustre_dir, const struct lu_fid *fid, __u32 id); int llapi_pool_unpin_file(const char *path, __u32 id); bool llapi_pool_is_pinned_fd(int fd, __u32 id); bool llapi_pool_is_pinned_file(const char *path, __u32 id); bool llapi_pool_is_pinned_fid(const char *lustre_dir, const struct lu_fid *fid, __u32 id);
These could be inspired from their pcc equivalents, see all the llapi_pcc_*pin* functions.
We would also need dedicated lfs commands, such as lfs pool pin/unpin/get. pin/unpin would take a path or FID and optionally a pool name. If no pool is specified, we would internally pin to the pool that is configured in the file layout as the primary mirror.
Apart from the userspace side of things, it is important for the lustre.pin xattr to be automatically inherited for files and subdirectories when set on the parent directory. That would allow setting it for instance once at the project directory or group directory level, and then automatically benefit from it for all files and directories created in there. We do inherit layout already, which is basically an xattr we process and then write, so we would have to implement a similar behavior for the lustre.pin xattr.