see also LU-10345

Another requirement collected from real user is:

People would like to allocate the files to different pools according to the range of the IDs. For example, project ID from 0 to 1000, to pool 0, pool ID from 1000 to 2000, to pool 1. We need support rule like:

lctl set_param lod.*.layout_policy=\ "add rule_vip_users pool=pool1 uid={[0-000]}"

This would benefit the setting of TBF rule too, since we can't set a rule that list all 1000 IDs.

Instead of specifying just the OST pool for files matching the policy, it would be much more useful to base this on patch https://review.whamcloud.com/28972 "LU-9982 lustre: Clients striping from mapped FID in nodemap" and allow specifying the FID that is the source for the file layout. That allows specifying a pool, but more importantly allows specifying the rest of the layout (stripe count, PFL components with multiple different pools, FLR mirroring, etc.). Getting patch 28972 landed would itself be useful in any case for nodemap/fileset/isolation purposes. Once that patch lands, I think the changes to 33126 would be relatively straight forward, adding a fid=[SEQ:OID:VER] option to the matching rules.

Rather than make all of these policies global, they should be implemented as part of a nodemap, even if it is the" default" nodemap to start with. This will allow different policies to be implemented for different clients if needed (eg. prefer local OST pool for clients connected over WAN).

As discussed with Teddy and Ihara, we want to implement something like:

1) Use Jobid/UID/GID to define expressions of rules (Just like TBF rules).

2) Each rule has an operation, the current operation is simple, just to create the file in a dedicated OST pool. That means, the rule is a tuple of (expression, OST pool name)

3) Define a list of rules with order. The file creation process will walk through this list, trying to match the first ones of them. if matches, the file will be created on the defined OST pool.

4) The rule can be defined through /proc entry, just like what did for TBF.

A lot of codes can be shared by TBF and this new feature. Maybe we can put some the functions into shared library rather than copy and paste.

Teddy Zheng (jjkky@yahoo.com) uploaded a new patch: https://review.whamcloud.com/33126
Subject: LU-11234 lod: Choose pools to place file's objects by filename extension policies
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5615ef1ee1db30e362e61ecc29add9907637a1e2

There is nothing preventing users from allocating all of their files on the flash OSTs today, they can just use "lfs setstripe -o idx,idx,idx" directly if they want. We need to have OST quotas to prevent users from using these OSTs.

In that respect, allowing users some mechanism to specify "pools" or filename extension policies doesn't affect this, but could make like a lot easier for some applications and users.

One simple way to enable this would be to allow "lfs setstripe -o" to specify a smaller number of stripes than the OSTs listed. This would be treated the same way as a stripe count in a pool - the specified number of stripes is allocated to the file from the listed OSTs.

After that point, it would be possible for users to create their own pools, which are just named lists of OSTs, and "lfs setstripe" can just get those lists from _/.lustrerc or similar. It is a bit more tricky if the files are not being created via lfs, but eg. we could store the FID for the user /.lustrerc in the LOV EA so that the MDS can access it.

In the description, ‘user’ means ‘administrator’: application users are only allowed to propose requirements for rules, but applying the rules to the system should be allowed to execute by authoritative administrator. Otherwise, if application users can define and apply rules by themselves, most will prefer to put their data on the faster devices. Besides, applying the rules by application users will increase the rate of misconfiguration.

Indeed, I think, the rules of this feature deserves more careful designs. We can define rule conjunction, just like rules in TBF.  For example, rules with UID and filename extension will only allow control file’s layout of users with UID( in LU-9658, we have already add uid and gid in creat RPC).

I'll upload the basic patch that only supports filename extension rule for review.

When you write "user" here, you really mean "administrator", since it typically is not possible for a regular user to login on the MGS to set a policy. Also, it appears that this policy would be global to all users.

I think this proposal is a very useful one, we've been discussing it for years, but nobody has been working on it It would be nice if we could work out a mechanism for regular users to be able to create OST pools by themselves, and have per-user or per-directory policies for file creation. With the patch https://review.whamcloud.com/32814 "LU-11146 lustre: fix setstripe for specific osts upon dir" it is possible to set a directory default layout that selects specific OSTs, which is almost like a pool. The main difference is that the specific-OST layout currently requires the file to be striped over all OSTs, but it would be nice to allow striping overconly a subset of OSTs.

The patch https://review.whamcloud.com/28972 "UL-9982 lustre: Clients striping from mapped FID in nodemap" allows specifying an arbitrary source FID for the default layout for files in a subdirectory mount.

I think it would be possible to combine these ideas by having a $HOME/.lustre/defaults directory (nothing special there, just a regular directory) that users could create policies in or on (eg. different layouts for filename extensions), and then the $HOME/.lustre directory FID can be set as the default layout for all user directories if they want to apply this policy to their directories (it would be inherited by subdirectories automatically, or fetched from the fs root if needed. .

Since this is not a configuration parameter, the user does not need to have any special login permisssion. Since thos layout is explicitly applied to user directories (there can be arbitrarily many different layouts for a user, they just need to supply the directory FID).