[LU-13055] add ability for named Changelog consumers

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.12.8, Lustre 2.15.0
• Assignee: Mikhail Pershin
• Reporter: Andreas Dilger

Description

Currently Lustre Changelog consumers are always named e.g. "cl1" or "cl14". It would be useful to be able to declare the changelog username (e.g. "cl-rbh" or "cl-audit") so that it is clear who the Changelog users are, and to avoid duplicate changelog registrations. Otherwise, it can happen that the original Changelog user registration can be lost, and the application registers a new user, and the old user causes the Changelog records to accumulate.

Issue Links

• is duplicated by: LU-8517 Changelog Users
• is duplicated by: LU-13338 add changelog mask per registered user
• is related to: LU-14699 changelog garbage collection is too lax
• is related to: LU-16268 per-server changelog mask change doesn't change effective mask in some cases
• is related to: LU-13338 add changelog mask per registered user

Comments

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44022
Subject: LU-13055 doc: update changelog manpages
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7bdb0af8f93a70ebf7f465da6c0ca49109aae723

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43741/
Subject: LU-13055 libcfs: allow comma-separated masks
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6b6fde1026311a28595ea43af56392ca6ad24d79

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43892
Subject: LU-13055 mdd: make current changelog mask writable
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 11e30ad4770811df392870e09a92dda022574ee6

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43741
Subject: LU-13055 libcfs: allow comma-separated masks
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 17ed95152ff513dcfe4c1a8f36e66b8bfeeba895

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43710
Subject: LU-13055 mdd: don't assert on unknown changelog lrh_type
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 1259811054a0e41f7fe646e8ad05307d693549f7

[Mikhail Pershin]

Patch is updated. For now name uniqueness is not provided, if you think it must be - let me know. The another patch for b2_12 branches is on its way to provide better compatibility during downgrade

[Mikhail Pershin]

well, I was asking mostly because having unique names means changelog scan for all registered names upon new user registration, while right now the only ID is increased. That would require patch modification.

Another problem I have right now is backward compatibility. New name format like 'cl$ID-$NAME' means that our scripts and any other one on customer side will get changelog user name in that format from the new server and later call to 'lfs changelog_clear' would return error because of sscanf() format in chgl_write(). That is not problem to change it or 'lfs' to skip '-$NAME' addition and still use cl$ID part after all but that is client side changes which means old client is not compatible with a new server changelog names. I am not sure if a client wants to work with any name registered from other client, so probably that is not problem, but if it is problem then I have no good solution except compatibility patch to an older Lustre client

[Andreas Dilger]

Mike, the goal is for a service to be able to determine if it already has a registered changelog user or not at startup. Otherwise, if there are 2 changelog users "cl2" and "cl7" there is no way for the user or administrator to know whether those are being used by a particular service (audit, HSM, etc.), if a new user needs to be registered because the old one was idle and removed, or if the users are for some service that is no longer running. Allowing the service name in the username makes it totally clear that "cl2-audit" is for the audit service, "cl7-lamigo" is for lamigo, etc.

Allowing only a single named user for a given service makes sense, since otherwise it is again ambiguous if both are in use, or one of them is stale. Then, if the service starts up and doesn't know what its changelog user is, it could first scan for it, or just try to register with the name and be given the old user back. If some service really needs to register two changelog users on the same MDT, then it can pick two different names (e.g. "robinhood" and "rbh-tmp-scan" or whatever).

[Mikhail Pershin]

John, as for the same names:

we should ensure that $NAME is valid according to the same rules as above and that it is unique on the MDT. So if cl42-lamigo is already registered then we cannot register a new changelog with name lamigo on the same MDT.

if we are using cl$ID-$name format then 'name' could be the same because ID is always unique, and even with same name these are different changelog users, so what is the point to require name uniqueness? 

[John Hammond]

I think we should require a cl prefix on named changelogs. This will be less likely to break scripts that try to parse the changelog_users file.

> there are couple questions related to proposed functionality. First, what name format will we allow? For example, name 'user1' would cause combined result as 'user11' or similar which is undistinguished from 'user' with ID 11. So either we prohibit digits in name or need separator like '.' between them, so that would be 'user1.1' and 'user.11'

I had imagined that we would use something like cl-$NAME. For example cl-lamigo. The - makes splitting between numbered and named changelogs easier. To make it totally unambiguous we should require that $NAME starts with a letter and contains only letters, digits, and dashes.

Alternatively we could allow specifying $NAME when the changelog is registered. But the the actual changelog created is named cl$NUM-$NAME where $NUM is allocated sequentially as before. In this case we should ensure that $NAME is valid according to the same rules as above and that it is unique on the MDT. So if cl42-lamigo is already registered then we cannot register a new changelog with name lamigo on the same MDT.

> Another question is about deregister, right now tool demands it in form 'cl<ID>' and extract just ID to proceed with.

I think it's much better if we can support a deregister command that uses a name. But if we use the cl$NUM-$NAME scheme and cl42-lamigo is registered then I think changelog_deregister cl42 should deregister cl42-lamigo.

> So either we prohibit digits in name or need separator like '.' between them, so that would be 'user1.1' and 'user.11'

I don't like dots. Please don't use them or allow them here. They'll be terrible if we ever want to use the changelog name in a param.

[Mikhail Pershin]

there are couple questions related to proposed functionality. First, what name format will we allow? For example, name 'user1' would cause combined result as 'user11' or similar which is undistinguished from 'user' with ID 11. So either we prohibit digits in name or need separator like '.' between them, so that would be 'user1.1' and 'user.11' 

Another question is about deregister, right now tool demands it in form 'cl<ID>' and extract just ID to proceed with. While keeping old 'cl<ID>' format for compatibility is OK, wouldn't that be useful to support pure ID form: 'deregister ID' and/or name+ID form: 'deregister 'name<ID>'. In latter case we also have problem #1 with name and ID proper separation