[LU-3155] Permanent parameters with lctl set_param -P

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.5.0
• Fix Version/s: Lustre 2.5.0
• Type: Improvement
• Priority: Minor
• Reporter: Artem Blagodarenko (Inactive)
• Assignee: Emoly Liu
• Resolution: Fixed
• Labels: patch

Sub-Tasks

Key	Summary	Type	Status	Assignee
LU-3788	add "lctl set_param -P" to lctl.8 man...	Technical task	Resolved	Andreas Dilger
LU-3789	Add deprecation warning for "lctl con...	Technical task	Resolved	WC Triage

• Key: Summary
• Summary: Type
• Type: Status
• Status: Assignee

Issue Links

• is duplicated by: LU-2629 set_param and conf_param have differe...
• is related to: LU-3798 replay-single test_86: configuration ...
• is related to: LUDOC-166 Document new "lctl set_param -P" option
• is related to: LU-7004 fix "lctl set_param -P" to allow depr...

Comments

[Artem Blagodarenko (Inactive)]

Sorry, next information should be in issue's description.

With this patch permanent and temporary formats are identical:
Temporary:

set_param obdfilter.*.client_cache_seconds=15

Permanent:

set_param -P obdfilter.*.client_cache_seconds=15

1. Permanent parameter is added to single special param log called "params". Same format is using. Command "param2", target "general"
2. Params log updates get pulled by all the clients and servers. Separate ldlm lock is used for that.
3. MGC executes via upcall-type mechanism to userspace a local lctl set_param string=value
4. The protocol can be extended for supporting another upcalls, not only lctl

The command's format is next

#01 (224)marker   2 (flags=0x01, v2.3.63.0) general         'osc.*.max_dirty_mb' Wed Apr 10 23:32:13 2013-
#02 (112)param2 0:general  1:osc.*.max_dirty_mb=64  2:lctl  
#03 (224)marker   2 (flags=0x02, v2.3.63.0) general         'osc.*.max_dirty_mb' Wed Apr 10 23:32:13 2013-

Benefits of using set_param -P instead of conf_param:
1. Identical format for permanent and temporary commands
2. wildcarding in strings
3. no unimplemented paths (e.g. ptlrpc services)
4. simpler implementation

[Artem Blagodarenko (Inactive)]

Please, change topic to "Permanent parameters with lctl set_param -P". Thanks.

[Artem Blagodarenko (Inactive)]

The patch is here: http://review.whamcloud.com/6025

[Artem Blagodarenko (Inactive)]

Xyratex MRP-272

[Peter Jones]

Emoly

Could you please look after this one?

Thanks

Peter

[Emoly Liu]

OK.

[Nathan Rutman]

Originally LU-2629

[Nathan Rutman]

Looking for some input from Intel architects.

Alex Lyashkov see a scalability problem. This implementation shares a single llog over all targets and clients. Alex advizes splitting the configuration file into dedicated client and server versions. This save rpc's if only servers or clients are configured.
Nathan Rutman believes that scalability is virtually the same as current conf_param code - items that affect the client llog will dominate the worst-case scenario anyhow. I.e. the difference between updating all clients, and all clients plus all servers, is minor. And the increased complexity of a) determining what items need to go into each log, and b) the additional code required is not worth the slight RPC savings.

[Andreas Dilger]

I haven't looked at the patch in detail yet, so my comments are only generalities.

While I'm never fond of extra overhead imposed on e.g. the servers for something that should mostly impacts the client, I don't think this is a critical performance issue. The MGC llog processing on the OSS and MDS will be asynchronous, so will not affect server functionality, and the number of servers is small compared to the number of clients, so the load on the MGS should not be affected in any serious manner.

I think Nathan brought up the critical issue which might force a single log for all tunables - the MGS has no way to know whether e.g. ldlm.namespaces.*.lru_size is a parameter for a client or a server. Any number of similar ambiguities exist that mean the parameters need to be stored in a single configuration llog. It is up to the administrator to specify any wildcards in a manner so that they apply only to the client or server, if appropriate. The nodes that process the parameters should ignore ones that do not apply locally.

[Nathan Rutman]

Can we get this landed please? I really would like to see a simpler and more robust configuration, and we don't want to diverge our branch from mainstream too far.

[Emoly Liu]

The patch that Artem pushed is always hitting some test failures. After the patch passes the test and review, it will be landed.

[Artem Blagodarenko (Inactive)]

Test always fails on test_132 of sanity with message 'som is still disabled'.
As I understand som is enabled on MGS and checked on MDS (this servers are on different hosts) and I can't find code that offer check this setting on MDS. Should this scheme works?

Indeed, this test works on other branches. I can't understand why and how my patch blokes this.

[Artem Blagodarenko (Inactive)]

Bug is fixed. Waiting for testing results.

[Andreas Dilger]

I filed LUDOC-166 for tracking the required documentation update for this feature.

[Andreas Dilger]

It would be useful to have two additional patches for this change:

• add deprecation warnings for "conf_param" usage for a future release (e.g. LUSTRE_VERSION_CODE > OBD_OCD_VERSION(2, 7, 53, 0)) and disable the code entirely in some future release (e.g. > 3.0.53)
• remove the use of "lctl conf_param" from test scripts around 2.7.53 (want to keep these for a while for interop testing)

[Emoly Liu]

patch http://review.whamcloud.com/6025 landed for 2.5

[Andreas Dilger]

I noticed that the original patch is also missing an update to the lustre/doc/lctl.8 man page.

[Artem Blagodarenko (Inactive)]

Warnings and man page
http://review.whamcloud.com/7452
http://review.whamcloud.com/#/c/7436/

[Bob Glossman (Inactive)]

Artem, Please take a look at LU-3798 and comment.

[Artem Blagodarenko (Inactive)]

>Artem, Please take a look at LU-3798 and comment.
Thanks. Done.

[Emoly Liu]

Warnings and man page change landed for 2.5