[LU-7183] lctl set_param -P does not work for sptlrpc flavor

• Status: Closed
• Project: Lustre
• Affects Version/s: Lustre 2.9.0
• Type: Bug
• Priority: Minor
• Reporter: Jeremy Filizetti
• Assignee: John Hammond
• Resolution: Duplicate
• Labels: SSK, kerberos
• Severity: 3

Description

Using
lctl set_param -P lustre.srpc.flavor.default=skpi
instead of:
lctl conf_param lustre.srpc.flavor.default=skpi
does not work correctly. No sptlrpc flavor is used. It appears that this may be because the llog for sptlrpc at MGS at CONFIGS/fsname-sptlrpc is not created. Instead with "set_param -P" the parameters are placed in params.

Issue Links

• is related to: LU-9567 sptlrpc rules are not being updated
• is related to: LU-3289 IU Shared Secret Key authentication a...
• is related to: LU-10937 Use sysfs to fix up sptlrpc handling.
• is related to: LU-9399 lctl conf_param and lctl set_param -P...

Comments

[Jeremy Filizetti]

Linking to shared key though it will affect kerberos as well

[Andreas Dilger]

There are several other issues related to "lctl set_param -P" that will be fixed via LU-7004 for 2.9.0.

[James A Simmons]

I did some debugging over the weekend and have traced it in failing to parse correctly in mgc_process_llog. Will continue to debug. I see the reason for lctl set_param -P failing as well. In mgs_parse_param() it reports:

"setparam fs='general' device=general'

This is wrong. Its just copying the device 'general' over to the file system name which causes the failure.

[James A Simmons]

I looked into why this is failing. The reason is because the way lctl set_param -P work is that is send a tunable setting to lctl in userland that than then sets the proc file entry. The problem for sptlrpc is that the flavor settings are not exposed in the procfs/sysfs tree.

[James A Simmons]

So normally for lctl set_param -P the config change is sent to lctl in userland so it can write to to the proc,sysfs, or debugfs file. In this case its a bit tricker than most since two pieces of data want to be written out. One is flavor i.e skpi etc and the other is the direction. Since set_param -P format follows $subsystem.$device.$param in this case we will have by default the source direction. This leaves us with the destination direction and the flavor needing to be set.

Their are two ways we could handle this. First make srpc_info (debugfs or proc entry) writeable. This means pushing three values, the flavor, the destination direction, and the source net, o2ib1 for example, into srpc_info. Sadly source net is missing from srpc_info. One option is to push all the settings at the same time to srpc_info. The other option is create new individual files to handle each change in sysfs. In that case we run into making the change atomic. Any ideas?

[Artem Blagodarenko (Inactive)]

Hello James,

Just what to remind that params config file has reserved field to store which means is used to execute configuration string. For example:

#01 (224)marker   2 (flags=0x01, v2.8.55.0) general         'ldlm.namespaces.*mdc*.lru_size' Wed Nov 23 13:29:33 2016- #02 (128)set_param 0:general  1:ldlm.namespaces.*mdc*.lru_size=0  2:lctl   #03 (224)END   marker   2 (flags=0x02, v2.8.55.0) general         'ldlm.namespaces.*mdc*.lru_size' Wed Nov 23 13:29:33 201

Here "lctl" is stored in 2nd field which means configuration need to be execute using lctl set_param with given parameters. The original idea was to save the place to extend functionality and use any other method. But lctl set_param -P syntax need to be extended to store all permanent parameters in the same params file.

[James A Simmons]

I was looking for a way to set it with the upcall/udev event. So how failnode.nid is handled inspired me. We can use something like:

lctl set_param osc.lustre-OST*.srpc_info="default=skpi".

A patch should be coming shortly.

[James A Simmons]

New ticket replaces this work