[LU-13361] setregid or setreuid does not work as expected when cli2mdt SSK is on

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.12.3, Lustre 2.12.4
• Type: Bug
• Priority: Minor
• Reporter: Hans Henrik Happe
• Assignee: Sebastien Buisson
• Resolution: Unresolved

Environment

Centos 7.7. ZFS on both MDTs and OSTs. TCP network.

• Severity: 3

Description

During tests of gocryptfs 1.7 I found that it didn't work on-top of Lustre with cli2mdt SSK. After a bit of tracing I found that after some setreuid/setregid calls it hadn't got permission to open files.

I've attached a test program that causes the issue. The 501 uid/gid is arbitrary, but it must be run in a directory with with the same uid/gid as chosen here.

When cli2mdt SSK is of it works as expected. When cli2mdt SSK is on it returns:

"open: Permission denied"

Cheers,
Hans Henrik

Comments

[Sebastien Buisson]

Hi,

I am able to reproduce this behavior. It stems from the fact that when using SSK, Lustre makes use of the identity upcall that is defined for the MDT targets. You can check with the command:

# lctl get_param mdt.*.identity_upcall
mdt.lustre-MDT0000.identity_upcall=/usr/sbin/l_getidentity
mdt.lustre-MDT0001.identity_upcall=/usr/sbin/l_getidentity

By default, as shown above, l_getidentity is defined as the identity upcall. For it to handle permissions, you have to create a file named /etc/lustre/perm.conf on your MDS nodes, with the following syntax for each line:

permission file format is like this:
{nid} {uid} {perms}

'*' nid means any nid
'*' uid means any uid
the valid values for perms are:
setuid/setgid/setgrp		-- enable corresponding perm
nosetuid/nosetgid/nosetgrp	-- disable corresponding perm

In the case of your test program, you can insert a line with:

* 0 setuid,setgid

It will grant setuid and setgid permissions to user root, from any client node.

Once you have created the file, remember to flush the identity cache on your MDS nodes by doing:

lctl set_param mdt.*.identity_flush=-1

This way, new content in /etc/lustre/perm.conf will be taken into account.

Alternatively, you can disable identity upcall by doing:

lctl set_param mdt.*.identity_upcall=NONE

In this case, Lustre grants setuid, setgid and setgrp permissions.

[Hans Henrik Happe]

Thanks Sebastien, that works for me.

I would love to help document this, but I'm not sure why there is a difference when turning on SSK?

[Sebastien Buisson]

This is because when SSK is enabled, credentials checking is carried out a little bit differently on server side.