[LU-1441] mdt.nosquash_nids is not honored when the client is on the MDS

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.1.0
• Type: Bug
• Priority: Minor
• Reporter: Kilian Cavalotti
• Assignee: Cliff White (Inactive)
• Resolution: Fixed
• Labels: root_squash,
• Severity: 3

Description

When using root squash, if we add the MDS NID in the mdt.nosquash_nids list and mount the filesystem on the MDS, the nosquash_nids list is not honored, and the client gets ENOPERM when accessing root-only files.

To reproduce:

[root@gl1 ~]# shine status
FILESYSTEM COMPONENTS STATUS (gl)
+-----+--+----------+--------+
|type |# |  nodes   | status |
+-----+--+----------+--------+
|MGT  |1 |gl1       |online  |
|MDT  |1 |gl1       |online  |
|OST  |4 |gl[2-3]   |online  |
|CLI  |3 |gl[1,4-5] |mounted |
+-----+--+----------+--------+
[root@gl1 ~]# clush -w gl[1-5] lctl list_nids | sort
gl1: 10.2.0.102@tcp
gl2: 10.2.0.103@tcp
gl3: 10.2.0.104@tcp
gl4: 10.2.0.105@tcp
gl5: 10.2.0.106@tcp
[root@gl1 ~]# lctl get_param mdt.gl*.*squash*
mdt.gl-MDT0000.nosquash_nids=10.2.0.[102,105]@tcp
mdt.gl-MDT0000.root_squash=65534:65534}}

gl1 (MDS) and gl4 (client) are in the nosquash_nids list, and gl5 (client) is not.
We can check that everything is in order on gl4 and gl5: gl4 can access root-only content, and gl5 can not.

[root@gl1 ~]# ssh gl4 ls -al /gl/root
total 8
drwx------ 2 root root 4096 May 25 09:40 .
drwxr-xr-x 4 root root 4096 May 25 09:40 ..
[root@gl1 ~]# ssh gl4 ls -al /gl/
total 16
drwxr-xr-x   4 root root 4096 May 25 09:40 .
dr-xr-xr-x. 25 root root 4096 May 24 14:08 ..
drwxr-xr-x   2 root root 4096 May 25 09:37 .lustre
drwx------   2 root root 4096 May 25 09:40 root
[root@gl1 ~]# ssh gl5 ls -al /gl/
total 16
drwxr-xr-x   4 root root 4096 May 25 09:40 .
dr-xr-xr-x. 28 root root 4096 May 24 14:08 ..
drwxr-xr-x   2 root root 4096 May 25 09:37 .lustre
drwx------   2 root root 4096 May 25 09:40 root
[root@gl1 ~]# ssh gl5 ls -al /gl/root
ls: cannot open directory /gl/root: Permission denied

gl1 (MDS) is also in the nosquash_nids list, so it should get access. But it doesn't:

[root@gl1 ~]# ls -al /gl/
total 16
drwxr-xr-x   4 root root 4096 May 25 09:40 .
dr-xr-xr-x. 25 root root 4096 May 25 09:39 ..
drwxr-xr-x   2 root root 4096 May 25 09:37 .lustre
drwx------   2 root root 4096 May 25 09:40 root
[root@gl1 ~]# ls -al /gl/root/
ls: cannot open directory /gl/root/: Permission denied

We can verify that the nosquash_nids syntax is correct by adding gl5 to the list of authorized NIDs:

[root@gl1 ~]# lctl conf_param gl.mdt.nosquash_nids="10.2.0.[102,105,106]@tcp"
[root@gl1 ~]# ssh gl5 ls -al /gl/
total 16
drwxr-xr-x   4 root root 4096 May 25 09:40 .
dr-xr-xr-x. 28 root root 4096 May 24 14:08 ..
drwxr-xr-x   2 root root 4096 May 25 09:37 .lustre
drwx------   2 root root 4096 May 25 09:40 root
[root@gl1 ~]# ssh gl5 ls -al /gl/root/
total 8
drwx------ 2 root root 4096 May 25 09:40 .
drwxr-xr-x 4 root root 4096 May 25 09:40 ..

Works.

So it seems there's a corner case when nosquash_nids contains the MDS's NID.

Comments

[Johann Lombardi (Inactive)]

Did you try to add the loopback interface (i.e. 0@lo)? Just my 2 cents ...

[Kilian Cavalotti]

Ah, that works, indeed. That would probably be worth a note in the documentation, then. Thanks!

[Peter Jones]

Thanks Johann! Cliff, could you please ensure that the documentation covers this case?

[John Fuchs-Chesney (Inactive)]

This called for a document update.