[LU-6528] permission denied over NFS (intermittent

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.8.0
• Affects Version/s: Lustre 2.5.3
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Frederik Ferner (Inactive)

Description

(This might be related to LU-5730 but as the symptoms are different, I'm opening a new ticket.)

Our users are

Issue Links

• is related to: LU-7630 permission denied over NFS
• is related to: LU-5730 intermittent I/O errors for some directories
• is related to: LU-13791 Capabilities are not effective

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/14978/
Subject: LU-6528 mdt: keep FS capability for intent_getattr
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a55aebd70e9073ef1d1bd7de280c8393af0ea30d

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: http://review.whamcloud.com/14978
Subject: LU-6528 mdt: keep FS capability for intent_getattr
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b288a18408a59cf3ac8e3d9bd3ed074547689c57

[Frederik Ferner (Inactive)]

FWIW, I've been running our reproducer for nearly 24h after applying the patch. So far we have not seen the permission denied issue at all, so it looks good AFAICT.

[Lai Siyao]

Once it works(I'll wait until tomorrow), I'll put it on gerrit later.

[Frederik Ferner (Inactive)]

I've had the reproducer running over night so far it hasn't triggered the issue, so looking good. I'll leave the reproducer running...

I noticed the patch had been directly attached to the ticket and not to review.whamcloud.com as I would have expected, does this mean it's not yet a good idea to consider applying it to production systems?

[Frederik Ferner (Inactive)]

I've applied the proposed patch on the MDS and I'm now running the tests.... I guess we'll see tomorrow, though I'll keep running those tests for a t least a day continuously...

[Lai Siyao]

I just uploaded a possible patch LU-6528.diff for this, could you apply it on MDS and test again?

[Lai Siyao]

This patch will keep FS capability for intent getattr, this is because NFSD will raise FS capability to allow exportfs_decode_fh() to reconnect dentry into dcache, if MDS dropped this capability, reconnect path may fail with -EACCES.

[Frederik Ferner (Inactive)]

As the NFS export (on a RHEL6 machine) currently doesn't contain either "no_subtree_check" or "subtree_check" and by my reading of the documentation, it seems no_subtree_check is in force, yes. (also confirmed by running exportfs -v.)

I can try (at least on our test system) to re-export with explicit subtree_check and re-test.

Cheers,
Frederik

[Lai Siyao]

Is "no_subtree_check" set for the NFS export? It looks like if this feature is set (it's set by default), nfsd_set_fh_dentry() won't set fsuid correctly, but raise capability to allow reconnect_path(), but MDS cleared all FS privileges for lookup (see old_init_ucred()).

[Lai Siyao]

yes, it's fsuid that's used in permission check, and it looks like it's mistakenly set somewhere. I'm still checking related code, will update you later.