Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Lustre 2.4.0
-
3
-
9459
Description
mdd_acl_set() sets the ACL xattr synchronously if /proc/fs/lustre/mdd/lustre-MDT0000/sync_permission is set (it is set by default). This can have a large performance impact for operations like cp -a or tar --xattr. Should this be the default behavior? How does this relate to other permissions-related changed like chown, chmod, etc.? It would be good to explain the security implications of this setting in code comments and in the Lustre manual.
Some background information:
http://wiki.lustre.org/index.php/Architecture_-_Version_Based_Recovery#Permissions
https://projectlava.xyratex.com/show_bug.cgi?id=15390
http://git.whamcloud.com/gitweb?p=lustre.git;a=commit;h=0509770778396825ba1d7f1b42be5993aac1d6f3
LLNL-bug-ID: TOSS-2207
Attachments
Issue Links
Activity
Ned, your second assessment is correct, and I should have made that more clear in my comment. The danger is that the user thinks the chmod (or chown by root) succeeded, but it was lost during recovery, and this exposes files in the directory that shouldn't be visible to other users. Mike is correct that if file3/4 are created before MDS recovery (while the chmod/chown is still in the MDS cache) then they will be refused during replay because the dir1 permission change was lost when client1 failed and the creates will depend on the now-missing dir1 version. However, if file3/4 are created after MDS recovery (with chmod/chown lost) they will succeed.
I guess sync_permissions is over & above what POSIX requires even of a local filesystem, since a local filesystem would lose the chmod after a crash and the user might forget to re-do it, but at least there was a visible interruption to the user when the local node crashed. With a distributed filesystem, the failing node might be out of sight of the user.
Andreas, yes, chmod changed version of directory, so depended replays will be denied. But nothing prevents just regular creations in that directory after recovery, it will stay without permission for a while. So sync_permission flag is not just about recovery but more like policy when server guarantee all such changes to be done for sure, without sync_permission all works as before but permission might be lost, note this problem exists in all Lustre versions and is not about VBR. All further optimization are just attempts to find some balance between security and performance.
I think I understand the point of the example now. The touch is predicated on the knowledge that chmod succeeded and the directory has secure permissions. If the chmod is lost, the directory contents become exposed to unauthorized users. I was assuming in my last comment that client3 couldn't see the results of the chmod until it was committed to disk, but now I suspect that it could. Do I have that right?
Andreas, thanks for the explanation, but the pieces still aren't quite fitting together for me. In your example, dir1 perms change from 755 to 700, so non-owners lack write permission before and after the chmod. I'm not sure if that's what you intended.
Given the optimization recently landed, we're only interested in the case where permissions are reduced. In that case, I imagine the spirit of the example is that touch would normally succeed before the chmod, but fail after. But if the chmod isn't replayed in recovery, client3 cannot distinguish that from the case where client1 never ran chmod (because it crashed), and therefore it is perfectly consistent to allow the touch to succeed. The directory would still have the old permissions which permitted file creation.
Suggestion: That example (once the details have been confirmed) would make an excellent addition to the manual.
I filed LUDOC-180 to track the documentation for this /proc tunable, and this one can be closed since the patch to avoid sync operations for regular files and non-permission setattrs has landed for 2.5.0.
It could potentially also be cherry-picked for 2.4.x and 2.1.x.