[LU-1482] attribute not permitted on Lustre versions 2.x

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.1.0, Lustre 2.2.0, Lustre 2.1.1
• Fix Version/s: Lustre 2.9.0
• Type: Bug
• Priority: Major
• Reporter: Pierre Choukroun
• Assignee: Dmitry Eremin (Inactive)
• Resolution: Fixed

Environment

MDS Redhat 6 64 bit with Lustre version 2.2
3 OSSs Redhat 5 64 bit with Lustre version 2.2
Clients Redhat 5 64 bit with Lustre version 2.2

Description

Attribute are not permitted on version 2.x but works on version 1.8.x

to reproduce the problem

Lustre clients are mounted with option acl,user_xattr
wipp-mds2@tcp:/WIPP /mnt/Lustre lustre defaults,acl, 0 0

#!/bin/bash
rm -f dummy_file_test_xattr
touch dummy_file_test_xattr
chown nobody:nobody dummy_file_test_xattr
su -s/bin/bash nobody -c "setfacl -m g:bin:rw dummy_file_test_xattr"
su -s/bin/bash nobody -c "getfacl dummy_file_test_xattr"
su -s/bin/bash bin -c "setfattr -n user.test.xattr -v 123456 dummy_file_test_xattr"
su -s/bin/bash bin -c "getfattr -d dummy_file_test_xattr"

the output
setfattr: dummy_file_test_xattr: Operation not permitted

Issue Links

• is duplicated by: LU-1343 attribute not permitted on Lustre v...

Comments

[Andreas Dilger]

This does appear to be a valid test case. Looking at the kernel code, it seems that users that do not own a file should still be able to create user.* xattrs on a file they have write permission on. I suspect some strange interaction between ACLs and the xattr permission checking is the cause. Is this reproducible with a RHEL6 client (filesystem could be mounted temporarily on the MDS for testing purposes)?

However, I'm reducing the severity of this problem, since it is a relatively obscure use case, and is unlikely to be fixed in the short term.

[Pierre Choukroun]

The attribute problem may be a corner case for you, but it is a show stopper for users of the LHC Computing Grid’s STORM storage resource manager (http://storm.forge.cnaf.infn.it/) that want to upgrade to Lustre 2.X Attributes are used to store checksums of every file which, after every gridftp transfer, are compared between source and destination. As long as this bug remains unfixed, we cannot upgrade Lustre. I hope you will upgrade the priority for fixing this bug so that we and the other STORM sites around the world using Lustre can upgrade.

[Goncalo Filipe Mauricio dos Santos Borges]

This issue seems to be stalled since June 2012 with no evolution whatsoever. The attribute issue is really a showstopper in our case since it is avoiding the migration to lustre 2.1 and its interaction with the StoRM SRM system. We are a Tier-2 for WLCG with a storage capacity of about 600 TB. As we, there are a couple of more sites in the same situation, and therefore, the impact of this issue is not small.

I've just tried it with a lustre 1.8 client in RH6:

1. touch dummy_file_test_xattr
2. chown storm:storm dummy_file_test_xattr
3. su -s /bin/bash storm -c "setfacl -m g:auger:rw dummy_file_test_xattr"
4. su -s/bin/bash storm -c "getfacl dummy_file_test_xattr"
5. file: dummy_file_test_xattr
6. owner: storm
7. group: storm
   user::rw-
   group::r--
   group:auger:rw-
   mask::rw-
   other::r--
8. su -s/bin/bash auger001 -c "setfattr -n user.test.xattr -v 123456 dummy_file_test_xattr"
   setfattr: dummy_file_test_xattr: Operation not supported

[Goncalo Filipe Mauricio dos Santos Borges]

Dear All

After some investigation, mostly from two of my colleagues, we were able to pinpoint the problem.

The main difference between lustre 1.8 and 2.x series is that the acls are now checked on the server side, while in the older 1.8 versions, these was done in the client side. This lead to some restructure of the code, and the current implementation relies on very simple validations.

These very simple validations are done in mdd_xattr_sanity_check function under lustre/mdd/mdd_object.c. The fundamental piece of code is

if ((uc->mu_fsuid != tmp_la->la_uid) &&
!mdd_capable(uc, CFS_CAP_FOWNER))
RETURN(-EPERM);

which basically tells you that only the owner of the file is allowed to change the extended atributes.

We have developed a dirty hack to overcome this issue but we do not want to make it public since it may not be general enough, or may have hidden problems since I'm sure we do not have a full understanding of the code. I'm certain that Whamcloud people can do it much better, and now that we have identified the problem, maybe the fix could be delivered faster than expected.

Thank you
Goncalo

[Jeremy Maris (Inactive)]

We are also a Tier-2 for WLCG, and have held off upgrading from Lustre 1.8.9 because of this problem.
We really do need to move to Lustre 2.x!!

The bug is is still there in v2.4.1; tested with ext4/ldiskfs back-end:
-----------

1. rpm -qa | grep lustre-2
   lustre-2.4.1-2.6.32_358.18.1.el6_lustre.x86_64.x86_64

1. ./testscript
2. file: dummy_file_test_xattr
3. owner: nobody
4. group: nobody
   user::rw-
   group::r--
   group:bin:rw-
   mask::rw-
   other::r--

setfattr: dummy_file_test_xattr: Operation not permitted

-----------

Wil this problem ever be addressed?

Jeremy

[Wang Shilong (Inactive)]

I think maybe something like this should fix issue?

diff --git a/lustre/mdd/mdd_object.c b/lustre/mdd/mdd_object.c
index 5938bc9..441c3db 100644
--- a/lustre/mdd/mdd_object.c
+++ b/lustre/mdd/mdd_object.c
@@ -951,8 +951,8 @@ static int mdd_xattr_sanity_check(const struct lu_env *env,
                    !md_capable(uc, CFS_CAP_FOWNER))
                        RETURN(-EPERM);
        } else {
-               if ((uc->uc_fsuid != tmp_la->la_uid) &&
-                   !md_capable(uc, CFS_CAP_FOWNER))
+               if (uc->uc_fsuid != tmp_la->la_uid &&
+                     !md_capable(uc, CFS_CAP_CHOWN))
                        RETURN(-EPERM);
        }

We don't need OWNER rights always..

[Gerrit Updater]

Wang Shilong ([e-mail]) uploaded a new patch: http://review.whamcloud.com/15959
Subject: LU-1482 mdd: remove owner rights check for xattr permission
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: cd58efc71705edb7ab96bc64d295d22948312d28

[Gerrit Updater]

Wang Shilong ([e-mail]) uploaded a new patch: http://review.whamcloud.com/19258
Subject: LU-1482 mdd: Setting xattr are properly checked with and without ACLs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8d29a17482ed41ef654d538dd3a667834b5dcf37

[Gerrit Updater]

Dmitry Eremin ([e-mail]) uploaded a new patch: http://review.whamcloud.com/21496
Subject: LU-1482 mdd: Setting xattr are properly checked with and without ACLs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2bd076858bd40532985d7caa1609266a7fe88927

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/21496/
Subject: LU-1482 mdd: Setting xattr are properly checked with and without ACLs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 93e459f4d0604cccb31e5cc0a1677499d48fff0b

[Peter Jones]

Landed for 2.9