[LU-532] 2.1 does not filter extended attributes list based on permissions, showing entire list at all times

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.1.0, Lustre 2.2.0, Lustre 2.3.0, Lustre 2.1.2
• Fix Version/s: Lustre 2.3.0, Lustre 2.1.2
• Type: Bug
• Priority: Blocker
• Reporter: Lukasz Flis
• Assignee: nasf (Inactive)
• Resolution: Fixed

Environment

Client: lustre 1.8.6-wc1
Servers:
lustre-2.0.65-2.6.18_238.12.1.el5_lustre_ga34dd87
lustre-modules-2.0.65-2.6.18_238.12.1.el5_lustre_ga34dd87
lustre-ldiskfs-3.3.0-2.6.18_238.12.1.el5_lustre_ga34dd87

• Severity: 3

Description

There seems to be a problem with user_xattr when using 1.8.6-wc1 client with
newest lustre-master build:

cd /mnt/lustre/scratch/people/b14flis
touch test

mv test /tmp

mv: getting attribute `trusted.lma' of `trusted.lma': Operation not permitted
mv: getting attribute `trusted.link' of `trusted.link': Operation not permitted
mv: getting attribute `trusted.lov' of `trusted.lov': Operation not permitted
mv: getting attribute `trusted.lma' of `trusted.lma': Operation not permitted
mv: getting attribute `trusted.link' of `trusted.link': Operation not permitted
mv: getting attribute `trusted.lov' of `trusted.lov': Operation not permitted

strace dump:
...
fgetxattr(3, "trusted.link", 0x0, 0) = -1 EPERM (Operation not permitted)
fgetxattr(3, "trusted.lov", 0x0, 0) = -1 EPERM (Operation not permitted)
...
lgetxattr("test", "trusted.lma", 0x0, 0) = -1 EPERM (Operation not permitted)
...
lgetxattr("test", "lustre.lov", "\xd0\x0b\xd1\x0b\x01\x00\x00\x00E\x1d\x00\x00\x00\x00\x00\x00\xe2\x00/\x00\x02\x00\x00\x00\x00\x00\x10\x00\x01\x00\x00\x00nE\x0b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0c\x00\x00", 56) = 56
lsetxattr("/tmp/test", "lustre.lov", "\xd0\x0b\xd1\x0b\x01\x00\x00\x00E\x1d\x00\x00\x00\x00\x00\x00\xe2\x00/\x00\x02\x00\x00\x00\x00\x00\x10\x00\x01\x00\x00\x00nE\x0b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0c\x00\x00", 56, 0) = -1 EOPNOTSUPP (Operation not supported)
getxattr("test", "system.posix_acl_access", 0x7fff6edfae60, 132) = -1 ENODATA (No data available)

serwer side error:

Lustre: 10440:0:(mdt_xattr.c:374:mdt_reint_setxattr()) client miss to set OBD_MD_FLCTIME when setxattr: [object [0x2002f0053:0x38:0x0]] [valid 68719476736]

mount options:
MDT:
/dev/mapper/scratch-mdt on /mnt/lustre/scratch-mdt type lustre (rw,acl,user_xattr)

Client:
172.16.126.1@tcp:/scratch on /mnt/lustre/scratch type lustre (rw,nosuid,nodev,user_xattr,localflock,acl)

We can supply more information if needed.

–
Lukasz Flis
ACC Cyfronet AGH
Poland

Issue Links

• duplicates: LU-1838 1.8 client does not filter 2.x server...

Comments

[Andreas Dilger]

Is this test being run ad a regular user or as root?

For regular users, the "trusted.*" xattrs are inaccessible.

The server side error is definitely incorrect (though useful). It is legal (though unusual) for clients not to send timestamps with the RPCs, but that means the server timestamps will be used. The client should send the ctime with the setxattr, since we normally use the client timestamps for file updates to avoid clock-skew issues visible on the client.

[Lukasz Flis]

This test was done by regular user.
OS is RHEL5.6

The error message appears even when lustre is mounted w/o xattributes.
172.16.193.1@o2ib:/scratch on /mnt/lustre/scratch type lustre (rw,nosuid,nodev,flock)

We also have other Lustre filesystem mounted on the same machine but served via 1.8.6 servers and
the message doesn't appear.

[Oleg Drokin]

Ok, I think I tracked this one down.

The problem in 2.1 is all permission checks are done at MDD level and wthe user info does not filter down to ldiskfs it seems, so the ldiskfs check does not work and all attributes are always present in listxattr output.
Now of course the man page says that the FS is not supposed to always filter the list, so there is a bug in mv (or whatever lib it uses) too.

Additionally I am not sure if we want to do filtering that we want to do it on server as opposed to the filtering on client so that the client can always cache the full list and save RPCs on subsequent access

[Andreas Dilger]

I think it makes more sense to do the xattr list filtering on the client, since it is possible that root will want to access all the xattrs on the client at some point (e.g. for backup). Moving discussion about xattr cache to a separate bug LU-549.

I've submitted http://review.whamcloud.com/1161 to improve the MDS-side warning message to make this problem easier to debug in the future. The 1.8 client should also needs a patch so that it sends OBD_MD_FLCTIME for setxattr - looks like mdc_xattr_common() is the right place for it.

[Oleg Drokin]

Updated the title to indicate this is not an interop issue.

Lukasz, this does not affect the data integrity or anything similar, so I think it should not be a big area of concern for you at this time.

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el6,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,sles11,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el6,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,ubuntu1004,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,ofa #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,ofa #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el6,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,ofa #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,inkernel #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,ofa #329
LU-532 mdt: improve xattr ctime warning message (Revision 622b1b48a1fa9dc8c6954fa7ca596358b05af888)

Result = SUCCESS
Oleg Drokin : 622b1b48a1fa9dc8c6954fa7ca596358b05af888
Files :

• lustre/mdt/mdt_xattr.c

[Andreas Dilger]

This is still a problem for Lustre 2.3.0. Running sanity.sh test_102j() as a non-root user with a lustre-aware tar still shows lots of spurious errors:

tar: d0.sanity/d102: Warning: Cannot fgetxattr: Operation not permitted
tar: d0.sanity/d102: Warning: Cannot fgetxattr: Operation not permitted
tar: d0.sanity/d102: Warning: Cannot fgetxattr: Operation not permitted
tar: d0.sanity/d102/file3-1-1: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file3-1-1: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file3-1-1: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file3-0-2: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file3-0-2: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file3-0-2: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file1-0-1: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file1-0-1: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file1-0-1: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file1-1-3: Warning: Cannot lgetxattr: Operation not permitted
tar: d0.sanity/d102/file1-1-3: Warning: Cannot lgetxattr: Operation not permitted

Running this under strace shows it trying to access the trusted xattrs:

lgetxattr("d0.sanity/d102/file2-1-1", "trusted.lma", 0x229d690, 1024) = -1 EPERM (Operation not permitted)
lgetxattr("d0.sanity/d102/file2-1-1", "trusted.link", 0x229d690, 1024) = -1 EPERM (Operation not permitted)
lgetxattr("d0.sanity/d102/file2-1-1", "trusted.lov", 0x229d690, 1024) = -1 EPERM  (Operation not permitted)

This would essentially make the RHEL6 tar unusable on a 2.x client, so I'm elevating it to blocker status for 2.1.2 and 2.3.

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,sles11,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,client,el6,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,server,el6,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,client,el5,ofa #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,server,el5,ofa #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,el6,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,server,el6,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,el5,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,server,el5,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,server,el5,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,server,el5,ofa #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » x86_64,client,el5,ofa #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[Build Master (Inactive)]

Integrated in lustre-b2_1 » i686,client,el5,inkernel #41
LU-532 mdt: improve xattr ctime warning message (Revision a9388033cccf0bb5306560cfb76436dd1f1c2431)

Result = SUCCESS
Oleg Drokin : a9388033cccf0bb5306560cfb76436dd1f1c2431
Files :

• lustre/mdt/mdt_xattr.c

[nasf (Inactive)]

Patch for filtering out trusted. xattr for non-root users:

http://review.whamcloud.com/#change,2490

[Peter Jones]

Landed for 2.1.2 and 2.3

[Wojciech Turek (Inactive)]

I am still seeing these when moving a file as an ordinary user. I am using 2.1.2 server and 1.8.8 client

mv test0000 /scratch/mb435/
mv: getting attribute `trusted.lma' of `trusted.lma': Operation not permitted
mv: getting attribute `trusted.link' of `trusted.link': Operation not permitted
mv: getting attribute `trusted.lov' of `trusted.lov': Operation not permitted

On the server log shows

Aug 30 12:34:40 10.143.245.207 mds07 kernel: Lustre: 5499:0:(mdt_xattr.c:377:mdt_reint_setxattr()) lhome-MDT0000: client miss to set OBD_MD_FLCTIME when setxattr system.posix_acl_access: [object [0x200000bf7:0x9b:0x0]] [valid 68719476736]

[Andreas Dilger]

Filed LU-1838 to track landing for 1.8.9.