[LU-2185] setfacl not working correctly

• Type: Bug
• Resolution: Duplicate
• Priority: Minor
• Affects Version/s: Lustre 2.1.3
• Environment: sles11sp1 2.1.3 (NASA version) client with 2.1.2 (NAS version) server bits
• Severity: 3
• Assignee: Jian Yu
• Reporter: James Karellas (Inactive)

Description

looked at LU-974, but not exactly the same problem.

I did one test on lustre and the same test on /tmp (ext3). Started by clearing out all
setfacl's using "setfacl -R -b" on top level directory before each example is started.

=================
lustre filesystem
=================

[client1:workdir]$ pwd
/nobackup/workdir
[client1:workdir]$ setfacl -M setfacl.jdk .
[client1:workdir]$ ls -ld ../workdir/
drwxrwxr-x+ 5 user2 g26137 4096 Oct 5 13:10 ../workdir/

[client1:workdir]$ mkdir test1
[client1:workdir]$ touch test1/file1
[client1:workdir]$ getfacl test1/

1. file: test1
2. owner: user2
3. group: g26137
   user::rwx
   user:user1:rwx #effective:---
   user:user2:rwx #effective:---
   user:user3:rwx #effective:---
   user:user4:rwx #effective:---
   group::rwx #effective:---
   mask::---
   other::---
   default:user::rwx
   default:user:user1:rwx
   default:user:user2:rwx
   default:user:user3:rwx
   default:user:user4:rwx
   default:group::rwx
   default:mask::rwx
   default:other::r-x

[client1:workdir]$ getfacl test1/file1

1. file: test1/file1
2. owner: user2
3. group: g26137
   user::rw-
   user:user1:rwx #effective:---
   user:user2:rwx #effective:---
   user:user3:rwx #effective:---
   user:user4:rwx #effective:---
   group::rwx #effective:---
   mask::---
   other::---

setfacl file used:
[client1:workdir]$ cat setfacl.jdk

1. file: workdir
2. owner: user2
3. group: g26137
   user::rwx
   user:user1:rwx
   user:user2:rwx
   user:user3:rwx
   user:user4:rwx
   group::rwx
   mask::rwx
   other::r-x
   default:user::rwx
   default:user:user1:rwx
   default:user:user2:rwx
   default:user:user3:rwx
   default:user:user4:rwx
   default:group::rwx
   default:mask::rwx
   default:other::r-x

=================
tmp/ext3 filesystem
=================

[client1:workdir]$ pwd
/tmp/workdir
[client1:workdir]$ setfacl -M setfacl.jdk .
[client1:workdir]$ ls -ld ../workdir/
drwxrwxr-x+ 5 user2 g26137 4096 Oct 5 13:10 ../workdir/

[client1:workdir]$ mkdir test2
[client1:workdir]$ touch test2/file2
[client1:workdir]$ getfacl test2

1. file: test2
2. owner: user2
3. group: g26137
   user::rwx
   user:user1:rwx
   user:user2:rwx
   user:user3:rwx
   user:user4:rwx
   group::rwx
   mask::rwx
   other::r-x
   default:user::rwx
   default:user:user1:rwx
   default:user:user2:rwx
   default:user:user3:rwx
   default:user:user4:rwx
   default:group::rwx
   default:mask::rwx
   default:other::r-x

[client1:workdir]$ getfacl test2/file2

1. file: test2/file2
2. owner: user2
3. group: g26137
   user::rw-
   user:user1:rwx #effective:rw-
   user:user2:rwx #effective:rw-
   user:user3:rwx #effective:rw-
   user:user4:rwx #effective:rw-
   group::rwx #effective:rw-
   mask::rw-
   other::r--

setfacl file used:
[client1:workdir]$ cat setfacl.jdk

1. file: workdir
2. owner: user2
3. group: g26137
   user::rwx
   user:user1:rwx
   user:user2:rwx
   user:user3:rwx
   user:user4:rwx
   group::rwx
   mask::rwx
   other::r-x
   default:user::rwx
   default:user:user1:rwx
   default:user:user2:rwx
   default:user:user3:rwx
   default:user:user4:rwx
   default:group::rwx
   default:mask::rwx
   default:other::r-x
   =================

Let me know if you need more information.

jdk

Issue Links

• duplicates: LU-974 Lustre does not ignore umask when default ACL with mask is set

Comments

[Jian Yu]

I just verified that the patch in http://review.whamcloud.com/1972 for LU-974 fixed the above issue. Here is the test result with build http://build.whamcloud.com/job/lustre-reviews/10353/ on SLES11SP1 client:

user2@client-3:~> umask 77
user2@client-3:~> umask -S
u=rwx,g=,o=
user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    413640   7059392   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-11-19 22:38 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:rw-
user:user2:rwx                  #effective:rw-
user:user3:rwx                  #effective:rw-
user:user4:rwx                  #effective:rw-
group::rwx                      #effective:rw-
mask::rw-
other::r--

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

So, let's close this ticket as a duplicate of LU-974.

[Jian Yu]

Here is the test result with umask 77:

user2@client-3:~> umask 77
user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    410224   7063632   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-11-19 00:30 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx                  #effective:---
user:user2:rwx                  #effective:---
user:user3:rwx                  #effective:---
user:user4:rwx                  #effective:---
group::rwx                      #effective:---
mask::---
other::---
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:---
user:user2:rwx                  #effective:---
user:user3:rwx                  #effective:---
user:user4:rwx                  #effective:---
group::rwx                      #effective:---
mask::---
other::---

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

So, with umask 77, the test result is the same as James'. Let me take a look at the patch http://review.whamcloud.com/1972 for LU-974.

[Jian Yu]

Our default umask is 77. Please test with 77.

OK, will do.

[Jay Lan (Inactive)]

Our default umask is 77. Please test with 77.

[Jian Yu]

From my test result, it turned out to be the same issue as that in LU-974, which was just reopened because the real patches had not been landed yet.

While I performing the above test, the umask value was:

user2@client-3:~> umask
0022

After I changed the value to 0002 and performed the same test again, the result on Lustre filesystem became correct:

user2@client-3:~> umask 0002
user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    410228   7063628   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-11-01 04:42 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:rw-
user:user2:rwx                  #effective:rw-
user:user3:rwx                  #effective:rw-
user:user4:rwx                  #effective:rw-
group::rwx                      #effective:rw-
mask::rw-
other::r--

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x                  

Hi James,

Could you please take a look at the umask value on your test system? Thanks.

[Jian Yu]

From the above test result, the effective rights mask on Lustre filesystem was incorrect:

For directory, it was "mask::r-x", but should be "mask::rwx".
For regular file, it was "mask::r--", but should be "mask::rw-".

[Jian Yu]

Here is my test result:

Lustre Client: 2.1.3
Lustre Build: http://build.whamcloud.com/job/lustre-b2_1/121/
Distro/Arch: SLES11SP1/x86_64 (kernel version: 2.6.32.36-0.5-default)

Lustre Server: 2.1.2
Lustre Build: http://build.whamcloud.com/job/lustre-b2_1/91/
Distro/Arch: RHEL6.3/x86_64 (kernel version: 2.6.32-220.17.1.el6_lustre.x86_64)

client-3:~ # mount | grep lustre
fat-intel-4@tcp:/lustre on /nobackup/workdir type lustre (rw,acl,user_xattr)

[root@fat-intel-4 ~]# mount | grep lustre
/dev/sdc9 on /nobackup/mds1 type lustre (rw,acl,errors=panic,iopen_nopriv,user_xattr)
/dev/sdc5 on /nobackup/ost1 type lustre (rw)
/dev/sdc6 on /nobackup/ost2 type lustre (rw)
/dev/sdc7 on /nobackup/ost3 type lustre (rw)
/dev/sdc8 on /nobackup/ost4 type lustre (rw)

=================
lustre filesystem
=================

user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    410224   7063632   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-10-31 01:18 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx                  #effective:r-x
user:user2:rwx                  #effective:r-x
user:user3:rwx                  #effective:r-x
user:user4:rwx                  #effective:r-x
group::rwx                      #effective:r-x
mask::r-x
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:r--
user:user2:rwx                  #effective:r--
user:user3:rwx                  #effective:r--
user:user4:rwx                  #effective:r--
group::rwx                      #effective:r--
mask::r--
other::r--

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

=================
tmp/ext3 filesystem
=================

user2@client-3:~> df -T /tmp/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3    20635700   1492244  18095220   8% /
user2@client-3:~> setfacl -R -b /tmp/workdir
user2@client-3:~> cd /tmp/workdir
user2@client-3:/tmp/workdir> pwd
/tmp/workdir
user2@client-3:/tmp/workdir> setfacl -M setfacl.jdk .
user2@client-3:/tmp/workdir> ls -ld ../workdir/
drwxrwxr-x+ 2 user2 g26137 4096 2012-10-31 01:26 ../workdir/
user2@client-3:/tmp/workdir> mkdir test2
user2@client-3:/tmp/workdir> touch test2/file2
user2@client-3:/tmp/workdir> getfacl test2
# file: test2
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/tmp/workdir> getfacl test2/file2
# file: test2/file2
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:rw-
user:user2:rwx                  #effective:rw-
user:user3:rwx                  #effective:rw-
user:user4:rwx                  #effective:rw-
group::rwx                      #effective:rw-
mask::rw-
other::r--

user2@client-3:/tmp/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

[Jian Yu]

Sorry for the late response. I'll look into this right away.

[James Karellas (Inactive)]

lustre mounted with the following options:

acl,errors=panic,iopen_nopriv,user_xattr

[Peter Jones]

Yujian

Could you please look into this one?

Thanks

Peter