[LU-2185] setfacl not working correctly Created: 15/Oct/12  Updated: 20/Nov/12  Resolved: 20/Nov/12

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.1.3
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: James Karellas Assignee: Jian Yu
Resolution: Duplicate Votes: 0
Labels: None
Environment:

sles11sp1 2.1.3 (NASA version) client with 2.1.2 (NAS version) server bits

Issue Links:
Duplicate
duplicates LU-974 Lustre does not ignore umask when def... Resolved
Severity: 3
Rank (Obsolete): 5226

 Description   

looked at LU-974, but not exactly the same problem.

I did one test on lustre and the same test on /tmp (ext3). Started by clearing out all
setfacl's using "setfacl -R -b" on top level directory before each example is started.

=================
lustre filesystem
=================

[client1:workdir]$ pwd
/nobackup/workdir
[client1:workdir]$ setfacl -M setfacl.jdk .
[client1:workdir]$ ls -ld ../workdir/
drwxrwxr-x+ 5 user2 g26137 4096 Oct 5 13:10 ../workdir/

[client1:workdir]$ mkdir test1
[client1:workdir]$ touch test1/file1
[client1:workdir]$ getfacl test1/

  1. file: test1
  2. owner: user2
  3. group: g26137
    user::rwx
    user:user1:rwx #effective:---
    user:user2:rwx #effective:---
    user:user3:rwx #effective:---
    user:user4:rwx #effective:---
    group::rwx #effective:---
    mask::---
    other::---
    default:user::rwx
    default:user:user1:rwx
    default:user:user2:rwx
    default:user:user3:rwx
    default:user:user4:rwx
    default:group::rwx
    default:mask::rwx
    default:other::r-x

[client1:workdir]$ getfacl test1/file1

  1. file: test1/file1
  2. owner: user2
  3. group: g26137
    user::rw-
    user:user1:rwx #effective:---
    user:user2:rwx #effective:---
    user:user3:rwx #effective:---
    user:user4:rwx #effective:---
    group::rwx #effective:---
    mask::---
    other::---

setfacl file used:
[client1:workdir]$ cat setfacl.jdk

  1. file: workdir
  2. owner: user2
  3. group: g26137
    user::rwx
    user:user1:rwx
    user:user2:rwx
    user:user3:rwx
    user:user4:rwx
    group::rwx
    mask::rwx
    other::r-x
    default:user::rwx
    default:user:user1:rwx
    default:user:user2:rwx
    default:user:user3:rwx
    default:user:user4:rwx
    default:group::rwx
    default:mask::rwx
    default:other::r-x

=================
tmp/ext3 filesystem
=================

[client1:workdir]$ pwd
/tmp/workdir
[client1:workdir]$ setfacl -M setfacl.jdk .
[client1:workdir]$ ls -ld ../workdir/
drwxrwxr-x+ 5 user2 g26137 4096 Oct 5 13:10 ../workdir/

[client1:workdir]$ mkdir test2
[client1:workdir]$ touch test2/file2
[client1:workdir]$ getfacl test2

  1. file: test2
  2. owner: user2
  3. group: g26137
    user::rwx
    user:user1:rwx
    user:user2:rwx
    user:user3:rwx
    user:user4:rwx
    group::rwx
    mask::rwx
    other::r-x
    default:user::rwx
    default:user:user1:rwx
    default:user:user2:rwx
    default:user:user3:rwx
    default:user:user4:rwx
    default:group::rwx
    default:mask::rwx
    default:other::r-x

[client1:workdir]$ getfacl test2/file2

  1. file: test2/file2
  2. owner: user2
  3. group: g26137
    user::rw-
    user:user1:rwx #effective:rw-
    user:user2:rwx #effective:rw-
    user:user3:rwx #effective:rw-
    user:user4:rwx #effective:rw-
    group::rwx #effective:rw-
    mask::rw-
    other::r--

setfacl file used:
[client1:workdir]$ cat setfacl.jdk

  1. file: workdir
  2. owner: user2
  3. group: g26137
    user::rwx
    user:user1:rwx
    user:user2:rwx
    user:user3:rwx
    user:user4:rwx
    group::rwx
    mask::rwx
    other::r-x
    default:user::rwx
    default:user:user1:rwx
    default:user:user2:rwx
    default:user:user3:rwx
    default:user:user4:rwx
    default:group::rwx
    default:mask::rwx
    default:other::r-x
    =================

Let me know if you need more information.

jdk

 Comments   
Comment by James Karellas [ 15/Oct/12 ]

To (try to) clarify: When ACLs are set on a file (individually), in my limited testing it works as expected. When
I create a directory and then file, the new ACLs are not propogated correctly. Give it a try and let me know what you come up with.

Comment by Peter Jones [ 15/Oct/12 ]

Yujian

Could you please look into this one?

Thanks

Peter

Comment by James Karellas [ 15/Oct/12 ]

lustre mounted with the following options:

acl,errors=panic,iopen_nopriv,user_xattr

Comment by Jian Yu [ 30/Oct/12 ]

Sorry for the late response. I'll look into this right away.

Comment by Jian Yu [ 31/Oct/12 ]

Here is my test result:

Lustre Client: 2.1.3
Lustre Build: http://build.whamcloud.com/job/lustre-b2_1/121/
Distro/Arch: SLES11SP1/x86_64 (kernel version: 2.6.32.36-0.5-default)

Lustre Server: 2.1.2
Lustre Build: http://build.whamcloud.com/job/lustre-b2_1/91/
Distro/Arch: RHEL6.3/x86_64 (kernel version: 2.6.32-220.17.1.el6_lustre.x86_64)

client-3:~ # mount | grep lustre
fat-intel-4@tcp:/lustre on /nobackup/workdir type lustre (rw,acl,user_xattr)

[root@fat-intel-4 ~]# mount | grep lustre
/dev/sdc9 on /nobackup/mds1 type lustre (rw,acl,errors=panic,iopen_nopriv,user_xattr)
/dev/sdc5 on /nobackup/ost1 type lustre (rw)
/dev/sdc6 on /nobackup/ost2 type lustre (rw)
/dev/sdc7 on /nobackup/ost3 type lustre (rw)
/dev/sdc8 on /nobackup/ost4 type lustre (rw)

=================
lustre filesystem
=================

user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    410224   7063632   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-10-31 01:18 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx                  #effective:r-x
user:user2:rwx                  #effective:r-x
user:user3:rwx                  #effective:r-x
user:user4:rwx                  #effective:r-x
group::rwx                      #effective:r-x
mask::r-x
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:r--
user:user2:rwx                  #effective:r--
user:user3:rwx                  #effective:r--
user:user4:rwx                  #effective:r--
group::rwx                      #effective:r--
mask::r--
other::r--

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

=================
tmp/ext3 filesystem
=================

user2@client-3:~> df -T /tmp/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3    20635700   1492244  18095220   8% /
user2@client-3:~> setfacl -R -b /tmp/workdir
user2@client-3:~> cd /tmp/workdir
user2@client-3:/tmp/workdir> pwd
/tmp/workdir
user2@client-3:/tmp/workdir> setfacl -M setfacl.jdk .
user2@client-3:/tmp/workdir> ls -ld ../workdir/
drwxrwxr-x+ 2 user2 g26137 4096 2012-10-31 01:26 ../workdir/
user2@client-3:/tmp/workdir> mkdir test2
user2@client-3:/tmp/workdir> touch test2/file2
user2@client-3:/tmp/workdir> getfacl test2
# file: test2
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/tmp/workdir> getfacl test2/file2
# file: test2/file2
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:rw-
user:user2:rwx                  #effective:rw-
user:user3:rwx                  #effective:rw-
user:user4:rwx                  #effective:rw-
group::rwx                      #effective:rw-
mask::rw-
other::r--

user2@client-3:/tmp/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x
Comment by Jian Yu [ 31/Oct/12 ]

From the above test result, the effective rights mask on Lustre filesystem was incorrect:

For directory, it was "mask::r-x", but should be "mask::rwx".
For regular file, it was "mask::r--", but should be "mask::rw-".
Comment by Jian Yu [ 01/Nov/12 ]

From my test result, it turned out to be the same issue as that in LU-974, which was just reopened because the real patches had not been landed yet.

While I performing the above test, the umask value was:

user2@client-3:~> umask
0022

After I changed the value to 0002 and performed the same test again, the result on Lustre filesystem became correct:

user2@client-3:~> umask 0002
user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    410228   7063628   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-11-01 04:42 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:rw-
user:user2:rwx                  #effective:rw-
user:user3:rwx                  #effective:rw-
user:user4:rwx                  #effective:rw-
group::rwx                      #effective:rw-
mask::rw-
other::r--

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

Hi James,

Could you please take a look at the umask value on your test system? Thanks.

Comment by Jay Lan (Inactive) [ 15/Nov/12 ]

Our default umask is 77. Please test with 77.

Comment by Jian Yu [ 16/Nov/12 ]

Our default umask is 77. Please test with 77.

OK, will do.

Comment by Jian Yu [ 19/Nov/12 ]

Here is the test result with umask 77:

user2@client-3:~> umask 77
user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    410224   7063632   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-11-19 00:30 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx                  #effective:---
user:user2:rwx                  #effective:---
user:user3:rwx                  #effective:---
user:user4:rwx                  #effective:---
group::rwx                      #effective:---
mask::---
other::---
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:---
user:user2:rwx                  #effective:---
user:user3:rwx                  #effective:---
user:user4:rwx                  #effective:---
group::rwx                      #effective:---
mask::---
other::---

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

So, with umask 77, the test result is the same as James'. Let me take a look at the patch http://review.whamcloud.com/1972 for LU-974.

Comment by Jian Yu [ 20/Nov/12 ]

I just verified that the patch in http://review.whamcloud.com/1972 for LU-974 fixed the above issue. Here is the test result with build http://build.whamcloud.com/job/lustre-reviews/10353/ on SLES11SP1 client:

user2@client-3:~> umask 77
user2@client-3:~> umask -S
u=rwx,g=,o=
user2@client-3:~> df -T /nobackup/workdir
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
fat-intel-4@tcp:/lustre
            lustre     7874112    413640   7059392   6% /nobackup/workdir
user2@client-3:~> setfacl -R -b /nobackup/workdir
user2@client-3:~> cd /nobackup/workdir
user2@client-3:/nobackup/workdir> pwd
/nobackup/workdir
user2@client-3:/nobackup/workdir> setfacl -M setfacl.jdk .
user2@client-3:/nobackup/workdir> ls -ld ../workdir/
drwxrwxr-x+ 3 user2 g26137 4096 2012-11-19 22:38 ../workdir/
user2@client-3:/nobackup/workdir> mkdir test1
user2@client-3:/nobackup/workdir> touch test1/file1
user2@client-3:/nobackup/workdir> getfacl test1/
# file: test1/
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

user2@client-3:/nobackup/workdir> getfacl test1/file1
# file: test1/file1
# owner: user2
# group: g26137
user::rw-
user:user1:rwx                  #effective:rw-
user:user2:rwx                  #effective:rw-
user:user3:rwx                  #effective:rw-
user:user4:rwx                  #effective:rw-
group::rwx                      #effective:rw-
mask::rw-
other::r--

user2@client-3:/nobackup/workdir> cat setfacl.jdk
# file: workdir
# owner: user2
# group: g26137
user::rwx
user:user1:rwx
user:user2:rwx
user:user3:rwx
user:user4:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:user1:rwx
default:user:user2:rwx
default:user:user3:rwx
default:user:user4:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

So, let's close this ticket as a duplicate of LU-974.

Generated at Sat Feb 10 01:23:05 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.