[LU-9929] Use "setfacl" to set "default" setting fail when nodemap enabled

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.11.0, Lustre 2.10.2
• Environment: Lustre 2.9
• Severity: 3
• Assignee: Emoly Liu
• Reporter: sebg-crd-pm (Inactive)

Description

Hi ,

When we setfacl default in lustre directory, I got unmapping group id(getfacl) after first setting(setfacl) . Then we executed setfacl command again and got fail ( Operation not permitted).

Please help us to fix this problem. Thanks!

The detail information is listed below.

1.cat /etc/passwd
user1:x:1001:1001::/home/user1:/bin/bash
2.nodemap setting
nodemap.21b7e9f04fed448e.idmap=
[
.....

{ idtype: gid, client_id: 1001, fs_id: 23501 }

,
.....
]
3.setfacl steps
[root@hsm client]# mkdir hadoop3
[root@hsm client]# getfacl /mnt/client/hadoop3
getfacl: Removing leading '/' from absolute path names
file: mnt/client/hadoop3
owner: root
group: root
user::rwx
group::r-x
other::r-x
[root@hsm client]# setfacl -R -d -m group:user1:rwx /mnt/client/hadoop3
[root@hsm client]# getfacl /mnt/client/hadoop3
getfacl: Removing leading '/' from absolute path names
file: mnt/client/hadoop3
owner: root
group: root
user::rwx
group::r-x
other::r-x
default:user::rwx
default:group::r-x
default:group:23501:rwx
default:mask::rwx
default:other::r-x
[root@hsm client]# setfacl -R -d -m group:user1:rwx /mnt/client/hadoop3
setfacl: /mnt/client/hadoop3: Operation not permitted

Issue Links

• duplicates: LU-9759 setfacl default setting fail when nodemap enabled

Comments

[Emoly Liu]

Thanks, I can see this issue now. I will investigate it.

[sebg-crd-pm (Inactive)]

Hi Emoly,

Did you set your client ip to nodemap_test ? (lctl nodemap_add_range)?
The client will in nodemap.default if you did not set it to nodemap_test.
Please also check nodemap.active=1(lctl get_param nodemap.*) and "lctl set_param nodemap.nodemap_test.admin_nodemap=1" for root access permission. Thanks

Which nodes are there in your system and which nodes have user "user1"?
>> One server node 172.20.110.212(mgt/mdt/ost), One client node 172.20.110.211
Which steps did you run on which node?
>> Please see below
Can you provide all your nodemap information by the command "lctl get_param nodemap.$your_nodemap.*"?
>> Please see below
Can you collect some lustre logs on mgs node and client node during you test by the following commands:
>> see attached files.
What is your detailed lustre version?
>> Test it in Lustre 2.10 (lustre-release-58fd06e.tar.gz)/ one node(mgs/mds/oss ) follow these steps.

[setup lustre]
lctl set_param nodemap.active=1
lctl nodemap_add nodemap_test
lctl set_param nodemap.nodemap_test.admin_nodemap=1
lctl nodemap_add_idmap --name nodemap_test --idtype gid --idmap 1001:23501
lctl nodemap_add_range --name nodemap_test --range 172.20.110.[211-211]@o2ib
lctl get_param nodemap.*
lctl get_param nodemap.nodemap_test.*

[output]
nodemap.active=1
nodemap.nodemap_test.admin_nodemap=1
nodemap.nodemap_test.deny_unknown=0
nodemap.nodemap_test.exports=
[

{ nid: 172.20.110.211@o2ib, uuid: 2a980ffd-962a-eef8-37aa-cedf34253b31 }

,

{ nid: 172.20.110.211@o2ib, uuid: 2a980ffd-962a-eef8-37aa-cedf34253b31 }

,
]
nodemap.nodemap_test.fileset=

nodemap.nodemap_test.id=1
nodemap.nodemap_test.idmap=[

{ idtype: gid, client_id: 1001, fs_id: 23501 }

]
nodemap.nodemap_test.map_mode=both
nodemap.nodemap_test.ranges=
[

{ id: 1, start_nid: 172.20.110.211@o2ib, end_nid: 172.20.110.211@o2ib }

]

[client]
//client node user ppp: aclserver.log aclclient.log ppp:x:1001:1001::/home/ppp:/bin/bash

[root@hsm mnt]mount.lustre 172.20.110.212@o2ib:/jlustre /mnt/lustre
[root@hsm mnt]mkdir -p /mnt/lustre/hadoop3
[root@hsm mnt]# setfacl -R -d -m group:ppp:rwx /mnt/lustre/hadoop3
[root@hsm mnt]# getfacl /mnt/lustre/hadoop3
getfacl: Removing leading '/' from absolute path names

1. file: mnt/lustre/hadoop3
2. owner: root
3. group: root
   user::rwx
   group::r-x
   other::r-x
   default:user::rwx
   default:group::r-x
   default:group:23501:rwx
   default:mask::rwx
   default:other::r-x

[root@hsm mnt]# setfacl -R -d -m group:ppp:rwx /mnt/lustre/hadoop3
setfacl: /mnt/lustre/hadoop3: Operation not permitted

[Emoly Liu]

sebg-crd-pm,
I can't reproduce this issue in single node or multiple nodes tests. Here are my steps and output of my test:

#On client node
+ groupadd -g 1001 user1
+ useradd -g user1 -u 1001 user1
+ cat /etc/passwd
+ grep user1
user1:x:1001:1001::/home/user1:/bin/bash

#On MGS node
+ lctl nodemap_add nodemap_test
+ lctl nodemap_add_idmap --name nodemap_test --idtype gid --idmap 1001:23501
+ lctl get_param 'nodemap.nodemap_test.*'
nodemap.nodemap_test.admin_nodemap=0
nodemap.nodemap_test.deny_unknown=0
nodemap.nodemap_test.exports=[

]
nodemap.nodemap_test.fileset=

nodemap.nodemap_test.id=5
nodemap.nodemap_test.idmap=[
 { idtype: gid, client_id: 1001, fs_id: 23501 }
]
nodemap.nodemap_test.map_mode=both
nodemap.nodemap_test.ranges=[

]
nodemap.nodemap_test.squash_gid=99
nodemap.nodemap_test.squash_uid=99
nodemap.nodemap_test.trusted_nodemap=0

#On client node
+ mkdir -p /mnt/lustre/hadoop3
+ getfacl /mnt/lustre/hadoop3
getfacl: Removing leading '/' from absolute path names
# file: mnt/lustre/hadoop3
# owner: root
# group: root
user::rwx
group::r-x
other::r-x

+ echo '1st: setfacl'
1st: setfacl
+ setfacl -R -d -m group:user1:rwx /mnt/lustre/hadoop3
+ getfacl /mnt/lustre/hadoop3
getfacl: Removing leading '/' from absolute path names
# file: mnt/lustre/hadoop3
# owner: root
# group: root
user::rwx
group::r-x
other::r-x
default:user::rwx
default:group::r-x
default:group:user1:rwx
default:mask::rwx
default:other::r-x

+ echo '2nd: setfacl'
2nd: setfacl
+ setfacl -R -d -m group:user1:rwx /mnt/lustre/hadoop3
+ getfacl /mnt/lustre/hadoop3
getfacl: Removing leading '/' from absolute path names
# file: mnt/lustre/hadoop3
# owner: root
# group: root
user::rwx
group::r-x
other::r-x
default:user::rwx
default:group::r-x
default:group:user1:rwx
default:mask::rwx
default:other::r-x

Could you provide the following information?

• Which nodes are there in your system and which nodes have user "user1"?
• Which steps did you run on which node?
• Can you provide all your nodemap information by the command "lctl get_param nodemap.$your_nodemap.*"?
• Can you collect some lustre logs on mgs node and client node during you test by the following commands:

  #before test
  lctl set_param debug=-1 debug_mb=1000
  lctl dk > /dev/null
  #testing ...
  #after test
  lctl dk > $logfile
  Then please upload the logfile here.
  
  

• What is your detailed lustre version?

Thanks!

[Emoly Liu]

I will look into this issue.

[sebg-crd-pm (Inactive)]

Hi Emoly,

Do you have any comment about this issue?
We really need your help.

Thanks !

[Peter Jones]

Emoly

Could you please assist with this one?

Thanks

Peter