[LU-4673] Permission denied executing --x executable

• Status: Resolved
• Project: Lustre
• Type: Bug
• Priority: Major
• Reporter: Supporto Lustre Jnet2000 (Inactive)
• Assignee: Emoly Liu
• Resolution: Incomplete

Environment

System Red Hat Enterprise Linux Server release 5.7 (Tikanga)

kernel 2.6.18-348.3.1.el5

lustre-client-modules-1.8.9-2.6.18_348.3.1.el5_g7880158
lustre-client-1.8.9-2.6.18_348.3.1.el5_g7880158

• Severity: 3

Description

I try to execute the executable "sqlplus" on lustre filesystem,but i have this issue:

$ sqlplus

-bash: /home/opt/oracle/app/ora11g/product/11.2.0.3/client_1/bin/sqlplus: Permission denied

The executable has these permissions :

ls -l /home/opt/oracle/app/ora11g/product/11.2.0.3/client_1/bin/sqlplus

rwxr-x-x 1 ora11g dba 9221 Sep 24 14:57 /home/opt/oracle/app/ora11g/product/11.2.0.3/client_1/bin/sqlplus

If i add the "read" permission to "other" the executable work.

Comments

[Peter Jones]

Emoly

Could you please assist with this issue?

Thanks

Peter

[Emoly Liu]

Hi Jnet2000,

Do you run sqlplus as an Oracle user? Could you please run "which sqlplus" to see if $PATH is set correctly? And if possible, please post the corresponding lustre log here when this problem happens again.

Thanks.

[Supporto Lustre Jnet2000 (Inactive)]

i can run the sqlplus command and the path is correct, when i have the problem "permission denied" on the host, i haven't log on the client and i haven't log on the server.

regards

[Emoly Liu]

You can get the lustre logs by setting parameters debug and debug_mb before running the sqlplus command, like this
#lctl set_param debug=-1
#lctl set_param debug_mb=150

Then, run sqlplus, and after that, collect the logs by running the following command
#lctl dk > logfile

Could you please upload the logs here if you can get anything? Thanks.

[Supporto Lustre Jnet2000 (Inactive)]

We attached the logfile as requested.
Regards

[Supporto Lustre Jnet2000 (Inactive)]

Any news ?

regards

[Emoly Liu]

This client side log includes almost five months data. I should have suggested to do a "lctl dk" before running sqlplus.

I saw the following error many times.

(mdc_locks.c:845:mdc_finish_intent_lock()) D_IT dentry sqlplus intent: open status -13 disp 240002b rc 0

If possible, could you please upload the server side log as well? Thanks

[Supporto Lustre Jnet2000 (Inactive)]

I attached the log files requested. Before collecting them i issue the lctl flush command

[Supporto Lustre Jnet2000 (Inactive)]

the problem occurs with every executables that have the o=x permission

[b487000@osiride-lp-032 ~]$ ls -l bash
rwxr-x-x 1 root root 801528 May 13 2011 bash
[b487000@osiride-lp-032 ~]$ ./bash

in another window

[b487000@osiride-lp-032 ~]$ ./bash
-bash: ./bash: Permission denied

[Emoly Liu]

The server log showed that the mds didn't get the intent lock req@ffff81083d2f1000 sent by client, and as you said, all the executables can't be run, so I suspect this problem is related to flock.

By default, the flock utility is disabled. So could you please try to remount client with option "-o flock" and see what will happen?

Thanks.

[Supporto Lustre Jnet2000 (Inactive)]

the flock option is already enabled

[b487000@osiride-lp-032 ~]$ cat /etc/lustre.conf
#############################################################################

1. 1. 1. 1. 1. #####
            2. OSIRIDE #####
            3. #####
            4. Do not manually modify this file (it will be overwritten) #####
               #############################################################################
               LUSTRE="TEST"
               SLP_INIT=10
               SLP_ERR=30
               MGS_PRI=10.121.14.38
               MGS_SEC=10.121.14.69
               FILESYSTEM="/home"
               OPTIONS="flock,user_xattr"

[Emoly Liu]

Hi, can you run the following command to see group_upcall value and paste it here,
#lctl get_param mds.*.group_upcall

and also paste the result of "getfacl sqlplus"?

Thanks

[Supporto Lustre Jnet2000 (Inactive)]

[wisi281@osiride-lp-034 ~]$ sudo /usr/sbin/lctl get_param mds.*.group_upcall
mds.home-MDT0000.group_upcall=/usr/sbin/l_getgroups

[wisi281@osiride-lp-032 bin]$ sudo getfacl sqlplus

1. file: sqlplus
2. owner: ora11g
3. group: dba
   user::rwx
   group::r-x
   other::--x

regards

[Emoly Liu]

Since the server side log is not complete, I tried to reproduce this problem by running an executable file without read permission on NFS lustre client. Fortunately, I hit it.

It shows that we need read permission to run an executable file. So can you run "id" command to see if the user which has permission issue to run sqlplus belongs to group "dba"?

It's simple to verify this "o-r" by the following example:

[root@centos6-2 liuy]# cat foo.sh 
#!/bin/sh
echo "test permission"
ls -al *
[root@centos6-2 liuy]# chmod o-r foo.sh 
[root@centos6-2 liuy]# getfacl foo.sh 
# file: foo.sh
# owner: root
# group: root
user::rwx
group::r-x
other::--x
[root@centos6-2 liuy]# ./foo.sh 
test permission
-rwxr-x--x 1 root root   42 Mar 28 23:55 foo.sh
-rwxr-x--x 1 root root 7296 Mar 28 23:36 teststat
[root@centos6-2 liuy]# su - liuy
[liuy@centos6-2 ~]$ id
uid=500(liuy) gid=500(liuy) groups=500(liuy)
[liuy@centos6-2 ~]$ ./foo.sh 
/bin/sh: ./foo.sh: Permission denied

[Supporto Lustre Jnet2000 (Inactive)]

The r attribute is necessary to run bash script, for the binary this is not true.

[b487000@osiride-lp-002 bin]$ ls -l bash
--xx-x 1 b487000 b487000 801528 Apr 2 11:42 bash
[b487000@osiride-lp-002 bin]$ ./bash

[b487000@osiride-lp-002 bin]$ ls -l generate_lamp_packages.sh
--xx-x 1 b487000 b487000 866 Sep 19 2011 generate_lamp_packages.sh
[b487000@osiride-lp-002 bin]$ ./generate_lamp_packages.sh
/bin/bash: ./generate_lamp_packages.sh: Permission denied

Why the server log is incomplete ? We follow your instructions.

regards

[Emoly Liu]

Yes, scripts must be readable and executable but compiled programs need only be executable.

I want to check "id" and "r" attribute because the previous comments said:

The executable has these permissions :
ls -l /home/opt/oracle/app/ora11g/product/11.2.0.3/client_1/bin/sqlplus
rwxr-x-x 1 ora11g dba 9221 Sep 24 14:57 /home/opt/oracle/app/ora11g/product/11.2.0.3/client_1/bin/sqlplus
If i add the "read" permission to "other" the executable work.

the problem occurs with every executables that have the o=x permission
[b487000@osiride-lp-032 ~]$ ls -l bash
rwxr-x-x 1 root root 801528 May 13 2011 bash
[b487000@osiride-lp-032 ~]$ ./bash
in another window
[b487000@osiride-lp-032 ~]$ ./bash
-bash: ./bash: Permission denied

The "id" result will help me make sure if this is a lustre issue.

As for server log, the log happened at time 1395299404 was probably flushed if the clock between the client and server is synchronous, because there was nothing useful at that moment, and there was no EACCES(-13) error returned by mds_open(), even no xid=1448864745813986 found, sent by the client.

So if possible, could you please set debug_mb to a larger number(lctl set_param debug_mb=1000?), and synchronize the clock between the client and the server, and then collect the logs again?

Many thanks.

[John Fuchs-Chesney (Inactive)]

Hello Jnet2000 Support Team,

Is there any further way we can help you with this ticket?

If not, we would like to mark it as resolved.

Many thanks,
~ jfc.

[John Fuchs-Chesney (Inactive)]

Asked customer for guidance on this ticket on December 18, 2014.

Will now mark it as Resolved – Incomplete.

We can re-open the ticket if required.
Thanks,
~ jfc.