Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16210

Lustre on RHEL8 with selinux disabled tries to retrieve security.selinux xattr

Details

    • 3
    • 9223372036854775807

    Description

      On Centos7, selinux_is_enabled() was used to prevent this but on RHEL8 this interface disappears (on 5.1 kernel):

      static int ll_xattr_get_common()
      ...
       	/* LU-549:  Disable security.selinux when selinux is disabled */
       	if (handler->flags == XATTR_SECURITY_T && !selinux_is_enabled() &&
      	    !strcmp(name, "selinux"))
       		RETURN(-EOPNOTSUPP);
      

      The patch https://review.whamcloud.com/38480 "LU-12355 llite: include file linux/selinux.h removed" was submitted to add the compatibility to 5.1 kernel. But it considers selinux enabled if "selinux_is_enabled" is not found.

      So on RHEL8 with selinux disabled, we send unneeded RPCs on the MDS to retrieved security.selinux xattr. This has performance issues.

      Here the behavior with "ls":

      Centos 7 :
      getenforce 0
      Disabled
      strace -e getxattr,lgetxattr ls -lda “directory”
      lgetxattr("directory", "security.selinux", 0x1096100, 255) = -1 EOPNOTSUPP (Operation not supported)
      getxattr("directory", "system.posix_acl_access", NULL, 0) = -1 ENODATA (No data available)
      getxattr("directory", "system.posix_acl_default", NULL, 0) = -1 ENODATA (No data available)
       
      Centos 8 :
      getenforce 0
      Disabled
      lgetxattr("directory", "security.selinux", 0x1096100, 255) = -1 ENODATA (No data available)
      getxattr("directory", "system.posix_acl_access", NULL, 0) = -1 ENODATA (No data available)
      getxattr("directory", "system.posix_acl_default", NULL, 0) = -1 ENODATA (No data available)
      

      Here the "perf report" diff on getfattr:

      Centos 7 :
      +   10.59%     0.00%  getfattr  libc-2.17.so      [.] __GI___lxstat64 (inlined)                                                                                            
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] system_call                                                                                                          
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] sys_newlstat                                                                                                         
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] SYSC_newlstat                                                                                                        
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] vfs_fstatat                                                                                                          
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] user_path_at                                                                                                         
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] user_path_at_empty                                                                                                   
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] filename_lookup                                                                                                      
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] path_lookupat                                                                                                        
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] link_path_walk                                                                                                       
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] inode_permission                                                                                                     
      +   10.59%     0.00%  getfattr  [kernel.vmlinux]  [k] __inode_permission                                                                                                  
      +   10.59%     0.00%  getfattr  [lustre]          [k] ll_inode_permission                                                                                                  
      +   10.59%     0.00%  getfattr  [lustre]          [k] ll_inode_revalidate                                                                                                  
      +   10.59%     0.00%  getfattr  [lmv]             [k] lmv_intent_lock  
       
      Centos 8 :
      +   15.74%     0.00%  getfattr  libc-2.28.so       [.] __GI_getxattr (inlined)                                                                                             
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] path_getxattr                                                                                                       
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] getxattr                                                                                                           
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] vfs_getxattr                                                                                                      
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] __vfs_getxattr                                                                                                      
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] ll_xattr_get_common                                                                                                 
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] ll_xattr_list                                                                                                       
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] lmv_getxattr                                                                                                        
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] mdc_getxattr                                                                                                        
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] mdc_xattr_common         <----------                                                                                      
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] ptlrpc_queue_wait                                                                                                   
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] ptlrpc_set_wait                                                                                  
      +   15.74%     0.00%  getfattr  [kernel.kallsyms]  [k] ptlrpc_check_set.part
      

      Attachments

        Activity

          [LU-16210] Lustre on RHEL8 with selinux disabled tries to retrieve security.selinux xattr

          "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49630/
          Subject: LU-16210 llite: replace selinux_is_enabled()
          Project: fs/lustre-release
          Branch: b2_15
          Current Patch Set:
          Commit: 71143864865361d6b6012d75d8bb03a19a36b505

          gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49630/ Subject: LU-16210 llite: replace selinux_is_enabled() Project: fs/lustre-release Branch: b2_15 Current Patch Set: Commit: 71143864865361d6b6012d75d8bb03a19a36b505

          "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50404
          Subject: LU-16210 llite: replace selinux_is_enabled()
          Project: fs/lustre-release
          Branch: b2_12
          Current Patch Set: 1
          Commit: 4ca0eceec6182e4c611d23c386c7e36c38d8e1fc

          gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50404 Subject: LU-16210 llite: replace selinux_is_enabled() Project: fs/lustre-release Branch: b2_12 Current Patch Set: 1 Commit: 4ca0eceec6182e4c611d23c386c7e36c38d8e1fc

          "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49630
          Subject: LU-16210 llite: replace selinux_is_enabled()
          Project: fs/lustre-release
          Branch: b2_15
          Current Patch Set: 1
          Commit: 06b50ff6c162ccdf57ecc59c81d8c266a184d582

          gerrit Gerrit Updater added a comment - "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49630 Subject: LU-16210 llite: replace selinux_is_enabled() Project: fs/lustre-release Branch: b2_15 Current Patch Set: 1 Commit: 06b50ff6c162ccdf57ecc59c81d8c266a184d582
          pjones Peter Jones added a comment -

          Landed for 2.16

          pjones Peter Jones added a comment - Landed for 2.16

          "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/48875/
          Subject: LU-16210 llite: replace selinux_is_enabled()
          Project: fs/lustre-release
          Branch: master
          Current Patch Set:
          Commit: 1d8faaf6caf4acaf0e2d4943b51c024a96c80624

          gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/48875/ Subject: LU-16210 llite: replace selinux_is_enabled() Project: fs/lustre-release Branch: master Current Patch Set: Commit: 1d8faaf6caf4acaf0e2d4943b51c024a96c80624

          Those patches restore the behavior of the http://review.whamcloud.com/2503 when SElinux is disabled:

          commit 7cc542fd4c26ccb117ceb13a47ac8ced3107b9b3
          Author: Yevheniy Demchenko <zheka@uvt.cz>
          Date:   Tue Apr 10 22:01:14 2012 +0200
          
              LU-549 llite: Improve statfs performance if selinux is disabled
              
              Even if selinux is disabled, client still tries to get selinux
              attributes from MDS. As xattrs are not yet cached, this significantly
              slows down xattr heavy operations like ls -l. This patch forces
              to return -EOPNOTSUPP on the client side if selinux is disabled.
              It speeds up ls -l 25% for cold-cache case and 50% for hot-cache
              case.
          

          Tests with "strace -c ls -l" with 100000 files on root in a multi VMs env (on Rocky 9). FS is remounted for each tests (cache is cleaned) and selinux is disabled.

          Total time % lgetxattr statx
          Without the patch: 29% 51%
          With the patch: 0% 87%

          "ls -l" uses lgetxattr to get "security.selinux".

          eaujames Etienne Aujames added a comment - Those patches restore the behavior of the http://review.whamcloud.com/2503 when SElinux is disabled: commit 7cc542fd4c26ccb117ceb13a47ac8ced3107b9b3 Author: Yevheniy Demchenko <zheka@uvt.cz> Date: Tue Apr 10 22:01:14 2012 +0200 LU-549 llite: Improve statfs performance if selinux is disabled Even if selinux is disabled, client still tries to get selinux attributes from MDS. As xattrs are not yet cached, this significantly slows down xattr heavy operations like ls -l. This patch forces to return -EOPNOTSUPP on the client side if selinux is disabled. It speeds up ls -l 25% for cold-cache case and 50% for hot-cache case. Tests with "strace -c ls -l" with 100000 files on root in a multi VMs env (on Rocky 9). FS is remounted for each tests (cache is cleaned) and selinux is disabled. Total time % lgetxattr statx Without the patch: 29% 51% With the patch: 0% 87% "ls -l" uses lgetxattr to get "security.selinux".
          eaujames Etienne Aujames added a comment - - edited

          *https://review.whamcloud.com/48796 : implements a solution with a simple i_security check.
          This will only work if SELinux is disabled at boot time (adding "selinux=0" on kernel cmdline) for RHEL >= 8.5.
          *https://review.whamcloud.com/48875 : implements a more complex solution by retrieving and saving the security xattr name used by the LSM module at mount time (via security_inode_listsecurity()). Then it will use this stored value to filter out "security.selinux" (if SELinux is disabled, nothing is returned by security_inode_listsecurity()).
          This will work in all cases: SELinux is disabled via /etc/selinux/config or via kernel cmdline.

          eaujames Etienne Aujames added a comment - - edited * https://review.whamcloud.com/48796 : implements a solution with a simple i_security check. This will only work if SELinux is disabled at boot time (adding "selinux=0" on kernel cmdline) for RHEL >= 8.5. * https://review.whamcloud.com/48875 : implements a more complex solution by retrieving and saving the security xattr name used by the LSM module at mount time (via security_inode_listsecurity()). Then it will use this stored value to filter out "security.selinux" (if SELinux is disabled, nothing is returned by security_inode_listsecurity()). This will work in all cases: SELinux is disabled via /etc/selinux/config or via kernel cmdline.

          "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/48875
          Subject: LU-16210 llite: replace selinux_is_enabled()
          Project: fs/lustre-release
          Branch: master
          Current Patch Set: 1
          Commit: 19da032d77292cc068859e4dbe44a2376465515d

          gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/48875 Subject: LU-16210 llite: replace selinux_is_enabled() Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 19da032d77292cc068859e4dbe44a2376465515d

          "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/48796
          Subject: LU-16210 llite: replace selinux_is_enabled()
          Project: fs/lustre-release
          Branch: master
          Current Patch Set: 1
          Commit: b47eb08c932842af9aa10633b017e18153ad711d

          gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/48796 Subject: LU-16210 llite: replace selinux_is_enabled() Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: b47eb08c932842af9aa10633b017e18153ad711d

          eaujames please add me as a reviewer to the patch you are cooking.

          sebastien Sebastien Buisson added a comment - eaujames please add me as a reviewer to the patch you are cooking.

          People

            eaujames Etienne Aujames
            eaujames Etienne Aujames
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: