[LU-10444] l_getidentity keeps remount /sys/kernel/debug and reverting permissions.

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.11.0, Lustre 2.10.4
• Affects Version/s: Lustre 2.10.1
• Severity: 3
• Assignee: Oleg Drokin
• Reporter: Mahmoud Hanafi

Description

We change the permissions of /sys/kernel/debug to 755. But it kept revering to 700.

Using systemtap script

#!/usr/bin/env stap

probe kernel.function("debug_mount") {
 printf("mounting ")
 printf("pid %i %s %s %s \n", pid(),execname(), cmdline_str(), caller())
 print_backtrace()
}

probe kernel.function("debugfs_remount") {
 printf("remounting ")
 printf("pid %i %s %s %s \n", pid(),execname(), cmdline_str(), caller())
 print_backtrace()
}

I tracked this down to l_getidentity. Specificly cfs_get_param_paths keeps remounting /sys/kernel/debug.

# mount -o remount,mode=755 /sys/kernel/debug
debugfs on /sys/kernel/debug type debugfs (rw,relatime,mode=755)

Here is the output from the systemtap

nbp1-mds ~ # stap -v debug_mount.stp 
Pass 1: parsed user script and 468 library scripts using 122940virt/39548res/3192shr/36448data kb, in 340usr/20sys/351real ms.
Pass 2: analyzed script: 2 probes, 22 functions, 5 embeds, 0 globals using 161632virt/79356res/4388shr/75140data kb, in 580usr/80sys/665real ms.
Pass 3: translated to C into "/tmp/stapYTzkIl/stap_6d34b35e04d11e38ae0c8b364ac253de_10487_src.c" using 162160virt/80148res/4692shr/75668data kb, in 340usr/10sys/353real ms.
Pass 4: compiled C into "stap_6d34b35e04d11e38ae0c8b364ac253de_10487.ko" in 5660usr/860sys/5857real ms.
Pass 5: starting run.
mounting pid 9739 l_getidentity "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" mount_fs 0xffffffff811fec69 
 0xffffffff81289120 : debug_mount+0x0/0x20 [kernel]
 0xffffffff811fec69 : mount_fs+0x39/0x1b0 [kernel]
 0xffffffff8121b5c7 : vfs_kern_mount+0x67/0x110 [kernel]
 0xffffffff8121dad3 : do_mount+0x233/0xaf0 [kernel]
 0xffffffff8121e716 : SyS_mount+0x96/0xf0 [kernel]
 0xffffffff81695b89 : system_call_fastpath+0x16/0x1b [kernel]
remounting pid 9739 l_getidentity "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" do_remount_sb 0xffffffff811fe7c2 
 0xffffffff81289330 : debugfs_remount+0x0/0x50 [kernel]
 0xffffffff811fe7c2 : do_remount_sb+0x72/0x200 [kernel]
 0xffffffff811feb07 : mount_single+0x57/0xc0 [kernel]
 0xffffffff81289138 : debug_mount+0x18/0x20 [kernel]
 0xffffffff811fec69 : mount_fs+0x39/0x1b0 [kernel]
 0xffffffff8121b5c7 : vfs_kern_mount+0x67/0x110 [kernel]
 0xffffffff8121dad3 : do_mount+0x233/0xaf0 [kernel]
 0xffffffff8121e716 : SyS_mount+0x96/0xf0 [kernel]
 0xffffffff81695b89 : system_call_fastpath+0x16/0x1b [kernel]
mounting pid 9779 l_getidentity "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" mount_fs 0xffffffff811fec69 
 0xffffffff81289120 : debug_mount+0x0/0x20 [kernel]
 0xffffffff811fec69 : mount_fs+0x39/0x1b0 [kernel]
 0xffffffff8121b5c7 : vfs_kern_mount+0x67/0x110 [kernel]
 0xffffffff8121dad3 : do_mount+0x233/0xaf0 [kernel]
 0xffffffff8121e716 : SyS_mount+0x96/0xf0 [kernel]
 0xffffffff81695b89 : system_call_fastpath+0x16/0x1b [kernel]
remounting pid 9779 l_getidentity "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" do_remount_sb 0xffffffff811fe7c2 
 0xffffffff81289330 : debugfs_remount+0x0/0x50 [kernel]
 0xffffffff811fe7c2 : do_remount_sb+0x72/0x200 [kernel]
 0xffffffff811feb07 : mount_single+0x57/0xc0 [kernel]
 0xffffffff81289138 : debug_mount+0x18/0x20 [kernel]
 0xffffffff811fec69 : mount_fs+0x39/0x1b0 [kernel]
 0xffffffff8121b5c7 : vfs_kern_mount+0x67/0x110 [kernel]
 0xffffffff8121dad3 : do_mount+0x233/0xaf0 [kernel]
 0xffffffff8121e716 : SyS_mount+0x96/0xf0 [kernel]
 0xffffffff81695b89 : system_call_fastpath+0x16/0x1b [kernel]

Comments

[Oleg Drokin]

/proc/fs/lustre is going away, so don't depend on it being there for a long time.

The content is moving to /sys/fs/lustre for simple one file per value cases, everything else is removed or moved to debugfs.

[James A Simmons]

Can you tell me if lctl device_list works for you? You shouldn't be reading procfs/sysfs/debugfs files directly

[Mahmoud Hanafi]

We have nagios monitoring scripts that read the file directly. The script ran as a regular usr. I guess we could scan /proc/fs/lustre to get the same info.

[James A Simmons]

Andreas yes you can change the permissions with the mount() function by using the last parameter and passing in the mode value. The question is this a good thing. The problem is so much devices but the debugfs mount point itself. Its mounted for root access only by default. Their is a reason debugfs is not accessible to non-root users. I do understand what the problem is now. I still think Oleg's approach is better.

Mahmoud the file was moved due to a requirement from the linux kernel maintainers. All special files in /proc are in the process of being  moved into /sysfs in the linux kernel. In the case of "devices" it breaks the one item per file rule for sysfs so it has to be placed into debugfs. Now what you are doing is not the best idea for security reasons but I see why you are doing it. The question is do non root users really/ need to access "devices" or the other case stat files in debugfs? Other than that all the other files look administrative.

Mahmoud are you using lctl device_list or directly accessing /proc. The reason I ask is that lctl device_list will attempt to access the debugfs file first and if it fails call an ioctl. If lctl device_list doesn't work for you then its really broken.

Mahmoud are you using lctl device_list or directly accessing /proc. The reason I ask is that lctl device_list will attempt to access the 

[Andreas Dilger]

James, is it possible to create the devices file with 755 permissions from the start?

[Mahmoud Hanafi]

The only reason we needed to change  permissions on /sys/kernel/debug was, user level access need for  '/sys/kernel/debug/lustre/devices.' Why was this file moved from /proc/sys/fs/lustre/devices to /sys/kernel/debug/lustre/devices. Is this the correct place for this file? We think relocating this file should be considered.

-Mahmoud

 

[James A Simmons]

That is really strange. You would think it would return -EBUSY in that case instead of remounting.

[Gerrit Updater]

Oleg Drokin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30675
Subject: LU-10444 utils: Don't remount debugfs every time
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 64e656deebdc977593c6d43e7a5ee50c045803dc

[Oleg Drokin]

hm.. Yes, this is a bug in this patch https://review.whamcloud.com/25182.- we really should be checking if the fs is mounted before trying to mount it as the first step.

Let me see if I can make a simple patch.