https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11850 Lustre <p>For security reasons /sys/kernel/debug is restrict to root only so by relocating /proc/fs/lustre/ost &amp; mdt to /sys/kenrnel/debug/lustre breaks many tools such as 'performance co pilot" that run as non-privilege users. We rely on such tools to collect lustre metric.</p> <p>We could change the permissions on /sys/kernel/debug but that is not good security practice. Can there be a build option to selected the location?</p> LU-11850

Relocating /proc/fs/lustre/ost to /sys/kernel/debug/lustre/ost prevents non-root access

Bug Minor In Progress Unresolved James A Simmons Mahmoud Hanafi llnl Thu, 10 Jan 2019 17:19:41 +0000 Fri, 26 Jan 2024 17:41:26 +0000 Lustre 2.12.0 Upstream 1 16 <p>I think the question we should ask is "should the information revealed in lustre/ost and lustre/mdt be kept away from non-privileged users?" If not, maybe we can reconsider the decision and move the information back to procfs? Or other sysfs location that do not require root privilege to read?</p> <p>Proc is the worst place for this kind of information. Not only is frowned on by the kernel maintainers but if we every placed a large amount of data into an seq_file, which is used by procfs, the node would be brought to its knees by the cpu load. Actually this has been under discussion on the lustre-devel mailing list. If you are talking about "stats" I'm working a netlink implementation which is way more flexible and has better performance. Give me a few days to roll something out.</p> <p>BTW here is an excellent article about a solution which I'm modeling my work after.</p> <p><a href="https://lwn.net/Articles/406975" class="external-link" target="_blank" rel="nofollow noopener">https://lwn.net/Articles/406975</a></p> <p>The article cover why stats in proc is bad.</p> <p>ok James</p> <blockquote> <p>We could change the permissions on /sys/kernel/debug but that is not good security practice.</p></blockquote> <p>There is no need to change the permissions for the whole of <tt>/sys/kernel/debug</tt> to be world readable. Currently, it looks like <tt>/sys/kernel/debug</tt> is itself the only directory that blocks access. It would be possible in the short term to recursively change the permissions of this tree to remove world-readable permissions ("<tt>chmod -R go-rw /sys/kernel/debug</tt>") and then enable group access permissions for the monitoring tools to the Lustre tree after the filesystem is mounted ("<tt>chmod -R g+rX /sys/kernel/debug/lustre; chgrp -R collectd /sys/kernel/debug/lustre</tt>" or similar).</p> <p>James, this is great! Performance is very important! Thanks!</p> <p>Why is <tt>/sys/kernel/debug/lustre</tt> not located at <tt>/sys/kernel/lustre?</tt></p> <p>Because all debugfs files go into /sys/kernel/debug. That is the mount point.</p> <p>But why are we considering /sys/kenrel/debug/lustre/ost/... part of "debugging"</p> <p> </p> <p>The kernel has rules about what can be in sysfs. An excellent article covering these rules is here:</p> <p><a href="https://lwn.net/Articles/378884" class="external-link" target="_blank" rel="nofollow noopener">https://lwn.net/Articles/378884</a></p> <p>Since Lustre has complex data files they are not allowed in sysfs. So the quick fix done for the linux client was moving it to debugfs . The point of this policy was due to proc becoming a dumpster. Now the dumpster is debugfs <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/> Note I have been avoiding the move of several files like stats to debugfs for the OpenSFS tree.</p> <p>No fear netlink will resolve these issues. I have a prototypes partially working. I just need to work out the nesting of data. I see its the ptlrpc service stats.</p> <p>I managed to get the basics working using netlink with obd stats. Just need to figure out how to link into the ptlrpc service.</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/34256" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34256</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11850" title="Relocating /proc/fs/lustre/ost to /sys/kernel/debug/lustre/ost prevents non-root access" class="issue-link" data-issue-key="LU-11850">LU-11850</a> obd: use netlink to get lustre stats<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 6c74e4ed15ad654b4a20925bd36b8cc0e014d34c</p> <p>Just pushed a prototype patch which I'm going to use to discsuss Netlink API with other developers. It does sort of work with just md_stats but more is needed.</p> <p>So I have been doing research into the different stat collectors out their. From what I see you can configure them to collect the data from the lustre utilities instead of attempting to read from the debugfs files directly. For example for collectd you would use:</p> <p>&lt;Plugin exec&gt;</p> <p>    Exec "myuser:mygroup" "myprog"</p> <p>   Exec "otheruser" "/path/to/another/binary" "arg0" "arg1"</p> <p>   NotificationExec "user" "/usr/lib/collectd/exec/handle_notification"</p> <p>&lt;/Plugin&gt;</p> <p>Looking at LMT and performance co pilot it looks to be the same case. If we can get are utilities to work without root access we should be in good shape.</p> <p> </p> <p>"James Simmons &lt;jsimmons@infradead.org&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51959" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51959</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11850" title="Relocating /proc/fs/lustre/ost to /sys/kernel/debug/lustre/ost prevents non-root access" class="issue-link" data-issue-key="LU-11850">LU-11850</a> lov: migrate completely to lu_tgt_descs API<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8e49bf0a866c9214ac72bb85e2c49557615a3dd4</p> <p>We would need to provide access to the statistics on the client side from non-root accesses. Such access is mandatory for many performance tools running in user space.</p> <p>Probably the patch is of modest size, is it possible to have a hot-fix?</p> <p>JT, the change of <tt>/sys/kernel/debug</tt> to root-only happened in the upstream kernel after Lustre started using it, so it would need a kernel patch on all clients (AFAIK), that I don't think anyone wants. </p> <p>I haven't if there is an easy way to restructure the code back to using <tt>/proc/fs/lustre</tt> to make these stats available again, but that would probably be the least disruptive code change. The other option would be to add a dedicated "<tt>lparamfs</tt>" to hold all the Lustre stats so we don't have to deal with the kernel restrictions at all. </p> <p>Hello. Since you brought this up I do have a patch - <a href="https://review.whamcloud.com/c/fs/lustre-release/+/34256." class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/34256.</a> After reading this I tracked down the crash I was seeing which was due to a really large message. Stats can be really huge amount of data. I need to adjust the skb. I do need to do more testing for what global_match() can handle.</p> <p>I updated patch 34256. Not perfect but is mostly works now. Give it a try. Note you will need the latest Lustre to make this work.</p> <p>Almost done with the patch for stats. Only bug left is if you grab ALL stats it overflows the liblnetconfig library. I do want to move the internal storage of the stats structures as an Xarray instead of a generic_radix struct.</p> Duplicate LU-11988 Related LU-8066 LU-9680 LU-17471 LU-11988 LU-12244 LU-12513 LU-12841 LU-17472 Development Rank 1|i0098v: Rank (Obsolete) 9223372036854775807 Severity