https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13353 Lustre <p>We run a script that checks the quota status of users and informs them if they are over quota. Currently we run lfs for each user and it can take a long time for a large system with a lot of users.</p> <p>So far I've found that I can use llapi_quotactl() (which lfs also uses) to get the necessary information. However, to make llapi_quotactl() run faster, I set dqb_valid to non-zero per the man page for llapi_quotactl(). This seems to work correctly in that I don't cause communication between the QMT and QSDs, but I don't get some of the important information (dqb_curspace and dqb_curinodes) that's used to determine if a user is exceeding a quota (also as expected, per the man page).</p> <p>However, I believe the struct lquota_entry.lqe_edquot flag has the information I want. It may not be completely up to date at all times, but that's ok because it's not being used in real time for this script. I can get a hold of it by passing it back to the llapi_quotactl() call in unused space in the struct obd_quotactl which comes out in user space as a struct if_quotactl. I'm just putting all the flags in the struct lquota_entry into the unused flags portion of the struct if_quotactl.</p> <p>Is there an easier way to get this information (relatively) quickly?</p> LU-13353

quickly determine if a quota is exceeded

Improvement Minor Open Unresolved Hongchao Zhang Gian-Carlo Defazio llnl Tue, 10 Mar 2020 23:06:32 +0000 Sat, 13 Jun 2020 14:58:46 +0000 0 9 <p>Hongchao, could you please comment.</p> <p>Hi,<br/> The relative quick way to check whether the users/groups is over quota limit is get it directly QMT(at MDT0000)<br/> for instance, check the status of users</p> <p>At MDT0000</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># cat /proc/fs/lustre/qmt/lustre-QMT0000/dt-0x0/glb-usr global_pool0_dt_usr - id: 0 limits: { hard: 0, soft: 0, granted: 0, time: 604800 } - id: 60000 limits: { hard: 102400, soft: 102400, granted: 32768, time: 0 } - id: 60001 limits: { hard: 102400, soft: 102400, granted: 8192, time: 0 } </pre> </div></div> <p>Print the users</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>#cat /proc/fs/lustre/qmt/lustre-QMT0000/dt-0x0/glb-usr | grep -B 1 "limits" | sed '$!N;s/\n/,/' | awk -F "[:,]" ' { if ($9 &gt; $5) print $2 " is over quota hard limit" ; else if ($9 &gt; $7) print $2 " is over quota soft limit" }' </pre> </div></div> <p>Hongchao, thanks for your response.</p> <p>Gian-Carlo is actually asking about an approach for an enhancement he is working on.  He's asking about sending the lquota_entry.lqe_edquot flag back from the QMT to the MDC and through to userspace.</p> <p>One of another interesting feature is to add 'lfs quota --list-all' an interface which try to get all existed quota type record from MDS, and even possible passing some flags to MDS(only filter overquota) and then return them back to clients, i heard several times that some people want this, this could save RPCs call for your case(mostly it might be just several RPCs to return back all quota informations).</p> <p>Great!</p> <blockquote><p>an interface which try to get all existed quota type record from MDS</p></blockquote> <p>and</p> <blockquote><p>only filter overquota</p></blockquote> <p>Those are the kind of options we have considered and would like your input on.</p> <p>Gian-Carlo is extracting the key bits out of a patch he's used for testing, and he'll push that so we can use it to talk about the options.</p> <p>Has anyone looked at quota_nld? The linux kernel sends netlink packets about the quota state. </p> <p>Hi Olaf,</p> <p>The "lquota_entry.lqe_edquot" can be packed into "dqi_flags" (struct obd_dqinfo) in QMT and return to client side,<br/> the more efficient way to get the quota information of all users/groups could be defining a new quota calling type to<br/> getting all quota information (the processing is similar as "cat /proc/fs/lustre/qmt/lustre-QMT0000/dt-0x0/glb-usr")<br/> from QMT and send it to client.</p> <p>Gian-Carlo DeFazio (defazio1@llnl.gov) uploaded a new patch: <a href="https://review.whamcloud.com/37908" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37908</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13353" title="quickly determine if a quota is exceeded" class="issue-link" data-issue-key="LU-13353">LU-13353</a> quota: quickly determine if quota exceeded<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8bd14f3e1e7cfab53ad045465483c7c6373fcf13</p> <p>Hi Gian-Carlo, can you change the "type" on this ticket to "Improvement"? For whatever reason I'm unable to do it. Thanks.</p> <p>Olaf, I'm unable to do that either. It might be something with Jira. I think Peter has the most admin privilege here, so hopefully he can change it.</p> <p>Thanks Andreas</p> <p>Well, I can't edit that either. <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=jgmitter" class="user-hover" rel="jgmitter">jgmitter</a> I know that we have seen this kind of thing in the past - is the workaround to move to the ticket to this same project? I did not want to do that without confirmation as it will mess up the commit message for the the patch in flight.</p> <p>I can't change the type either, but that's probably not a surprise at this point.</p> <p>Yes, that is correct Peter. That’s the only way to be able to reset the type from a question/request to either bug, improvement, or feature.</p> <p>Thanks <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=jgmitter" class="user-hover" rel="jgmitter">jgmitter</a>. Ah good - I expected it to increment the id in the LU project when I moved it but it retained the original one so it's now fixed without any side-effects. Really, not intuitive but it works...</p> <p>Gian-Carlo DeFazio (defazio1@llnl.gov) uploaded a new patch: <a href="https://review.whamcloud.com/38183" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38183</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13353" title="quickly determine if a quota is exceeded" class="issue-link" data-issue-key="LU-13353">LU-13353</a> quota: add man page for lfs quota<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7119836871013eb61b4f276a16a098786994283b</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38183/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38183/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13353" title="quickly determine if a quota is exceeded" class="issue-link" data-issue-key="LU-13353">LU-13353</a> quota: add man page for lfs quota<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: fba945ca0daa6a442a01f5174a0ba8b6d94294b9</p> Related Development Epic/Theme Performance Rank 1|i00vaf: Rank (Obsolete) 9223372036854775807