[LU-13353] quickly determine if a quota is exceeded

• Type: Improvement
• Resolution: Won't Do
• Priority: Minor
• Labels: llnl patch performance

Epic/Theme

Performance

• Assignee: Hongchao Zhang
• Reporter: Gian-Carlo Defazio

Description

We run a script that checks the quota status of users and informs them if they are over quota. Currently we run lfs for each user and it can take a long time for a large system with a lot of users.

So far I've found that I can use llapi_quotactl() (which lfs also uses) to get the necessary information. However, to make llapi_quotactl() run faster, I set dqb_valid to non-zero per the man page for llapi_quotactl(). This seems to work correctly in that I don't cause communication between the QMT and QSDs, but I don't get some of the important information (dqb_curspace and dqb_curinodes) that's used to determine if a user is exceeding a quota (also as expected, per the man page).

However, I believe the struct lquota_entry.lqe_edquot flag has the information I want. It may not be completely up to date at all times, but that's ok because it's not being used in real time for this script. I can get a hold of it by passing it back to the llapi_quotactl() call in unused space in the struct obd_quotactl which comes out in user space as a struct if_quotactl. I'm just putting all the flags in the struct lquota_entry into the unused flags portion of the struct if_quotactl.

Is there an easier way to get this information (relatively) quickly?

Comments

[Gerrit Updater]

Gian-Carlo DeFazio ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38183
Subject: LU-13353 quota: add man page for lfs quota
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7119836871013eb61b4f276a16a098786994283b

[Peter Jones]

Thanks jgmitter. Ah good - I expected it to increment the id in the LU project when I moved it but it retained the original one so it's now fixed without any side-effects. Really, not intuitive but it works...

[Joseph Gmitter (Inactive)]

Yes, that is correct Peter. That’s the only way to be able to reset the type from a question/request to either bug, improvement, or feature.

[Gian-Carlo Defazio]

I can't change the type either, but that's probably not a surprise at this point.

[Peter Jones]

Well, I can't edit that either. jgmitter I know that we have seen this kind of thing in the past - is the workaround to move to the ticket to this same project? I did not want to do that without confirmation as it will mess up the commit message for the the patch in flight.

[Olaf Faaland]

Thanks Andreas

[Andreas Dilger]

Olaf, I'm unable to do that either. It might be something with Jira. I think Peter has the most admin privilege here, so hopefully he can change it.

[Olaf Faaland]

Hi Gian-Carlo, can you change the "type" on this ticket to "Improvement"? For whatever reason I'm unable to do it. Thanks.

[Gerrit Updater]

Gian-Carlo DeFazio ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37908
Subject: LU-13353 quota: quickly determine if quota exceeded
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8bd14f3e1e7cfab53ad045465483c7c6373fcf13

[Hongchao Zhang]

Hi Olaf,

The "lquota_entry.lqe_edquot" can be packed into "dqi_flags" (struct obd_dqinfo) in QMT and return to client side,
the more efficient way to get the quota information of all users/groups could be defining a new quota calling type to
getting all quota information (the processing is similar as "cat /proc/fs/lustre/qmt/lustre-QMT0000/dt-0x0/glb-usr")
from QMT and send it to client.

[James A Simmons]

Has anyone looked at quota_nld? The linux kernel sends netlink packets about the quota state.