[LU-4530] Mainline kernel client (3.12-3.14): lfs quota -u ... -> Permission denied

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Labels: patch
• Severity: 3
• Assignee: Oleg Drokin
• Reporter: Cédric Dufour

Description

Hello,

'lfs quota -u $(whoami)...' reports "Permission denied' for regular users. It works if the same user queries the quota for another user:

$ whoami
cdufour

$ lfs quota -u $(whoami) /idiap/temp/$(whoami)
Permission denied.

$ lfs quota -u formaz /idiap/temp/formaz
Disk quotas for user formaz (uid 1005):
Filesystem kbytes quota limit grace files quota limit grace
/idiap/temp/formaz
113371820 1072693248 1073741824 - 452499 9999900 10000000 -

(patch follows as soon as I have the LU-<ID> for this bug)

Best,

Cédric

Issue Links

• is related to: LU-4011 problems with upstream lustre client code
• is related to: LU-4416 support for 3.12 linux kernel
• is related to: LU-6215 Sync Lustre external tree with lustre linux kernel client

Comments

[Roland Fehrenbacher]

Thanks for your detailed answer to my questions. I am glad to see you are putting further
effort into the vanilla kernel client. Looking forward to the days, when the server code will
compile against recent vanilla kernels as well

[Cédric Dufour]

Thanks for the GIT pointers. Having some "insider's" insights on which patch to "backport" in priority (before they make it to the kernel) will definitely help (and provide opportunities for feedback from us users before it does).

Will/should patches always be linked to Jira IDs and should Jira be the place to provide feedback (in case issues remain) ?
Will a notice be given in Jira when a patch is submitted to mainline ?

Also, are you the one to link new mainline kernel issues with LU-4416 or can/should we reporter do it ?
(note: actually haven't found how to link one issue to another in Jira).

As for jobstats: I hadn't heard of them until your mention but I will look into it; it seems it would allow us to address some questions we have with some load patterns we experience with our (SGE) grid.

Best,

Cédric

[Oleg Drokin]

Thanks for this info. I am glad to hear you find lustre useful and that even vanilla kernel client works for you.

I uploaded a stream of patches I plan to push upstream to https://github.com/verygreen/linux/tree/lustre-next
With it and updated LU-4209 patch for the tools sanity testing almost completely succeeds, and more importantly, nothing crashes anymore.

I know Peng Tao has his tree at https://github.com/bergwolf/linux/compare/master-sync-round-5 that has some more patches, though not as critical for normal operations.

BTW, do you happen to use jobstats feature?

[Cédric Dufour]

Hello Oleg,

Thank you for those precisions.

Thanks to Roland - who originally set up our Lustre cluster server-wise - we have been using Lustre for 5 years now, as a general purpose temp space for our users and our computation grid. We're quite happy with it, especially after evaluating several commercial solutions and finding none up to Lustre so far. Lately, we grew plagued by the requirement to stick with the 2.6.32 kernel (supporting new hardware and new distributions was becoming a nightmare). As we migrated from Lustre 1.8 to 2.4 - thanks to Roland again - we decided to use the in-kernel client (actually, using the in-kernel client was the pre-requisite for this migration). We're happy to see that it works. There are few bugs of course, but fortunately none blocking. Thanks to a very few patches added to 3.13/14 kernel - namely those addressing LU-4209, 4231, 4400, 4530 and hopefully soon 4520 - our Lustre cluster is functional production-wise.

We believe that proper in-kernel support can bring a lot of momentum to Lustre. We hope that Lustre in-kernel client may one day be as notorious as other Intel's contributions to the kernel. Let us know if we can help in some ways.

Best,

Cédric

[Oleg Drokin]

• The mainline-tracking repo is currently being setup (so, no, there is not one yet).
  For now the best you could do is send patches to Greg KH directly, and CC us, so we can pick those up once (either immediately, or once we actually add support for such modern kernels).

• The flow from master to mainline kernel is not currently working all that well, we are currently thinking how to overcome this. The present state is such that the upstream kernel lustre code is quite a bit behind in some respects and ahead in others.

• The official QA/testing will be happening in the repo from item #1 once it's ready, we are working on it. Until then, it's just a few developers have their own testbeds. I set up mine just this week and the state is quite sorry, I am feeding important bugfixes upstream to Greg now to make the client at least work in most cases (I guess see my patches from yesterday in lkml as a startign point, also do not use 3.12, for best results the staging tree is what has the most uptodate client).
  Also, do not enable the additional lustre debug checks, those don't work at present and crash needlessly, this will be fixed eventually.

I agree currently in-kernel stuff is in a somewhat strange state now (for a variety of reasons, not all of which are actually under our control), we are thinking on how to better turn it around.

[Roland Fehrenbacher]

Couple of additional questions/comments concerning this:

• Is there no internal GIT repo for this at Intel (if yes, please make it available publicly)?
• How do you manage the flow from the master Lustre branch to the in-kernel code? It seems
  a lot of fixes don't make it to the in-kernel code.
• Is there any QA/testing on the in-kernel code (the 3.12 stuff crashes as soon as you load the modules)?
• I think, if the in-kernel stuff is supposed to attract any serious users, there should be a much
  better workflow and clear policy regarding its management.

[Oleg Drokin]

Yes, the upstream kernel patches you just send to Greg & general Linux mailing lists (and CC us too, of course), as we have no commit right to staging tree ourselves.

Thanks!

[Cédric Dufour]

Hello again,

Problem was introduced by changes made in https://lkml.org/lkml/2013/7/15/218, which got the "uid_eq" check the wrong way around.

Attached patch solves the issue:

$ whoami
cdufour

$ lfs quota -u $(whoami) /idiap/temp/$(whoami)
Disk quotas for user cdufour (uid 1224):
Filesystem kbytes quota limit grace files quota limit grace
/idiap/temp/cdufour
20 1072693248 1073741824 - 5 9999900 10000000 -

$ lfs quota -u formaz /idiap/temp/formaz
Permission denied.

Best,

Cédric

PS: what is the recommended GIT source to work on the mainline kernel client of Lustre ?
(I'm currently working with git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git)