Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-4530

Mainline kernel client (3.12-3.14): lfs quota -u ... -> Permission denied

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • None
    • None
    • 3
    • 12388

    Description

      Hello,

      'lfs quota -u $(whoami)...' reports "Permission denied' for regular users. It works if the same user queries the quota for another user:

      $ whoami
      cdufour

      $ lfs quota -u $(whoami) /idiap/temp/$(whoami)
      Permission denied.

      $ lfs quota -u formaz /idiap/temp/formaz
      Disk quotas for user formaz (uid 1005):
      Filesystem kbytes quota limit grace files quota limit grace
      /idiap/temp/formaz
      113371820 1072693248 1073741824 - 452499 9999900 10000000 -

      (patch follows as soon as I have the LU-<ID> for this bug)

      Best,

      Cédric

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-4011 problems with upstream lustre client code

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. LU-4416 support for 3.12 linux kernel

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-6215 Sync Lustre external tree with lustre linux kernel client

          • Major - Major loss of function.
          • Resolved

          Activity

            [LU-4530] Mainline kernel client (3.12-3.14): lfs quota -u ... -> Permission denied
            cdufour Cédric Dufour added a comment -

            Thanks for the GIT pointers. Having some "insider's" insights on which patch to "backport" in priority (before they make it to the kernel) will definitely help (and provide opportunities for feedback from us users before it does).

            Will/should patches always be linked to Jira IDs and should Jira be the place to provide feedback (in case issues remain) ?
            Will a notice be given in Jira when a patch is submitted to mainline ?

            Also, are you the one to link new mainline kernel issues with LU-4416 or can/should we reporter do it ?
            (note: actually haven't found how to link one issue to another in Jira).

            As for jobstats: I hadn't heard of them until your mention but I will look into it; it seems it would allow us to address some questions we have with some load patterns we experience with our (SGE) grid.

            Best,

            Cédric

            cdufour Cédric Dufour added a comment - Thanks for the GIT pointers. Having some "insider's" insights on which patch to "backport" in priority (before they make it to the kernel) will definitely help (and provide opportunities for feedback from us users before it does). Will/should patches always be linked to Jira IDs and should Jira be the place to provide feedback (in case issues remain) ? Will a notice be given in Jira when a patch is submitted to mainline ? Also, are you the one to link new mainline kernel issues with LU-4416 or can/should we reporter do it ? (note: actually haven't found how to link one issue to another in Jira). As for jobstats: I hadn't heard of them until your mention but I will look into it; it seems it would allow us to address some questions we have with some load patterns we experience with our (SGE) grid. Best, Cédric
            green Oleg Drokin added a comment -

            Thanks for this info. I am glad to hear you find lustre useful and that even vanilla kernel client works for you.

            I uploaded a stream of patches I plan to push upstream to https://github.com/verygreen/linux/tree/lustre-next
            With it and updated LU-4209 patch for the tools sanity testing almost completely succeeds, and more importantly, nothing crashes anymore.

            I know Peng Tao has his tree at https://github.com/bergwolf/linux/compare/master-sync-round-5 that has some more patches, though not as critical for normal operations.

            BTW, do you happen to use jobstats feature?

            green Oleg Drokin added a comment - Thanks for this info. I am glad to hear you find lustre useful and that even vanilla kernel client works for you. I uploaded a stream of patches I plan to push upstream to https://github.com/verygreen/linux/tree/lustre-next With it and updated LU-4209 patch for the tools sanity testing almost completely succeeds, and more importantly, nothing crashes anymore. I know Peng Tao has his tree at https://github.com/bergwolf/linux/compare/master-sync-round-5 that has some more patches, though not as critical for normal operations. BTW, do you happen to use jobstats feature?
            cdufour Cédric Dufour added a comment -

            Hello Oleg,

            Thank you for those precisions.

            Thanks to Roland - who originally set up our Lustre cluster server-wise - we have been using Lustre for 5 years now, as a general purpose temp space for our users and our computation grid. We're quite happy with it, especially after evaluating several commercial solutions and finding none up to Lustre so far. Lately, we grew plagued by the requirement to stick with the 2.6.32 kernel (supporting new hardware and new distributions was becoming a nightmare). As we migrated from Lustre 1.8 to 2.4 - thanks to Roland again - we decided to use the in-kernel client (actually, using the in-kernel client was the pre-requisite for this migration). We're happy to see that it works. There are few bugs of course, but fortunately none blocking. Thanks to a very few patches added to 3.13/14 kernel - namely those addressing LU-4209, 4231, 4400, 4530 and hopefully soon 4520 - our Lustre cluster is functional production-wise.

            We believe that proper in-kernel support can bring a lot of momentum to Lustre. We hope that Lustre in-kernel client may one day be as notorious as other Intel's contributions to the kernel. Let us know if we can help in some ways.

            Best,

            Cédric

            cdufour Cédric Dufour added a comment - Hello Oleg, Thank you for those precisions. Thanks to Roland - who originally set up our Lustre cluster server-wise - we have been using Lustre for 5 years now, as a general purpose temp space for our users and our computation grid. We're quite happy with it, especially after evaluating several commercial solutions and finding none up to Lustre so far. Lately, we grew plagued by the requirement to stick with the 2.6.32 kernel (supporting new hardware and new distributions was becoming a nightmare). As we migrated from Lustre 1.8 to 2.4 - thanks to Roland again - we decided to use the in-kernel client (actually, using the in-kernel client was the pre-requisite for this migration). We're happy to see that it works. There are few bugs of course, but fortunately none blocking. Thanks to a very few patches added to 3.13/14 kernel - namely those addressing LU-4209 , 4231, 4400, 4530 and hopefully soon 4520 - our Lustre cluster is functional production-wise. We believe that proper in-kernel support can bring a lot of momentum to Lustre. We hope that Lustre in-kernel client may one day be as notorious as other Intel's contributions to the kernel. Let us know if we can help in some ways. Best, Cédric
            green Oleg Drokin added a comment -
            • The mainline-tracking repo is currently being setup (so, no, there is not one yet).
              For now the best you could do is send patches to Greg KH directly, and CC us, so we can pick those up once (either immediately, or once we actually add support for such modern kernels).
            • The flow from master to mainline kernel is not currently working all that well, we are currently thinking how to overcome this. The present state is such that the upstream kernel lustre code is quite a bit behind in some respects and ahead in others.
            • The official QA/testing will be happening in the repo from item #1 once it's ready, we are working on it. Until then, it's just a few developers have their own testbeds. I set up mine just this week and the state is quite sorry, I am feeding important bugfixes upstream to Greg now to make the client at least work in most cases (I guess see my patches from yesterday in lkml as a startign point, also do not use 3.12, for best results the staging tree is what has the most uptodate client).
              Also, do not enable the additional lustre debug checks, those don't work at present and crash needlessly, this will be fixed eventually.

            I agree currently in-kernel stuff is in a somewhat strange state now (for a variety of reasons, not all of which are actually under our control), we are thinking on how to better turn it around.

            green Oleg Drokin added a comment - The mainline-tracking repo is currently being setup (so, no, there is not one yet). For now the best you could do is send patches to Greg KH directly, and CC us, so we can pick those up once (either immediately, or once we actually add support for such modern kernels). The flow from master to mainline kernel is not currently working all that well, we are currently thinking how to overcome this. The present state is such that the upstream kernel lustre code is quite a bit behind in some respects and ahead in others. The official QA/testing will be happening in the repo from item #1 once it's ready, we are working on it. Until then, it's just a few developers have their own testbeds. I set up mine just this week and the state is quite sorry, I am feeding important bugfixes upstream to Greg now to make the client at least work in most cases (I guess see my patches from yesterday in lkml as a startign point, also do not use 3.12, for best results the staging tree is what has the most uptodate client). Also, do not enable the additional lustre debug checks, those don't work at present and crash needlessly, this will be fixed eventually. I agree currently in-kernel stuff is in a somewhat strange state now (for a variety of reasons, not all of which are actually under our control), we are thinking on how to better turn it around.
            rfehren Roland Fehrenbacher added a comment - - edited

            Couple of additional questions/comments concerning this:

            • Is there no internal GIT repo for this at Intel (if yes, please make it available publicly)?
            • How do you manage the flow from the master Lustre branch to the in-kernel code? It seems
              a lot of fixes don't make it to the in-kernel code.
            • Is there any QA/testing on the in-kernel code (the 3.12 stuff crashes as soon as you load the modules)?
            • I think, if the in-kernel stuff is supposed to attract any serious users, there should be a much
              better workflow and clear policy regarding its management.
            rfehren Roland Fehrenbacher added a comment - - edited Couple of additional questions/comments concerning this: Is there no internal GIT repo for this at Intel (if yes, please make it available publicly)? How do you manage the flow from the master Lustre branch to the in-kernel code? It seems a lot of fixes don't make it to the in-kernel code. Is there any QA/testing on the in-kernel code (the 3.12 stuff crashes as soon as you load the modules)? I think, if the in-kernel stuff is supposed to attract any serious users, there should be a much better workflow and clear policy regarding its management.
            green Oleg Drokin added a comment - - edited

            Yes, the upstream kernel patches you just send to Greg & general Linux mailing lists (and CC us too, of course), as we have no commit right to staging tree ourselves.

            Thanks!

            green Oleg Drokin added a comment - - edited Yes, the upstream kernel patches you just send to Greg & general Linux mailing lists (and CC us too, of course), as we have no commit right to staging tree ourselves. Thanks!
            cdufour Cédric Dufour added a comment -

            Hello again,

            Problem was introduced by changes made in https://lkml.org/lkml/2013/7/15/218, which got the "uid_eq" check the wrong way around.

            Attached patch solves the issue:

            $ whoami
            cdufour

            $ lfs quota -u $(whoami) /idiap/temp/$(whoami)
            Disk quotas for user cdufour (uid 1224):
            Filesystem kbytes quota limit grace files quota limit grace
            /idiap/temp/cdufour
            20 1072693248 1073741824 - 5 9999900 10000000 -

            $ lfs quota -u formaz /idiap/temp/formaz
            Permission denied.

            Best,

            Cédric

            PS: what is the recommended GIT source to work on the mainline kernel client of Lustre ?
            (I'm currently working with git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git)

            cdufour Cédric Dufour added a comment - Hello again, Problem was introduced by changes made in https://lkml.org/lkml/2013/7/15/218 , which got the "uid_eq" check the wrong way around. Attached patch solves the issue: $ whoami cdufour $ lfs quota -u $(whoami) /idiap/temp/$(whoami) Disk quotas for user cdufour (uid 1224): Filesystem kbytes quota limit grace files quota limit grace /idiap/temp/cdufour 20 1072693248 1073741824 - 5 9999900 10000000 - $ lfs quota -u formaz /idiap/temp/formaz Permission denied. Best, Cédric PS: what is the recommended GIT source to work on the mainline kernel client of Lustre ? (I'm currently working with git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git)

            People

              green Oleg Drokin
              cdufour Cédric Dufour
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: