Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-2587

Quota error after upgrade from 2.1.4 to 2.4

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • Lustre 2.4.0
    • Lustre 2.4.0
    • before upgrade server and client: 2.1.4
      after upgrade server and client: lustre-master tag-2.3.58 build#1142
    • 3
    • 6030

    Description

      The test steps is:
      1.on lustre-2.1.4, set quota limit for quota_2usr and run IOZONE to use up the limit
      2.stop lustre, upgrade the whole system to 2.4, enable quota on the new system
      3.run IOZONE again on lustre-2.4 with the same user, should expect EDQUOT but the user can still write data exceed quota limit a lot.

      Lustre: DEBUG MARKER: upgrade-downgrade : @@@@@@ FAIL: iozone did not fail with EDQUOT

      Quota info on lustre-2.1.4 filesystem before running IOZONE:

      Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
      client-5: Disk quotas for user quota_2usr (uid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre       0       0 1048576       -       0       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       0       -  131072       -       0       -    1280       -
      client-5: lustre-OST0000_UUID
      client-5:                       0       -   65536       -       -       -       -       -
      
      client-5: Disk quotas for group quota_2usr (gid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre       0       0 1048576       -       0       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       0       -  131072       -       0       -    1280       -
      client-5: lustre-OST0000_UUID
      client-5:                       0       -   65536       -       -       -       -       -
      

      Quota info on lustre-2.1.4 filesystem after running IOZONE :

      Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
      client-5: Disk quotas for user quota_2usr (uid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre 1049108*      0 1048576       -       3       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       4       -    1024       -       3       -     640       -
      client-5: lustre-OST0000_UUID
      client-5:                 1049104*      - 1047552       -       -       -       -       -
      
      client-5: Disk quotas for group quota_2usr (gid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre 1049108*      0 1048576       -       3       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       4       -    1024       -       3       -     640       -
      client-5: lustre-OST0000_UUID
      client-5:                 1049104*      - 1047552       -       -       -       -       -
      

      Quota info after upgrade from 2.1.4 to 2.4, before running IOZONE:

      Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
      client-5: Disk quotas for user quota_2usr (uid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre 1049108*      0 1048576       -       3       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       4       -       0       -       3       -       0       -
      client-5: lustre-OST0000_UUID
      client-5:                 1049104       -       0       -       -       -       -       -
      
      client-5: Disk quotas for group quota_2usr (gid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre 1049108*      0 1048576       -       3       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       4       -       0       -       3*      -       3       -
      client-5: lustre-OST0000_UUID
      client-5:                 1049104       -       0       -       -       -       -       -
      

      Quota info on lustre-2.4 filesystem after running IOZONE:

      Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
      client-5: Disk quotas for user quota_2usr (uid 60001):
      client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      client-5:     /mnt/lustre 3146264*      0 1048576       -       0       0    5120       -
      client-5: lustre-MDT0000_UUID
      client-5:                       0       -       0       -       0       -    1024       -
      client-5: lustre-OST0000_UUID
      client-5:                 3146264       -       0       -       -       -       -       -
      

      Attachments

        Activity

          [LU-2587] Quota error after upgrade from 2.1.4 to 2.4
          sarah Sarah Liu made changes -
          Status Original: Resolved [ 5 ] New: Closed [ 6 ]
          sarah Sarah Liu added a comment -

          verified, close.

          sarah Sarah Liu added a comment - verified, close.
          niu Niu Yawei (Inactive) made changes -
          Fix Version/s New: Lustre 2.4.0 [ 10154 ]
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

          landed for 2.4

          niu Niu Yawei (Inactive) added a comment - landed for 2.4
          jlevi Jodi Levi (Inactive) made changes -
          Labels Original: MB New: HB
          jlevi Jodi Levi (Inactive) made changes -
          Labels New: MB
          Priority Original: Minor [ 4 ] New: Blocker [ 1 ]
          adilger Andreas Dilger made changes -
          Summary Original: Quota error ater upgrade from 2.1.4 to 2.4 New: Quota error after upgrade from 2.1.4 to 2.4
          sarah Sarah Liu made changes -
          Description Original: Lustre: DEBUG MARKER: upgrade-downgrade : @@@@@@ FAIL: iozone did not fail with EDQUOT

          Quota info on lustre-2.1.4 filesystem before running IOZONE:
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 0 0 1048576 - 0 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 0 - 131072 - 0 - 1280 -
          client-5: lustre-OST0000_UUID
          client-5: 0 - 65536 - - - - -

          client-5: Disk quotas for group quota_2usr (gid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 0 0 1048576 - 0 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 0 - 131072 - 0 - 1280 -
          client-5: lustre-OST0000_UUID
          client-5: 0 - 65536 - - - - -
          {noformat}

          Quota info on lustre-2.1.4 filesystem after running IOZONE :
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 1024 - 3 - 640 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104* - 1047552 - - - - -

          client-5: Disk quotas for group quota_2usr (gid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 1024 - 3 - 640 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104* - 1047552 - - - - -
          {noformat}

          Quota info after upgrade from 2.1.4 to 2.4, before running IOZONE:
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 0 - 3 - 0 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104 - 0 - - - - -

          client-5: Disk quotas for group quota_2usr (gid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 0 - 3* - 3 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104 - 0 - - - - -
          {noformat}

          Quota info on lustre-2.4 filesystem after running IOZONE:
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 3146264* 0 1048576 - 0 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 0 - 0 - 0 - 1024 -
          client-5: lustre-OST0000_UUID
          client-5: 3146264 - 0 - - - - -
          {noformat}
          New: The test steps is:
          1.on lustre-2.1.4, set quota limit for quota_2usr and run IOZONE to use up the limit
          2.stop lustre, upgrade the whole system to 2.4, enable quota on the new system
          3.run IOZONE again on lustre-2.4 with the same user, should expect EDQUOT but the user can still write data exceed quota limit a lot.

          Lustre: DEBUG MARKER: upgrade-downgrade : @@@@@@ FAIL: iozone did not fail with EDQUOT

          Quota info on lustre-2.1.4 filesystem before running IOZONE:
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 0 0 1048576 - 0 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 0 - 131072 - 0 - 1280 -
          client-5: lustre-OST0000_UUID
          client-5: 0 - 65536 - - - - -

          client-5: Disk quotas for group quota_2usr (gid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 0 0 1048576 - 0 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 0 - 131072 - 0 - 1280 -
          client-5: lustre-OST0000_UUID
          client-5: 0 - 65536 - - - - -
          {noformat}

          Quota info on lustre-2.1.4 filesystem after running IOZONE :
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 1024 - 3 - 640 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104* - 1047552 - - - - -

          client-5: Disk quotas for group quota_2usr (gid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 1024 - 3 - 640 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104* - 1047552 - - - - -
          {noformat}

          Quota info after upgrade from 2.1.4 to 2.4, before running IOZONE:
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 0 - 3 - 0 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104 - 0 - - - - -

          client-5: Disk quotas for group quota_2usr (gid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 1049108* 0 1048576 - 3 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 4 - 0 - 3* - 3 -
          client-5: lustre-OST0000_UUID
          client-5: 1049104 - 0 - - - - -
          {noformat}

          Quota info on lustre-2.4 filesystem after running IOZONE:
          {noformat}
          Listing quotas for user "quota_2usr" and group "quota_2usr" on client-5,client-6...
          client-5: Disk quotas for user quota_2usr (uid 60001):
          client-5: Filesystem kbytes quota limit grace files quota limit grace
          client-5: /mnt/lustre 3146264* 0 1048576 - 0 0 5120 -
          client-5: lustre-MDT0000_UUID
          client-5: 0 - 0 - 0 - 1024 -
          client-5: lustre-OST0000_UUID
          client-5: 3146264 - 0 - - - - -
          {noformat}
          sarah Sarah Liu added a comment -

          Sarah, did you enable quota on the MGS via lctl conf_param? If not, enforcement won't be enabled and this behavior is expected.

          I did lctl conf_param lustre.quota.mdt/ost=ug

          sarah Sarah Liu added a comment - Sarah, did you enable quota on the MGS via lctl conf_param? If not, enforcement won't be enabled and this behavior is expected. I did lctl conf_param lustre.quota.mdt/ost=ug
          niu Niu Yawei (Inactive) added a comment - http://review.whamcloud.com/4969

          People

            niu Niu Yawei (Inactive)
            sarah Sarah Liu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: