[LU-3861] Quota issues after upgrade from 2.1.4 to 2.4

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.4.0
• Type: Bug
• Priority: Major
• Reporter: Mahmoud Hanafi
• Assignee: Niu Yawei (Inactive)
• Resolution: Not a Bug
• Severity: 3

Description

filesystem was create with lustre 2.1.4 with quota off. Files were created. The the filesystem was upgraded.

Did run tunefs.lustre --quota on all osts and mdt.
and lctl conf_param fsname.quota.ost|mdt=u|g|ug|none

inode quota doesn't release when you go over the remove the files. Also how do you run a "quotacheck" to account for all the files created when quota was off

Issue Links

• is related to: LU-3784 Quota issue on system upgraded to 2.4.x

Comments

[Mahmoud Hanafi]

ok we figured out why this was. inodes are aquired 1024 at a time. But the other question remains why is not account for the files that where created before the upgrade.

[Peter Jones]

Niu

Could you please advise here?

Thanks

Peter

[Niu Yawei (Inactive)]

ok we figured out why this was. inodes are aquired 1024 at a time. But the other question remains why is not account for the files that where created before the upgrade.

The 'quotacheck' would be run when you enable the quota feature by 'tunefs.lustre --quota', and quota accounting will always be enabled form then on.

Do you mean the files created before upgrade were not accounted? Could you show me the out of 'lfs quota' and the real file count?

[Mahmoud Hanafi]

yes files created pre-upgrade are not accounted for in the inode count. As you can see I have > 20 files but lfs quota says i have 4. And those are the 4 I created post upgrade.

pfe21 /nobackupp7 # find /nobackupp7 -user mhanafi -type f | wc -l
24615
pfe21 /nobackupp7 # lfs quota -u mhanafi /nobackupp7
Disk quotas for user mhanafi (uid 11312):
Filesystem kbytes quota limit grace files quota limit grace
/nobackupp7 34640729864 0 346350607480 - 4 4000 5000 -

[Niu Yawei (Inactive)]

What's the e2fsprogs version on MDS? Was there any error messages when running 'tune2fs.lustre --quota'? Could you paste the output of 'dumpe2fs met_device' and try to collect debug log when start mdt and when running 'lfs quota'?

[Niu Yawei (Inactive)]

I believe it's the same problem described in LU-3784.

[Mahmoud Hanafi]

No errors where seen during the tune2fs.lustre --quota.
Running version of e2fsprogs are
nbp7-mds1 ~ # rpm -qa |grep e2fs
e2fsprogs-debuginfo-1.42.6.wc2-7.el6.x86_64
e2fsprogs-devel-1.42.7.wc1-7.el6.x86_64
e2fsprogs-1.42.7.wc1-7.el6.x86_64
e2fsprogs-static-1.42.7.wc1-7.el6.x86_64
e2fsprogs-libs-1.42.7.wc1-7.el6.x86_64

attach is debug logs while running quota command

[Niu Yawei (Inactive)]

Hi, Mahmoud

Could install the e2fsprogs from http://build.whamcloud.com/job/e2fsprogs-reviews/173/ (see http://review.whamcloud.com/#/c/7556/), and disable/enable quota for all your mdt and ost devices by:

• tune2fs -O ^quota $dev
• tune2fs -O quota $dev

Then setup lustre to see if the problem is resloved? Thanks.

[Jay Lan (Inactive)]

Which branch should I use to build for lustre-2.4? Thanks!

[Jay Lan (Inactive)]

Please ignore my question earlier wrt branch of e2fsprogs git.

I pulled things down and did a 'make rpm'. It failed here...

154 tests succeeded 1 tests failed
Tests failed: m_quota
make[2]: *** [test_post] Error 1
make[2]: Leaving directory `/usr/src/redhat/BUILD/e2fsprogs-1.42.7.wc1/tests'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/usr/src/redhat/BUILD/e2fsprogs-1.42.7.wc1'
error: Bad exit status from /var/tmp/rpm-tmp.10sTlL (%check)

What mistake could I have made? Please advise. Thanks!

[Jay Lan (Inactive)]

The tests/m_quota/expect.1 file generated during 'make rpm'. This test failed.

The source is at https://github.com/jlan/e2fsprogs-nas. Branch nas-2.4.0.

[Niu Yawei (Inactive)]

Hi, Jay

You can download rpm from http://build.whamcloud.com/job/e2fsprogs-reviews/173/ directly. If you want to build by yourself, you'd build patchset2, and if m_quota test failed, it probably because the users/groups on your building system is different with ours, you can just check the m_quota.failed file and correct the m_quota/expect.1.

[Jay Lan (Inactive)]

Attached the m_quota.failed file. It does not seem to be the users/groups ID problem. Can you check the file and let me know what the failure was about? Thanks!

[Niu Yawei (Inactive)]

Jay, you can just correct your m_quota/expect.1 as the m_quota.failed shows.

[Jay Lan (Inactive)]

Hi Niu, yeah, that worked. Thanks!

Hmm, I would assume the tests/m_quota/expect.1 file specifies expected values out of a good test run. If I change the expected values to get tests passed, I am wondering what's the point of running the tests?

[Niu Yawei (Inactive)]

I'm not quite sure why the free blocks are not constant on different building system when quota feature enabled.

[Mahmoud Hanafi]

Installing the new e2fsprogs and running tune2fs -O ^quota $device/tune2fs -O quota $device had fixed the issue. lfs quota is now reporting correctly.

mhanafi@pfe21:~> lfs quota -u mhanafi /nobackupp7
Disk quotas for user mhanafi (uid 11312):
Filesystem kbytes quota limit grace files quota limit grace
/nobackupp7 34640747452 0 346350607480 - 24751* 4000 5000 -

[Mahmoud Hanafi]

This case can be closed

[Peter Jones]

Thanks Mahmoud