[#LU-2815] quota not correct

Comment by Johann Lombardi (Inactive) [ 14/Feb/13 ]

Disk quotas for user efares (uid 12137):
Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
/nobackupp2 56990092684* 50000000000 75000000000 1w4d18h25m10s   98038* 100000  200000 1w6d23h49m47s

Although the inode usage is below the soft limit, the MDT still owns 102400 inodes from the master as shown below:

nbp2-MDT0000_UUID
                 287960       -  393216       -   98038       -  102400       -

Therefore the quota master started the grace time. The issue is that the quota allocation algorithm only relies on the hard limit and does not take into account the soft limit. The new quota implementation in 2.4 should address this problem.
Meanwhile, you can decrease iunit/itune:

$ cat /proc/fs/lustre/mds/lustre-MDT0000/quota_iunit_sz 
5120
$ cat /proc/fs/lustre/mds/lustre-MDT0000/quota_itune_sz 
2560
$ echo 640 > /proc/fs/lustre/mds/lustre-MDT0000/quota_itune_sz
$ echo 1280 > /proc/fs/lustre/mds/lustre-MDT0000/quota_iunit_sz

Comment by Peter Jones [ 14/Feb/13 ]

Assigning to Niu for any follow-on questions

Comment by Johann Lombardi (Inactive) [ 15/Feb/13 ]

When removing a file, the inode count should drop (provided that the file isn't opened somewhere else) from 113869 to 113868.
As for the quota allocation, the MDS won't release any quota space until inode_usage < quota_owned_by_mds - itune.

Comment by Mahmoud Hanafi [ 15/Feb/13 ]

if the file is open then you shouldn't be able to delete it. The user was able to delete the file. This "inode_usage < quota_owned_by_mds - itune" is used for inode quota or block quota. We are having a lot of reports such issue with quotas.

We need a better explanation of how this can occur and why Lustre quota are getting out of sync with real usage.

Thanks,
Mahmoud

Comment by Johann Lombardi (Inactive) [ 15/Feb/13 ]

if the file is open then you shouldn't be able to delete it.

Not at all. You can definitely delete an opened file. The file is unlinked from the namespace (it is actually moved to a special directory called PENDING on the MDT) and then finally deleted once the last opener closes the file. Please see the man page of unlink(2) for more information.

This "inode_usage < quota_owned_by_mds - itune" is used for inode quota or block quota.

I deliberately used the word "inode" above to tell that this logic applies to inode quota management. The same logic is used for blocks on the OST.

We are having a lot of reports such issue with quotas.

Have you decreased iunit and itune as suggested above?

We need a better explanation of how this can occur and why Lustre quota are getting out of sync with real usage.

Could you please try to create a file with this user, run lfs quota -v, delete the file and run lfs quota -v again?

Comment by Mahmoud Hanafi [ 01/Mar/13 ]

Here is another example of quota issue. du and lfs quota don't match

nbp6-mds /proc/fs/lustre/lquota/mdd_obd-nbp6-MDT0000 # lfs quota -q -u dposselt /nobackupp6
/nobackupp6 6493023000 7000000000 10000000000 - 164963 200000 500000 -

pfe1 /nobackupp6/dposselt # du --apparent-size -sk
5565676998 .

Comment by Niu Yawei (Inactive) [ 05/Mar/13 ]

Mahmoud, "du --apparent-size" is usually smaller than actual disk usage, because apparent-size doesn't count the full block size at the end of file, but disk usage does. Quota usage is actual disk usage (maybe not exactly equal to the output of du), could you check the output of "du -sk", I think it should be closer to the output of 'lfs quota'?

Comment by Mahmoud Hanafi [ 29/Oct/13 ]

This can be closed

Comment by Peter Jones [ 29/Oct/13 ]

Thanks Mahmoud

[LU-2815] quota not correct Created: 14/Feb/13 Updated: 29/Oct/13 Resolved: 29/Oct/13
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	Lustre 2.1.3
Fix Version/s:	None

[LU-2815] quota not correct Created: 14/Feb/13 Updated: 29/Oct/13 Resolved: 29/Oct/13