Details
-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
-
Lustre 2.16.1
-
None
-
Lustre 2.15 / EL7 or Lustre 2.16 / EL9
-
2
-
9223372036854775807
Description
Hello! I just realized that posix_fallocate() allows users to bypass Lustre quota enforcement. Tested against Lustre 2.15.5 and 2.16.1.
I'm attaching a simple fallocate helper fallocate.c that I used below.
We assume quota enforcement is enabled here (osd-ldiskfs.lustre-OST*.quota_slave_dt.enabled=up):
User quota enforcement:
$ lfs setquota -u sthiell -B 1T /elm $ ./fallocate bigfalloc-user $(( 2 * 1024 * 1024 * 1024 * 1024 )) $ stat bigfalloc-user File: bigfalloc-user Size: 2199023255552 Blocks: 4294968080 IO Block: 4194304 regular file Device: 899af214h/2308633108d Inode: 144115441093058211 Links: 1 Access: (0644/-rw-r--r--) Uid: (282232/ sthiell) Gid: ( 100/ users) Access: 2025-05-14 15:39:29.000000000 -0700 Modify: 2025-05-14 15:39:29.000000000 -0700 Change: 2025-05-14 15:39:29.000000000 -0700 Birth: 2025-05-14 15:39:29.000000000 -0700 $ lfs quota -u sthiell -h /elm/ Disk quotas for usr sthiell (uid 282232): Filesystem used quota limit grace files quota limit grace /elm/ 2T* 0k 1T - 5 0 0 -
On another system, I was able to bypass project quotas in the same way.
$ ./fallocate /scratch/users/sthiell/bigfalloc $(( 10 * 1024 * 1024 * 1024 * 1024 )) Successfully allocated 10995116277760 bytes for file /scratch/users/sthiell/bigfalloc $ lfs quota -p 282232 -h /scratch/ Disk quotas for prj 282232 (pid 282232): Filesystem used quota limit grace files quota limit grace /scratch/ 112T* 0k 100T - 4849 0 104857600 - pid 282232 is using default block quota setting
Originally discovered with dtar which makes use of posix_fallocate() to preallocate the resulting tar file. It seems pretty important to fix this ASAP.
Thanks scherementsev. for you inputs.
This implies that, it appears to allocate first and then report the usage afterward? What I understand. fallocate declares quota usage through osd_declare_inode_qid() it may not query the QMT(Master) before proceeding?
I am tracing through osd_declare_inode_qid() and it did seem to call qsd_op_begin().
osd_declare_inode_qid
->qsd_op_begin()
Small snip of call graph (just tracing lquota.ko)
I can confirm I have hit this too. This means it is querying QMT and reporting error. What conditions are met here? Any hints.