Whamcloud Community JIRA

Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us

9.4.14

940014

05-12-2023

[LU-1842] Quota enforcement landing https://jira.whamcloud.com/browse/LU-1842 Lustre <p>Tracker bug for landing orion_quota to master.</p> LU-1842

Quota enforcement landing

New Feature Minor Resolved Fixed Johann Lombardi Johann Lombardi Thu, 6 Sep 2012 05:39:54 +0000 Tue, 20 Nov 2018 21:49:54 +0000 Wed, 23 Apr 2014 08:09:46 +0000 Lustre 2.4.0 Lustre 2.4.0 Lustre 2.1.4 Lustre 1.8.9 0 11 <p>quota: define quota records for glb/slv indexes<br/> <a href="http://review.whamcloud.com/#change,3886" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3886</a></p> <p>ldlm: support for sending GL ASTs to multiple locks<br/> <a href="http://review.whamcloud.com/3887" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3887</a></p> <p>quota: quota identifier unification<br/> <a href="http://review.whamcloud.com/3888" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3888</a></p> <p>quota: ldiskfs local enforcement<br/> <a href="http://review.whamcloud.com/3915" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3915</a></p> <p>quota: zfs local enforcement<br/> <a href="http://review.whamcloud.com/3933" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3933</a></p> <p>protocol: add support for OBD_IDX_READ<br/> <a href="http://review.whamcloud.com/#change,3942" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3942</a></p> <p>quota: helper routines to manage FID_SEQ_QUOTA_GLB<br/> <a href="http://review.whamcloud.com/#change,3937" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3937</a></p> <p>mdt: add lvbo support to MDT namespace<br/> <a href="http://review.whamcloud.com/#change,3951" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3951</a></p> <p>protocol: new quota RPC format<br/> <a href="http://review.whamcloud.com/3977" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3977</a></p> <p>iam: store key & rec in LE order for quota IAM files<br/> <a href="http://review.whamcloud.com/#change,3989" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3989</a></p> <p>lfs: lfs quota commands changes<br/> <a href="http://review.whamcloud.com/4022" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4022</a></p> <p>test: t-f changes for the new quota<br/> <a href="http://review.whamcloud.com/4031" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4031</a></p> <p>test: new s-q<br/> <a href="http://review.whamcloud.com/4040" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4040</a></p> <p>quota: add quota disk operations<br/> <a href="http://review.whamcloud.com/4042" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4042</a></p> <p>osp: osp-on-ost device<br/> <a href="http://review.whamcloud.com/4068" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4068</a></p> <p>quota: migrate old quota admin files<br/> <a href="http://review.whamcloud.com/4077" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4077</a></p> <p>quota: remove quota code<br/> <a href="http://review.whamcloud.com/#change,4092" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4092</a></p> <p>quota: lquota entry<br/> <a href="http://review.whamcloud.com/4101" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4101</a></p> <p>ldlm: add support for quota LVB<br/> <a href="http://review.whamcloud.com/4107" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4107</a></p> <p>quota: qsd request<br/> <a href="http://review.whamcloud.com/4109" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4109</a></p> <p>quota: remove quota code from lvfs<br/> <a href="http://review.whamcloud.com/#change,4110" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4110</a></p> <p>quota: qsd entry operations<br/> <a href="http://review.whamcloud.com/4119" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4119</a></p> <p>quota: setup/shutdown qmt device<br/> <a href="http://review.whamcloud.com/#change,4126" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4126</a></p> <p>quota: qsd lock<br/> <a href="http://review.whamcloud.com/4129" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4129</a></p> <p>quota: add per-filesystem information<br/> <a href="http://review.whamcloud.com/4156" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4156</a></p> <p>quota: add quotactl support on qmt<br/> <a href="http://review.whamcloud.com/4160" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4160</a></p> <p>quota: add quota locks support on QMT<br/> <a href="http://review.whamcloud.com/#change,4166" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4166</a></p> <p>quota: add acq/rel logic on QMT<br/> <a href="http://review.whamcloud.com/#change,4178" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4178</a></p> <p>quota: add core QSD code<br/> <a href="http://review.whamcloud.com/#change,4182" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4182</a></p> <p>test: re-enable all sanity-quota tests<br/> <a href="http://review.whamcloud.com/#change,4200" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4200</a></p> <p>Patch <a href="http://review.whamcloud.com/#change,4156" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4156</a> introduced a kernel oops when mounting a MGT on a stand alone MGS system. The oops is:</p> <p>2012-10-05 09:52:05 [ 355.128991] Lustre: DEBUG MARKER: mkdir -p /tmp/lustre/mgs<br/> 2012-10-05 09:52:05 [ 355.485941] Lustre: DEBUG MARKER: test -b /dev/sda1<br/> 2012-10-05 09:52:05 [ 355.830922] Lustre: DEBUG MARKER: mkdir -p /tmp/lustre/mgs; mount -t lustre ^I^I /dev/sda1 /tmp/lustre/mgs<br/> 2012-10-05 09:52:06 [ 356.265280] LDISKFS-fs (sda1): mounted filesystem with ordered data mode. quota=on. Opts: <br/> 2012-10-05 09:52:06 [ 356.381871] LustreError: 8414:0:(qsd_lib.c:355:qsd_init()) MGS: fail to extract filesystem name<br/> 2012-10-05 09:52:06 [ 356.449872] BUG: unable to handle kernel NULL pointer dereference at (null)<br/> 2012-10-05 09:52:06 [ 356.457029] IP: <span class="error">[<ffffffffa0cdf43a>]</span> osd_oi_table_put+0x10a/0x1d0 <span class="error">[osd_ldiskfs]</span><br/> 2012-10-05 09:52:06 [ 356.464534] PGD 5c85dc067 PUD 5c96f8067 PMD 0 <br/> 2012-10-05 09:52:06 [ 356.469223] Oops: 0000 <a href="#1" target="_blank" rel="noopener">1</a> SMP <br/> 2012-10-05 09:52:06 [ 356.472645] last sysfs file: /sys/module/ldiskfs/initstate<br/> 2012-10-05 09:52:06 [ 356.478213] CPU 2 <br/> 2012-10-05 09:52:06 [ 356.480088] Modules linked in: fsfilt_ldiskfs(U) exportfs osd_ldiskfs(U) mdd(U) ldiskfs(U) mbcache jbd2 lustre(U) lov(U) osc(U) mdc(U) lquo<br/> ta(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ko2iblnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) autofs4 ib_srp scsi_transport_srp scsi_tgt ipmi_devintf ipmi_si ipmi_msghandler nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iptable_filter ip_tables ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ib_sa mlx4_ib ib_mad ib_core dm_mirror dm_region_hash dm_log dm_multipath dm_mod sg power_meter sr_mod cdrom sd_mod crc_t10dif ses enclosure microcode dcdbas serio_raw ata_generic pata_acpi ata_piix iTCO_wdt iTCO_vendor_support megaraid_sas i7core_edac edac_core ipv6 nfs lockd fscache nfs_acl auth_rpcgss sunrpc mlx4_en mlx4_core bnx2 <span class="error">[last unloaded: scsi_wait_scan]</span><br/> 2012-10-05 09:52:06 [ 356.565516] <br/> 2012-10-05 09:52:06 [ 356.567099] Pid: 8414, comm: mount.lustre Not tainted 2.6.32-279.9.1.el6.head.x86_64 #1 Dell Inc. PowerEdge R610/0F0XJ6<br/> 2012-10-05 09:52:06 [ 356.578177] RIP: 0010:<span class="error">[<ffffffffa0cdf43a>]</span> <span class="error">[<ffffffffa0cdf43a>]</span> osd_oi_table_put+0x10a/0x1d0 <span class="error">[osd_ldiskfs]</span><br/> 2012-10-05 09:52:06 [ 356.588109] RSP: 0018:ffff8805c463d948 EFLAGS: 00010246<br/> 2012-10-05 09:52:06 [ 356.593503] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000<br/> 2012-10-05 09:52:06 [ 356.600717] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffff8805c4590000<br/> 2012-10-05 09:52:06 [ 356.607930] RBP: ffff8805c463d978 R08: 0000000000000000 R09: ffff8805f0091c80<br/> 2012-10-05 09:52:06 [ 356.615156] R10: ffff880330096440 R11: 0000000000000000 R12: 0000000000000000<br/> 2012-10-05 09:52:06 [ 356.622371] R13: 0000000000000040 R14: 0000000000000000 R15: 5a5a5a5a5a5a5a5a<br/> 2012-10-05 09:52:06 [ 356.629590] FS: 00007f371446e700(0000) GS:ffff88033ac20000(0000) knlGS:0000000000000000<br/> 2012-10-05 09:52:06 [ 356.637820] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b<br/> 2012-10-05 09:52:06 [ 356.643649] CR2: 0000000000000000 CR3: 000000031701a000 CR4: 00000000000006e0<br/> 2012-10-05 09:52:06 [ 356.650864] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000<br/> 2012-10-05 09:52:06 [ 356.658077] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400<br/> 2012-10-05 09:52:06 [ 356.665297] Process mount.lustre (pid: 8414, threadinfo ffff8805c463c000, task ffff8805c466f500)<br/> 2012-10-05 09:52:06 [ 356.674219] Stack:<br/> 2012-10-05 09:52:06 [ 356.676322] ffff8805c463d968 ffff8805ca6da000 ffff8805c463da08 00000000ffffffea<br/> 2012-10-05 09:52:06 [ 356.683788] <d> ffff8805ca6da000 ffff8805ca6db4e8 ffff8805c463d998 ffffffffa0cdf51d<br/> 2012-10-05 09:52:06 [ 356.691845] <d> ffff8805c463da08 ffff8805ca6da000 ffff8805c463d9e8 ffffffffa0cd373e<br/> 2012-10-05 09:52:06 [ 356.701234] Call Trace:<br/> 2012-10-05 09:52:06 [ 356.703777] <span class="error">[<ffffffffa0cdf51d>]</span> osd_oi_fini+0x1d/0x120 <span class="error">[osd_ldiskfs]</span><br/> 2012-10-05 09:52:06 [ 356.710395] <span class="error">[<ffffffffa0cd373e>]</span> osd_device_alloc+0x43e/0x810 <span class="error">[osd_ldiskfs]</span><br/> 2012-10-05 09:52:06 [ 356.717559] <span class="error">[<ffffffffa055acf7>]</span> obd_setup+0x1d7/0x2f0 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.723846] <span class="error">[<ffffffffa055b018>]</span> class_setup+0x208/0x890 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.730304] <span class="error">[<ffffffffa05625cc>]</span> class_process_config+0xc0c/0x1ca0 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.737678] <span class="error">[<ffffffffa03a8be0>]</span> ? cfs_alloc+0x30/0x60 <span class="error">[libcfs]</span><br/> 2012-10-05 09:52:06 [ 356.743791] <span class="error">[<ffffffffa0569b63>]</span> ? lustre_cfg_new+0x353/0x710 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.750685] <span class="error">[<ffffffffa056a4b9>]</span> do_lcfg+0x149/0x480 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.756796] <span class="error">[<ffffffffa056a884>]</span> lustre_start_simple+0x94/0x200 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.763862] <span class="error">[<ffffffffa057b0a3>]</span> lustre_fill_super+0x1093/0x1af0 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.770996] <span class="error">[<ffffffff8116a48c>]</span> ? pcpu_alloc+0x3ac/0xa50<br/> 2012-10-05 09:52:06 [ 356.776570] <span class="error">[<ffffffff8127a22a>]</span> ? strlcpy+0x4a/0x60<br/> 2012-10-05 09:52:06 [ 356.781715] <span class="error">[<ffffffff8117de33>]</span> ? sget+0x3e3/0x480<br/> 2012-10-05 09:52:06 [ 356.786764] <span class="error">[<ffffffff8117cf10>]</span> ? set_anon_super+0x0/0x100<br/> 2012-10-05 09:52:06 [ 356.792527] <span class="error">[<ffffffffa057a010>]</span> ? lustre_fill_super+0x0/0x1af0 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.799572] <span class="error">[<ffffffff8117e37f>]</span> get_sb_nodev+0x5f/0xa0<br/> 2012-10-05 09:52:06 [ 356.804993] <span class="error">[<ffffffffa05656a5>]</span> lustre_get_sb+0x25/0x30 <span class="error">[obdclass]</span><br/> 2012-10-05 09:52:06 [ 356.811430] <span class="error">[<ffffffff8117dfdb>]</span> vfs_kern_mount+0x7b/0x1b0<br/> 2012-10-05 09:52:06 [ 356.817090] <span class="error">[<ffffffff8117e182>]</span> do_kern_mount+0x52/0x130<br/> 2012-10-05 09:52:06 [ 356.822665] <span class="error">[<ffffffff8119c862>]</span> do_mount+0x2d2/0x8d0<br/> 2012-10-05 09:52:06 [ 356.827890] <span class="error">[<ffffffff8119cef0>]</span> sys_mount+0x90/0xe0<br/> 2012-10-05 09:52:06 [ 356.833030] <span class="error">[<ffffffff8100b0f2>]</span> system_call_fastpath+0x16/0x1b<br/> 2012-10-05 09:52:06 [ 356.839120] Code: f8 b9 19 00 00 00 48 89 d7 f3 48 ab 48 8b 3b e8 7d 97 6c ff 48 c7 03 00 00 00 00 41 83 c4 01 45 39 e5 76 3d 49 63 c4 49 8d 1c c6 <48> 8b 03 48 85 c0 74 e8 48 8b 38 48 85 ff 74 35 48 3b 78 08 0f <br/> 2012-10-05 09:52:06 [ 356.861652] RIP <span class="error">[<ffffffffa0cdf43a>]</span> osd_oi_table_put+0x10a/0x1d0 <span class="error">[osd_ldiskfs]</span><br/> 2012-10-05 09:52:06 [ 356.869243] RSP <ffff8805c463d948><br/> 2012-10-05 09:52:06 [ 356.872820] CR2: 0000000000000000<br/> 2012-10-05 09:52:07 [ 0.000000] Initializing cgroup subsys cpuset<br/> 2012-10-05 09:52:07 [ 0.000000] Initializing cgroup subsys cpu</p> <p>Hi James,</p> <p>Could you please give a try to this patch? <a href="http://review.whamcloud.com/4203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4203</a></p> <p>It should address the qsd setup issue.<br/> That said, there is still a bug related to oi scrub since it should definitely properly clean up if the quota slave instance failed to be started. I will talk to Fanyong.</p> <p>Thanks in advance.</p> <p>For the record, i have verified that the patch fixes the problem.<br/> Thanks James!</p> <p>utils: tunefs.lustre --quota<br/> <a href="http://review.whamcloud.com/420" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/420</a></p> <blockquote> <p>That said, there is still a bug related to oi scrub since it should definitely properly clean up if the quota slave instance failed to be started. I will talk to Fanyong.</p></blockquote> <p>Patch for the oi scrub init issue (independent of quota) is available here:<br/> b2_3: <a href="http://review.whamcloud.com/#change,4218" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4218</a><br/> master: <a href="http://review.whamcloud.com/#change,4219" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4219</a></p> <p>For the record, <a href="https://jira.whamcloud.com/browse/LUDOC-89" title="Quota Enforcement Doc Changes" class="issue-link" data-issue-key="LUDOC-89"><del>LUDOC-89</del></a> is the bug to track changes to the manual.</p> <p>Appears master + <a href="http://review.whamcloud.com/#change,4218" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4218</a> fixed my problem.</p> <p>Hello,</p> <p>I am seeing a problem with ldiskfs and quotas in 2.4. <a href="http://review.whamcloud.com/4098" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4098</a> turned on quotas by default when mounting. However, when my ldiskfs goes to mount, it will fails since quotas and usage accounting are contraindicated. from the "first class accounting" ldiskfs patch:</p> <p>+#ifdef CONFIG_QUOTA<br/> + /* Enable space tracking during mount, enforcement can be enabled/disable<br/> + * later with quota_on/off */<br/> + if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_QUOTA) &&<br/> + !(sb->s_flags & MS_RDONLY)) {<br/> + ret = ext4_acct_on(sb);<br/> + if (ret) </p> { + ext4_msg(sb, KERN_ERR, "Can't enable usage tracking on " + "a filesystem with the QUOTA feature set"); + goto failed_mount4; + } <p>+ }</p> <p>am i confused about what is going on here?</p> <p>This code snippet just enables space accounting automatically at mount time if the quota feature is enabled on the backend ldiskfs filesystem.</p> <p>Could you please tell us more about the problem you see with ldiskfs and quota in 2.4?</p> <p>i ported the ldiskfs patches to sles11sp2's kernel (basically 3.0). so it is possible to run quota and usage accounting at the same time? i guess i need to look harder at the ldiskfs patches.</p> <p>Quota enforcement actually requires accounting to be enabled. The only possible combinations are:</p> <ul class="alternate" type="square"> <li>no accounting & no limit enforcement</li> <li>space accounting enabled & no limit enforcement</li> <li>space accounting enabled & quota limit enforced</li> </ul> <p>how clever. apparently i have to have the quota_v2 module loaded for this feature. however, ldiskfs doesnt depend on this module and doesnt seem to automagically load it either. this doesnt appear to be happening because QFMT_VFS_V1 isn't mapped to any particular module name in linux/quota.h:</p> <p>#define INIT_QUOTA_MODULE_NAMES {\<br/> </p> {QFMT_VFS_OLD, "quota_v1"} <p>,\<br/> </p> {QFMT_VFS_V0, "quota_v2"} <p>,\<br/> {0, NULL}}</p> <p>For the older OS(SLES11 SP1) that does not have QFMT_VFS_V1 support, is there a problem enabling it with QFMT_VFS_V0 instead of QFMT_VFS_V1?</p> <p>Looking at the new quota code it appears to be independent of what version of linux quota you are using. From Lustre 2.1.X I see the main dependency was from the lustre quota ioctl in the fsfilt layer which did vary between linux quota versions. Now that code is linux independent.</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33694" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33694</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-1842" title="Quota enforcement landing" class="issue-link" data-issue-key="LU-1842"><del>LU-1842</del></a> ptlrpc: clean up ptlrpc_client setup<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: fa1de82769802f5b772ff625e738e15f23b15848</p> Duplicate LU-1300 Related LU-1789 LU-1318 LU-5975 LU-5978 LU-5980 LUDOC-89 LU-1318 LU-1789 LU-1818 LU-1830 LU-1920 Development Rank 1|hzv5c7: Rank (Obsolete) 4399