[LU-2350] Unexpected quotactl error after upgrade from 1.8.8-wc1 to master: Operation not supported

• Status: Closed
• Project: Lustre
• Affects Version/s: Lustre 2.4.0
• Type: Bug
• Priority: Minor
• Reporter: Sarah Liu
• Assignee: Niu Yawei (Inactive)
• Resolution: Won't Fix
• Labels: sdsc

Environment

Before upgrade:
server 1.8.8-wc1 RHEL5
client 1.8.8-wc1 RHEL5/RHEL6

After upgrade:
server lustre-master build #1043 RHEL6
client luster-master build #1043 RHEL6

• Severity: 3

Description

Here is console log from client:

===== Check Lustre quotas usage/limits ======================================
quotacheck failed: Operation not supported
Listing quotas for user "quota_usr" and group "quota_usr" on client-5,client-6...
client-5: Disk quotas for user quota_usr (uid 60000):
client-5: Unexpected quotactl error: Operation not supported
client-5:      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
client-5:     /mnt/lustre     [0]     [0]     [0]       -     [0]     [0]     [0]       -
client-5: Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.

dmesg from client-5

Lustre: DEBUG MARKER: ===== Check Lustre quotas usage/limits ======================================
Lustre: DEBUG MARKER: cancel_lru_locks mdc start
Lustre: DEBUG MARKER: cancel_lru_locks osc start
LustreError: 4995:0:(osc_quota.c:277:osc_quotactl()) ptlrpc_queue_wait failed, rc: -95
LustreError: 4995:0:(mdc_request.c:1431:mdc_quotactl()) ptlrpc_queue_wait failed, rc: -95
LustreError: 4995:0:(lmv_obd.c:3125:lmv_quotactl()) getquota on mdt 0 failed. -95
LustreError: 4995:0:(mdc_request.c:1431:mdc_quotactl()) ptlrpc_queue_wait failed, rc: -95
LustreError: 4995:0:(osc_quota.c:277:osc_quotactl()) ptlrpc_queue_wait failed, rc: -95

Issue Links

• is related to: LU-4761 console quota message spew

Comments

[Johann Lombardi (Inactive)]

Sarah, quotacheck isn't supported any more in 2.4. Instead, tunefs.lustre --quota should be run against the targets formatted with 1.8. Also the 1.8 clients won't be fully interoperable with 2.4 servers as far as quota is concerned. Customers will have to migrate to 1.8.9.

[Haisong Cai (Inactive)]

upgraded Lustre server from 1.8.7 to 2.4.2.
Did what was recommented, "tunefs.lustre --quota <devices>".

lctl get_param osd-..quota_slave.info

osd-ldiskfs.puma-OST000e.quota_slave.info=
target name: puma-OST000e
pool ID: 0
type: dt
quota enabled: none
conn to master: not setup yet
space acct: ug
user uptodate: glb[0],slv[0],reint[1]
group uptodate: glb[0],slv[0],reint[1]

showed OSTs quota noenable. Then ran:

lctl conf_param meerkat.quota.ost=ug

[root@puma-oss-13-2 ~]# lctl get_param osd-..quota_slave.info
osd-ldiskfs.puma-OST0006.quota_slave.info=
target name: puma-OST0006
pool ID: 0
type: dt
quota enabled: ug
conn to master: not setup yet
space acct: ug
user uptodate: glb[0],slv[0],reint[1]
group uptodate: glb[0],slv[0],reint[1]

But on client side, I don't see usages prior to upgrading anymore.

Haisong

[Peter Jones]

Niu

Could you please advise on this one?

Thanks

Peter

[Niu Yawei (Inactive)]

lctl conf_param meerkat.quota.ost=ug
[root@puma-oss-13-2 ~]# lctl get_param osd-..quota_slave.info
osd-ldiskfs.puma-OST0006.quota_slave.info=
target name: puma-OST0006
pool ID: 0
type: dt
quota enabled: ug
conn to master: not setup yet
space acct: ug
user uptodate: glb[0],slv[0],reint[1]
group uptodate: glb[0],slv[0],reint[1]

quota slave on OST didn't connect to master yet, did you upgrade MDT and enable quota on MDT by tunefs as well?

But on client side, I don't see usages prior to upgrading anymore.

{qutoe}

as Johann mentioned, you need to upgrade client to 1.8.9 or later to interoperate with the new quota in 2.4.

BTW, you need to upgrade the e2fsprogs to 1.42.7-wc2 or later (which has a important fix for quota).

[Haisong Cai (Inactive)]

Hi Yawei,

MDT was enabled with quota as well, even though I though it was optional because it was only for the inode quota and we didn't want to enable inode quota.

2 questions:

1) what does it mean "OST didn't connect to master yet"? How to connect to master?

2) Does upgrading e2fsprogs require offline the file-system?

thanks,
Haisong

[Bob Glossman (Inactive)]

As far as question 2), need not offline filesystems to upgrade e2fsprogs. It is merely a set of user space tools and utilities. Nothing in it runs continuously while lustre is running.

[Niu Yawei (Inactive)]

1) what does it mean "OST didn't connect to master yet"? How to connect to master?

The quota_slave.info shows "conn to master: not setup yet", which means slave(ost) didn't connect to master(mdt) yet, and quota can't be enforced. Slave will connect to master automatically.

[Niu Yawei (Inactive)]

Close old 1.8 issue.