[LU-340] system hang when running sanity-quota on RHEL5-x86_64-OFED

• Type: Bug
• Resolution: Duplicate
• Priority: Minor
• Affects Version/s: Lustre 2.1.0, Lustre 2.1.1
• Environment: lustre-master/RHEL5-x86_64/#120/ofa build
• Severity: 3
• Assignee: Niu Yawei (Inactive)
• Reporter: Sarah Liu

Description

system hang when running sanity-quota on RHEL5-x86_64-ofa build. Please see the attachment for all the logs.

Issue Links

• is related to: LU-1782 Ignore sb_has_quota_active() in OFED's header
• Trackbacks: Lustre 2.1.0 release testing tracker Lustre 2.1.0 RC0 Tag: v2100RC0 Created Date: 20110820 The difference between RC0 and RC1 is only a date change in lustre/ChangeLog. Lustre 2.1....
• Trackbacks: Lustre 2.1.1 release testing tracker Lustre 2.1.1 RC2 Tag: v2110RC2 Build:

Comments

[Niu Yawei (Inactive)]

Thank you, Sarah. I think the debug_log confirmed that dqacq_handler failed for group quota not enabled or fail_loc set.

Could you try the following commands on client-5 to see what will happen? (quotacheck then set group quota):
lfs quotacheck -ug lustre_dir
lfs setquota -g group_name -b 0 -B 0 -i 0 -I 0 lustre_dir

[Niu Yawei (Inactive)]

I think the default + D_QUOTA will be fine, thank you, Sarah.

[Sarah Liu]

Is the D_QUOTA enabled?

no. I can give you debug log tomorrow. please tell me the debug mask

[Niu Yawei (Inactive)]

Is the D_QUOTA enabled? can we get the debug log on MDS?

[Sarah Liu]

Is it repeatable? What's the /proc/fs/lustre/fail_loc on mds? Thanks.

yes, it can be reproduced.
[root@fat-intel-1 ~]# more /proc/sys/lustre/fail_loc
0

[Niu Yawei (Inactive)]

From the log we can see all pdflush threads on client were waiting on page lock, whereas the dd thread was holding the page lock to do synchronous IO, because of something wrong with group quota, the synchronous I/O can't finish in time, which caused the pdflush threads stalled.

What confused me is that there were lots of "dqacq/dqrel failed! (rc:-5)" errors while setting group quota, but setting user quota was done successfully, and the user quota limit tests passed also. Looks there are only two possible cases that dqacq_handler() return -EIO, one is OBD_FAIL_OBD_DQACQ and another is ll_sb_has_quota_active() checking fails.

Hi, Sarah

Is it repeatable? What's the /proc/fs/lustre/fail_loc on mds? Thanks.

[Peter Jones]

Niu

Please look into this quotas issue when you get a chance

Thanks

Peter