[LU-1493] assertion in dqacq_completion() (count < *hardlimit) failed

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.3.0, Lustre 2.1.3
• Affects Version/s: Lustre 2.1.1
• Severity: 3
• Assignee: Niu Yawei (Inactive)
• Reporter: Alexandre Louvet (Inactive)

Description

System did assert with following trace :

LustreError: 12908:0:(quota_context.c:683:dqacq_completion()) ASSERTION(count < *hardlimit) failed: id(10912) flag(22) type(u) isblk(b) count(134217728) qd_qunit(134217728) hardlimit(131072).
LustreError: 12908:0:(quota_context.c:683:dqacq_completion()) LBUG
Pid: 12908, comm: mdt_01

Call Trace:
 [<ffffffffa04df855>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
 [<ffffffffa04dfe95>] lbug_with_loc+0x75/0xe0 [libcfs]
 [<ffffffffa07d6d06>] dqacq_completion+0x1606/0x1610 [lquota]
 [<ffffffffa07eb84d>] ? quota_search_lqs+0x9d/0x5f0 [lquota]
 [<ffffffffa04e09ae>] ? cfs_free+0xe/0x10 [libcfs]
 [<ffffffff81085bf0>] ? getnstimeofday+0x60/0xf0
 [<ffffffffa07d4738>] schedule_dqacq+0xa08/0x19d0 [lquota]
 [<ffffffffa04e09ae>] ? cfs_free+0xe/0x10 [libcfs]
 [<ffffffffa07d15e8>] ? check_cur_qunit+0x448/0xb70 [lquota]
 [<ffffffffa07dc8ca>] ? quota_is_set+0x6a/0x2f0 [lquota]
 [<ffffffffa07d79c9>] qctxt_adjust_qunit+0x109/0x350 [lquota]
 [<ffffffff81085cea>] ? do_gettimeofday+0x1a/0x50
 [<ffffffffa07e299d>] mds_quota_adjust+0x2ad/0x3b0 [lquota]
 [<ffffffffa09f2001>] ? mdd_lov_create_finish+0x61/0xd0 [mdd]
 [<ffffffffa0a07982>] mdd_create+0x6c2/0x1db0 [mdd]
 [<ffffffffa04ee649>] ? cfs_hash_bd_add_locked+0x29/0x90 [libcfs]
 [<ffffffffa05a96de>] ? lu_object_find_at+0x3fe/0x770 [obdclass]
 [<ffffffffa09e5f08>] ? mdd_version_get+0x68/0xa0 [mdd]
 [<ffffffffa0a902bc>] cml_create+0xbc/0x280 [cmm]
 [<ffffffffa0a4d746>] ? mdt_version_save+0x96/0x170 [mdt]
 [<ffffffffa0a65747>] mdt_reint_open+0x1f67/0x2d90 [mdt]
 [<ffffffff81003ace>] ? common_interrupt+0xe/0x13
 [<ffffffffa0a0e586>] ? md_ucred+0x26/0x60 [mdd]
 [<ffffffffa0a305f5>] ? mdt_ucred+0x15/0x20 [mdt]
 [<ffffffffa0a4786f>] ? mdt_root_squash+0x2f/0x450 [mdt]
 [<ffffffffa0a4cabf>] mdt_reint_rec+0x3f/0x100 [mdt]
 [<ffffffffa069cd74>] ? lustre_msg_get_flags+0x34/0xa0 [ptlrpc]
 [<ffffffffa0a44f64>] mdt_reint_internal+0x6d4/0x9f0 [mdt]
 [<ffffffffa0a32cde>] ? mdt_intent_fixup_resent+0x4e/0x270 [mdt]
 [<ffffffffa0a455e5>] mdt_intent_reint+0x245/0x600 [mdt]
 [<ffffffffa04ef615>] ? cfs_hash_bd_lookup_intent+0xe5/0x130 [libcfs]
 [<ffffffffa069e170>] ? lustre_swab_ldlm_intent+0x0/0x20 [ptlrpc]
 [<ffffffffa0a3d630>] mdt_intent_policy+0x3c0/0x6b0 [mdt]
 [<ffffffff810f18c6>] ? __perf_event_task_sched_out+0x36/0x50
 [<ffffffffa0587441>] ? class_handle_hash+0xa1/0x280 [obdclass]
 [<ffffffffa0655afa>] ldlm_lock_enqueue+0x2da/0xa50 [ptlrpc]
 [<ffffffffa0674495>] ? ldlm_export_lock_get+0x15/0x20 [ptlrpc]
 [<ffffffffa04ee682>] ? cfs_hash_bd_add_locked+0x62/0x90 [libcfs]
 [<ffffffffa067c577>] ldlm_handle_enqueue0+0x447/0x1090 [ptlrpc]
 [<ffffffffa0a313a1>] ? mdt_unpack_req_pack_rep+0x51/0x5d0 [mdt]
 [<ffffffffa0a3d0ca>] mdt_enqueue+0x4a/0x110 [mdt]
 [<ffffffffa0a37865>] mdt_handle_common+0x8d5/0x1810 [mdt]
 [<ffffffffa069a4f4>] ? lustre_msg_get_opc+0x94/0x100 [ptlrpc]
 [<ffffffffa0a38875>] mdt_regular_handle+0x15/0x20 [mdt]
 [<ffffffffa06ab239>] ptlrpc_main+0xc79/0x19d0 [ptlrpc]
 [<ffffffff810017bc>] ? __switch_to+0x1ac/0x320
 [<ffffffffa06aa5c0>] ? ptlrpc_main+0x0/0x19d0 [ptlrpc]
 [<ffffffff810041aa>] child_rip+0xa/0x20
 [<ffffffffa06aa5c0>] ? ptlrpc_main+0x0/0x19d0 [ptlrpc]
 [<ffffffffa06aa5c0>] ? ptlrpc_main+0x0/0x19d0 [ptlrpc]
 [<ffffffff810041a0>] ? child_rip+0x0/0x20

After the MDS went back to production it did failed again with the same trace, same assertion (with same parameters), but req->rq_peer didn't reference the same client.

Any idea ?

Thanks

Issue Links

• Trackbacks: Changelog 2.1 Changes from version 2.1.2 to version 2.1.3 Server support for kernels: 2.6.18308.13.1.el5 (RHEL5) 2.6.32279.2.1.el6 (RHEL6) Client support for unpatched kernels: 2.6.18308.13.1.el5 (RHEL5) 2.6.32279.2.1....