[LU-17356] qmt_sarr_get_idx()) ASSERTION( arr_idx < qpi->qpi_sarr.osts.op_count && arr_idx >= 0 ) failed: idx invalid 0 op_count 0

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.16.0
• Affects Version/s: Upstream
• Severity: 3
• Assignee: Sergey Cheremencev
• Reporter: Alexander Boyko

Description

Something like next commands lead to crash

$lctl set_param osd-ldiskfs.*.quota_slave.enabled=p
$ lfs project /mnt/lustre
1 - /mnt/lustre/d144c.recovery-small
0 - /mnt/lustre/test1

$lfs project -p 1 /mnt/lustre

[282708.586862] LustreError: 941728:0:(qsd_reint.c:635:qqi_reint_delayed()) lustre-MDT0001: Delaying reintegration for qtype:2 until pending updates are flushed.
[282826.999640] LustreError: 906153:0:(qmt_pool.c:1672:qmt_sarr_get_idx()) ASSERTION( arr_idx < qpi->qpi_sarr.osts.op_count && arr_idx >= 0 ) failed: idx invalid 0 op_count 0
[282827.000369] LustreError: 906153:0:(qmt_pool.c:1672:qmt_sarr_get_idx()) LBUG  
[282827.000372] Pid: 906153, comm: mdt_rdpg01_000 4.18.0-348.12.2.x6.1r.27.x86_64 #1 SMP Wed Jan 25 18:39:55 MST 2023
[282827.002089] Call Trace TBD:                                                  
[282827.008953] [<0>] libcfs_call_trace+0x63/0x80 [libcfs]                       
[282827.011406] [<0>] lbug_with_loc+0x43/0x80 [libcfs]                           
[282827.011993] [<0>] qmt_sarr_get_idx+0x4f/0x50 [lquota]                        
[282827.013089] [<0>] qmt_setup_lqe_gd+0xa9/0x1a0 [lquota]                       
[282827.013456] [<0>] qmt_lvbo_init+0x335/0x840 [lquota]                         
[282827.013707] [<0>] ldlm_lvbo_init+0x5e/0x1c0 [ptlrpc]                         
[282827.033033] [<0>] ldlm_handle_enqueue+0x1c5/0x1410 [ptlrpc]                  
[282827.033388] [<0>] tgt_enqueue+0xa8/0x220 [ptlrpc]                            
[282827.033517] [<0>] tgt_request_handle+0x378/0x18f0 [ptlrpc]                   
[282827.033517] [<0>] ptlrpc_server_handle_request+0x3d1/0xc20 [ptlrpc]          
[282827.035233] [<0>] ptlrpc_main+0xc5d/0x15d0 [ptlrpc]                          
[282827.036144] [<0>] kthread+0x116/0x130                                        
[282827.037214] [<0>] ret_from_fork+0x1f/0x40   

[166802.199879] Lustre: Lustre: Build Version: 2.15.59_53_gc06b54e_dirty 
$ git log --oneline -n10 c06b54e
c06b54efba LU-17306 ofd: return error for reconnection
de352465eb (origin/master, origin/HEAD) LU-17046 tests: fix write success in 1g
0ef4e5b0c1 LU-16518 lnet: fix uninitialized variable in api-ni.c
350dfbcfa8 LU-17293 kernel: update SLES15 SP5 [5.14.21-150500.55.36.1]
57217b7e4e LU-17280 scrub: skip dir stripes with OI
698498b563 LU-17275 kernel: RHEL 8.9 client and server support
9eb87e7ef3 LU-17274 kernel: new kernel [RHEL 9.3 5.14.0-362.8.1.el9_3]
f3b45a0547 LU-17278 ldlm: don't grant failed lock
c5aa16db17 LU-17265 tests: allow margin for sanity/39r
6897dbe67c LU-17230 socklnd: treat UNKNOWN netif operstate as UP

Issue Links

• is related to: LU-19005 qmt_uuid2idx can never be less than 0 now

Comments

[Peter Jones]

Merged for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55019/
Subject: LU-17356 quota: fix qmt_pool_new_conn
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b64bc2c6f35f9fee35da6eec1e476eab76ad7646

[Gerrit Updater]

"Sergey Cheremencev <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55019
Subject: LU-17356 quota: fix qmt_pool_new_conn
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b7bd78526b921b87b998f50d7cb46730fc26fdba

[Gerrit Updater]

"Sergey Cheremencev <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55004
Subject: LU-17356 tests: enable project quota for files
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 05dadf09750ebbf71745d1c73830a619a31bdac8

[Sergey Cheremencev]

reproducer:

MDSCOUNT=2 bash ./llmount.sh

lctl set_param debug=+quota
lctl set_param debug=+trace
lctl set_param debug_mb=200

for i in `seq 1 5`;
do
	dd if=/dev/urandom of=/mnt/lustre/f$i bs=1M count=10
done

lfs project -p 0 /mnt/lustre
lfs setquota -p 0 -I1K /mnt/lustre
lfs setquota -p 1 -I1K /mnt/lustre
lfs project /mnt/lustre
lfs quota -p 1 /mnt/lustre

echo "set quota project"
lctl set_param osd*.*.quota_slave.enabled=p