[LU-17933] sanity test_63a: FAIL: failed grant check: client:700186624 server:368967680 - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Fixed
Priority: Minor
Fix Version/s: Lustre 2.17.0
Affects Version/s: Lustre 2.16.0
Labels:
- zfs
Environment:
RHEL 9.4 zfs server

Severity:
3
Rank (Obsolete):
9223372036854775807

Description

This issue was created by maloo for jianyu <yujian@whamcloud.com>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/faabd6b5-a956-42bd-a631-867ad6924185

test_63a failed with the following error:

failed grant check: client:700186624 server:368967680

Test session details:
clients: https://build.whamcloud.com/job/lustre-reviews/105370 - 5.14.0-362.24.1.el9_3.x86_64
servers: https://build.whamcloud.com/job/lustre-reviews/105370 - 5.14.0-427.20.1_lustre.el9.x86_64

<<Please provide additional information about the failure here>>

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity test_63a - failed grant check: client:700186624 server:368967680

Attachments

Issue Links

is related to

LU-14601 sanity test_64e: init grants: grants mismatch: 0, expected 2146304

Open

LU-17404 Support RHEL 9.4

Resolved

mentioned in: Page Loading...

Activity

[LU-17933] sanity test_63a: FAIL: failed grant check: client:700186624 server:368967680

Alex Zhuravlev added a comment - 14/May/25 11:06 AM

vsaveliev shouldn't tgt_grant_dealloc() reset oa->o_grant ?

Alex Zhuravlev added a comment - 14/May/25 11:06 AM vsaveliev shouldn't tgt_grant_dealloc() reset oa->o_grant ?

Peter Jones added a comment - 08/May/25 2:45 PM

green should we revert this patch?

Peter Jones added a comment - 08/May/25 2:45 PM green should we revert this patch?

Alex Zhuravlev added a comment - 04/May/25 4:27 AM

with this patch landed:

Subject: LU-17933 target: do not break grants on RPC failure

I hit this problem:

[ 4745.188760] Lustre: DEBUG MARKER: == sanity-quota test 58: project ID should be kept for new mirrors created by FID ========================================================== 23:17:31 (1746314251)
[ 4770.124961] LustreError: 208496:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9e1973355000: inode [0x200000402:0xac:0x0] mdc close failed: rc = -22
[ 4805.518750] LustreError: 207592:0:(tgt_grant.c:747:tgt_grant_check()) lustre-OST0001: cli 86e86cb6-1593-4c0f-9b46-63889d50caa5 claims 1703936 GRANT, real grant 0
[ 4805.571439] LustreError: 207592:0:(tgt_grant.c:560:tgt_grant_incoming()) lustre-OST0001: cli 86e86cb6-1593-4c0f-9b46-63889d50caa5/ffff9e1965267800 dirty 10223616 pend 0 grant -3407872
[ 4805.571632] LustreError: 207592:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG
[ 4805.571689] CPU: 1 PID: 207592 Comm: ll_ost_io00_006 Tainted: G           O     --------- -  - 4.18.0 #12
[ 4805.571746] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014
[ 4805.571805] Call Trace:
[ 4805.571851]  dump_stack+0x6e/0xa0
[ 4805.571888]  lbug_with_loc.cold.4+0x5/0x4e [libcfs]
[ 4805.571932]  tgt_grant_incoming.isra.1+0x575/0x580 [ptlrpc]
[ 4805.572065]  tgt_grant_prepare_write+0x397/0xf30 [ptlrpc]
[ 4805.572194]  ofd_preprw+0xa47/0x27f0 [ofd]
[ 4805.572278]  tgt_brw_write+0xda5/0x2720 [ptlrpc]
[ 4805.572400]  ? tgt_request_preprocess.isra.1+0xdd/0x1440 [ptlrpc]
[ 4805.572531]  tgt_request_handle+0x36c/0x1af0 [ptlrpc]
[ 4805.572648]  ptlrpc_main+0x109b/0x3570 [ptlrpc]
[ 4805.572765]  ? trace_hardirqs_on+0x1c/0xe0
[ 4805.572795]  ? _raw_spin_unlock_irqrestore+0x43/0x70
[ 4805.572829]  ? ptlrpc_wait_event+0x530/0x530 [ptlrpc]
[ 4805.572944]  kthread+0x16e/0x1a0
[ 4805.572977]  ? set_kthread_struct+0x40/0x40
[ 4805.573008]  ret_from_fork+0x24/0x30
[ 4805.573042] Kernel panic - not syncing: LBUG

bisection:

COMMIT          TESTED  PASSED  FAILED          COMMIT DESCRIPTION
8dbf049479      1       0       1       BAD     LU-17427 mdt: reduce hold time for BFL rename lock
9fe3951c44      7       3       4       BAD     LU-16350 ldiskfs: update for kernel 6.12
df2b5d99ad      6       2       4       BAD     LU-17933 target: do not break grants on RPC failure
a2e3a2f5a3      10      10      0       GOOD    LU-14590 utils: merge similar list_param params

Alex Zhuravlev added a comment - 04/May/25 4:27 AM with this patch landed: Subject: LU-17933 target: do not break grants on RPC failure I hit this problem: [ 4745.188760] Lustre: DEBUG MARKER: == sanity-quota test 58: project ID should be kept for new mirrors created by FID ========================================================== 23:17:31 (1746314251) [ 4770.124961] LustreError: 208496:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9e1973355000: inode [0x200000402:0xac:0x0] mdc close failed: rc = -22 [ 4805.518750] LustreError: 207592:0:(tgt_grant.c:747:tgt_grant_check()) lustre-OST0001: cli 86e86cb6-1593-4c0f-9b46-63889d50caa5 claims 1703936 GRANT, real grant 0 [ 4805.571439] LustreError: 207592:0:(tgt_grant.c:560:tgt_grant_incoming()) lustre-OST0001: cli 86e86cb6-1593-4c0f-9b46-63889d50caa5/ffff9e1965267800 dirty 10223616 pend 0 grant -3407872 [ 4805.571632] LustreError: 207592:0:(tgt_grant.c:562:tgt_grant_incoming()) LBUG [ 4805.571689] CPU: 1 PID: 207592 Comm: ll_ost_io00_006 Tainted: G O --------- - - 4.18.0 #12 [ 4805.571746] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014 [ 4805.571805] Call Trace: [ 4805.571851] dump_stack+0x6e/0xa0 [ 4805.571888] lbug_with_loc.cold.4+0x5/0x4e [libcfs] [ 4805.571932] tgt_grant_incoming.isra.1+0x575/0x580 [ptlrpc] [ 4805.572065] tgt_grant_prepare_write+0x397/0xf30 [ptlrpc] [ 4805.572194] ofd_preprw+0xa47/0x27f0 [ofd] [ 4805.572278] tgt_brw_write+0xda5/0x2720 [ptlrpc] [ 4805.572400] ? tgt_request_preprocess.isra.1+0xdd/0x1440 [ptlrpc] [ 4805.572531] tgt_request_handle+0x36c/0x1af0 [ptlrpc] [ 4805.572648] ptlrpc_main+0x109b/0x3570 [ptlrpc] [ 4805.572765] ? trace_hardirqs_on+0x1c/0xe0 [ 4805.572795] ? _raw_spin_unlock_irqrestore+0x43/0x70 [ 4805.572829] ? ptlrpc_wait_event+0x530/0x530 [ptlrpc] [ 4805.572944] kthread+0x16e/0x1a0 [ 4805.572977] ? set_kthread_struct+0x40/0x40 [ 4805.573008] ret_from_fork+0x24/0x30 [ 4805.573042] Kernel panic - not syncing: LBUG bisection: COMMIT TESTED PASSED FAILED COMMIT DESCRIPTION 8dbf049479 1 0 1 BAD LU-17427 mdt: reduce hold time for BFL rename lock 9fe3951c44 7 3 4 BAD LU-16350 ldiskfs: update for kernel 6.12 df2b5d99ad 6 2 4 BAD LU-17933 target: do not break grants on RPC failure a2e3a2f5a3 10 10 0 GOOD LU-14590 utils: merge similar list_param params

Peter Jones added a comment - 02/May/25 12:13 PM

Merged for 2.17

Peter Jones added a comment - 02/May/25 12:13 PM Merged for 2.17

Gerrit Updater added a comment - 02/May/25 2:17 AM

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56547/
Subject: ~~LU-17933~~ target: do not break grants on RPC failure
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: df2b5d99adfc6117a327f8e9102ba30076253213

Gerrit Updater added a comment - 02/May/25 2:17 AM "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56547/ Subject: LU-17933 target: do not break grants on RPC failure Project: fs/lustre-release Branch: master Current Patch Set: Commit: df2b5d99adfc6117a327f8e9102ba30076253213

Gerrit Updater added a comment - 01/Oct/24 2:55 PM

"Vladimir Saveliev <vladimir.saveliev@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56547
Subject: ~~LU-17933~~ target: do not break grants on RPC failure
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 166ad18f8168190882d96a45723b6f12ed92eebb

Gerrit Updater added a comment - 01/Oct/24 2:55 PM "Vladimir Saveliev <vladimir.saveliev@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56547 Subject: LU-17933 target: do not break grants on RPC failure Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 166ad18f8168190882d96a45723b6f12ed92eebb

People

Assignee:: Alex Zhuravlev

Reporter:: Maloo

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 11/Jun/24 5:15 PM

Updated:: 14/May/25 11:06 AM

Resolved:: 02/May/25 12:13 PM