[LU-16514] sanity test_398b: DIO and buffer IO race - uncovered page!

Mon, 30 Jan 2023 02:26:28 +0000

This issue was created by maloo for Qian Yingjin <qian@ddn.com>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/82edbdf8-d9d8-4a4e-b6a9-6839d24a8266

test_398b failed with the following error:

onyx-73vm1 crashed during sanity test_398b

Test session details:
clients: https://build.whamcloud.com/job/lustre-reviews/91983 - 4.18.0-348.7.1.el8_5.x86_64
servers: https://build.whamcloud.com/job/lustre-reviews/91983 - 4.18.0-348.23.1.el8_lustre.x86_64

<<[ 8921.171353] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398b: DIO and buffer IO race ============== 05:09:48 (1674968988)
[ 8921.513054] Lustre: DEBUG MARKER: == sanity test 398b: DIO and buffer IO race ============== 05:09:48 (1674968988)
[ 9014.390751] LustreError: 571887:0:(osc_object.c:397:osc_req_attr_set()) page@00000000b4319127[2 0000000068fd64ec 4 1 00000000dac5039d]

[ 9014.399227] LustreError: 571887:0:(osc_object.c:397:osc_req_attr_set()) vmpage @0000000096debca8 fffffc0001001 3:0 ffff9dbb25f218c0 1716 lru

[ 9014.401491] LustreError: 571887:0:(osc_object.c:397:osc_req_attr_set()) osc-page@0000000036495006 180: 1< 1 + + > 2< 737280 0 4096 0x7 0x9 | 00000000dac5039d 00000000aa4682af 00000000213d223c > 3< 1 0 > 4< 0 0 8 349954048 - | - - + + > 5< - - + + | 0 - | 48 - ->

[ 9014.405415] LustreError: 571887:0:(osc_object.c:397:osc_req_attr_set()) end page@00000000b4319127

[ 9014.407145] LustreError: 571887:0:(osc_object.c:397:osc_req_attr_set()) uncovered page!
[ 9014.408529] LustreError: 571887:0:(ldlm_resource.c:1796:ldlm_resource_dump()) — Resource: [0x10c43:0x0:0x0].0x0 (000000002369d41d) refcount = 11
[ 9014.410675] LustreError: 571887:0:(ldlm_resource.c:1799:ldlm_resource_dump()) Granted locks (in reverse order):
[ 9014.412397] LustreError: 571887:0:(ldlm_resource.c:1802:ldlm_resource_dump()) ### ### ns: lustre-OST0005-osc-ffff9dbb05488800 lock: 00000000b5744ed4/0x7b75909dd9641159 lrc: 1/0,0 mode: PW/PW res: [0x10c43:0x0:0x0].0x0 rrc: 12 type: EXT [557056->573439] (req 557056->573439) gid 0 flags: 0x800020000000000 nid: local remote: 0x826a7f1c1793c6e0 expref: -99 pid: 898097 timeout: 0 lvb_type: 1
[ 9014.417765] Pid: 571887, comm: ptlrpcd_01_01 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021
[ 9014.419437] Call Trace TBD:
[ 9014.420243] [<0>] libcfs_call_trace+0x6f/0x90 [libcfs]
[ 9014.421203] [<0>] osc_req_attr_set+0x49a/0x550 [osc]
[ 9014.422259] [<0>] cl_req_attr_set+0x5e/0x150 [obdclass]
[ 9014.423148] [<0>] osc_build_rpc+0x518/0x1310 [osc]
[ 9014.423974] [<0>] osc_check_rpcs+0x12b2/0x18f0 [osc]
[ 9014.424824] [<0>] osc_io_unplug0+0xc0/0x110 [osc]
[ 9014.425637] [<0>] brw_queue_work+0x2e/0xc0 [osc]
[ 9014.426688] [<0>] work_interpreter+0x32/0x110 [ptlrpc]
[ 9014.427595] [<0>] ptlrpc_check_set+0x5b8/0x1f60 [ptlrpc]
[ 9014.428518] [<0>] ptlrpcd+0x6c6/0xa50 [ptlrpc]
[ 9014.429322] [<0>] kthread+0x116/0x130
[ 9014.429986] [<0>] ret_from_fork+0x35/0x40
[ 9014.430689] LustreError: 571887:0:(osc_object.c:410:osc_req_attr_set()) LBUG
[ 9014.431846] Pid: 571887, comm: ptlrpcd_01_01 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021
[ 9014.433437] Call Trace TBD:
[ 9014.433954] [<0>] libcfs_call_trace+0x6f/0x90 [libcfs]
[ 9014.434833] [<0>] lbug_with_loc+0x43/0x80 [libcfs]
[ 9014.435655] [<0>] osc_req_attr_set+0x4b0/0x550 [osc]
[ 9014.436508] [<0>] cl_req_attr_set+0x5e/0x150 [obdclass]
[ 9014.437395] [<0>] osc_build_rpc+0x518/0x1310 [osc]
[ 9014.438222] [<0>] osc_check_rpcs+0x12b2/0x18f0 [osc]
[ 9014.439068] [<0>] osc_io_unplug0+0xc0/0x110 [osc]
[ 9014.439881] [<0>] brw_queue_work+0x2e/0xc0 [osc]
[ 9014.440691] [<0>] work_interpreter+0x32/0x110 [ptlrpc]
[ 9014.441584] [<0>] ptlrpc_check_set+0x5b8/0x1f60 [ptlrpc]
[ 9014.442501] [<0>] ptlrpcd+0x6c6/0xa50 [ptlrpc]
[ 9014.443264] [<0>] kthread+0x116/0x130
[ 9014.443922] [<0>] ret_from_fork+0x35/0x40>>

This panic occurs on rhel8.5 for direct I/O and buffered I/O race, it seems not same as ~~LU-16412~~...

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity test_398b - onyx-73vm1 crashed during sanity test_398b

Whamcloud Community JIRA

[LU-16514] sanity test_398b: DIO and buffer IO race - uncovered page!