[LU-11276] racer: mdc_dev.c:1346:mdc_req_attr_set()) uncovered page

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.14.0, Lustre 2.12.6
• Affects Version/s: Lustre 2.12.0
• Labels: DoM2 LTS12
• Severity: 3
• Assignee: Mikhail Pershin
• Reporter: Oleg Drokin

Description

Saw this in master-next on Ag 12th first once, and then in the past two day many more occurences all of a sudden.

LU-10910 related? Most likely DOM related.

[27046.472503] LustreError: 12291:0:(ldlm_resource.c:1725:ldlm_resource_dump()) --- Resource: [0x200000406:0xe97:0x0].0x0 (ffff88000cdd1c80) refcount = 4
[27046.474149] LustreError: 12291:0:(ldlm_resource.c:1728:ldlm_resource_dump()) Granted locks (in reverse order):
[27046.475547] LustreError: 12291:0:(ldlm_resource.c:1731:ldlm_resource_dump()) ### ### ns: lustre-MDT0000-mdc-ffff880050df3800 lock: ffff880008d22d80/0x85f55ba2c9d6a1c8 lrc: 3/1,0 mode: CR/CR res: [0x200000406:0xe97:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x0 nid: local remote: 0x85f55ba2c9d6a1cf expref: -99 pid: 12291 timeout: 0 lvb_type: 3
[27046.479190] LustreError: 12291:0:(ldlm_resource.c:1731:ldlm_resource_dump()) ### ### ns: lustre-MDT0000-mdc-ffff880050df3800 lock: ffff8800a46fad80/0x85f55ba2c9d6a040 lrc: 1/0,0 mode: PR/PR res: [0x200000406:0xe97:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x0 nid: local remote: 0x85f55ba2c9d6a04e expref: -99 pid: 12291 timeout: 0 lvb_type: 0
[27046.482737] Pid: 12291, comm: dd 3.10.0-7.5-debug #2 SMP Sat Jun 2 18:41:57 EDT 2018
[27046.484162] Call Trace:
[27046.484763]  [<ffffffffa01b77dc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
[27046.485651]  [<ffffffffa01b7836>] libcfs_debug_dumpstack+0x26/0x30 [libcfs]
[27046.486498]  [<ffffffffa088781b>] mdc_get_lock_handle+0xcb/0xe0 [mdc]
[27046.487299]  [<ffffffffa0887c50>] mdc_req_attr_set+0x90/0x170 [mdc]
[27046.487994]  [<ffffffffa0379170>] cl_req_attr_set+0x60/0x150 [obdclass]
[27046.488818]  [<ffffffffa07fb4e6>] osc_build_rpc+0x496/0xfe0 [osc]
[27046.489621]  [<ffffffffa0815597>] osc_io_unplug0+0xe17/0x1920 [osc]
[27046.490470]  [<ffffffffa081a050>] osc_cache_writeback_range+0xc20/0x1260 [osc]
[27046.491804]  [<ffffffffa0884c96>] mdc_io_fsync_start+0x56/0x170 [mdc]
[27046.493774]  [<ffffffffa03793e5>] cl_io_start+0x65/0x130 [obdclass]
[27046.494603]  [<ffffffffa08accd5>] lov_io_call.isra.5+0x85/0x140 [lov]
[27046.495669]  [<ffffffffa08ace96>] lov_io_start+0x56/0x150 [lov]
[27046.496550]  [<ffffffffa03793e5>] cl_io_start+0x65/0x130 [obdclass]
[27046.497361]  [<ffffffffa037b6a1>] cl_io_loop+0x131/0xb30 [obdclass]
[27046.498087]  [<ffffffffa144e9eb>] cl_sync_file_range+0x2db/0x380 [lustre]
[27046.500281]  [<ffffffffa1494fd4>] vvp_prune+0x64/0x2b0 [lustre]
[27046.501184]  [<ffffffffa03718e5>] cl_object_prune+0x55/0x110 [obdclass]
[27046.503100]  [<ffffffffa08c1e01>] lov_conf_set+0x531/0xa70 [lov]
[27046.504495]  [<ffffffffa0371a00>] cl_conf_set+0x60/0x120 [obdclass]
[27046.505423]  [<ffffffffa1450e81>] ll_layout_conf+0x81/0x410 [lustre]
[27046.507428]  [<ffffffffa14517c5>] ll_layout_refresh+0x5b5/0xa60 [lustre]
[27046.508276]  [<ffffffffa149468f>] vvp_io_init+0x35f/0x460 [lustre]
[27046.509231]  [<ffffffffa03797e8>] cl_io_init0.isra.17+0x88/0x160 [obdclass]
[27046.526478]  [<ffffffffa0379983>] cl_io_init+0x43/0x80 [obdclass]
[27046.527149]  [<ffffffffa037a383>] cl_io_rw_init+0x93/0x2a0 [obdclass]
[27046.527843]  [<ffffffffa1447d3d>] ll_file_io_generic+0x20d/0xd10 [lustre]
[27046.528490]  [<ffffffffa1448db1>] ll_file_aio_write+0x391/0x830 [lustre]
[27046.529163]  [<ffffffffa14492f4>] ll_file_write+0xa4/0x170 [lustre]
[27046.529567]  [<ffffffff8120daad>] vfs_write+0xbd/0x1e0
[27046.530010]  [<ffffffff8120e8cf>] SyS_write+0x7f/0xf0
[27046.530450]  [<ffffffff817838e9>] system_call_fastpath+0x16/0x1b
[27046.530895]  [<ffffffffffffffff>] 0xffffffffffffffff
[27046.531358] LustreError: 12291:0:(mdc_dev.c:1346:mdc_req_attr_set()) page@ffff88000dd4ae00[2 ffff880018299718 2 1           (null)]

[27046.532734] LustreError: 12291:0:(mdc_dev.c:1346:mdc_req_attr_set()) vvp-page@ffff88000dd4ae50(0:0) vm@ffffea00020884e0 1fffff0000282c 2:0 ffff88000dd4ae00 0 lru

[27046.534974] LustreError: 12291:0:(mdc_dev.c:1346:mdc_req_attr_set()) lov-page@ffff88000dd4ae90, comp index: 0, gen: 2

[27046.536852] LustreError: 12291:0:(mdc_dev.c:1346:mdc_req_attr_set()) osc-page@ffff88000dd4aec8 0: 1< 0x845fed 258 0 + + > 2< 0 0 4096 0x5 0x520 |           (null) ffff88000990ac48 ffff880008e60e60 > 3< 1 64 0 > 4< 0 0 8 11886592 - | - - + - > 5< - - + - | 0 - | 0 - ->

[27046.540751] LustreError: 12291:0:(mdc_dev.c:1346:mdc_req_attr_set()) end page@ffff88000dd4ae00

[27046.542709] LustreError: 12291:0:(mdc_dev.c:1346:mdc_req_attr_set()) uncovered page!
[27046.544045] LustreError: 12291:0:(mdc_dev.c:1347:mdc_req_attr_set()) LBUG
[27046.544824] Pid: 12291, comm: dd 3.10.0-7.5-debug #2 SMP Sat Jun 2 18:41:57 EDT 2018
[27046.546276] Call Trace:
[27046.546712]  [<ffffffffa01b77dc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
[27046.547165]  [<ffffffffa01b788c>] lbug_with_loc+0x4c/0xa0 [libcfs]
[27046.547704]  [<ffffffffa0887d24>] mdc_req_attr_set+0x164/0x170 [mdc]
[27046.548206]  [<ffffffffa0379170>] cl_req_attr_set+0x60/0x150 [obdclass]
[27046.548675]  [<ffffffffa07fb4e6>] osc_build_rpc+0x496/0xfe0 [osc]
[27046.549099]  [<ffffffffa0815597>] osc_io_unplug0+0xe17/0x1920 [osc]
[27046.549834]  [<ffffffffa081a050>] osc_cache_writeback_range+0xc20/0x1260 [osc]
[27046.550585]  [<ffffffffa0884c96>] mdc_io_fsync_start+0x56/0x170 [mdc]
[27046.551013]  [<ffffffffa03793e5>] cl_io_start+0x65/0x130 [obdclass]
[27046.551433]  [<ffffffffa08accd5>] lov_io_call.isra.5+0x85/0x140 [lov]
[27046.551839]  [<ffffffffa08ace96>] lov_io_start+0x56/0x150 [lov]
[27046.552439]  [<ffffffffa03793e5>] cl_io_start+0x65/0x130 [obdclass]
[27046.553102]  [<ffffffffa037b6a1>] cl_io_loop+0x131/0xb30 [obdclass]
[27046.553823]  [<ffffffffa144e9eb>] cl_sync_file_range+0x2db/0x380 [lustre]
[27046.554653]  [<ffffffffa1494fd4>] vvp_prune+0x64/0x2b0 [lustre]
[27046.555426]  [<ffffffffa03718e5>] cl_object_prune+0x55/0x110 [obdclass]
[27046.556088]  [<ffffffffa08c1e01>] lov_conf_set+0x531/0xa70 [lov]
[27046.556528]  [<ffffffffa0371a00>] cl_conf_set+0x60/0x120 [obdclass]
[27046.556978]  [<ffffffffa1450e81>] ll_layout_conf+0x81/0x410 [lustre]
[27046.557386]  [<ffffffffa14517c5>] ll_layout_refresh+0x5b5/0xa60 [lustre]
[27046.557836]  [<ffffffffa149468f>] vvp_io_init+0x35f/0x460 [lustre]
[27046.558475]  [<ffffffffa03797e8>] cl_io_init0.isra.17+0x88/0x160 [obdclass]
[27046.559128]  [<ffffffffa0379983>] cl_io_init+0x43/0x80 [obdclass]
[27046.559655]  [<ffffffffa037a383>] cl_io_rw_init+0x93/0x2a0 [obdclass]
[27046.560233]  [<ffffffffa1447d3d>] ll_file_io_generic+0x20d/0xd10 [lustre]
[27046.560650]  [<ffffffffa1448db1>] ll_file_aio_write+0x391/0x830 [lustre]
[27046.561267]  [<ffffffffa14492f4>] ll_file_write+0xa4/0x170 [lustre]
[27046.561948]  [<ffffffff8120daad>] vfs_write+0xbd/0x1e0
[27046.562603]  [<ffffffff8120e8cf>] SyS_write+0x7f/0xf0
[27046.563264]  [<ffffffff817838e9>] system_call_fastpath+0x16/0x1b
[27046.563704]  [<ffffffffffffffff>] 0xffffffffffffffff
[27046.564385] Kernel panic - not syncing: LBUG

Issue Links

• is related to: LU-15893 replay-dual test_30: ldlm_cli_cancel on converting lock
• is related to: LU-13927 MDS crash when increasing max_rpcs_in_flight to 256
• is related to: LU-12124 slower unlinks with DOM files due to disabled ELC
• is related to: LU-12321 Unlink speed needs to be improved in DoM configuration