https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3729 Lustre <p>This is potentially another hit of <a href="https://jira.whamcloud.com/browse/LU-1650" title="crash of lustre clients in osc_req_attr_set() routine" class="issue-link" data-issue-key="LU-1650"><del>LU-1650</del></a>?</p> <p>Hit this during racer on tip of master:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>&lt;4&gt;[597951.119031] Lustre: 19544:0:(vvp_io.c:866:vvp_io_read_page()) page@ffff880098850df0[2 ffff880041514f90:0 ^(null)_ffff8800692f7df0 1 0 1 ffff88005f85ceb8 (null) 0x0] &lt;4&gt;[597951.119877] Lustre: 19544:0:(vvp_io.c:866:vvp_io_read_page()) without a lock: -61 &lt;3&gt;[597951.120398] LustreError: 19544:0:(osc_io.c:795:osc_req_attr_set()) lock@ffff880070e24ee0[1 2 0 1 1 00000000] R(1):[0, 18446744073709551615]@[0x100000000:0x2bbb:0x0] { &lt;3&gt;[597951.121234] LustreError: 19544:0:(osc_io.c:795:osc_req_attr_set()) lovsub@ffff8800378214b0: [0 ffff880060813230 P(0):[0, 18446744073709551615]@[0x200000403:0xb287:0x0]] &lt;3&gt;[597951.122095] LustreError: 19544:0:(osc_io.c:795:osc_req_attr_set()) osc@ffff880027c4ff38: ffff8800546fcdb8 0x20000001001 0x59b8e65062a2d012 3 (null) size: 0 mtime: 0 atime: 0 ctime: 0 blocks: 0 &lt;3&gt;[597951.122945] LustreError: 19544:0:(osc_io.c:795:osc_req_attr_set()) } lock@ffff880070e24ee0 &lt;3&gt;[597951.123453] LustreError: 19544:0:(osc_io.c:795:osc_req_attr_set()) no cover page! &lt;3&gt;[597951.123949] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) page@ffff880098850df0[4 ffff880041514f90:0 ^(null)_ffff8800692f7df0 3 0 1 (null) ffff8800383022a0 0x0] &lt;3&gt;[597951.124752] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) page@ffff8800692f7df0[2 ffff880054617fa8:0 ^ffff880098850df0_(null) 3 0 1 (null) (null) 0x0] &lt;3&gt;[597951.125578] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) vvp-page@ffff880098850ec8(0:0:0) vm@ffffea0001de5878 20000000000801 4:0 ffff880098850df0 0 lru &lt;3&gt;[597951.126487] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) lov-page@ffff880098850f18 &lt;3&gt;[597951.127006] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) osc-page@ffff8800692f7ef0: 1&lt; 0x845fed 257 0 + + &gt; 2&lt; 0 0 4096 0x7 0x108 | (null) ffff8800b75cab18 ffff880047ff4ec0 &gt; 3&lt; + ffff880028c963c0 1 1 0 &gt; 4&lt; 0 0 8 0 - | - - - + &gt; 5&lt; - - - + | 0 - | 0 - -&gt; &lt;3&gt;[597951.128148] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) end page@ffff8800692f7df0 &lt;3&gt;[597951.128663] LustreError: 19544:0:(osc_io.c:797:osc_req_attr_set()) dump uncover page! &lt;4&gt;[597951.129258] Pid: 19544, comm: cat &lt;4&gt;[597951.130485] &lt;4&gt;[597951.130486] Call Trace: &lt;4&gt;[597951.130950] [&lt;ffffffffa0d108a5&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] &lt;4&gt;[597951.131267] [&lt;ffffffffa04be51f&gt;] osc_req_attr_set+0x55f/0x5b0 [osc] &lt;4&gt;[597951.131599] [&lt;ffffffffa0fbdd79&gt;] cl_req_attr_set+0xd9/0x230 [obdclass] &lt;4&gt;[597951.131904] [&lt;ffffffffa04afc14&gt;] osc_build_rpc+0x484/0x17f0 [osc] &lt;4&gt;[597951.132204] [&lt;ffffffffa0d21041&gt;] ? libcfs_debug_msg+0x41/0x50 [libcfs] &lt;4&gt;[597951.132510] [&lt;ffffffffa04c9e2c&gt;] osc_io_unplug0+0x12ac/0x1f30 [osc] &lt;4&gt;[597951.132819] [&lt;ffffffffa0d20a63&gt;] ? libcfs_debug_vmsg2+0x513/0xab0 [libcfs] &lt;4&gt;[597951.133149] [&lt;ffffffffa04cc641&gt;] osc_io_unplug+0x11/0x20 [osc] &lt;4&gt;[597951.133442] [&lt;ffffffffa04cc822&gt;] osc_queue_sync_pages+0x1d2/0x360 [osc] &lt;4&gt;[597951.133754] [&lt;ffffffffa04bea8b&gt;] osc_io_submit+0x1cb/0x490 [osc] &lt;4&gt;[597951.134068] [&lt;ffffffffa0fbd6dc&gt;] cl_io_submit_rw+0x6c/0x160 [obdclass] &lt;4&gt;[597951.134387] [&lt;ffffffffa053a889&gt;] lov_io_submit+0x209/0xb70 [lov] &lt;4&gt;[597951.134710] [&lt;ffffffffa0fbd6dc&gt;] cl_io_submit_rw+0x6c/0x160 [obdclass] &lt;4&gt;[597951.135063] [&lt;ffffffffa0fbfbfe&gt;] cl_io_read_page+0xae/0x170 [obdclass] &lt;4&gt;[597951.135410] [&lt;ffffffffa0fb3a77&gt;] ? cl_page_assume+0xf7/0x220 [obdclass] &lt;4&gt;[597951.135749] [&lt;ffffffffa0a501c6&gt;] ll_readpage+0x96/0x1a0 [lustre] &lt;4&gt;[597951.136043] [&lt;ffffffff81119955&gt;] generic_file_aio_read+0x215/0x740 &lt;4&gt;[597951.136349] [&lt;ffffffffa0a763d4&gt;] ? cl_glimpse_lock+0x2a4/0x490 [lustre] &lt;4&gt;[597951.136693] [&lt;ffffffffa0a7fef9&gt;] vvp_io_read_start+0x259/0x470 [lustre] &lt;4&gt;[597951.137130] [&lt;ffffffffa0fbd83a&gt;] cl_io_start+0x6a/0x140 [obdclass] &lt;4&gt;[597951.137479] [&lt;ffffffffa0fc1e34&gt;] cl_io_loop+0xb4/0x1b0 [obdclass] &lt;4&gt;[597951.137795] [&lt;ffffffffa0a20f5f&gt;] ll_file_io_generic+0x33f/0x610 [lustre] &lt;4&gt;[597951.138109] [&lt;ffffffffa0a2136f&gt;] ll_file_aio_read+0x13f/0x2c0 [lustre] &lt;4&gt;[597951.138434] [&lt;ffffffffa0a21c0c&gt;] ll_file_read+0x16c/0x2a0 [lustre] &lt;4&gt;[597951.138759] [&lt;ffffffff81181bd5&gt;] vfs_read+0xb5/0x1a0 &lt;4&gt;[597951.139028] [&lt;ffffffff81181d11&gt;] sys_read+0x51/0x90 &lt;4&gt;[597951.139294] [&lt;ffffffff8100b0b2&gt;] system_call_fastpath+0x16/0x1b &lt;4&gt;[597951.139669] &lt;0&gt;[597951.139904] LustreError: 19544:0:(osc_io.c:799:osc_req_attr_set()) LBUG &lt;4&gt;[597951.140366] Pid: 19544, comm: cat &lt;4&gt;[597951.140657] &lt;4&gt;[597951.140657] Call Trace: &lt;4&gt;[597951.141149] [&lt;ffffffffa0d108a5&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] &lt;4&gt;[597951.141466] [&lt;ffffffffa0d10ea7&gt;] lbug_with_loc+0x47/0xb0 [libcfs] &lt;4&gt;[597951.141773] [&lt;ffffffffa04be535&gt;] osc_req_attr_set+0x575/0x5b0 [osc] &lt;4&gt;[597951.142101] [&lt;ffffffffa0fbdd79&gt;] cl_req_attr_set+0xd9/0x230 [obdclass] &lt;4&gt;[597951.142457] [&lt;ffffffffa04afc14&gt;] osc_build_rpc+0x484/0x17f0 [osc] &lt;4&gt;[597951.142847] [&lt;ffffffffa0d21041&gt;] ? libcfs_debug_msg+0x41/0x50 [libcfs] &lt;4&gt;[597951.143164] [&lt;ffffffffa04c9e2c&gt;] osc_io_unplug0+0x12ac/0x1f30 [osc] &lt;4&gt;[597951.143468] [&lt;ffffffffa0d20a63&gt;] ? libcfs_debug_vmsg2+0x513/0xab0 [libcfs] &lt;4&gt;[597951.143786] [&lt;ffffffffa04cc641&gt;] osc_io_unplug+0x11/0x20 [osc] &lt;4&gt;[597951.144079] [&lt;ffffffffa04cc822&gt;] osc_queue_sync_pages+0x1d2/0x360 [osc] &lt;4&gt;[597951.144385] [&lt;ffffffffa04bea8b&gt;] osc_io_submit+0x1cb/0x490 [osc] &lt;4&gt;[597951.144710] [&lt;ffffffffa0fbd6dc&gt;] cl_io_submit_rw+0x6c/0x160 [obdclass] &lt;4&gt;[597951.145040] [&lt;ffffffffa053a889&gt;] lov_io_submit+0x209/0xb70 [lov] &lt;4&gt;[597951.145356] [&lt;ffffffffa0fbd6dc&gt;] cl_io_submit_rw+0x6c/0x160 [obdclass] &lt;4&gt;[597951.145680] [&lt;ffffffffa0fbfbfe&gt;] cl_io_read_page+0xae/0x170 [obdclass] &lt;4&gt;[597951.146001] [&lt;ffffffffa0fb3a77&gt;] ? cl_page_assume+0xf7/0x220 [obdclass] &lt;4&gt;[597951.146324] [&lt;ffffffffa0a501c6&gt;] ll_readpage+0x96/0x1a0 [lustre] &lt;4&gt;[597951.146624] [&lt;ffffffff81119955&gt;] generic_file_aio_read+0x215/0x740 &lt;4&gt;[597951.146928] [&lt;ffffffffa0a763d4&gt;] ? cl_glimpse_lock+0x2a4/0x490 [lustre] &lt;4&gt;[597951.147241] [&lt;ffffffffa0a7fef9&gt;] vvp_io_read_start+0x259/0x470 [lustre] &lt;4&gt;[597951.147566] [&lt;ffffffffa0fbd83a&gt;] cl_io_start+0x6a/0x140 [obdclass] &lt;4&gt;[597951.147885] [&lt;ffffffffa0fc1e34&gt;] cl_io_loop+0xb4/0x1b0 [obdclass] &lt;4&gt;[597951.148188] [&lt;ffffffffa0a20f5f&gt;] ll_file_io_generic+0x33f/0x610 [lustre] &lt;4&gt;[597951.148497] [&lt;ffffffffa0a2136f&gt;] ll_file_aio_read+0x13f/0x2c0 [lustre] &lt;4&gt;[597951.148809] [&lt;ffffffffa0a21c0c&gt;] ll_file_read+0x16c/0x2a0 [lustre] &lt;4&gt;[597951.149112] [&lt;ffffffff81181bd5&gt;] vfs_read+0xb5/0x1a0 &lt;4&gt;[597951.149386] [&lt;ffffffff81181d11&gt;] sys_read+0x51/0x90 &lt;4&gt;[597951.149681] [&lt;ffffffff8100b0b2&gt;] system_call_fastpath+0x16/0x1b &lt;4&gt;[597951.149973] &lt;0&gt;[597951.217252] Kernel panic - not syncing: LBUG </pre> </div></div> LU-3729

uncovered page in osc_req_attr_set

Bug Major Resolved Cannot Reproduce WC Triage Oleg Drokin Fri, 9 Aug 2013 07:44:26 +0000 Sun, 10 Oct 2021 22:35:22 +0000 Sun, 10 Oct 2021 22:35:22 +0000 Lustre 2.5.0 0 5 <p>I just hit this again on master while running master.</p> <p>Crashdump in /exports/crashdumps/192.168.10.210-2015-08-17-01\:29\:24/<br/> tag in my tree master-20150816</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>&lt;3&gt;[52056.764891] LustreError: 7366:0:(osc_io.c:976:osc_req_attr_set()) page@ffff88006d92cdf0[2 ffff8800b4adcfa0 3 0 1 (null) ffff880067f70508] &lt;3&gt;[52056.764892] &lt;3&gt;[52056.765524] LustreError: 7366:0:(osc_io.c:976:osc_req_attr_set()) vvp-page@ffff88006d92ce58(1:0) vm@ffffea000135f6e8 20000000000821 2:0 ffff88006d92cdf0 256 lru &lt;3&gt;[52056.765525] &lt;3&gt;[52056.766311] LustreError: 7366:0:(osc_io.c:976:osc_req_attr_set()) lov-page@ffff88006d92ce98, raid0 &lt;3&gt;[52056.766311] &lt;3&gt;[52056.766905] LustreError: 7366:0:(osc_io.c:976:osc_req_attr_set()) osc-page@ffff88006d92cf00 0: 1&lt; 0x845fed 257 0 + + &gt; 2&lt; 0 0 4096 0x7 0x108 | (null) ffff88002d9f2ac8 ffff880095990ea0 &gt; 3&lt; 1 9 0 &gt; 4&lt; 0 0 8 36995072 - | - - + + &gt; 5&lt; - - - + | 256 + | 0 - -&gt; &lt;3&gt;[52056.766906] &lt;3&gt;[52056.768140] LustreError: 7366:0:(osc_io.c:976:osc_req_attr_set()) end page@ffff88006d92cdf0 &lt;3&gt;[52056.768141] &lt;3&gt;[52056.768833] LustreError: 7366:0:(osc_io.c:976:osc_req_attr_set()) uncovered page! &lt;3&gt;[52056.769315] LustreError: 7366:0:(ldlm_resource.c:1448:ldlm_resource_dump()) --- Resource: [0xe05:0x0:0x0].0 (ffff88006d854ef8) refcount = 2 &lt;4&gt;[52056.770013] Pid: 7366, comm: ptlrpcd_01_00 &lt;4&gt;[52056.770260] &lt;4&gt;[52056.770260] Call Trace: &lt;4&gt;[52056.770679] [&lt;ffffffffa0896885&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] &lt;4&gt;[52056.771025] [&lt;ffffffffa04d5e6a&gt;] osc_req_attr_set+0x55a/0x720 [osc] &lt;4&gt;[52056.771339] [&lt;ffffffffa0da0ab9&gt;] cl_req_attr_set+0xc9/0x220 [obdclass] &lt;4&gt;[52056.771634] [&lt;ffffffffa04c6ecf&gt;] osc_build_rpc+0x84f/0x1280 [osc] &lt;4&gt;[52056.772018] [&lt;ffffffffa04e1d1e&gt;] osc_io_unplug0+0xe3e/0x1ae0 [osc] &lt;4&gt;[52056.772314] [&lt;ffffffff8151e3ee&gt;] ? thread_return+0x4e/0x7e0 &lt;4&gt;[52056.772584] [&lt;ffffffffa08a2c31&gt;] ? libcfs_debug_msg+0x41/0x50 [libcfs] &lt;4&gt;[52056.772926] [&lt;ffffffffa04e4f40&gt;] osc_io_unplug+0x10/0x20 [osc] &lt;4&gt;[52056.773363] [&lt;ffffffffa04bf65b&gt;] brw_queue_work+0x3b/0xf0 [osc] &lt;4&gt;[52056.773752] [&lt;ffffffffa1ad03f0&gt;] work_interpreter+0x30/0x100 [ptlrpc] &lt;4&gt;[52056.774203] [&lt;ffffffffa1ad9693&gt;] ptlrpc_check_set+0x613/0x1bf0 [ptlrpc] &lt;4&gt;[52056.774527] [&lt;ffffffff81522574&gt;] ? _spin_lock_irqsave+0x24/0x30 &lt;4&gt;[52056.774866] [&lt;ffffffffa1b076e3&gt;] ptlrpcd_check+0x3e3/0x630 [ptlrpc] &lt;4&gt;[52056.775281] [&lt;ffffffffa1b07c4b&gt;] ptlrpcd+0x31b/0x500 [ptlrpc] &lt;4&gt;[52056.775563] [&lt;ffffffff81061630&gt;] ? default_wake_function+0x0/0x20 &lt;4&gt;[52056.775875] [&lt;ffffffffa1b07930&gt;] ? ptlrpcd+0x0/0x500 [ptlrpc] &lt;4&gt;[52056.776255] [&lt;ffffffff8109ce4e&gt;] kthread+0x9e/0xc0 &lt;4&gt;[52056.776510] [&lt;ffffffff8100c24a&gt;] child_rip+0xa/0x20 &lt;4&gt;[52056.776772] [&lt;ffffffff8109cdb0&gt;] ? kthread+0x0/0xc0 &lt;4&gt;[52056.777040] [&lt;ffffffff8100c240&gt;] ? child_rip+0x0/0x20 &lt;4&gt;[52056.777372] &lt;0&gt;[52056.777565] LustreError: 7366:0:(osc_io.c:986:osc_req_attr_set()) LBUG &lt;4&gt;[52056.777875] Pid: 7366, comm: ptlrpcd_01_00 &lt;4&gt;[52056.778198] &lt;4&gt;[52056.778199] Call Trace: &lt;4&gt;[52056.778601] [&lt;ffffffffa0896885&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] &lt;4&gt;[52056.778917] [&lt;ffffffffa0896e87&gt;] lbug_with_loc+0x47/0xb0 [libcfs] &lt;4&gt;[52056.779281] [&lt;ffffffffa04d5e80&gt;] osc_req_attr_set+0x570/0x720 [osc] &lt;4&gt;[52056.779583] [&lt;ffffffffa0da0ab9&gt;] cl_req_attr_set+0xc9/0x220 [obdclass] &lt;4&gt;[52056.779888] [&lt;ffffffffa04c6ecf&gt;] osc_build_rpc+0x84f/0x1280 [osc] &lt;4&gt;[52056.780258] [&lt;ffffffffa04e1d1e&gt;] osc_io_unplug0+0xe3e/0x1ae0 [osc] &lt;4&gt;[52056.781567] [&lt;ffffffff8151e3ee&gt;] ? thread_return+0x4e/0x7e0 &lt;4&gt;[52056.781857] [&lt;ffffffffa08a2c31&gt;] ? libcfs_debug_msg+0x41/0x50 [libcfs] &lt;4&gt;[52056.782246] [&lt;ffffffffa04e4f40&gt;] osc_io_unplug+0x10/0x20 [osc] &lt;4&gt;[52056.782521] [&lt;ffffffffa04bf65b&gt;] brw_queue_work+0x3b/0xf0 [osc] &lt;4&gt;[52056.782817] [&lt;ffffffffa1ad03f0&gt;] work_interpreter+0x30/0x100 [ptlrpc] &lt;4&gt;[52056.783231] [&lt;ffffffffa1ad9693&gt;] ptlrpc_check_set+0x613/0x1bf0 [ptlrpc] &lt;4&gt;[52056.783520] [&lt;ffffffff81522574&gt;] ? _spin_lock_irqsave+0x24/0x30 &lt;4&gt;[52056.783816] [&lt;ffffffffa1b076e3&gt;] ptlrpcd_check+0x3e3/0x630 [ptlrpc] &lt;4&gt;[52056.784154] [&lt;ffffffffa1b07c4b&gt;] ptlrpcd+0x31b/0x500 [ptlrpc] &lt;4&gt;[52056.784498] [&lt;ffffffff81061630&gt;] ? default_wake_function+0x0/0x20 &lt;4&gt;[52056.784807] [&lt;ffffffffa1b07930&gt;] ? ptlrpcd+0x0/0x500 [ptlrpc] &lt;4&gt;[52056.785108] [&lt;ffffffff8109ce4e&gt;] kthread+0x9e/0xc0 &lt;4&gt;[52056.785457] [&lt;ffffffff8100c24a&gt;] child_rip+0xa/0x20 &lt;4&gt;[52056.785710] [&lt;ffffffff8109cdb0&gt;] ? kthread+0x0/0xc0 &lt;4&gt;[52056.785992] [&lt;ffffffff8100c240&gt;] ? child_rip+0x0/0x20 &lt;4&gt;[52056.786301] &lt;0&gt;[52057.013098] Kernel panic - not syncing: LBUG </pre> </div></div> <p>Just hit once again.<br/> dump is in /exports/crashdumps/192.168.10.223-2015-09-17-10\:31\:51/</p> <p>On master or 2.5?</p> Development Rank 1|hzvxlr: Rank (Obsolete) 9623 Severity