https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16362 Lustre <p>Async readhead have skip a "locking" phase</p> <p>c2791674260 (Wang Shilong 2019-01-21 20:23:47 +0800 658) io-&gt;ci_state = CIS_LOCKED;</p> <p>But separate cl io created, this caused pages sends outside of original page lock and parallel blocking AST caused an "uncovered page" panic hit.</p> LU-16362

landing async readhead caused a "uncovered page" panic during sanityN runs

Bug Critical Open Unresolved Alexey Lyashkov Alexey Lyashkov Fri, 2 Dec 2022 09:35:57 +0000 Wed, 10 Jan 2024 13:56:33 +0000 0 8 <p><a href="https://jira.whamcloud.com/browse/LU-16332" title="LBUG with osc_req_attr_set() uncovered page!" class="issue-link" data-issue-key="LU-16332">LU-16332</a> is a similar issue.</p> <p>"Alexey Lyashkov &lt;alexey.lyashkov@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51677" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51677</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16362" title="landing async readhead caused a &quot;uncovered page&quot; panic during sanityN runs" class="issue-link" data-issue-key="LU-16362">LU-16362</a> lov: dont't allow RA outside of active stripe<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 01966622f7b4f1c7626bc5980f9adc0d01338e1c</p> <p>once first patch version found an issues, I spent more time to investing it.<br/> Current picture is:</p> <p>application started to read a data from a file. It's multi stripe file, but read want to read just first stripe only, and cl io had created to offset range assigned to first stripe.<br/> once stripe size is 4M now - RA started to full rpc with this size and finished at end of stripe offset.<br/> On next read, RA window moved outside of IO region requested by cl_io and outside of active locks pined by this IO.<br/> In the current code, it caused a situation osc will search an old lock for the second stripe. lock might exist from past, it's OK. but RA had pined this lock just to send a prepare read for one page - and release it after it via cl_read_ahead_release().<br/> It caused a situation when some pages prepared to send read before lock cancel, but some not and error returned from cl_io_read_ahead(), but previous pages still in osc extents and ready to send rpc. It caused a panic in the osc_req_attr_set when osc have a time to send.<br/> This is easy reproduce with simple assertion and sanity 44A+44a (START_AT=44A).</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> @@ -1195,22 +1198,27 @@ <span class="code-keyword">static</span> <span class="code-object">int</span> lov_io_read_ahead(<span class="code-keyword">const</span> struct lu_env *env, <span class="code-keyword">if</span> (unlikely(!r0-&gt;lo_sub[stripe])) RETURN(-EIO); sub = lov_sub_get(env, lio, lov_comp_index(index, stripe)); <span class="code-keyword">if</span> (IS_ERR(sub)) RETURN(PTR_ERR(sub)); <span class="code-comment">/* no RA outside of active stripe */</span> + <span class="code-keyword">if</span> (sub-&gt;sub_io.ci_state != CIS_IO_GOING &amp;&amp; + sub-&gt;sub_io.ci_state != CIS_LOCKED) + LBUG(); </pre> </div></div> <p>I'm not sure what is best in this situation - just disable an RA outside of active region. or make lock pinning better to avoid situation with submitting without lock held.</p> <p>I looks I understand an issue finally.<br/> readahead loop collects a locked pages during loop, but ldlm lock released after each iteration. So just PG_lock prevents a lock canceling during submit, bit osc_make_rpc had release a page lock after rpc created but don't send. it open window when lock might be canceled and osc_req_attr_set complains lock don't exist.<br/> Lock might be canceled due aging or conlfict on the different node.<br/> Right code should create an special cl_io to collect a lock and cl_io_submit_sync should be called during lock held. osc extent will hold a ldlm lock (after Bobi Jam patch <a href="https://jira.whamcloud.com/browse/LU-16160" title="take ldlm lock when queue sync pages" class="issue-link" data-issue-key="LU-16160"><del>LU-16160</del></a> osc: take ldlm lock when queue sync pages) it will prevent to cancel a lock until it will don't needs.</p> <p>It might be artifact of PG_writeback change. page don't unlocked until IO done before it and panic don't hit. </p> <p>"Alexey Lyashkov &lt;alexey.lyashkov@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/53635" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/53635</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16362" title="landing async readhead caused a &quot;uncovered page&quot; panic during sanityN runs" class="issue-link" data-issue-key="LU-16362">LU-16362</a> readahead: simplification<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: b41d1363be6ee45e2b19ec708be43954dde69e86</p> Related LU-16332 LU-16401 Development Rank 1|i036zj: Rank (Obsolete) 9223372036854775807 Severity