https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3448 Lustre <p>Running a modified racer (<a href="https://jira.whamcloud.com/browse/LU-3072" title="add more operations to racer" class="issue-link" data-issue-key="LU-3072"><del>LU-3072</del></a>, + smaller dds + sleep in file_create.sh) I can reproduce this.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LustreError: 23518:0:(osc_cache.c:2379:osc_teardown_async_page()) extent ffff8800abb54c60@{[0 -&gt; 1/255], [3|1|-|active|wi|ffff8800aa8b4688], [8192|2|+|-|ffff8800ac9a2e78|256|(null)]} trunc at 0. LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) page@ffff8800acab8600[2 ffff8800ae3bb630:0 ^(null)_ffff880112410800 4 0 1 (null) (null) 0x0] LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) page@ffff880112410800[2 ffff8800a99e8508:0 ^ffff8800acab8600_(null) 4 0 1 (null) (null) 0x0] LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) vvp-page@ffff8800acab86c0(0:0:0) vm@ffffea0002c25250 20000000000035 3:0 0 0 lru LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) lov-page@ffff8800acab8710 LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) osc-page@ffff8801124108e8: 1&lt; 0x845fed 258 0 + - &gt; 2&lt; 0 0 4096 0x0 0x520 | (null) ffff8800b135e600 ffff8800aa8b4688 &gt; 3&lt; + ffff8800ade74080 0 0 0 &gt; 4&lt; 0 0 8 33980416 - | - - + - &gt; 5&lt; - - + - | 0 - | 7 - -&gt; LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) end page@ffff8800acab8600 LustreError: 23518:0:(osc_page.c:430:osc_page_delete()) Trying to teardown failed: -16 LustreError: 23518:0:(osc_page.c:431:osc_page_delete()) ASSERTION( 0 ) failed: LustreError: 23518:0:(osc_page.c:431:osc_page_delete()) LBUG Pid: 23518, comm: cp Call Trace: [&lt;ffffffffa02ae895&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] [&lt;ffffffffa02aee97&gt;] lbug_with_loc+0x47/0xb0 [libcfs] [&lt;ffffffffa0854701&gt;] osc_page_delete+0x311/0x320 [osc] [&lt;ffffffffa0468bb5&gt;] cl_page_delete0+0xc5/0x4e0 [obdclass] [&lt;ffffffffa0469012&gt;] cl_page_delete+0x42/0x120 [obdclass] [&lt;ffffffffa0ce766d&gt;] ll_invalidatepage+0x8d/0x160 [lustre] [&lt;ffffffff81131ae5&gt;] do_invalidatepage+0x25/0x30 [&lt;ffffffff81131e02&gt;] truncate_inode_page+0xa2/0xc0 [&lt;ffffffff811322d2&gt;] truncate_inode_pages_range+0x292/0x500 [&lt;ffffffffa02afa4e&gt;] ? cfs_mem_cache_free+0xe/0x10 [libcfs] [&lt;ffffffff81143b62&gt;] ? unmap_mapping_range+0x72/0x140 [&lt;ffffffff811325d5&gt;] truncate_inode_pages+0x15/0x20 [&lt;ffffffff8113262f&gt;] truncate_pagecache+0x4f/0x70 [&lt;ffffffff811aa84a&gt;] simple_setsize+0x3a/0x50 [&lt;ffffffff811aa8a0&gt;] simple_setattr+0x40/0x70 [&lt;ffffffffa0cc1416&gt;] ll_setattr_raw+0x2a6/0x1090 [lustre] [&lt;ffffffffa0cc225b&gt;] ll_setattr+0x5b/0xf0 [lustre] [&lt;ffffffff8119fdc8&gt;] notify_change+0x168/0x340 [&lt;ffffffff811807e4&gt;] do_truncate+0x64/0xa0 [&lt;ffffffff8121e52f&gt;] ? security_inode_permission+0x1f/0x30 [&lt;ffffffff811946e4&gt;] do_filp_open+0x844/0xdd0 [&lt;ffffffff8104757c&gt;] ? __do_page_fault+0x1ec/0x480 [&lt;ffffffff811a0ca2&gt;] ? alloc_fd+0x92/0x160 [&lt;ffffffff8117f559&gt;] do_sys_open+0x69/0x140 [&lt;ffffffff8117f670&gt;] sys_open+0x20/0x30 [&lt;ffffffff8100b072&gt;] system_call_fastpath+0x16/0x1b </pre> </div></div> LU-3448

osc_page_delete()) ASSERTION(0) failed running racer

Bug Blocker Resolved Fixed John Hammond John Hammond osc Mon, 10 Jun 2013 17:29:15 +0000 Wed, 24 Jul 2013 11:05:41 +0000 Fri, 21 Jun 2013 22:08:13 +0000 Lustre 2.5.0 Lustre 2.5.0 0 6 <p>This appears to have been introduced by a61ff59. After reverting that commit I no longer see it.</p> <p>On master I can see it using a non-modified racer as well.</p> <p>My bad, this is a race. After the truncate finished in ll_setattr_ost(), it released inode mutex so that new write produced more pages beyond truncate size. </p> <p>John, if you get a chance, can you please restore the change in <a href="http://review.whamcloud.com/#patch,sidebyside,4816,18,lustre/llite/llite_lib.c" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#patch,sidebyside,4816,18,lustre/llite/llite_lib.c</a> and give it a try?</p> <p>This works for me. I reverted the first 3 of the 4 hunks in llite_lib.c and the crash has gone away. I don't follow your explanation however. I seems more likely to me that the difference is that the old way cleared ATTR_SIZE from valid in simple_setattr(). Please correct me if I'm wrong.</p> <p>Please see <a href="http://review.whamcloud.com/6643" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6643</a>.</p> <p>Restoring the call to simple_setattr() fixed the LBUG but causes the added test (sanity 229) to fail. Shall I delete the test as well?</p> <p>On master (2.4.50-79-gaed8203) which has the patch from <a href="https://jira.whamcloud.com/browse/LU-2482" title="Define new layout for released file" class="issue-link" data-issue-key="LU-2482"><del>LU-2482</del></a> I see that the effect of truncate on a released file is not seen consistently across clients:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># MOUNT_2=y llmount.sh # multiop /mnt/lustre/f0 H2c # truncate --size=42 /mnt/lustre/f0 # stat /mnt/lustre/f0 File: `/mnt/lustre/f0' Size: 42 Blocks: 0 IO Block: 4194304 regular file ... # stat /mnt/lustre2/f0 File: `/mnt/lustre2/f0' Size: 0 Blocks: 0 IO Block: 4194304 regular empty file ... # stat /mnt/lustre/f0 File: `/mnt/lustre/f0' Size: 42 Blocks: 0 IO Block: 4194304 regular file ... </pre> </div></div> <p>Can someone explain why we support truncate (to non-zero size) on released files? Truncate to zero seems somewhat defensible, but to non-zero just asks for trouble. But in either case why bother? In practice won't truncate almost always be followed by write (requiring a restore)?</p> <p>Originally this is worked out for an optimization. For example, when a released file is truncated, we just change the size on the MDT but restore it later. But here is a problem that if a release file is truncated down to size A, then up to size B. The file content in <span class="error">&#91;A, B&#93;</span> should contain zero.</p> <p>I think it's okay to remove the truncate part in the test case.</p> <p>Well that would make an even stronger case for disallowing truncates on released files.</p> <p>CEA colleagues, do you have opinions to share on this?</p> <p>I don't think we ever intended to support such an optimization. In general, truncate should trigger a restore. The only case we want to "optimize" is truncate to 0 where we can just discard the HSM copy.</p> <p>I confirm Johann comment, truncate trigs a restore and blocks up to end of full restore. Later with partial restore we can optimize this.<br/> Today the only optimization is truncate to 0 (no restore)</p> <p>It appears I misunderstood you guys. Because when you asked me to add the test case for truncate to released file, I thought this is what you will do.</p> <p>To make it safe, let's deny the truncate to HSM released file.</p> <p>John, can you please add this fix in your patch? The extra fix would be:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[jinxiong@intel mdc]$ git diff ../lov/lov_io.c diff --git a/lustre/lov/lov_io.c b/lustre/lov/lov_io.c index 6f6ea84..bec9fea 100644 --- a/lustre/lov/lov_io.c +++ b/lustre/lov/lov_io.c @@ -984,12 +984,12 @@ <span class="code-object">int</span> lov_io_init_released(<span class="code-keyword">const</span> struct lu_env *env, struct cl_object *obj, LASSERTF(0, <span class="code-quote">"invalid type %d\n"</span>, io-&gt;ci_type); <span class="code-keyword">case</span> CIT_MISC: <span class="code-keyword">case</span> CIT_FSYNC: - <span class="code-keyword">case</span> CIT_SETATTR: result = +1; <span class="code-keyword">break</span>; <span class="code-keyword">case</span> CIT_READ: <span class="code-keyword">case</span> CIT_WRITE: <span class="code-keyword">case</span> CIT_FAULT: + <span class="code-keyword">case</span> CIT_SETATTR: <span class="code-comment">/* TODO: need to restore the file. */</span> result = -EBADF; <span class="code-keyword">break</span>; </pre> </div></div> <p>Without this fix, it will have problem to handle the size of released file with truncate.</p> <p>Patch landed to master.</p> <p>Is there any plan to rehabilitate truncate to 0 (or O_TRUNC) for released files?</p> <p>today we have 2 cases:<br/> 1) a released file (just created with a release layout as in test 229)<br/> 2) a released file associated with an archive (after hsm_archive + hsm_release or an import)</p> <p>truncate 0 or &gt; 0 on 1) make an error<br/> truncate &gt; 0 on 2), restores the archived and truncates the file<br/> truncate 0 on 2) should work without a restore (still need to be done, see <a href="https://jira.whamcloud.com/browse/LU-3454" title="open (O_TRUNC) and truncate(0) on a released file must not trig a restore " class="issue-link" data-issue-key="LU-3454"><del>LU-3454</del></a>)</p> <p>If we change to have truncate 0 works on 1), we can also change to have truncate &gt; 0 also works. This is only philosophy on "what is a released file without an HSM archive"</p> Related LU-2482 LU-2531 Development Rank 1|hzvsyn: Rank (Obsolete) 8619 Severity