<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 03:15:48 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-15139] sanity test_160h: dt_record_write() ASSERTION( dt-&gt;do_body_ops-&gt;dbo_write ) failed</title>
                <link>https://jira.whamcloud.com/browse/LU-15139</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;This issue was created by maloo for Andreas Dilger &amp;lt;adilger@whamcloud.com&amp;gt;&lt;/p&gt;

&lt;p&gt;This issue relates to the following test suite run: &lt;a href=&quot;https://testing.whamcloud.com/test_sets/a9a49ddc-ce2e-4d36-a040-a2522bc7bfaa&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://testing.whamcloud.com/test_sets/a9a49ddc-ce2e-4d36-a040-a2522bc7bfaa&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;test_160h failed with the following error:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[ 7154.952072] LustreError: 27898:0:(dt_object.c:503:dt_record_write()) ASSERTION( dt-&amp;gt;do_body_ops-&amp;gt;dbo_write ) failed: 
[ 7154.954429] LustreError: 27898:0:(dt_object.c:503:dt_record_write()) LBUG
[ 7154.955686] Pid: 27898, comm: mdt_out00_000 3.10.0-1160.15.2.el7_lustre.ddn12.x86_64 #1 SMP Tue Feb 23 18:25:44 UTC 2021
[ 7154.957907] Call Trace:
[ 7154.958488]  [&amp;lt;ffffffffc08197cc&amp;gt;] libcfs_call_trace+0x8c/0xc0 [libcfs]
[ 7154.960155]  [&amp;lt;ffffffffc081987c&amp;gt;] lbug_with_loc+0x4c/0xa0 [libcfs]
[ 7154.961662]  [&amp;lt;ffffffffc0a50272&amp;gt;] dt_record_write+0xb2/0x120 [obdclass]
[ 7154.963451]  [&amp;lt;ffffffffc0da7b69&amp;gt;] out_tx_write_exec+0x169/0x2f0 [ptlrpc]
[ 7154.965451]  [&amp;lt;ffffffffc0d9ff51&amp;gt;] out_tx_end+0xe1/0x5c0 [ptlrpc]
[ 7154.967053]  [&amp;lt;ffffffffc0da4172&amp;gt;] out_handle+0x1452/0x1bc0 [ptlrpc]
[ 7154.968687]  [&amp;lt;ffffffffc0d98a3e&amp;gt;] tgt_request_handle+0xaee/0x15f0 [ptlrpc]
[ 7154.970322]  [&amp;lt;ffffffffc0d3f40b&amp;gt;] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
[ 7154.972268]  [&amp;lt;ffffffffc0d42d74&amp;gt;] ptlrpc_main+0xb34/0x1470 [ptlrpc]
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Another test had a slightly different crash that is also new that may be related:&lt;br/&gt;
&lt;a href=&quot;https://testing.whamcloud.com/test_sets/d3ede9b0-ecaa-484e-8840-891e62bdb314&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://testing.whamcloud.com/test_sets/d3ede9b0-ecaa-484e-8840-891e62bdb314&lt;/a&gt;&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[86376.168111] LustreError: 31898:0:(mdt_handler.c:5316:mdt_fini()) ASSERTION( atomic_read(&amp;amp;d-&amp;gt;ld_ref) == 0 ) failed: 
[86376.169205] LustreError: 31898:0:(mdt_handler.c:5316:mdt_fini()) LBUG
[86376.169811] Pid: 31898, comm: umount 3.10.0-1160.15.2.el7_lustre.ddn12.x86_64 #1 SMP Tue Feb 23 03:03:21 UTC 2021
[86376.170763] Call Trace:
[86376.171066]  [&amp;lt;ffffffffc0bed7cc&amp;gt;] libcfs_call_trace+0x8c/0xc0 [libcfs]
[86376.171857]  [&amp;lt;ffffffffc0bed87c&amp;gt;] lbug_with_loc+0x4c/0xa0 [libcfs]
[86376.172537]  [&amp;lt;ffffffffc148b5ea&amp;gt;] mdt_device_fini+0x8fa/0x930 [mdt]
[86376.173227]  [&amp;lt;ffffffffc0e0ea68&amp;gt;] class_cleanup+0x9a8/0xc40 [obdclass]
[86376.174052]  [&amp;lt;ffffffffc0e0f98c&amp;gt;] class_process_config+0x65c/0x2830 [obdclass]
[86376.174793]  [&amp;lt;ffffffffc0e11d26&amp;gt;] class_manual_cleanup+0x1c6/0x710 [obdclass]
[86376.175574]  [&amp;lt;ffffffffc0e446a5&amp;gt;] server_put_super+0xa35/0x1150 [obdclass]
[86376.176295]  [&amp;lt;ffffffff8c4506fd&amp;gt;] generic_shutdown_super+0x6d/0x100
[86376.177103]  [&amp;lt;ffffffff8c450b02&amp;gt;] kill_anon_super+0x12/0x20
[86376.177684]  [&amp;lt;ffffffffc0e14932&amp;gt;] lustre_kill_super+0x32/0x50 [obdclass]
[86376.178360]  [&amp;lt;ffffffff8c450ede&amp;gt;] deactivate_locked_super+0x4e/0x70
[86376.179043]  [&amp;lt;ffffffff8c451666&amp;gt;] deactivate_super+0x46/0x60
[86376.179630]  [&amp;lt;ffffffff8c470cff&amp;gt;] cleanup_mnt+0x3f/0x80
[86376.180163]  [&amp;lt;ffffffff8c470d92&amp;gt;] __cleanup_mnt+0x12/0x20
[86376.180745]  [&amp;lt;ffffffff8c2c294b&amp;gt;] task_work_run+0xbb/0xe0
[86376.181296]  [&amp;lt;ffffffff8c22cc65&amp;gt;] do_notify_resume+0xa5/0xc0
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;





&lt;p&gt;VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV&lt;br/&gt;
sanity test_160h - onyx-44vm4 crashed during sanity test_160h&lt;/p&gt;</description>
                <environment></environment>
        <key id="63086">LU-15139</key>
            <summary>sanity test_160h: dt_record_write() ASSERTION( dt-&gt;do_body_ops-&gt;dbo_write ) failed</summary>
                <type id="1" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11303&amp;avatarType=issuetype">Bug</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="5" iconUrl="https://jira.whamcloud.com/images/icons/statuses/resolved.png" description="A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.">Resolved</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="1">Fixed</resolution>
                                        <assignee username="bzzz">Alex Zhuravlev</assignee>
                                    <reporter username="maloo">Maloo</reporter>
                        <labels>
                    </labels>
                <created>Fri, 26 Feb 2021 19:59:44 +0000</created>
                <updated>Tue, 7 Nov 2023 07:16:53 +0000</updated>
                            <resolved>Wed, 2 Nov 2022 13:28:07 +0000</resolved>
                                    <version>Lustre 2.14.0</version>
                    <version>Lustre 2.15.0</version>
                                    <fixVersion>Lustre 2.16.0</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>11</watches>
                                                                            <comments>
                            <comment id="293237" author="adilger" created="Fri, 26 Feb 2021 20:01:21 +0000"  >&lt;p&gt;It is possible that these failures relate to &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-14341&quot; title=&quot;hard LOCKUP lustre servers with kernel-3.10.0-1160.11.1&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-14341&quot;&gt;&lt;del&gt;LU-14341&lt;/del&gt;&lt;/a&gt; since we are now testing on 3.10.0-1160.15.2.el7 kernels, and that problem started with 3.10.0-1160.8.1.el7 kernels.&lt;/p&gt;</comment>
                            <comment id="317157" author="adilger" created="Mon, 1 Nov 2021 20:56:01 +0000"  >&lt;p&gt;+1 on master:&lt;br/&gt;
&lt;a href=&quot;https://testing.whamcloud.com/test_sets/0804055c-4eea-45f5-a43c-66079e3f3436&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://testing.whamcloud.com/test_sets/0804055c-4eea-45f5-a43c-66079e3f3436&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="317758" author="adilger" created="Tue, 9 Nov 2021 17:20:31 +0000"  >&lt;p&gt;+1 on master: &lt;a href=&quot;https://testing.whamcloud.com/test_sets/9c12f7a3-252e-48ae-aa7f-804bbd117895&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://testing.whamcloud.com/test_sets/9c12f7a3-252e-48ae-aa7f-804bbd117895&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="317761" author="bzzz" created="Tue, 9 Nov 2021 18:05:58 +0000"  >&lt;p&gt;I can reproduce the same assertion with recovery-small/149, very rarely, collecting data, but IMO this has a different cause. anyway, this is on my radar.&lt;/p&gt;</comment>
                            <comment id="330007" author="adilger" created="Wed, 23 Mar 2022 18:46:58 +0000"  >&lt;p&gt;+12 crashes in recovery-small test_149 in the past 4 weeks with the &lt;tt&gt;out_tx_write_exec&lt;/tt&gt; stack, but only 2 in the past few days, the rest are over 2 weeks ago.&lt;/p&gt;</comment>
                            <comment id="330009" author="bzzz" created="Wed, 23 Mar 2022 18:59:08 +0000"  >&lt;p&gt;yes, I also hit this locally few times a week, adding more and more debug ...&lt;/p&gt;</comment>
                            <comment id="330967" author="bzzz" created="Mon, 4 Apr 2022 16:36:37 +0000"  >&lt;p&gt;I was able to reproduce this and have log. and the log is confusing:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
00000004:00000010:0.0:1649084555.961084:0:6155:0:(mdt_handler.c:6301:mdt_object_alloc()) slab-alloced &lt;span class=&quot;code-quote&quot;&gt;&apos;mo&apos;&lt;/span&gt;: 912 at 00000000092425e9.
00000004:00000040:0.0:1649084555.961088:0:6155:0:(mdt_handler.c:6338:mdt_object_init()) object init, fid = [0x240001b70:0x1:0x0]
00000004:00000010:0.0:1649084555.961095:0:6155:0:(lod_dev.c:214:lod_object_alloc()) slab-alloced &lt;span class=&quot;code-quote&quot;&gt;&apos;lod_obj&apos;&lt;/span&gt;: 280 at 0000000012d03b04.
00080000:00000010:0.0:1649084555.961107:0:6155:0:(osd_handler.c:420:osd_object_alloc()) kmalloced &lt;span class=&quot;code-quote&quot;&gt;&apos;(mo)&apos;&lt;/span&gt;: 544 at 000000000ab89345.
00000004:00000040:0.0:1649084555.961102:0:6155:0:(lod_dev.c:169:lod_fld_lookup()) lustre-MDT0001-mdtlov: got tgt 1 &lt;span class=&quot;code-keyword&quot;&gt;for&lt;/span&gt; sequence: 0x240001b70

00000020:00000002:0.0:1649084555.962456:0:6155:0:(lu_object.c:224:lu_object_put()) Add 000000000ab89345/00000000092425e9 to site lru. bkt: 00000000cf2c67a7

00000004:00000001:0.0:1649084555.973169:0:6155:0:(mdt_handler.c:6299:mdt_object_alloc()) &lt;span class=&quot;code-object&quot;&gt;Process&lt;/span&gt; entered
00000004:00000010:0.0:1649084555.973174:0:6155:0:(mdt_handler.c:6301:mdt_object_alloc()) slab-alloced &lt;span class=&quot;code-quote&quot;&gt;&apos;mo&apos;&lt;/span&gt;: 912 at 000000000b6883b5.
00000004:00000040:0.0:1649084555.973177:0:6155:0:(mdt_handler.c:6338:mdt_object_init()) object init, fid = [0x240001b70:0x1:0x0]
00000004:00000010:0.0:1649084555.973179:0:6155:0:(mdd_object.c:277:mdd_object_alloc()) slab-alloced &lt;span class=&quot;code-quote&quot;&gt;&apos;mdd_obj&apos;&lt;/span&gt;: 96 at 00000000a2579935.
00000004:00000040:0.0:1649084555.973194:0:6155:0:(lod_dev.c:169:lod_fld_lookup()) lustre-MDT0001-mdtlov: got tgt 1 &lt;span class=&quot;code-keyword&quot;&gt;for&lt;/span&gt; sequence: 0x240001b70
00000004:00000001:0.0:1649084555.973195:0:6155:0:(lod_dev.c:171:lod_fld_lookup()) &lt;span class=&quot;code-object&quot;&gt;Process&lt;/span&gt; leaving (rc=0 : 0 : 0)
00080000:00000010:0.0:1649084555.973200:0:6155:0:(osd_handler.c:420:osd_object_alloc()) kmalloced &lt;span class=&quot;code-quote&quot;&gt;&apos;(mo)&apos;&lt;/span&gt;: 544 at 00000000b65f9684.
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;</comment>
                            <comment id="331169" author="gerrit" created="Wed, 6 Apr 2022 08:11:46 +0000"  >&lt;p&gt;&quot;Alex Zhuravlev &amp;lt;bzzz@whamcloud.com&amp;gt;&quot; uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/47003&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/47003&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-15139&quot; title=&quot;sanity test_160h: dt_record_write() ASSERTION( dt-&amp;gt;do_body_ops-&amp;gt;dbo_write ) failed&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-15139&quot;&gt;&lt;del&gt;LU-15139&lt;/del&gt;&lt;/a&gt; osp: block reads until writes are done&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: a09bc918bf4c18a4ff1c948237da5d5b3d391078&lt;/p&gt;</comment>
                            <comment id="331287" author="bzzz" created="Thu, 7 Apr 2022 08:42:25 +0000"  >&lt;p&gt;the patch above survived ~30 hours. w/o the patch it usually takes few hours to hit the assertion.&lt;br/&gt;
one of the early version confirmat that it&apos;s possible to hit reads during write-in-flight in OSP - basically at recovery time MDS&apos;s thread cleaning orphans from PENDING/ race with the recovery thread fetching updates from the remote llogs.&lt;/p&gt;</comment>
                            <comment id="335432" author="nangelinas" created="Wed, 18 May 2022 20:16:45 +0000"  >&lt;p&gt;+1 on master in recovery-small test_149: &lt;a href=&quot;https://testing.whamcloud.com/test_sets/68f3e91b-d748-4a92-89d1-babf4dcd7fb3&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://testing.whamcloud.com/test_sets/68f3e91b-d748-4a92-89d1-babf4dcd7fb3&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="336464" author="ys" created="Tue, 31 May 2022 18:41:50 +0000"  >&lt;p&gt;+1 on master: &lt;a href=&quot;https://testing.whamcloud.com/test_sets/88d891a9-d784-432b-a414-ef82751ac528&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://testing.whamcloud.com/test_sets/88d891a9-d784-432b-a414-ef82751ac528&lt;/a&gt;&lt;br/&gt;
I have decoded the vmcore:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;[ 7977.738850] LustreError: 14410:0:(dt_object.c:502:dt_record_write()) ASSERTION( dt-&amp;gt;do_body_ops-&amp;gt;dbo_write ) failed:
[ 7977.740884] LustreError: 14410:0:(dt_object.c:502:dt_record_write()) LBUG
[ 7977.742136] Pid: 14410, comm: mdt_out00_003 4.18.0-348.2.1.el8_lustre.x86_64 #1 SMP Fri May 6 20:34:00 UTC 2022
[ 7977.743950] Call Trace TBD:
[ 7977.744707] [&amp;lt;0&amp;gt;] libcfs_call_trace+0x6f/0x90 [libcfs]
[ 7977.745679] [&amp;lt;0&amp;gt;] lbug_with_loc+0x43/0x80 [libcfs]
[ 7977.746842] [&amp;lt;0&amp;gt;] dt_record_write+0x10a/0x110 [obdclass]
[ 7977.748285] [&amp;lt;0&amp;gt;] out_tx_write_exec+0x92/0x380 [ptlrpc]
[ 7977.749324] [&amp;lt;0&amp;gt;] out_tx_end+0x166/0x5f0 [ptlrpc]
[ 7977.750300] [&amp;lt;0&amp;gt;] out_handle+0x17ef/0x2180 [ptlrpc]
[ 7977.751308] [&amp;lt;0&amp;gt;] tgt_request_handle+0xc93/0x1a40 [ptlrpc]
[ 7977.752407] [&amp;lt;0&amp;gt;] ptlrpc_server_handle_request+0x323/0xbd0 [ptlrpc]
[ 7977.753643] [&amp;lt;0&amp;gt;] ptlrpc_main+0xc06/0x1560 [ptlrpc]
[ 7977.754607] [&amp;lt;0&amp;gt;] kthread+0x116/0x130
[ 7977.755345] [&amp;lt;0&amp;gt;] ret_from_fork+0x35/0x40
[ 7977.756132] Kernel panic - not syncing: LBUG

&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;The osd_object:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;struct osd_object {
  oo_dt = {
    do_lu = {
      lo_header = 0xffff96c0fb0a2930, 
      lo_dev = 0xffff96c0f1f40000, 
      lo_ops = 0xffffffffc116a440 &amp;lt;__func__.70851&amp;gt;, 
      lo_linkage = {
        next = 0xffff96c0fb0a2968, 
        prev = 0xffff96c0e6c87a30
      }, 
      lo_dev_ref = {&amp;lt;No data fields&amp;gt;}
    }, 
    do_ops = 0xffffffffc116a320 &amp;lt;__func__.71148+16&amp;gt;, 
    do_body_ops = 0xffffffffc116b760 &amp;lt;__flags.79514&amp;gt;, 
    do_index_ops = 0x0
  }, 
  oo_inode = 0x0, 
  oo_hl_head = 0x0, 
  oo_ext_idx_sem = {
    count = {
      counter = 256
    }, 
crash&amp;gt; sym 0xffffffffc116b760
ffffffffc116b760 (r) osd_body_ops_new [osd_ldiskfs] 
crash&amp;gt; 


&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;The oo_inode was NULL, since osd_fid_lookup return from:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;
00000004:00000001:0.0:1653480139.386555:0:14410:0:(mdt_handler.c:6335:mdt_object_init()) Process entered
00000004:00000040:0.0:1653480139.386555:0:14410:0:(mdt_handler.c:6338:mdt_object_init()) object init, fid = [0x240001b70:0x1:0x0]
00000004:00000010:0.0:1653480139.386557:0:14410:0:(mdd_object.c:277:mdd_object_alloc()) slab-alloced &apos;mdd_obj&apos;: 96 at 00000000b01970bb.
00000004:00000001:0.0:1653480139.386558:0:14410:0:(mdt_handler.c:6347:mdt_object_init()) Process leaving (rc=0 : 0 : 0)
00000004:00000001:0.0:1653480139.386558:0:14410:0:(mdd_object.c:299:mdd_object_init()) Process entered
00000004:00000001:0.0:1653480139.386559:0:14410:0:(lod_dev.c:212:lod_object_alloc()) Process entered
00000004:00000010:0.0:1653480139.386560:0:14410:0:(lod_dev.c:214:lod_object_alloc()) slab-alloced &apos;lod_obj&apos;: 152 at 0000000059b054cd.
00000004:00000001:0.0:1653480139.386560:0:14410:0:(lod_dev.c:224:lod_object_alloc()) Process leaving (rc=18446628353982626328 : -115719726925288 : ffff96c0e6c87a18)
00000004:00000001:0.0:1653480139.386561:0:14410:0:(mdd_object.c:309:mdd_object_init()) Process leaving (rc=0 : 0 : 0)
00000004:00000001:0.0:1653480139.386562:0:14410:0:(lod_object.c:9014:lod_object_init()) Process entered
00000004:00000001:0.0:1653480139.386563:0:14410:0:(lod_dev.c:128:lod_fld_lookup()) Process entered
80000000:00000001:0.0:1653480139.386563:0:14410:0:(fld_handler.c:247:fld_server_lookup()) Process entered
80000000:00000001:0.0:1653480139.386564:0:14410:0:(fld_handler.c:211:fld_local_lookup()) Process entered
80000000:00000001:0.0:1653480139.386565:0:14410:0:(fld_cache.c:468:fld_cache_lookup()) Process entered
80000000:00000001:0.0:1653480139.386565:0:14410:0:(fld_cache.c:487:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0)
80000000:00000001:0.0:1653480139.386566:0:14410:0:(fld_handler.c:228:fld_local_lookup()) Process leaving (rc=0 : 0 : 0)
80000000:00000001:0.0:1653480139.386566:0:14410:0:(fld_handler.c:251:fld_server_lookup()) Process leaving (rc=0 : 0 : 0)
00000004:00000040:0.0:1653480139.386567:0:14410:0:(lod_dev.c:169:lod_fld_lookup()) lustre-MDT0001-mdtlov: got tgt 1 for sequence: 0x240001b70
00000004:00000001:0.0:1653480139.386568:0:14410:0:(lod_dev.c:171:lod_fld_lookup()) Process leaving (rc=0 : 0 : 0)
00080000:00000010:0.0:1653480139.386569:0:14410:0:(osd_handler.c:420:osd_object_alloc()) kmalloced &apos;(mo)&apos;: 224 at 00000000b474d102.
00000004:00000001:0.0:1653480139.386570:0:14410:0:(lod_object.c:9057:lod_object_init()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386571:0:14410:0:(osd_handler.c:1126:osd_fid_lookup()) Process entered
00080000:00000001:0.0:1653480139.386571:0:14410:0:(osd_oi.c:576:fid_is_on_ost()) Process entered
80000000:00000001:0.0:1653480139.386572:0:14410:0:(fld_handler.c:211:fld_local_lookup()) Process entered
80000000:00000001:0.0:1653480139.386572:0:14410:0:(fld_cache.c:468:fld_cache_lookup()) Process entered
80000000:00000001:0.0:1653480139.386573:0:14410:0:(fld_cache.c:487:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0)
80000000:00000001:0.0:1653480139.386573:0:14410:0:(fld_handler.c:228:fld_local_lookup()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386574:0:14410:0:(osd_oi.c:615:fid_is_on_ost()) Process leaving (rc=0 : 0 : 0)
80000000:00000001:0.0:1653480139.386572:0:14410:0:(fld_handler.c:211:fld_local_lookup()) Process entered
80000000:00000001:0.0:1653480139.386572:0:14410:0:(fld_cache.c:468:fld_cache_lookup()) Process entered
80000000:00000001:0.0:1653480139.386573:0:14410:0:(fld_cache.c:487:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0)
80000000:00000001:0.0:1653480139.386573:0:14410:0:(fld_handler.c:228:fld_local_lookup()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386574:0:14410:0:(osd_oi.c:615:fid_is_on_ost()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386575:0:14410:0:(osd_oi.c:546:osd_oi_iam_lookup()) Process entered
00000001:00000010:0.0:1653480139.386576:0:14410:0:(osd_dynlocks.c:109:dynlock_lock()) slab-alloced &apos;nhl&apos;: 72 at 0000000006b44178.
00000001:00000010:0.0:1653480139.386580:0:14410:0:(osd_dynlocks.c:193:dynlock_unlock()) slab-freed &apos;hl&apos;: 72 at 0000000006b44178.
00080000:00000001:0.0:1653480139.386582:0:14410:0:(osd_oi.c:568:osd_oi_iam_lookup()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386582:0:14410:0:(osd_oi.c:576:fid_is_on_ost()) Process entered
80000000:00000001:0.0:1653480139.386583:0:14410:0:(fld_handler.c:211:fld_local_lookup()) Process entered
80000000:00000001:0.0:1653480139.386583:0:14410:0:(fld_cache.c:468:fld_cache_lookup()) Process entered
80000000:00000001:0.0:1653480139.386584:0:14410:0:(fld_cache.c:487:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0)
80000000:00000001:0.0:1653480139.386584:0:14410:0:(fld_handler.c:228:fld_local_lookup()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386585:0:14410:0:(osd_oi.c:615:fid_is_on_ost()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:0.0:1653480139.386586:0:14410:0:(osd_handler.c:1186:osd_fid_lookup()) Process leaving via out (rc=0 : 0 : 0x0).  &amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;-----

1179         /* Search order: 3. OI files. */
1180         result = osd_oi_lookup(info, dev, fid, id, OI_CHECK_FLD);
1181         if (result == -ENOENT) {
1182                 if (!(fid_is_norm(fid) || fid_is_igif(fid)) ||
1183                     fid_is_on_ost(info, dev, fid, OI_CHECK_FLD) ||
1184                     !ldiskfs_test_bit(osd_oi_fid2idx(dev, fid),
1185                                       sf-&amp;gt;sf_oi_bitmap))
1186                         GOTO(out, result = 0);
1187 
1188                 goto trigger;

&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Look at the sf_oi_bitmap:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;      os_file = {
        sf_uuid = {
          b = &quot;\202%\024\\_EL\322\240\220\060\322\250\244\357k&quot;
        },
        sf_flags = 0x0,
        sf_magic = 0x4c5fd252,
        sf_status = 0x0,
        sf_param = 0x0,
        sf_time_last_complete = 0x0,
        sf_time_latest_start = 0x0,
        sf_time_last_checkpoint = 0x0,
        sf_pos_latest_start = 0x0,
        sf_pos_last_checkpoint = 0x0,
        sf_pos_first_inconsistent = 0x0,
        sf_items_checked = 0x0,
        sf_items_updated = 0x0,
        sf_items_failed = 0x0,
        sf_items_updated_prior = 0x0,
        sf_items_noscrub = 0x0,
        sf_items_igif = 0x0,
        sf_run_time = 0x0,
        sf_success_count = 0x0,
        sf_oi_count = 0x40,
        sf_internal_flags = 0x1,
        sf_reserved_1 = 0x0,
        sf_reserved_2 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
        sf_oi_bitmap = &quot;\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000&quot;
      },

&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;</comment>
                            <comment id="351466" author="gerrit" created="Wed, 2 Nov 2022 07:10:50 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/c/fs/lustre-release/+/47003/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/c/fs/lustre-release/+/47003/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-15139&quot; title=&quot;sanity test_160h: dt_record_write() ASSERTION( dt-&amp;gt;do_body_ops-&amp;gt;dbo_write ) failed&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-15139&quot;&gt;&lt;del&gt;LU-15139&lt;/del&gt;&lt;/a&gt; osp: block reads until the object is created&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 4f2914537cc32fe89c4781bcfc87c38e3fe4419c&lt;/p&gt;</comment>
                            <comment id="351504" author="pjones" created="Wed, 2 Nov 2022 13:28:07 +0000"  >&lt;p&gt;Landed for 2.16&lt;/p&gt;</comment>
                            <comment id="377351" author="gerrit" created="Tue, 4 Jul 2023 13:48:05 +0000"  >&lt;p&gt;&quot;Etienne AUJAMES &amp;lt;eaujames@ddn.com&amp;gt;&quot; uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/c/fs/lustre-release/+/51558&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/c/fs/lustre-release/+/51558&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-15139&quot; title=&quot;sanity test_160h: dt_record_write() ASSERTION( dt-&amp;gt;do_body_ops-&amp;gt;dbo_write ) failed&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-15139&quot;&gt;&lt;del&gt;LU-15139&lt;/del&gt;&lt;/a&gt; osp: block reads until the object is created&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: b2_15&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 4afaf1249439ea7a27910b0f77a84232c268df61&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                                        </outwardlinks>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="70275">LU-15849</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                            <outwardlinks description="is related to ">
                                        <issuelink>
            <issuekey id="57980">LU-13195</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="73627">LU-16398</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="70092">LU-15809</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="78740">LU-17255</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="69839">LU-15769</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="65610">LU-14932</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="69273">LU-15695</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="70717">LU-15938</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i01nw7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10060" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Severity</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10022"><![CDATA[3]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>