https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14549 Lustre <p>4/10 times running the following I see the old layout returned from getfattr:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>rm -f f0 echo XXX &gt; f0 lfs mirror extend -N -p fast f0 cat f0 lfs mirror split --delete --pool=fast f0 getfattr -n trusted.lov -e text --only-values f0 | hexdump -C lfs getstripe f0 </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># rm -f f0 # echo XXX &gt; f0 # lfs mirror extend -N -p fast f0 # cat f0 XXX # lfs mirror split --delete --pool=fast f0 # getfattr -n trusted.lov -e text --only-values f0 | hexdump -C 00000000 d0 0b d6 0b 20 01 00 00 02 00 00 00 01 00 02 00 |.... ...........| 00000010 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000020 01 00 01 00 10 00 00 00 00 00 00 00 00 00 00 00 |................| 00000030 ff ff ff ff ff ff ff ff 80 00 00 00 38 00 00 00 |............8...| 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 01 00 02 00 10 00 00 00 00 00 00 00 00 00 00 00 |................| 00000060 ff ff ff ff ff ff ff ff b8 00 00 00 48 00 00 00 |............H...| 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000080 d0 0b d1 0b 01 00 00 00 00 00 00 00 00 00 00 00 |................| 00000090 00 00 00 00 00 00 00 00 00 00 10 00 01 00 00 00 |................| 000000a0 23 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |#...............| 000000b0 00 00 00 00 01 00 00 00 d0 0b d3 0b 01 00 00 00 |................| 000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000d0 00 00 10 00 01 00 00 00 66 61 73 74 00 00 00 00 |........fast....| 000000e0 00 00 00 00 00 00 00 00 24 00 00 00 00 00 00 00 |........$.......| 000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00000120 # lfs getstripe f0 f0 lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x23:0x0] } </pre> </div></div> LU-14549

getxattr for lustre.lov and trusted.lov sometimes wrong after mirror split

Bug Minor Resolved Fixed Zhenyu Xu John Hammond bjhpflr Wed, 24 Mar 2021 15:10:19 +0000 Fri, 16 Jul 2021 18:13:36 +0000 Thu, 27 May 2021 19:07:01 +0000 Lustre 2.15.0 0 6 <p>Bobi Jam (bobijam@hotmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/43716" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43716</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14549" title="getxattr for lustre.lov and trusted.lov sometimes wrong after mirror split" class="issue-link" data-issue-key="LU-14549"><del>LU-14549</del></a> llite: refresh layout after mirror merge/split<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: affedc28e95b2a63db920e0a1d76514325522867</p> <p>Strangely, I was testing this on my local filesystem (2.12 client, 2.14 server, without the 43718 patch) the layout generation was reset to "1" after "<tt>lfs mirror extend</tt>":</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> $ lfs getstripe /myth/tmp/adilger/flr-multi lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lmm_layout_gen: 0 : lcme_id: 65538 lcme_mirror_id: 1 lmm_layout_gen: 0 : lcme_id: 65539 lcme_mirror_id: 1 lmm_layout_gen: 0 : lcme_id: 131076 lcme_mirror_id: 2 : $ lfs mirror extend -N -c 1 /myth/tmp/adilger/flr-multi $ getfattr -n lustre.lov --only-values /myth/tmp/adilger/flr-multi | od -tx4 |&amp; awk '/000000 / { print "0x"$4; exit; }' 0000000 0bd60bd0 000002a8 00000001 00050001 $ $ lfs getstripe /myth/tmp/adilger/flr-multi /myth/tmp/adilger/flr-multi lcm_layout_gen: 1 lcm_mirror_count: 3 lcm_entry_count: 5 </pre> </div></div> <p>somehow the <tt>lcm_layout_gen</tt> went from 8 to 1 on a file that hadn't been modified in a long time (created 2021-04-07, and both client and server have been rebooted since then)? If I add new mirrors to that file now, <tt>lcm_layout_gen</tt> is being incremented properly, but it seems like it isn't being read from disk the first time?</p> <p>My theory of layout gen went from 8 to 1 is this, in current LOD code, the layout merge increases the layout generation and write to LOVEA</p> <div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader" style="border-bottom-width: 1px;"><b>lod_declare_layout_merge</b></div><div class="codeContent panelContent"> <pre class="code-java"> ... 3379 <span class="code-comment">/* fixup layout information */</span> 3380 lod_obj_inc_layout_gen(lo); 3381 lcm-&gt;lcm_layout_gen = cpu_to_le32(lo-&gt;ldo_layout_gen); 3382 lcm-&gt;lcm_size = cpu_to_le32(size); 3383 lcm-&gt;lcm_entry_count = cpu_to_le16(cur_entry_count + merge_entry_count); 3384 lcm-&gt;lcm_mirror_count = cpu_to_le16(mirror_count); 3385 <span class="code-keyword">if</span> ((le16_to_cpu(lcm-&gt;lcm_flags) &amp; LCM_FL_FLR_MASK) == LCM_FL_NONE) 3386 lcm-&gt;lcm_flags = cpu_to_le32(LCM_FL_RDONLY); 3387 3388 rc = lod_striping_reload(env, lo, buf); 3389 <span class="code-keyword">if</span> (rc) 3390 GOTO(out, rc); 3391 </pre> </div></div> <p>where the lod_object (@lo) could possibly be not initialized and line 3380 increases the layout_gen to 1 and line 3388 set the lod_object's in-memory layout.</p> <p>While patch <a href="https://review.whamcloud.com/43472/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43472/</a> fixes this as it set the lod_object's in-memory layout first, then increases its layout generation</p> <div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader" style="border-bottom-width: 1px;"><b>lod_declare_layout_merege in #43472</b></div><div class="codeContent panelContent"> <pre class="code-java"> 3409 <span class="code-comment">/* fixup layout information */</span> 3410 lcm-&gt;lcm_size = cpu_to_le32(size); 3411 lcm-&gt;lcm_entry_count = cpu_to_le16(cur_entry_count + merge_entry_count); 3412 lcm-&gt;lcm_mirror_count = cpu_to_le16(mirror_count); 3413 <span class="code-keyword">if</span> ((le16_to_cpu(lcm-&gt;lcm_flags) &amp; LCM_FL_FLR_MASK) == LCM_FL_NONE) 3414 lcm-&gt;lcm_flags = cpu_to_le32(LCM_FL_RDONLY); 3415 3416 rc = lod_striping_reload(env, lo, buf); 3417 <span class="code-keyword">if</span> (rc) 3418 GOTO(out, rc); 3419 3420 lod_obj_inc_layout_gen(lo); 3421 lcm-&gt;lcm_layout_gen = cpu_to_le32(lo-&gt;ldo_layout_gen); 3422 </pre> </div></div> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/43716/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43716/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14549" title="getxattr for lustre.lov and trusted.lov sometimes wrong after mirror split" class="issue-link" data-issue-key="LU-14549"><del>LU-14549</del></a> llite: refresh layout after mirror merge/split<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: bd7a20f8be4644ebce6a7225560ae933204f543d</p> <p>Landed for 2.15</p> Gantt End to End LU-14526 Related Development Rank 1|i01qdz: Rank (Obsolete) 9223372036854775807 Severity