https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13375 Lustre <p>Running "<tt>lfs mirror extend</tt>" on an existing file doesn't use the existing layout for the new file, nor the default filesystem layout on the root directory. Instead it seems to always create a plain layout using the filesystem default <tt>stripe_count</tt> and <tt>stripe_size</tt>. For example, the <tt>lcme_mirror_id: 1</tt> components were initially created from the filesystem default layout, and the <tt>lcme_mirror_id: 2</tt> component was created with "<tt>lfs mirror extend -N1</tt>" (default <tt>stripe_count=4</tt>): </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>/mnt/testfs/pfl0.2 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pool: ddn_ssd lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x12a:0x0] } lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 134217728 lcme_extent.e_end: 4294967296 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pool: ddn_ssd lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x109:0x0] } - 1: { l_ost_idx: 1, l_fid: [0x100010000:0x12b:0x0] } lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 4294967296 lcme_extent.e_end: EOF lmm_stripe_count: -1 lmm_stripe_size: 1048576 lmm_pool: ddn_hdd lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 4 lmm_stripe_size: 1048576 lmm_pool: ddn_hdd lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x12c:0x0] } - 1: { l_ost_idx: 3, l_fid: [0x100030000:0xc8:0x0] } - 2: { l_ost_idx: 2, l_fid: [0x100020000:0x88:0x0] } - 3: { l_ost_idx: 0, l_fid: [0x100000000:0x10a:0x0] } </pre> </div></div> <p>At a minimum, "<tt>lfs mirror extend -N file</tt>" should use the <tt>stripe_count</tt> from the last initialized component, since it is 99% likely that the file will never be modified after it is first written and closed. It would be even better to copy the <tt>stripe_size</tt> as well, but not the pool.</p> LU-13375

lfs mirror extend does not take file layout into account

Bug Minor Resolved Fixed Emoly Liu Andreas Dilger Thu, 19 Mar 2020 21:35:27 +0000 Sun, 19 Apr 2020 14:14:31 +0000 Sun, 19 Apr 2020 14:14:31 +0000 Lustre 2.14.0 Lustre 2.12.5 Lustre 2.14.0 0 4 <p>This issue happens because the mirror layout doesn't inherit any stripe option from the file's component before running mirror_extend() in lfs_setstripe_internal(). I have made some changes and now the patch can work as follows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@centos7-3 tests]# lfs setstripe -E 6M -S 1M -c 3 -E -1 -S 2m -c 2 /mnt/lustre/pfl [root@centos7-3 tests]# lfs mirror extend -N1 /mnt/lustre/pfl [root@centos7-3 tests]# lfs getstripe /mnt/lustre/pfl /mnt/lustre/pfl lcm_layout_gen: 3 lcm_mirror_count: 2 lcm_entry_count: 3 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 6291456 lmm_stripe_count: 3 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x3:0x0] } - 1: { l_ost_idx: 1, l_fid: [0x100010000:0x2:0x0] } - 2: { l_ost_idx: 2, l_fid: [0x100020000:0x4:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 6291456 lcme_extent.e_end: EOF lmm_stripe_count: 2 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 2 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x4:0x0] } - 1: { l_ost_idx: 1, l_fid: [0x100010000:0x3:0x0] } </pre> </div></div> <p>One thing I want to make sure is that the man page of "lfs mirror extend" says "If setstripe_options are not specified, then the stripe options inherited from the previous component will be used." So, we should inherit the stripe options from "the last initialized component" but not the last component, right? (The example above used the last component.)</p> <p>I think there are two reasonable behaviors for "<tt>lfs mirror extend -N1</tt>" on an existing file:</p> <ul class="alternate" type="square"> <li>copy the whole PFL layout of the existing file so that each mirror is the same. This is probably mostly what users "expect" when mirroring a file.</li> <li>copy the layout of the last <b>initialized</b> (used) component of the existing file. This is more efficient, from the point of view that it doesn't create extra objects for early components that are not very helpful, and it avoids e.g. DoM/flash components at the start of the file that would not work (DoM) or waste valuable resources (flash).</li> </ul> <p>The chance of a file being modified after it is written and then mirrored is very low, so if the simpler/better solution is to use only the last initialized component then I'm happy with that for now.</p> <p>Emoly Liu (emoly@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/38079" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38079</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13375" title="lfs mirror extend does not take file layout into account" class="issue-link" data-issue-key="LU-13375"><del>LU-13375</del></a> flr: inherit the correct layout by "mirror extend -N"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 15c47ef03a0a1ff978efa96c9238a87c0b984580</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38079/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38079/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13375" title="lfs mirror extend does not take file layout into account" class="issue-link" data-issue-key="LU-13375"><del>LU-13375</del></a> flr: inherit the correct layout by "mirror extend -N"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: f2a1c6e06f4dc5250cb72b1a6f4720f20486d502</p> <p>Landed for 2.14</p> Related Development Rank 1|i00vxj: Rank (Obsolete) 9223372036854775807 Severity