https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-8417 Lustre <p>the -o option to setstripe where you can specify the OSTs to use works on files but gets an error when you do the same thing on directories.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@Lustre-TG1 lustrefs]# mkdir testdir [root@Lustre-TG1 lustrefs]# lfs setstripe -o 0-3 -c 4 testfile [root@Lustre-TG1 lustrefs]# lfs getstripe testfile testfile lmm_stripe_count: 4 lmm_stripe_size: 1048576 lmm_pattern: 1 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 9925 0x26c5 0 1 1606 0x646 0 2 1608 0x648 0 3 1609 0x649 0 [root@Lustre-TG1 lustrefs]# lfs setstripe -o 0-3 -c 4 testdir error on ioctl 0x4008669a for 'testdir' (3): Invalid argument error: setstripe: create stripe file 'testdir' failed </pre> </div></div> LU-8417

setstripe -o does not work on directories

Improvement Major Open Unresolved WC Triage Gary Hagensen easy Wed, 20 Jul 2016 14:31:08 +0000 Thu, 30 Nov 2023 19:48:01 +0000 Lustre 2.7.0 0 9 <p>This would need to store the full uninitialized LOV EA on the MDS directory as the template for the file layout. Based on discussion with Gary, this is needed for testing OSS performance. For "real world" usage, it would probably be a "poor man's OST pools" in the sense that it would be possible to specify a list of OSTs, and then a stripe count less than the total OST count, and it should be possible to select a subset of OSTs to create files on.</p> <p>One issue with using "-o ... -c" to implement "temporary pools" is that there is no place (currently) to store the object allocation state across file creates as there is with a proper pool, so it can't be smart about round-robin allocation (e.g. 0+1, 2+3, 4+5, 6+7, 1+2, 3+4, ... to track the last OST index used and to avoid having stripe 0 on the same OST repeatedly).</p> <p>The MDS would somehow need to dynamically allocate an internal allocation state based on the specified OST list and then keep that in memory for some time to handle allocations with the same group of OSTs. It is OK if the same OST list is shared by different directories, since the file creation and IO load on the OSTs is also shared. There is no significant issue if the allocation state is dropped after some idle time (e.g. a couple of minutes), since the load on the OSTs is also transient.</p> <p>Ideally this would be implemented as part of <a href="https://jira.whamcloud.com/browse/LU-9" title="Optimize weighted QOS Round-Robin allocator" class="issue-link" data-issue-key="LU-9">LU-9</a> so that the imbalance of file creates on a subset of OSTs does not cause global imbalance across other OSTs.</p> <p>Link to original <a href="https://jira.whamcloud.com/browse/LU-4665" title="utils: lfs setstripe to specify OSTs" class="issue-link" data-issue-key="LU-4665"><del>LU-4665</del></a> ticket that introduced this feature.</p> <p>It looks like <a href="https://review.whamcloud.com/12275" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/12275</a> implements this to some extent.</p> <p>It looks like specifying explicit OSTs is working since at least 2.14:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># lfs setstripe -o 2,3,2,3 /mnt/testfs/specific # touch /mnt/testfs/specific/fff # lfs getstripe --yaml /mnt/testfs/specific stripe_count: 4 stripe_size: 1048576 pattern: raid0,overstriped stripe_offset: 2 lmm_stripe_count: 4 lmm_stripe_size: 1048576 lmm_pattern: raid0,overstriped lmm_layout_gen: 0 lmm_stripe_offset: 2 lmm_objects: - l_ost_idx: 2 l_fid: 0x380000401:0x79:0x0 - l_ost_idx: 3 l_fid: 0x3c0000401:0x79:0x0 - l_ost_idx: 2 l_fid: 0x380000401:0x7a:0x0 - l_ost_idx: 3 l_fid: 0x3c0000401:0x7a:0x0 </pre> </div></div> <p>The one remaining issue is that "<tt>lfs getstripe</tt>" on the directory does not print the specific layout properly. It should print the specific OST indices and not just the stripe count.</p> Related LU-4665 LU-9 LU-6135 LU-5170 Development Rank 1|hzyhzr: Rank (Obsolete) 9223372036854775807 Severity