<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 02:53:12 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-12507] llapi: llapi_layout_pool_name_set() should &quot;nuke&quot; the ost objects in the layout structure</title>
                <link>https://jira.whamcloud.com/browse/LU-12507</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;The llapi provides setters and getters around the &lt;tt&gt;struct llapi_layout&lt;/tt&gt;. One of them is &lt;tt&gt;llapi_layout_pool_name_set()&lt;/tt&gt;. The function&apos;s description gives the impression that it can be used to change an already existing layout&apos;s pool name to something else. In practice, there is information in the &lt;tt&gt;struct llapi_layout&lt;/tt&gt; that is tied to the pool&apos;s name that must be erased after (or before) a pool name is set. Here is the definition of &lt;tt&gt;struct llapi_layout&lt;/tt&gt;:&lt;/p&gt;

&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;/**
 * Layout component, which contains all attributes of a plain
 * V1/V3 layout.
 */
struct llapi_layout_comp {
	uint64_t	llc_pattern;
	uint64_t	llc_stripe_size;
	uint64_t	llc_stripe_count;
	uint64_t	llc_stripe_offset;
	/* Add 1 so user always gets back a null terminated string. */
	char		llc_pool_name[LOV_MAXPOOLNAME + 1];
	/** Number of objects in llc_objects array if was initialized. */
	uint32_t	llc_objects_count;
	struct		lov_user_ost_data_v1 *llc_objects;
	/* fields used only for composite layouts */
	struct lu_extent	llc_extent;	/* [start, end) of component */
	uint32_t		llc_id;		/* unique ID of component */
	uint32_t		llc_flags;	/* LCME_FL_* flags */
	uint64_t		llc_timestamp;	/* snapshot timestamp */
	struct list_head	llc_list;	/* linked to the llapi_layout
						   components list */
};

/**
 * An Opaque data type abstracting the layout of a Lustre file.
 */
struct llapi_layout {
	uint32_t	llot_magic; /* LLAPI_LAYOUT_MAGIC */
	uint32_t	llot_gen;
	uint32_t	llot_flags;
	bool		llot_is_composite;
	uint16_t	llot_mirror_count;
	/* Cursor pointing to one of the components in llot_comp_list */
	struct llapi_layout_comp *llot_cur_comp;
	struct list_head	  llot_comp_list;
};
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;&#160;&lt;br/&gt;
&lt;tt&gt;llc_objects_count&lt;/tt&gt; and &lt;tt&gt;llc_objects&lt;/tt&gt; are the fields that must be reset. This can be done with a call to &lt;tt&gt;llapi_layout_ost_index_set()&lt;/tt&gt; with &lt;tt&gt;stripe_number = 0&lt;/tt&gt; and &lt;tt&gt;ost_index = LLAPI_LAYOUT_DEFAULT&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;I think llapi_pool_name_set() should do this work itself. What do you think? Are there use cases to keeping a component&apos;s object list after a pool change?&lt;/p&gt;</description>
                <environment></environment>
        <key id="56270">LU-12507</key>
            <summary>llapi: llapi_layout_pool_name_set() should &quot;nuke&quot; the ost objects in the layout structure</summary>
                <type id="2" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11311&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="1" iconUrl="https://jira.whamcloud.com/images/icons/statuses/open.png" description="The issue is open and ready for the assignee to start work on it.">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="pjones">Peter Jones</assignee>
                                    <reporter username="cealustre">CEA</reporter>
                        <labels>
                    </labels>
                <created>Thu, 4 Jul 2019 15:54:59 +0000</created>
                <updated>Wed, 5 Aug 2020 13:50:24 +0000</updated>
                                            <version>Upstream</version>
                                                        <due></due>
                            <votes>0</votes>
                                    <watches>5</watches>
                                                                            <comments>
                            <comment id="250688" author="pfarrell" created="Thu, 4 Jul 2019 16:04:56 +0000"  >&lt;p&gt;If I understand this correctly, we need to be careful here.&lt;/p&gt;

&lt;p&gt;At least when a layout has been read from an existing file, llc_objects should point to the actual OST objects on disk.&#160; We can&apos;t just blow away that field without corrupting the existing layout by removing our association with those objects.&lt;/p&gt;

&lt;p&gt;What is stored in llc_objects at the time you want to remove it?&#160; The same question applies to llc_object_count - What is stored there when you want to remove it?&#160; It kind of sounds like it&apos;s the OST index, which I would believe.&#160; But again, for an instantiated layout, one that has been read up from disk, this is the number of actual objects and cannot be changed.&lt;/p&gt;

&lt;p&gt;What is your use case here?&#160; Are you copying the layout of an existing file and modifying it...?&#160; Why are you changing the pool of a component with an object list?&#160; For an instantiated component, changing the pool doesn&apos;t do anything because the OSTs are already selected.&lt;/p&gt;

&lt;p&gt;I am not saying you&apos;re wrong, just that I&apos;m a little confused by what you&apos;re doing.&lt;/p&gt;</comment>
                            <comment id="250689" author="bougetq" created="Thu, 4 Jul 2019 17:12:46 +0000"  >&lt;p&gt;&amp;gt; What is your use case here?&lt;/p&gt;

&lt;p&gt;The migration of a single PFL component. But maybe that does not make sense...&lt;/p&gt;</comment>
                            <comment id="250690" author="pfarrell" created="Thu, 4 Jul 2019 17:38:35 +0000"  >&lt;p&gt;Actually, that makes perfect sense and it&apos;s a bit of feature functionality that we&apos;ve been interested in for a while.&lt;/p&gt;

&lt;p&gt;So as part of that, clearing those components of a layout makes sense, but not just as part as setting the pool name.&#160; There&apos;s no clear link to just setting the pool name.&#160; (You might want to migrate but not from pool to pool.)&lt;/p&gt;

&lt;p&gt;I think it would be better to - as part of your migrate operation, which of course has to also delete the old component &amp;amp; its associated objects - simply clear these fields explicitly in the new layout.&#160; Not link it in to some particular existing API call.&lt;/p&gt;

&lt;p&gt;So this is mostly obvious, but the migration would be something like (assuming locking similar to that used for full file migrate, maybe exactly the same):&lt;/p&gt;

&lt;p&gt;Create a copy of the existing file layout, in every detail including objects&lt;br/&gt;
 For the component you are migrating, in the new layout, convert that component to be your &apos;new&apos; component.&#160; This probably means removing anything that is only for instantiated components (the flag, the OST objects &amp;amp; object count, possibly some of the timestamps...?), and then replacing any fields you are changing with migration (like the pool name or stripe count).&lt;br/&gt;
 Apply this layout to a temp file.&#160; This will require some work because you have to make sure that connecting this new file to the existing objects works OK - This means basically providing that info from userspace.&#160; For the migrated component, you could leave it uninitialized or initialize it at this time.&#160; Both would require tweaking the existing code, but neither would be overwhelming.&#160; (I would argue for leaving it uninstantiated because you&apos;re about to write to it, and that mechanism can take care of it.&#160; This would require allowing gaps in layouts, at least temporarily...)&lt;br/&gt;
 Then, for the component you&apos;re migrating, do a data copy from the relevant region of the existing layout to the new one&lt;br/&gt;
 Then, and this is just a shortcut to make removing the migrated objects easily, you could remove all components from the existing layout except the one you migrated&lt;br/&gt;
 Write this layout back to the file&lt;/p&gt;

&lt;p&gt;rm the original file&lt;/p&gt;

&lt;p&gt;Rename the temp file, and component migrate is complete.&lt;/p&gt;

&lt;p&gt;Note the importance of removing the existing objects corresponding to the migrated component.&lt;/p&gt;</comment>
                            <comment id="250691" author="pfarrell" created="Thu, 4 Jul 2019 17:40:48 +0000"  >&lt;p&gt;I don&apos;t think we&apos;ve got a ticket for &quot;migrate a single component of a PFL/composite&quot; file yet.&#160; If you&apos;re interested in working on that, Quentin, maybe we could just make this that ticket?&#160; Like I said, I know various people are interested in this already.&lt;/p&gt;</comment>
                            <comment id="250694" author="adilger" created="Thu, 4 Jul 2019 19:31:30 +0000"  >&lt;p&gt;This is very similar to &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-9961&quot; title=&quot;FLR2: Relocating objects to a new OST&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-9961&quot;&gt;LU-9961&lt;/a&gt;, which is related to migrating a single object vs. migrating a single component that is described here.&lt;/p&gt;

&lt;p&gt;The main complexity that exists in all such approaches is that we cannot allow a user/client application to specify the exact OST objects to be put into the layout, since this could allow file data to be corrupted (connecting the wrong objects into a file or into multiple files), or allow file access to bypass permission checks (connecting to objects of a file the user does not have access permission on).&lt;/p&gt;

&lt;p&gt;Doing per-component rather than per-object migration seems relatively straight-forward.  We would need to allow adding a new &quot;mirror&quot; component to an existing file (marked &lt;tt&gt;STALE&lt;/tt&gt;) without adding a &lt;em&gt;full&lt;/em&gt; mirror, and using the existing &quot;&lt;tt&gt;lfs mirror resync&lt;/tt&gt;&quot; command to resync the new component, then remove the original component (verifying that the removed component has an exact extent start/end match with the new component.&lt;/p&gt;</comment>
                            <comment id="251398" author="bougetq" created="Mon, 15 Jul 2019 13:21:15 +0000"  >&lt;p&gt;Thank you for the detailed answers. We will keep them in mind in our future developments.&lt;br/&gt;
Since this item is not particularly high-priority for us, I doubt we implement it anytime soon though.&lt;/p&gt;

&lt;p&gt;Thanks again.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                            <outwardlinks description="is related to ">
                                        <issuelink>
            <issuekey id="48237">LU-9961</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i00j7b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>