<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 02:38:08 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-10782] Enable tiny write append for singly striped non-composite file</title>
                <link>https://jira.whamcloud.com/browse/LU-10782</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;Append and tiny writes are incompatible in general (see &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10681&quot; title=&quot;Disable tiny writes for O_APPEND&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10681&quot;&gt;&lt;del&gt;LU-10681&lt;/del&gt;&lt;/a&gt;) without doing a lot of expensive LDLM lock requests, which defeat the point of tiny writes, but there is one exception.&lt;/p&gt;

&lt;p&gt;Singly striped files with simple (non-composite) layouts can use it safely.&lt;/p&gt;

&lt;p&gt;Tiny writes depend on finding an already dirty page and using its existence as a guarantee that various conditions have been met.  Unfortunately, a page can be created without locks appropriate for protecting file size as required by append, so we cannot append safely in general.&lt;/p&gt;

&lt;p&gt;However, a singly striped file presents a special case.  When appending to a singly striped file, we must take an LDLM lock on the whole file.  If a partial page is created as part of this, we could mark such a page as &quot;created by append&quot;.  This page is protected by an append-appropriate LDLM lock, and as long as it exists, we know the LDLM lock still exists, so size is &quot;owned&quot; by this client.&lt;/p&gt;

&lt;p&gt;This requires knowing that the file meets the criteria.  For this purpose, a flag can be set in the inode indicating that it meets the striping criteria and that the most recent i/o should have created an appropriate page.  This flag is only advisory - If it is present, we will attempt to find an appropriate page, if it is not, we won&apos;t bother.  Correctness depends on checking the inode flag to confirm the layout is simple, and then checking the page level flag to verify the page was indeed created by an append operation.&lt;/p&gt;

&lt;p&gt;I believe it is not possible to convert a simple layout to a composite one without removing the inode, but if that&apos;s not the case, then it should be a simple matter of clearing the inode flag.  This will require some locking that is not done in the current version of the patch.&lt;/p&gt;</description>
                <environment></environment>
        <key id="51149">LU-10782</key>
            <summary>Enable tiny write append for singly striped non-composite file</summary>
                <type id="1" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11303&amp;avatarType=issuetype">Bug</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="1" iconUrl="https://jira.whamcloud.com/images/icons/statuses/open.png" description="The issue is open and ready for the assignee to start work on it.">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="wc-triage">WC Triage</assignee>
                                    <reporter username="paf">Patrick Farrell</reporter>
                        <labels>
                    </labels>
                <created>Tue, 6 Mar 2018 19:52:47 +0000</created>
                <updated>Mon, 8 Jan 2024 20:13:08 +0000</updated>
                                                                                <due></due>
                            <votes>0</votes>
                                    <watches>3</watches>
                                                                            <comments>
                            <comment id="222615" author="gerrit" created="Tue, 6 Mar 2018 19:59:31 +0000"  >&lt;p&gt;Patrick Farrell (paf@cray.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/31553&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/31553&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10782&quot; title=&quot;Enable tiny write append for singly striped non-composite file&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10782&quot;&gt;LU-10782&lt;/a&gt; llite: Simple layout append tiny write&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: ebd325799c96b783b737b57fcafb3250c2ac94fc&lt;/p&gt;</comment>
                            <comment id="222914" author="adilger" created="Fri, 9 Mar 2018 01:19:22 +0000"  >&lt;blockquote&gt;
&lt;p&gt;I believe it is not possible to convert a simple layout to a composite one without removing the inode, but if that&apos;s not the case, then it should be a simple matter of clearing the inode flag. This will require some locking that is not done in the current version of the patch.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The &quot;&lt;tt&gt;lfs migrate&lt;/tt&gt;&quot; code will do an atomic layout swap with a &quot;victim&quot; inode that got a copy of the file data, so the file layout may change without the inode being removed. That said, the layout lock would be revoked in that case, so you could just clear the &quot;append safe&quot; flag when the layout bit is cancelled, and re-set the &quot;append safe&quot; flag when the client gets the layout lock again and the layout has not changed.&lt;/p&gt;</comment>
                            <comment id="222923" author="paf" created="Fri, 9 Mar 2018 03:50:17 +0000"  >&lt;p&gt;Ah, I see, thanks for pointing that out...  So we&apos;d tie the append bit to layout lock cancellation. all right.&lt;/p&gt;

&lt;p&gt;Uh, what, if anything, would happen to dirty data on the original layout when the new one is brought in?  I believe migrate uses group locks, with the intention of excluding other i/o.  So I&apos;m thinking that if there were such data, it would just be lost - but there&apos;s no way for another process to get dirty data there unless I wrote my own thing that did migrate without a group lock.  Is that correct?&lt;/p&gt;</comment>
                            <comment id="253950" author="adilger" created="Fri, 30 Aug 2019 16:46:18 +0000"  >&lt;p&gt;With the advent of patch: &lt;a href=&quot;https://review.whamcloud.com/35617&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/35617&lt;/a&gt; &quot;&lt;tt&gt;&lt;a href=&quot;https://jira.whamcloud.com/browse/LU-9341&quot; title=&quot;PFL: append should not instantiate full layout&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-9341&quot;&gt;&lt;del&gt;LU-9341&lt;/del&gt;&lt;/a&gt; lod: Add special O_APPEND striping&lt;/tt&gt;&quot; it is likely that most files using &lt;tt&gt;O_APPEND&lt;/tt&gt; will now be 1-stripe plain files, so this optimization may be worthwhile to look at again.&lt;/p&gt;

&lt;p&gt;As for the question of append vs. migrate, I think that should be a non-issue.  The same problem of racing write vs. layout change exists for non-appending writes and is handled by migrate by checking whether the file data version (== hash of last transaction each object in the layout was modified in) has changed from the start of the migration to the end.  If there was any change to an object, the &lt;tt&gt;transno&lt;/tt&gt; stored with the object will be updated, and the data version will be changed, and the migration will fail during layout swap.&lt;/p&gt;</comment>
                            <comment id="398870" author="adilger" created="Mon, 8 Jan 2024 20:13:08 +0000"  >&lt;p&gt;Another possibility is to start using &quot;&lt;tt&gt;MDS_INODELOCK_DOM&lt;/tt&gt;&quot; to protect the append state (even if there is no DoM component on the file)?  That would avoid ping-pong of the OST DLM locks across OSTs, but still serialize the writers.  It would need agreement across all clients writing the file, so there might need to be a connect flag check if an old client writing to the file doesn&apos;t understand this feature, and then the MDS would revoke &lt;tt&gt;MDS_INODELOCK_DOM&lt;/tt&gt; from the client on behalf of the old client (which would need to get all of the OST locks if it was doing an append).&lt;/p&gt;

&lt;p&gt;Just an idea I&apos;m throwing into the ring, it is not fully baked, but might give us a path to something reasonable here.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="58668">LU-13420</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="45499">LU-9341</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzztz3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10060" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Severity</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10022"><![CDATA[3]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>