<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 03:24:20 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-16138] remove need for T10-PI block integrity patches from server kernel</title>
                <link>https://jira.whamcloud.com/browse/LU-16138</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;Kernels since &lt;a href=&quot;https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0f8087ecdeac&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;v4.3-rc4-30-g0f8087ecdeac&lt;/a&gt; separate the block integrity verify and generate functions into a separate &lt;tt&gt;struct blk_integrity_profile&lt;/tt&gt; instead of copying the pointers into &lt;tt&gt;struct blk_integrity&lt;/tt&gt;.  This might be used to facilitate replacing the &lt;tt&gt;blk_integrity_profile&lt;/tt&gt; pointer on the block device for the duration that osd-ldiskfs is mounting it, and potentially avoid the need to patch the kernel for T10-PI at all.&lt;/p&gt;

&lt;p&gt;As a starting point, we would need to call &lt;tt&gt;blk_integrity_register()&lt;/tt&gt; using a template holding the integrity methods provided by &lt;tt&gt;osd-ldiskfs&lt;/tt&gt; - &lt;tt&gt;osd_dif_type&amp;lt;1,3&amp;gt;&lt;em&gt;generate&lt;/em&gt;&amp;lt;crc,ip&amp;gt;()&lt;/tt&gt; and &lt;tt&gt;osd_dif_type&amp;lt;1,3&amp;gt;&lt;em&gt;verify&lt;/em&gt;&amp;lt;crc,ip&amp;gt;()&lt;/tt&gt;.  It looks like this function is just copying the individual fields from the provided template into the disk queue&apos;s &lt;tt&gt;blk_integrity&lt;/tt&gt; fields, so we should save the original &lt;tt&gt;blk_integrity&lt;/tt&gt; to be restored when osdldiskfs unmounts the target, otherwise the kernel would crash trying to access the old function pointers if there is any block device IO after &lt;tt&gt;osd-ldiskfs&lt;/tt&gt; is removed from the kernel. &lt;/p&gt;

&lt;p&gt;It seems prudent to call &lt;tt&gt;blk_integrity_unregister()&lt;/tt&gt; before unloading the module, as that would force all block device IO using the &lt;tt&gt;osd-ldiskfs.ko&lt;/tt&gt; methods to be finished, and then &lt;tt&gt;blk_integrity_register()&lt;/tt&gt; the saved values so that T10-PI is still functional on the device.  &lt;/p&gt;

&lt;p&gt;The &lt;tt&gt;osd_dif_generate()&lt;/tt&gt; and &lt;tt&gt;osd_dif_verify()&lt;/tt&gt; functions need to handle being called on block read/write calls that are unrelated to osd-ldiskfs BRW IOs, since that will now happen for &lt;b&gt;every&lt;/b&gt; block in the filesystem whether or not it came from osd-ldiskfs.  One way to separate OSD vs. non-OSD IO might be by putting a magic at the end of &lt;tt&gt;struct osd_bio_private&lt;/tt&gt; that can be used to reliably detect if this is a BRW IO or if it is IO on some other ldiskfs block.  Non-BRW blocks would return NULL from &lt;tt&gt;find_lnb()&lt;/tt&gt; and not dereference &lt;tt&gt;bio_private&lt;/tt&gt; if NULL, and it looks like the rest of the code will &quot;just work&quot;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="72213">LU-16138</key>
            <summary>remove need for T10-PI block integrity patches from server kernel</summary>
                <type id="4" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11310&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.whamcloud.com/images/icons/priorities/major.svg">Major</priority>
                        <status id="4" iconUrl="https://jira.whamcloud.com/images/icons/statuses/reopened.png" description="This issue was once resolved, but the resolution was deemed incorrect. From here issues are either marked assigned or resolved.">Reopened</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="dongyang">Dongyang Li</assignee>
                                    <reporter username="adilger">Andreas Dilger</reporter>
                        <labels>
                            <label>medium</label>
                    </labels>
                <created>Tue, 6 Sep 2022 21:24:49 +0000</created>
                <updated>Mon, 2 Oct 2023 23:38:02 +0000</updated>
                                                            <fixVersion>Lustre 2.16.0</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>5</watches>
                                                                            <comments>
                            <comment id="347240" author="gerrit" created="Tue, 20 Sep 2022 18:21:02 +0000"  >&lt;p&gt;&quot;Jian Yu &amp;lt;yujian@whamcloud.com&amp;gt;&quot; uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/48608&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/48608&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-16138&quot; title=&quot;remove need for T10-PI block integrity patches from server kernel&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-16138&quot;&gt;LU-16138&lt;/a&gt; kernel: preserve RHEL8.x server kABI for block integrity&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: ec555c17c0254662c5886188dd2f7c0447fd4aef&lt;/p&gt;</comment>
                            <comment id="347241" author="gerrit" created="Tue, 20 Sep 2022 18:31:18 +0000"  >&lt;p&gt;&quot;Jian Yu &amp;lt;yujian@whamcloud.com&amp;gt;&quot; uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/48609&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/48609&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-16138&quot; title=&quot;remove need for T10-PI block integrity patches from server kernel&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-16138&quot;&gt;LU-16138&lt;/a&gt; kernel: preserve RHEL8.x server kABI for block integrity&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: b2_15&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: aba52c68e750710f9b799b87d7e03221b3d1e52a&lt;/p&gt;</comment>
                            <comment id="349079" author="gerrit" created="Mon, 10 Oct 2022 05:36:41 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/c/fs/lustre-release/+/48608/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/c/fs/lustre-release/+/48608/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-16138&quot; title=&quot;remove need for T10-PI block integrity patches from server kernel&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-16138&quot;&gt;LU-16138&lt;/a&gt; kernel: preserve RHEL8.x server kABI for block integrity&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: b90c0100dd93b56f3bfaee037b3bdd077523f43e&lt;/p&gt;</comment>
                            <comment id="349106" author="pjones" created="Mon, 10 Oct 2022 12:44:22 +0000"  >&lt;p&gt;Landed for 2.16&lt;/p&gt;</comment>
                            <comment id="349145" author="yujian" created="Mon, 10 Oct 2022 16:28:53 +0000"  >&lt;p&gt;Have to reopen this issue because removing the block integrity patches has not been done.&lt;br/&gt;
The kABI issue is resolved.&lt;/p&gt;</comment>
                            <comment id="350804" author="gerrit" created="Wed, 26 Oct 2022 04:42:04 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/c/fs/lustre-release/+/48609/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/c/fs/lustre-release/+/48609/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-16138&quot; title=&quot;remove need for T10-PI block integrity patches from server kernel&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-16138&quot;&gt;LU-16138&lt;/a&gt; kernel: preserve RHEL8.x server kABI for block integrity&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: b2_15&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: f582cd01fba4a32c6861a3be56fb320c69376064&lt;/p&gt;</comment>
                            <comment id="356913" author="dongyang" created="Tue, 20 Dec 2022 00:15:04 +0000"  >&lt;p&gt;Andreas, I think patching the kernel is inevitable. We could register our own integrity_profile, or we could save the generate/verify functions and replace them with our versions, then restore them when umount osd.&lt;/p&gt;

&lt;p&gt;But without at least applying block-pass-bio-into-integrity_processing_fn the struct blk_integrity_exchg/blk_integrity_iter doesn&apos;t know which bio are we working on, without the bio we could not access the struct osd_bio_private, and could not use find_lnb() to get the guard buffers.&lt;/p&gt;</comment>
                            <comment id="357066" author="adilger" created="Wed, 21 Dec 2022 01:30:41 +0000"  >&lt;p&gt;Some potential options for discussion:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;could we use &quot;&lt;tt&gt;container_of(iter, struct bio, bi_iter)&lt;/tt&gt;&quot; to access the bio?  It isn&apos;t very obvious that this would work, because there look like lots of &quot;struct bvec_iter&quot; copies being made and we aren&apos;t sure to be using the one in the bio.&lt;/li&gt;
	&lt;li&gt;we could use the hole in &lt;tt&gt;struct blk_integrity_iter&lt;/tt&gt; between &quot;&lt;tt&gt;interval/tuple_size&lt;/tt&gt;&quot; and &quot;&lt;tt&gt;disk_name&lt;/tt&gt;&quot; directly without patching the kernel.  In el8 this hole is 2 bytes (used for &lt;tt&gt;bi_idx&lt;/tt&gt; in the current patch), but in newer kernels it is only 1 byte.  If we had a maximum of 255 OST threads we could use this byte as an index into an array to get per-thread data, but there can be more than 255 threads&lt;/li&gt;
	&lt;li&gt;&lt;tt&gt;iter.prot_buf&lt;/tt&gt; is constant for the whole call to &lt;tt&gt;bio_integrity_process()&lt;/tt&gt; and could potentially be used as a hash key to look up the private data?&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="357073" author="dongyang" created="Wed, 21 Dec 2022 02:16:04 +0000"  >&lt;p&gt;Looking up &lt;tt&gt;bio&lt;/tt&gt; using the &lt;tt&gt;bi_iter&lt;/tt&gt; won&apos;t work, we are looking at different iter. the &lt;tt&gt;generate/verify_fn&lt;/tt&gt; is using &lt;tt&gt;struct blk_integrity_iter&lt;/tt&gt; and &lt;tt&gt;bio&lt;/tt&gt; has &lt;tt&gt;struct bvec_iter&lt;/tt&gt;. The &lt;tt&gt;blk_integrity_iter&lt;/tt&gt; is also declared on stack of &lt;tt&gt;bio_integrity_process()&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;One challenge of using &lt;tt&gt;iter.prot_buf&lt;/tt&gt; as a hash key with unpatched kernel is the buf is allocated by &lt;tt&gt;bio_integrity_prep()&lt;/tt&gt;, and &lt;tt&gt;bio_integrity_prep()&lt;/tt&gt; will generate the guards for the write case immediately,&lt;br/&gt;
so our &lt;tt&gt;generate_fn&lt;/tt&gt; will see the &lt;tt&gt;iter.prot_buf&lt;/tt&gt; before we setup the private data?&lt;/p&gt;</comment>
                            <comment id="357176" author="adilger" created="Wed, 21 Dec 2022 21:21:11 +0000"  >&lt;p&gt;OK, what about just having a per-device hash table using the device offsets?  The &lt;tt&gt;generate_fn&lt;/tt&gt; is at least called with &quot;&lt;tt&gt;sector_t seed&lt;/tt&gt;&quot; and &quot;&lt;tt&gt;disk_name&lt;/tt&gt;&quot; that could be mapped to the &lt;tt&gt;niobuf&lt;/tt&gt;.  It definitely isn&apos;t ideal, but could potentially work and we could see the overhead of that method vs. patching the kernel.&lt;/p&gt;

&lt;p&gt;I also see that there is a &quot;&lt;tt&gt;prepare_fn&lt;/tt&gt;&quot; callback in the &lt;tt&gt;blk_integrity_profile&lt;/tt&gt; on the device (e.g. &lt;tt&gt;t10_pi_type1_ip.t10_pi_type1_prepare&lt;/tt&gt;) that has access to the &lt;tt&gt;bio&lt;/tt&gt;, which might be helpful.&lt;/p&gt;</comment>
                            <comment id="357278" author="dongyang" created="Thu, 22 Dec 2022 23:43:16 +0000"  >&lt;p&gt;The per-device hash table could work, if the sector won&apos;t be remapped after we &lt;tt&gt;submit_bio()&lt;/tt&gt;.&lt;br/&gt;
I&apos;m trying to find what else we can make use of on the unpatched kernel to map the &lt;tt&gt;prot_buf&lt;/tt&gt; to &lt;tt&gt;niobuf&lt;/tt&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="72196">LU-16137</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="73682">LU-16413</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i02zan:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>