<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 02:31:59 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-10092] PCC: Lustre Persistent Client Cache</title>
                <link>https://jira.whamcloud.com/browse/LU-10092</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;PCC is a new framework which provides a group of local cache on Lustre client side. No global namespace will be provided by PCC. Each client uses its own SSD as a local cache for it self. Local file system is used on the SSD to manage the data on local caches. Cached I/O is directed to local file system while normal I/O is directed to OSTs.&lt;/p&gt;

&lt;p&gt;PCC uses HSM for data synchronization. It uses HSM copytool to restore file from local caches to Lustre OSTs. Each PCC has a copytool instance running with unique archive number. Any remote access from another Lustre client would trigger the data synchronization. If a client with PCC goes offline, the cached data becomes inaccessible for other client temporally. And after the PCC client reboots and the copytool restarts, the data will be accessible again.&lt;/p&gt;

&lt;p&gt;Following is what will happen in different conditions:&lt;/p&gt;

&lt;p&gt;&#9658;&lt;b&gt;When file is being created on PCC&lt;/b&gt;&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;A normal file is created on MDT;&lt;/li&gt;
	&lt;li&gt;An empty mirror file is created on local cache;&lt;/li&gt;
	&lt;li&gt;The HSM status of the Lustre file will be set to archived and released;&lt;/li&gt;
	&lt;li&gt;The archive number will be set to the proper value.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#9658;&lt;b&gt;When file is being **prefetched&lt;/b&gt;* to PCC*&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;An mirror file is copied to local cache;&lt;/li&gt;
	&lt;li&gt;The HSM status of the Lustre file will be set to archived and released;&lt;/li&gt;
	&lt;li&gt;The archive number will be set to the proper value.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#9658;&lt;b&gt;When file is being accessed from PCC&lt;/b&gt;&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Data will be read directly from local cache;&lt;/li&gt;
	&lt;li&gt;Metadata will be read from MDT, except file size;&lt;/li&gt;
	&lt;li&gt;File size will be got from local cache.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;PCC should be able to accelerate some applications with certain I/O patterns.&lt;/p&gt;

&lt;p&gt;For more information, please check the presentation during LUG&apos;18 (&lt;a href=&quot;http://wiki.lustre.org/images/0/04/LUG2018-Lustre_Persistent_Client_Cache-Xi.pdf&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://wiki.lustre.org/images/0/04/LUG2018-Lustre_Persistent_Client_Cache-Xi.pdf&lt;/a&gt;).&lt;/p&gt;</description>
                <environment></environment>
        <key id="48631">LU-10092</key>
            <summary>PCC: Lustre Persistent Client Cache</summary>
                <type id="2" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11311&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="5" iconUrl="https://jira.whamcloud.com/images/icons/statuses/resolved.png" description="A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.">Resolved</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="1">Fixed</resolution>
                                        <assignee username="lixi_wc">Li Xi</assignee>
                                    <reporter username="lixi">Li Xi</reporter>
                        <labels>
                            <label>patch</label>
                    </labels>
                <created>Fri, 6 Oct 2017 07:23:05 +0000</created>
                <updated>Tue, 25 Aug 2020 22:18:23 +0000</updated>
                            <resolved>Fri, 30 Aug 2019 17:11:43 +0000</resolved>
                                                    <fixVersion>Lustre 2.13.0</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>27</watches>
                                                                            <comments>
                            <comment id="210473" author="gerrit" created="Fri, 6 Oct 2017 07:26:23 +0000"  >&lt;p&gt;Li Xi (lixi@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/29347&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/29347&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; llite: Add cache on client support&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: a79a5d8115bfcf240479b363695feae3bc06756c&lt;/p&gt;</comment>
                            <comment id="210518" author="jay" created="Fri, 6 Oct 2017 16:13:40 +0000"  >&lt;p&gt;The benchmark seems to be problematic because the bottleneck should be on the gigabit network. This is an alternative to local cachefs in kernel. Good work.&lt;/p&gt;</comment>
                            <comment id="210784" author="adilger" created="Wed, 11 Oct 2017 02:52:47 +0000"  >&lt;p&gt;I&apos;d long thought of implementing something similar using the cachefs in the kernel, but there was always the problem of how to manage cache coherency and data migration for files written into the client cache.&#160; Using the HSM infrastructure to do this makes a lot of sense, since this correctly handles the case of files written into the client cache first, and only migrated into the filesystem afterward.&#160; Using regular DLM locking is potentially problematic, since the client may be evicted before the cached file is flushed, but this is not a problem with &quot;restoring&quot; a file from the HSM archive.&lt;/p&gt;

&lt;p&gt;This potentially also integrate nicely with composite files and FLR if we enhanced the Lustre layout to include an &quot;HSM layout&quot; component (equivalent to LOV_MAGIC_V1).&#160; The &quot;LOV_MAGIC_HSM&quot; component describes a file in an HSM archive, storing the HSM archive number, &quot;UUID&quot; of the file within the archive, and other parameters (e.g. archive timestamp) needed to identify the file.&#160; The archive timestamp could be useful for storing multiple replicas of the file in HSM and using it for file versioning, along with the FLR &lt;tt&gt;mirror_io&lt;/tt&gt; equivalent to open up a specific component to access an older version of the file.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;A note on naming, the &quot;LCOC&quot; acronym might have some bad connotations in English, so it probably makes sense to rename this feature to something like &quot;Lustre Client Cache (LCC)&quot; or &quot;Persistent Client Cache (PCC)&quot; or similar.&lt;/p&gt;</comment>
                            <comment id="210837" author="simmonsja" created="Wed, 11 Oct 2017 17:27:23 +0000"  >&lt;p&gt;Doesn&apos;t this overlap with the linux kernel DAX api?&lt;/p&gt;</comment>
                            <comment id="210919" author="lixi" created="Thu, 12 Oct 2017 02:08:24 +0000"  >&lt;p&gt;Hi Andreas,&lt;/p&gt;

&lt;p&gt;Thanks for you advice. We will change the name to PCC for better description.&lt;/p&gt;

&lt;p&gt;And integrate FLR with PCC sounds great.&lt;/p&gt;</comment>
                            <comment id="210920" author="lixi" created="Thu, 12 Oct 2017 02:09:51 +0000"  >&lt;p&gt;Hi James,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I have limited knowlege about Linux Kernel DAX, but I think DAX requires a specific kind of block device which supports DAX? We are thinking of using any kind of SSD/NVME/DISK to accelerate Lustre.&lt;/p&gt;</comment>
                            <comment id="211354" author="adilger" created="Wed, 18 Oct 2017 01:35:55 +0000"  >&lt;p&gt;It seems possible that the same PCC mechanism could be used as a way of managing data stored in NVRAM and accessed via DAX.  This wouldn&apos;t work for SSD/NVMe storage on the client, but that is OK as well.  Lustre would have to prestage/migrate data from the backend storage into the PCC (&lt;tt&gt;lfs hsm archive&lt;/tt&gt;) so that it can be accessed via low-latency IO operations (DAX/mmap), then flushed back to the backing storage when it is no longer in use.&lt;/p&gt;</comment>
                            <comment id="214563" author="lixi" created="Fri, 24 Nov 2017 02:50:50 +0000"  >&lt;p&gt;Hi Andreas,&lt;/p&gt;

&lt;p&gt;Thanks for reviewing the patch! It is really helpful!&lt;/p&gt;

&lt;p&gt;To discuss on your comments in the patch, I feel it would be better to post some of my thoughts here.&lt;/p&gt;

&lt;p&gt;&amp;gt; do we want to go directly to mounting whole filesystem images that are accessed directly on the client, or should the &quot;automatic mount of ext4 filesystem image&quot; be a separate (optional) feature of files stored in Lustre (regardless of PCC or remote)&lt;/p&gt;

&lt;p&gt;I feel that should be in a seperate patch.Without &quot;automatic mount of ext4 filesystem image&quot;, the current implementation can alreay accelerate some applications which do a lot of single client write/read.&lt;/p&gt;

&lt;p&gt;And also, another thing that I would like to do is to implement a patch which improves the stat/getattr() performance by using PCC. Currently, the patch will not cache any attributes of file in Lustre, which means, file stat performance will not be able to accelerated. In the next patch, the attribute updates will be applied first to Lustre MDT and then to local file system, thus attributes can read directly from cache which should be able to accelerate the performance of&#160;reading metadata.&lt;/p&gt;

&lt;p&gt;But indeed &quot;automatic mount of ext4 filesystem image&quot;&#160; would be a fancy feature which will accelerates all data and metadata operations, which should be our final goal.&lt;/p&gt;

&lt;p&gt;&amp;gt; is PCC duplicating fscache/cachefs/cachefiles too much, and we should look at using those to avoid code duplication and ease upstream acceptance, or is PCC faster and more flexible for our use?&lt;/p&gt;

&lt;p&gt;I agree it would be a concern when pushing the patch to upstream Linux kernel. But I don&apos;t think PCC duplicates fscache/cachefs/cachefiles. The PCC code only has 1500 lines including user space tool.&#160;By contrat fs-cache has almost 6000 lines, and in order to integrate NFS with fs-cache, another 1000 lines are needed. Most of the codes of PCC are only Lustre related. And no cache management or&#160;indexing mechanism is introduced into PCC either. PCC codes only connects the existing interfaces of Lustre HSM and&#160; the existing file system methods (struct file_operations etc.) of local file system together, using very limited externtion of inode in memory. I think the current implementation is almost the simplest implementation of local cache possible. That means, if we want to combine PCC with fscache or bcache or any other kind of mechanism, the implementation would be much more complicated, and more likely less efficient, and thus might be more difficult to be accepted by upstream Linux kernel. And it is hard to integrate PCC with fs-cache (or other cache machanism like bcache or so), because PCC is managing cache in granularity of files, yet fs-cache is managing cache in pages.&lt;/p&gt;

&lt;p&gt;I am not a expert of fs-cache. But judging from a introduction (&lt;a href=&quot;http://people.redhat.com/~dhowells/fscache/FS-Cache.pdf),&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://people.redhat.com/~dhowells/fscache/FS-Cache.pdf&lt;/a&gt;) and the codes, I feel fs-cache is less efficient than PCC. PCC is writing and reading (and getattr() in the later versions) directly to/from local file system which could be highly optimized. Instead, fs-cache manages cached data in pages. That means, in order to lookup local cache of file data, indexing of pages are needed. And searching indexes (through radix tree) will certainly introduce overhead. Also, PCC can be used on top of any kind of local file systems, which means PCC is able to utilize the newest techonologies of local file systems for NVME/SSD, which I think is another advantage over fs-cache.&lt;/p&gt;</comment>
                            <comment id="214713" author="simmonsja" created="Mon, 27 Nov 2017 18:41:32 +0000"  >&lt;p&gt;I don&apos;t think its a fair comparison for the line count between PCC and fscache since PCC depends on a lot of Lustre infrastructure already present. A more fair comparison is if you were to create a functional PCC layer that could work with any file system. In that case I suspect more code would be involved. If PCC is truly a more efficient solution than fs-cache I would recommend making a general implementation that would work with ext4 as a example and push it upstream to the VFS maintainers. From your last sentence it appears to be the case that PCC can be truly independent of Lustre. This would boost DDN + lustre standing with the linux kernel community as well as getting back valuable feed back. Also by having it apart of the core kernel code it would be properly maintained instead you managing it until the end of time with an out of source tree. Please consider this approach. I would gladly help with the process of pushing it upstream, especially since I test newer kernels all the time.&lt;/p&gt;</comment>
                            <comment id="214894" author="wangshilong" created="Wed, 29 Nov 2017 02:17:56 +0000"  >&lt;p&gt;I just wrote some idea of mine, Generally speaking, i think PCC is different from FSCache, and i think it hard&lt;br/&gt;
to make PCC truly independent of Lustre.&lt;/p&gt;

&lt;p&gt;I implement a simple version of FScache support for Lustre, see it from:&lt;br/&gt;
&lt;a href=&quot;https://github.com/wangshilong/lustre/tree/master_fscache&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;FScache for Lustre&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I haven&apos;t got benchmark for it, but here is what i learned already,&lt;br/&gt;
FsCache could be page range, that mean it could cache partial of file which is good advantage vs PCC, Generally, fscache use backend filesystem layout and hook read/write, it will use bmap to check whether one page cached etc.&lt;/p&gt;

&lt;p&gt;Here is some limit to use Fscache:&lt;br/&gt;
1) only buffer read.&lt;br/&gt;
2) any write will make file cache invalid.&lt;br/&gt;
3)double space usage for Cache, and write to cachefiles will happen during first cache miss.&lt;/p&gt;

&lt;p&gt;FsCache is desgined as Generic, but limitation is obvious, compared to that, core desgin of PCC&lt;br/&gt;
Rely on Lustre HSM, and it hook read/write directly to Lustre read/write function, compared to&lt;br/&gt;
Fscache, advantange will be:&lt;/p&gt;

&lt;p&gt;1)both read/write, buffer or direct IO could be easily support.&lt;br/&gt;
2)only one copy of space will be consumed.&lt;br/&gt;
3)read performance will be a little better than FsCache, due to shorter IO path compared to FsCache.&lt;/p&gt;

&lt;p&gt;But PCC also have Limitation:&lt;br/&gt;
1)Do not support partial file cache compared to FsCache&lt;br/&gt;
2)Number of clients limited to 32 due to HSM limitation.&lt;br/&gt;
3)Different Lustre clients could not do parallel cache read for one file?&lt;/p&gt;</comment>
                            <comment id="214919" author="adilger" created="Wed, 29 Nov 2017 08:54:21 +0000"  >&lt;p&gt;I think the 32-archive limit could be removed from Lustre fairly easily. Instead, there would probably be a limit of 2^32-1 archives.&lt;/p&gt;

&lt;p&gt;I also discussed the multi-client cache with Li Xi in Denver.&#160; One of the things I&apos;ve been thinking about for HSM for a while is to move the &quot;hsm&quot; xattr (see &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-7207&quot; title=&quot;HSM: Add Archive UUID to delete changelog records&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-7207&quot;&gt;LU-7207&lt;/a&gt;) to be part of a composite file layout as a new type of component (e.g. &lt;tt&gt;LOV_MAGIC_HSM&lt;/tt&gt; or similar).&#160; That would allow a file to have many different read-only mirrors of the data in different archives, though at most one would be writable.&#160; When the client is doing the read, it could compare the Archive ID with the local Archive ID to see which of the mirrors it should be reading from.&lt;/p&gt;

&lt;p&gt;In a similar way, it would be possible to have an HSM component that didn&apos;t cover the whole file, if the file was very large.&#160; The HSM component could have a specific&#160;&lt;tt&gt;extent_start&lt;/tt&gt; and &lt;tt&gt;extent_end&lt;/tt&gt; like any other component.&#160; This would also be useful for regular HSM archives if e.g. the file is larger than what could fit onto a single tape, or if it is desirable to archive different parts of the file in parallel.&#160; It wouldn&apos;t allow page-granular cache of files in PCC, but that wouldn&apos;t make much sense for Lustre anyway - it would be better to have each PCC handle a large chunk of the file and use collective IO to map the IO processes to the appropriate client node.&lt;/p&gt;</comment>
                            <comment id="216195" author="jay" created="Wed, 13 Dec 2017 20:58:22 +0000"  >&lt;p&gt;Hi Li Xi,&lt;/p&gt;

&lt;p&gt;Is the I/O pattern detector that mentioned in your slides open-sourced?&lt;/p&gt;</comment>
                            <comment id="218066" author="lixi" created="Fri, 12 Jan 2018 02:12:19 +0000"  >&lt;p&gt;Hi Jinshan,&lt;/p&gt;

&lt;p&gt;Sorry for my late reply. The I/O pattern detector is still under design and development. We are trying to implement some kind of file heat detector in Lustre client...&lt;/p&gt;</comment>
                            <comment id="218068" author="gerrit" created="Fri, 12 Jan 2018 02:38:39 +0000"  >&lt;p&gt;Li Xi (lixi@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/30844&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/30844&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; llite: improve stat using pcc&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 42f212ac303917959c6cb1bd4c4c7e4a230741cb&lt;/p&gt;</comment>
                            <comment id="221370" author="jhammond" created="Wed, 21 Feb 2018 17:15:24 +0000"  >&lt;p&gt;How will clients avoid choosing conflicting archive ids?&lt;/p&gt;</comment>
                            <comment id="221413" author="adilger" created="Thu, 22 Feb 2018 03:34:18 +0000"  >&lt;p&gt;I think the archive ID should be assigned by the MDS initially (lctl interface to generate a new ID), and then be assigned to the archive permanently. It should not relate to the NID of the client. With 32-bit IDs that shouldn&apos;t be too limiting, and they can be re-used if needed (round-robin).&lt;/p&gt;</comment>
                            <comment id="221588" author="jhammond" created="Fri, 23 Feb 2018 17:43:39 +0000"  >&lt;p&gt;Sounds like kind of a headache for this use case.&lt;/p&gt;</comment>
                            <comment id="229946" author="gerrit" created="Thu, 5 Jul 2018 07:42:17 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/32787&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/32787&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Non-blocking PCC caching&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: f8f1af1a6e60bee60d9f3561baa3d477ba18a39f&lt;/p&gt;</comment>
                            <comment id="229947" author="qian" created="Thu, 5 Jul 2018 07:50:18 +0000"  >&lt;p&gt;Current PCC uses refcount of PCC inode to determine whether a previous PCC-attached file can be detached. If a file is being used (refcount &amp;gt; 1), the detaching will return -EBUSY.&lt;br/&gt;
 Each open on the PCC-cached file will increase the refcount of the PCC inode; Each close on the PCC-cached file will decrease the refcount of the PCC inode;&lt;br/&gt;
 When another client accesses the PCC-cached file, it will trigger the restore progress as the file is HSM released. During restore, the Agent needs to detach the PCC-cached file.&lt;br/&gt;
 Thus, if a PCC-attached file is keeping opened but not closed for a long time, the restore request will always return failure. It is unacceptable for some application.&lt;br/&gt;
 In this patch, we implement a non-blocking PCC caching mechanism for Lustre. After attach the file into PCC, the client acquires the layout lock for the file, and the layout generation is maintained in the PCC inode. Under the layout lock protection, the PCC caching state is valid and all I/O are directing into PCC caching. When the layout lock is revoked, in the blocking AST it will invalidate the PCC caching state and detach the file automatically.&lt;br/&gt;
 This patch is also helpful to handle the ENOSPC error for PCC write by fallback to normal I/O path which will restore the file data into OSTs (as the file is in HSM released state) and redo the write again.&lt;/p&gt;</comment>
                            <comment id="231688" author="gerrit" created="Thu, 9 Aug 2018 02:44:31 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/32963&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/32963&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; llite: Add persistent cache on client&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 3c5dacbf218146df42cf822c90d911c7b08e3cfa&lt;/p&gt;</comment>
                            <comment id="231693" author="gerrit" created="Thu, 9 Aug 2018 05:58:02 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/32965&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/32965&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Use lease lock with intent to attach a file&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 20739c4806e7f43f538b3cf0e1aa0a86849a70b8&lt;/p&gt;</comment>
                            <comment id="231694" author="gerrit" created="Thu, 9 Aug 2018 06:20:35 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/32966&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/32966&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Non-blocking PCC caching&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: ab2e4429eb2f035d5ad30825ee3c548430ff0e4f&lt;/p&gt;</comment>
                            <comment id="231711" author="simmonsja" created="Thu, 9 Aug 2018 14:54:23 +0000"  >&lt;p&gt;Since you are going to land this even with its collision with other kernel features I suggest that you only enable this when CONFIG_FSCACHE and CONFIG_FS_DAX are both turn off. Otherwise this could cause all kinds of havoc on deployed systems. Sadly you can&apos;t force those features off when being out of tree.&lt;/p&gt;</comment>
                            <comment id="231720" author="paf" created="Thu, 9 Aug 2018 16:26:07 +0000"  >&lt;p&gt;James,&lt;/p&gt;

&lt;p&gt;Would you explain how this conflicts with those?&#160; Unless I&apos;ve really missed something, it has no runtime conflicts, and while it&apos;s got &lt;b&gt;some&lt;/b&gt; similarities/overlap, it also offers significantly different functionality.&lt;/p&gt;

&lt;p&gt;Given that it doesn&apos;t appear to have runtime conflicts, only enabling this when those are turned off seems pointlessly restrictive.&lt;/p&gt;</comment>
                            <comment id="231729" author="simmonsja" created="Thu, 9 Aug 2018 17:47:47 +0000"  >&lt;p&gt;My concern is that if lustre runs in an environment with other file systems that does use fscache or dax. Since fscache / dax are external subsystem they should be managing the node local storage between the different classes of file systems so they don&apos;t stomp on each other. I can see a case where PCC comes in and just smashes all over the local storage device. Does that make sense to you or maybe you have a far greater understanding of fscache and dax to ease my concern with potential conflicts with other files systems.&lt;/p&gt;

&lt;p&gt;Second pushing this into the linux kernel when other solution exist might be a hard sell. Will DDN live with this work never going mainstream? Maybe they will never care.&lt;/p&gt;</comment>
                            <comment id="231744" author="paf" created="Thu, 9 Aug 2018 18:25:49 +0000"  >&lt;p&gt;Huh?&#160; There are plenty of ways to configure your file systems so one stomps on another.&#160; DAX and fscache add some new ones, but there are plenty already.&#160; Setting up PCC to use a storage device that it shouldn&apos;t is no more (or less) dangerous than doing that with any other file system...?&#160; PCC requires formatting the local storage device anyway!&#160; So it&apos;s exactly as dangerous as anything else.&lt;/p&gt;

&lt;p&gt;That second issue is a separate one.&#160; To determine whether or not it&apos;s upstreamable as is would require a detailed analysis of offered functionality.&#160; Certainly, PCC has a lot of Lustre integration that fscache and DAX obviously don&apos;t have.&#160; In fact, other than it&apos;s usable as a cache, I don&apos;t see that much in common...?&lt;/p&gt;</comment>
                            <comment id="231747" author="simmonsja" created="Thu, 9 Aug 2018 18:52:10 +0000"  >&lt;p&gt;Excellent, you seem to have a much better grasp than me on those potential conflicts. Since that is the case could you provide detail&#160;documentation on what potential conflicts could exist. This way sites can avoid potential data corruption. Also they should be made aware that no solutions to avoid these conflicts are on the table at this time.&lt;/p&gt;

&lt;p&gt;Also since you seem to have a very good understanding of DAX, FSCache and this new PCC it would be nice if you take up the mantle and do the detail analysis.&#160;&lt;/p&gt;</comment>
                            <comment id="231758" author="paf" created="Thu, 9 Aug 2018 19:11:18 +0000"  >&lt;p&gt;There is no need - Unless I&apos;ve badly misunderstood, PCC requires &lt;b&gt;reformatting the block device it will use&lt;/b&gt;, in a manner exactly like any other file system.&#160; The implications of this are well understood and shared with any other file system.&#160; It has no special interaction with DAX or fscache.&#160; They&apos;re just other things that could potentially be used - well, fscache, anyway, I don&apos;t think DAX is, I believe it&apos;s not a caching layer - to achieve some of the same goals.&#160; But they have no direct interaction with PCC.&lt;/p&gt;

&lt;p&gt;There is no complex relationship to unpack here.&lt;/p&gt;</comment>
                            <comment id="237321" author="gerrit" created="Wed, 21 Nov 2018 08:40:19 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/33698&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33698&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Non-blocking PCC caching&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: db4610b8734e33e0d52d5d024799e8e6d0504b83&lt;/p&gt;</comment>
                            <comment id="237975" author="gerrit" created="Wed, 5 Dec 2018 03:24:17 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/33787&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33787&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: auto attach during open for valid cache&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 6125ce9802fee1866c81670b7a7f41de4d2e1088&lt;/p&gt;</comment>
                            <comment id="238511" author="gerrit" created="Thu, 13 Dec 2018 03:02:36 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/33844&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33844&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Add manual and remove options for detach&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 211495b07fe808a1b832e1d66354f6d50b2d4510&lt;/p&gt;</comment>
                            <comment id="239516" author="gerrit" created="Tue, 8 Jan 2019 02:38:33 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/33982&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33982&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Add RW-PCC support for non-root user&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 588cc597a9260c933b3750cb2b58ef740f69db3f&lt;/p&gt;</comment>
                            <comment id="242996" author="gerrit" created="Thu, 28 Feb 2019 03:39:18 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/34341&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/34341&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Add a new connect flag for PCC&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 57689020c3b68666274e5528ef74f69519595366&lt;/p&gt;</comment>
                            <comment id="243137" author="gerrit" created="Fri, 1 Mar 2019 07:23:55 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/34356&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/34356&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Reserve a new connection flag for PCC&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 26dd48459dbe3c108ead5964065d4f33ea262427&lt;/p&gt;</comment>
                            <comment id="245555" author="gerrit" created="Thu, 11 Apr 2019 02:50:55 +0000"  >&lt;p&gt;Yingjin Qian (qian@ddn.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/34637&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/34637&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: security and permission for non-root user access&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 07867e40b8dbd242146c304049e72f87e3bab664&lt;/p&gt;</comment>
                            <comment id="246040" author="gerrit" created="Thu, 18 Apr 2019 22:09:44 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/34356/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/34356/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Reserve a new connection flag for PCC&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 93aa684046699a1d8802524003115ebaf07758ca&lt;/p&gt;</comment>
                            <comment id="249177" author="gerrit" created="Thu, 13 Jun 2019 04:29:13 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/32963/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/32963/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; llite: Add persistent cache on client&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: f172b116885753d0f316549a2fb9d451e9b4bd2e&lt;/p&gt;</comment>
                            <comment id="249178" author="gerrit" created="Thu, 13 Jun 2019 04:32:05 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/32966/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/32966/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: Non-blocking PCC caching&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 58d744e3eaab358ef346e51ff4aa17e9f08efbb3&lt;/p&gt;</comment>
                            <comment id="249179" author="gerrit" created="Thu, 13 Jun 2019 04:33:01 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/34637/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/34637/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: security and permission for non-root user access&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 2102c86e0d0ae735aed9ee8c1c6a77b63eda6037&lt;/p&gt;</comment>
                            <comment id="249181" author="gerrit" created="Thu, 13 Jun 2019 04:35:07 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/33787/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33787/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: auto attach during open for valid cache&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: e29ecb659e51dd67758c5b0adb542210e7aeddb1&lt;/p&gt;</comment>
                            <comment id="249182" author="gerrit" created="Thu, 13 Jun 2019 04:36:11 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/33844/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33844/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; pcc: change detach behavior and add keep option&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: pcc&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 2dadefb4148f753dd93ee1dbebb3aac49bda2f8d&lt;/p&gt;</comment>
                            <comment id="249183" author="gerrit" created="Thu, 13 Jun 2019 04:37:59 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/35214&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/35214&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; First phase of persistent client cache project merging&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 478f97b212714bc3af9a9a104efab314ca942758&lt;/p&gt;</comment>
                            <comment id="249184" author="gerrit" created="Thu, 13 Jun 2019 05:58:16 +0000"  >&lt;p&gt;Oleg Drokin (green@whamcloud.com) merged in patch &lt;a href=&quot;https://review.whamcloud.com/35214/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/35214/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-10092&quot; title=&quot;PCC: Lustre Persistent Client Cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-10092&quot;&gt;&lt;del&gt;LU-10092&lt;/del&gt;&lt;/a&gt; First phase of persistent client cache project merging&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 478f97b212714bc3af9a9a104efab314ca942758&lt;/p&gt;</comment>
                            <comment id="249206" author="simmonsja" created="Thu, 13 Jun 2019 14:42:28 +0000"  >&lt;p&gt;Is it ready for testing yet?&lt;/p&gt;</comment>
                            <comment id="253952" author="jgmitter" created="Fri, 30 Aug 2019 17:11:43 +0000"  >&lt;p&gt;All patches targeted for V1 have been landed.  Patches being worked on for V2 are moved to &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-12714&quot; title=&quot;PCC Phase 2&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-12714&quot;&gt;LU-12714&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10120">
                    <name>Blocker</name>
                                            <outwardlinks description="is blocking">
                                                        </outwardlinks>
                                                                <inwardlinks description="is blocked by">
                                        <issuelink>
            <issuekey id="50196">LU-10499</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10323">
                    <name>Gantt End to End</name>
                                            <outwardlinks description="has to be finished together with">
                                        <issuelink>
            <issuekey id="54726">LUDOC-432</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="32303">LU-7207</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="57803">LU-13137</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="50522">LU-10602</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="50588">LU-10606</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="53214">LU-11333</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="51845">LU-10918</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="60486">LU-13924</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="48689">LU-10114</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="33972" name="LUG2018-Lustre_Persistent_Client_Cache-Xi.pdf" size="3432308" author="adilger" created="Thu, 5 Dec 2019 19:56:18 +0000"/>
                    </attachments>
                <subtasks>
                            <subtask id="54727">LU-11908</subtask>
                    </subtasks>
                <customfields>
                                                                                                                                                                <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzzlgn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>