https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16479 Lustre <p>We have the obdfilter.testfs-OST000*.degraded value set/unset by zedlets (/etc/zfs/zed.d/statechange-lustre.sh) based on zpool being DEGRADED/ONLINE, We'd like to have this behavior enabled/disabled through an option so that we have I/O or newer allocations to DEGRADED OST's as well and hence there is no degradation in net bandwidth of the filesystem due to the degraded OSTs.<br/>  <br/> Introduce a new Lustre-specific ZFS dataset user property (lustre:autodegrade=on|off) for this purpose. Update the Lustre zedlet and also extend the mkfs.lustre utility to add this property by default when creating a new Lustre server(only for ZFS OSTs). The default behavior would remain the same (lustre:autodegrade=on) which disables new allocations to DEGRADED OSTs.<br/> Creating a user property has a few advantages:</p> <ol> <li>User properties are a generic ZFS feature and won't be interpreted by ZFS itself. No ZFS changes are needed.</li> <li>The property can be set per dataset providing more granularity.</li> <li>The property is persistent and will survive reboots.</li> </ol> Lustre filesystem with ZFS as backend filesystem for OST&#39;s. LU-16479

Automatically manage/control DEGRADED ZFS OST's

Improvement Minor Resolved Fixed Akash B Akash B Mon, 16 Jan 2023 10:36:36 +0000 Fri, 17 Feb 2023 13:19:50 +0000 Tue, 14 Feb 2023 06:26:42 +0000 Upstream Lustre 2.15.0 Lustre 2.16.0 0 7 <p>"Akash B &lt;akash-b@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49660" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49660</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16479" title="Automatically manage/control DEGRADED ZFS OST&#39;s" class="issue-link" data-issue-key="LU-16479"><del>LU-16479</del></a> utils: Add option to manage degraded ZFS OST<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ccf612d181952744e1dd13a842c02bb28ba181d1</p> <p>Hi Akash,<br/> What motivated this? Did you see disk failures during testing that affected results unnecessarily?<br/> thanks</p> <p>Hi Olaf,</p> <p>By default for ZFS OSTs, the lustre zedlet(statechange-lustre.sh) does the automatic manipulation of Lustre degraded state. While writing to multiple OSS nodes where one or two OSTs are in a degraded state (<tt>D seen in lfs df -h at the client side</tt>), reduces the overall bandwidth(performance degradation) of the filesystem.</p> <p>We'd want to have this behavior enabled/disabled through an option. Talked to Brian Behlendorf about this behavior for having this as a user property which administrators can use to enable/disable this behavior. </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Hey Akash,   On occasion we've also wanted to be able to administratively disable <span class="code-keyword">this</span> behavior, supporting <span class="code-keyword">this</span> sounds great.   I agree that splitting portions of the patch between Lustre and ZFS is awkward.  It<span class="code-quote">'d be nice to handle it entirely on the Lustre side.  An alternate solution would be to introduce a <span class="code-keyword">new</span> Lustre specific ZFS dataset user property.  As I'</span>m sure you've noticed Lustre already adds the following dataset properties which are used <span class="code-keyword">for</span> configuration   kern3/ost1  lustre:flags           4130                   local  kern3/ost1  lustre:fsname         lslide                 local kern3/ost1  lustre:version        1                      local kern3/ost1  lustre:mgsnode        7@kfi:9@kfi              local kern3/ost1  lustre:index          0                      local kern3/ost1  lustre:failover.node  21@kfi:3@kfi             local kern3/ost1  lustre:svname         lslide-OST0000         local   We could add a <span class="code-keyword">new</span> lustre:autodegrade=&lt;on|off&gt; user property (See "User Properties in zfsprops(7)).  The statechange-lustre.sh zedlet could then check <span class="code-keyword">this</span> property on the dataset to control the behavior.  This has a few advantages: 1. User properties are a <span class="code-keyword">generic</span> ZFS feature and won't be interpreted by ZFS itself.  No ZFS changes are needed. 2. The property can be set per dataset providing more granularity. 3. The property is persistent and will survive reboots.   4. This mechanism is already used within the zedlet to identify Lustre datasets; e.g.: ZFS get -rH -s local -t filesystem -o name lustre:svname ${ZEVENT_POOL} 5. You can add the property at any time to an existing MDT/OST What <span class="code-keyword">do</span> you think?  If you were to implement <span class="code-keyword">this</span> I'd suggest not only updating the Lustre zedlet, but also extending the lustre.mkfs utility to add <span class="code-keyword">this</span> property by <span class="code-keyword">default</span> when creating a <span class="code-keyword">new</span> Lustre server.   Thanks, Brian </pre> </div></div> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/49660/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/49660/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16479" title="Automatically manage/control DEGRADED ZFS OST&#39;s" class="issue-link" data-issue-key="LU-16479"><del>LU-16479</del></a> utils: Add option to manage degraded ZFS OST<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: a2de6af65d21bff0d9357c30e6eb4ba049ff2059</p> Development Epic server zfs Epic/Theme zfs Rank 1|i03a47: Rank (Obsolete) 9223372036854775807