https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12998 Lustre <p>In order to allow draining an MDT for removal from the filesystem, or to disable it temporarily, it makes sense to add an "<tt>mdt.&#42;.no_create</tt>" parameter on the MDT similar to "<tt>obdfilter.&#42;.no_precreate</tt>" on the OST, and a matching mount option "<tt>-o no_create</tt>". The <tt>OS_STATE_NOPRECREATE</tt>" flag should be set in <tt>obd_statfs</tt> from the MDT to let clients/MDS know that it should be skipped. </p> <p>This should result in new stripes directories skipping the MDT during selection for "<tt>lfs mkdir -i -1</tt>", auto striping, etc. </p> LU-12998

DNE3: tunable to disable directory creation on MDT

Improvement Major Resolved Fixed Andreas Dilger Andreas Dilger medium Fri, 22 Nov 2019 04:21:25 +0000 Thu, 25 Jan 2024 08:36:54 +0000 Sun, 19 Nov 2023 00:38:36 +0000 Lustre 2.14.0 Lustre 2.12.4 Lustre 2.16.0 0 8 <p>The <tt>obdfilter.&#42;.no_precreate</tt> functionality was added for OSTs in patch <a href="https://review.whamcloud.com/35029" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35029</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-12025" title="Adding OST may cause EIO - delay activation of new OSTs on existing filesystem" class="issue-link" data-issue-key="LU-12025"><del>LU-12025</del></a> osp: allow OS_STATE_* flags from OSTs</tt>" and patch <a href="https://review.whamcloud.com/36716" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36716</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-12036" title="Add option to create new OSTs inactive" class="issue-link" data-issue-key="LU-12036"><del>LU-12036</del></a> ofd: add 'no_precreate' mount option</tt>".</p> <p>It would probably be best to use "<tt>no_create</tt>" for the mount option and parameter on the MDT, since "<tt>no_precreate</tt>" doesn't make much sense for the MDT. For consistency, the OST code should <b>also</b> add a "<tt>no_create</tt>" mount option and parameter, and rename <tt>OS_STATFS_NOPRECREATE</tt> to <tt>OS_STATFS_NOCREATE</tt>, and then deprecate the "<tt>no_precreate</tt>" option, since it was only added to master in commit 2.14.0, and the b2_12 backport patch <a href="https://review.whamcloud.com/37133" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37133</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-12036" title="Add option to create new OSTs inactive" class="issue-link" data-issue-key="LU-12036"><del>LU-12036</del></a> ofd: add "no_precreate" mount option</tt>" has not landed, though it appears the backport patch <a href="https://review.whamcloud.com/36872" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36872</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-12025" title="Adding OST may cause EIO - delay activation of new OSTs on existing filesystem" class="issue-link" data-issue-key="LU-12025"><del>LU-12025</del></a> osp: allow OS_STATE_* flags from OSTs</tt>" was included in 2.12.4.</p> <p>Another important issue that needs to be addressed is how the <tt>no_create</tt> state is returned to the clients? With patch "<tt><a href="https://jira.whamcloud.com/browse/LU-13440" title="DNE3: limit directory default layout inheritance" class="issue-link" data-issue-key="LU-13440"><del>LU-13440</del></a> lmv: add default LMV inherit depth</tt>" having the clients use the MDT <tt>OBD_STATFS</tt> data to make space balancing decisions, returning <tt>OS_STATFS_NOCREATE</tt> to the client can definitely influence the client MDT selection, but I'd think that <tt>no_create=1</tt> would be a hard restriction against MDT usage (e.g. draining the MDT).</p> <p>As such, if a client <b>does</b> try to create a file or directory on the MDT, the MDT should return an error (<tt>-EREMOTE</tt> if that would work with the existing clients, or preferably a unique code like <tt>-ENOANO</tt> or <tt>-EBADRQC</tt>) that makes it clear the client should use a different MDT for creation. We <em>shouldn't</em> use <tt>-EROFS</tt> to indicate the MDT cannot be used, since that would be returned to the application and the create would fail. We might also consider <tt>-ENOSPC</tt> to have the client try a different MDT, since we would also want the client to do this if the MDT was actually out of space. </p> <p>Creating remote subdirectories to avoid a specific MDT is fairly easy today, but avoiding the creation of files or directory entries <b>in</b> an existing directory is much harder, and we might consider that as a second stage. If the initial goal is to prevent "new" use of an MDT, the existing directory would not necessarily count, and if MDT evacuation is the goal then some external action would be needed to migrate the existing directories off the MDT, so the added files could also be migrated at that time.</p> <p>A more complete solution would be to turn the directory into a striped directory with a shard on another MDT with the <tt>LMV_HASH_FLAG_MIGRATION</tt> flag set, and then create <b>all</b> new inodes on the new MDT shard (both files and directories). That wouldn't affect existing inodes in the directory, but would prevent all new inodes from being allocated on that MDT in that directory. Something similar would need to be done during file migration anyway, so a later "<tt>lfs migrate -m</tt>" would "resume" the migration and move any remaining inodes on the disabled MDT.</p> <p>"Andreas Dilger &lt;adilger@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47124" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47124</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12998" title="DNE3: tunable to disable directory creation on MDT" class="issue-link" data-issue-key="LU-12998"><del>LU-12998</del></a> mds: add no_create parameter to stop creates<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 21b06ef06a84f248fd4a592d2cd46bc7874883f2</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/47124/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/47124/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12998" title="DNE3: tunable to disable directory creation on MDT" class="issue-link" data-issue-key="LU-12998"><del>LU-12998</del></a> mds: add no_create parameter to stop creates<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 1dbcd0bab881fac38d8a5e4ef1559f12618f8f0e</p> <p>The disabling of new files on the MDT is tracked under <a href="https://jira.whamcloud.com/browse/LU-17299" title="DNE3: disable new regular file creation on MDTs mounted with &#39;no_create&#39;" class="issue-link" data-issue-key="LU-17299">LU-17299</a>. </p> <p>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/53437" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/53437</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12998" title="DNE3: tunable to disable directory creation on MDT" class="issue-link" data-issue-key="LU-12998"><del>LU-12998</del></a> lod: statfs upon nocreate check<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 066262a04cb8e0cbf49a20b7bf036d4484399afe</p> Cloners LU-17299 Related LU-12025 LU-16024 LU-7668 LU-17300 LU-17334 Development Epic Link EX-4481 Rank 1|i00psn: Rank (Obsolete) 9223372036854775807