Details
-
Improvement
-
Resolution: Unresolved
-
Major
-
None
-
Lustre 2.14.0, Lustre 2.12.4
-
9223372036854775807
Description
In order to allow draining an MDT for removal from the filesystem, or to disable it temporarily, the "mdt.*.no_create" parameter should be set on the MDT to let clients/MDS know that it should be skipped.
However, the no_create parameter currently only stops new directory creates on that MDT. If a client does try to create a file or directory on the MDT, it will currently be allowed.
Instead, the MDT should return an error like -EREMOTE if that would work with the existing clients, or preferably a unique code like -ENOANO or -EBADRQC that makes it clear the client should use a different MDT for creation. We should not use -EROFS to indicate the MDT cannot be used, since that would be returned to the application and the create would fail. We might also consider -ENOSPC to have the client try a different MDT, since we would also want the client to do this if the MDT was actually out of space.
Creating remote subdirectories to avoid a specific MDT was implemented in LU-12998, but avoiding the creation of new files or directory entries in an existing directory still needs to be implemented.
One solution would be to turn the directory into a striped directory with a shard on another MDT with the LMV_HASH_FLAG_MIGRATION flag set, and then create all new inodes and directory entries in the new MDT shard. That wouldn't affect existing inodes in the directory, but would prevent all new inodes from being allocated on that MDT in that directory. Something similar would need to be done during file migration anyway, so a later "lfs migrate -m" of that directory would "resume" the migration and move any remaining inodes on the disabled MDT.
Attachments
Issue Links
- Clones
-
-
- Resolved
-
Activity
| Link | New: This issue is related to LR-11 [ LR-11 ] |
| Summary | Original: DNE3: disable file creation on MDTs with no_create | New: DNE3: disable new regular file creation on MDTs mounted with 'no_create' |
| Labels | Original: medium | New: dne3 medium |
| Labels | Original: exap medium | New: medium |
| Description |
Original:
In order to allow draining an MDT for removal from the filesystem, or to disable it temporarily, it makes sense to add an "{{mdt.\*.no_create}}" parameter on the MDT similar to "{{obdfilter.\*.no_precreate}}" on the OST, and a matching mount option "{{-o no_create}}". The {{OS_STATE_NOPRECREATE}}" flag should be set in {{obd_statfs}} from the MDT to let clients/MDS know that it should be skipped.
This should result in new stripes directories skipping the MDT during selection for "{{lfs mkdir -i -1}}", auto striping, etc. |
New:
In order to allow draining an MDT for removal from the filesystem, or to disable it temporarily, the "{{mdt.\*.no_create}}" parameter should be set on the MDT to let clients/MDS know that it should be skipped.
However, the {{no_create}} parameter currently only stops new *directory* creates on that MDT. If a client does try to create a file or directory on the MDT, it will currently be allowed. Instead, the MDT should return an error like {{-EREMOTE}} if that would work with the existing clients, or preferably a unique code like {{-ENOANO}} or {{-EBADRQC}} that makes it clear the client should use a different MDT for creation. We should *not* use {{-EROFS}} to indicate the MDT cannot be used, since that would be returned to the application and the create would fail. We might also consider {{-ENOSPC}} to have the client try a different MDT, since we would also want the client to do this if the MDT was actually out of space. Creating remote subdirectories to avoid a specific MDT was implemented in One solution would be to turn the directory into a striped directory with a shard on another MDT with the {{LMV_HASH_FLAG_MIGRATION}} flag set, and then create all new inodes and directory entries in the new MDT shard. That wouldn't affect existing inodes in the directory, but would prevent all new inodes from being allocated on that MDT in that directory. Something similar would need to be done during file migration anyway, so a later "{{lfs migrate -m}}" of that directory would "resume" the migration and move any remaining inodes on the disabled MDT. |
| Link | New: This issue is related to DDN-2946 [ DDN-2946 ] |
| Link | New: This issue is related to DDN-2949 [ DDN-2949 ] |
| Link | New: This issue is related to DDN-3885 [ DDN-3885 ] |
| Link | New: This issue is related to BGI-167 [ BGI-167 ] |
| Link | New: This issue is related to EX-4480 [ EX-4480 ] |
| Link | New: This issue is related to DDN-3050 [ DDN-3050 ] |
| Link | New: This issue is related to EX-4481 [ EX-4481 ] |
| Link | New: This issue is related to DDN-3852 [ DDN-3852 ] |