[LU-13417] DNE3: mkdir() automatically create remote directory on MDS which has more space

• Type: Improvement
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.15.0
• Affects Version/s: Lustre 2.13.0
• Labels: dne3
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Andreas Dilger

Description

Since the patches from LU-11213 landed for 2.13.0, I thought "lfs setdirstripe -i -1 /mnt/lustre" on the root or other existing directory would allow creation of remote directories on other MDTs using plain "mkdir" commands. This is different than the case of "lfs setdirstripe -i -1 -c N /mnt/lustre" selecting stripes on less-full MDTs that was landed via patch https://review.whamcloud.com/35825 "LU-12624 lod: alloc dir stripes by QoS", but this patch also removed the "space" hash, so I thought that regular mkdir of a directory could be allowed to balance across MDTs?

However, I can't seem to get this to work. On current master (2.13.52-259, just before 2.13.53) I'm not able to use "lfs setdirstripe -i -1 /path/to/dir" on an existing directory. It seems to select the less-full MDT if I explicitly run "lfs mkdir -i -1" for a new directory, but that was also true in 2.12 using patch https://review.whamcloud.com/30598 "LU-10277 utils: 'lfs mkdir -i -1' pick the less full MDTs", so it isn't clear how to enable the LU-11213 functionality to balance directories across MDTs?

There should be a way for "mkdir(2)" from a normal application (not "lfs mkdir -i -1") to be able to create remote (1-stripe) directories in the filesystem, and it should be possible to set this by default on the root directory (per LU-11213). This is critical for being able to use multiple MDTs effectively without users knowing the details of how to configure striped/remote directories manually, or be forced to set all directories as striped (unwelcome due to performance overhead).

The default mdt_qos_threshold_rr value should be reduced significantly (e.g. 1-2% and/or modified so that some amount of MDT balancing is active when the filesystem is balanced, at least in the root directory by default. Otherwise, without users understanding the details of DNE MDT0000 will hold all of the inodes, when it would be better if the top 1 or 2 levels of directories should be distributed across MDTs.

Maybe this is mostly a documentation issue, and the "lfs-setdirstripe.1" man page needs to be updated to be more clear so I can understand what needs to be done to enable this? (also the usage message for setdirstripe/mkdir should remove the "This can only be done on MDT0 with the right of administrator" message.)

Issue Links

• is duplicated by: LU-10784 DNE3: mkdir() automatically create remote directory on MDS which has more space
• is related to: LU-17300 Avoid creating new dir/file/object on newly added MDT/OST
• is related to: LU-13439 DNE3: MDT QOS tuning to avoid full MDTs completely
• is related to: LU-14898 sanity test_413a: (max - min) * 100 / min: division by 0 (error token is "min")
• is related to: LU-14909 LU-13417 patch breaks few recovery tests
• is related to: LU-17334 Client should handle dir/file/object created on newly added MDT/OST
• is related to: LU-18343 interop: sanity test_160a: FAIL: mkdir parent '/mnt/lustre/d160a.sanity/pics/2008' failed
• is related to: LU-14792 DNE3: enable filesystem-wide default LMV
• is related to: LU-15856 "lfs setdirstripe -D ... <dir>" should create directory if missing
• is related to: LU-13440 DNE3: limit directory default layout inheritance
• is related to: LU-13560 'lfs mkdir -i N' should be 'sticky' on a particular MDT
• is related to: LU-15216 improve MDT QOS space balance
• is related to: LU-11025 DNE3: directory restripe
• is related to: LU-12624 DNE3: striped directory allocate stripes by QoS
• is related to: LU-11213 DNE3: remote mkdir() in ROOT/ by default
• is related to: LU-15850 MDT QOS should always be used for round-robin directories.

Comments

[Gerrit Updater]

Andreas Dilger ([e-mail]) merged in patch https://review.whamcloud.com/44384/
Subject: LU-13417 test: generate uneven MDTs early for sanity 413
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 233344d451e567c71726bcb071f45cf8f1c6ef3e

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44384
Subject: LU-13417 test: generate uneven MDTs early for sanity 413
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: efa31b02f470cf90281504ca49185a37006523d8

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44315
Subject: LU-13417 test: mkdir_on_mdt0() in more tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2bd9a10b17db3df7f87b4068a3746bc114069b46

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43492/
Subject: LU-13417 test: use mkdir_on_mdt0() in replay-dual
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ce179e97767936ff76282fd06df063b386851fe7

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43491/
Subject: LU-13417 test: use mkdir_on_mdt0() in misc tests
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: de62c8c7ef5d627da872260686d9279cbb60736e

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43489/
Subject: LU-13417 test: add mkdir_on_mdt0()
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 54fb8458db0bff4fdfe42ba7476de3129d7606cd

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43492
Subject: LU-13417 test: use mkdir_on_mdt0() in replay-dual
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4196abb88570a008839c9f249b80bb1c357c2723

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43491
Subject: LU-13417 test: use mkdir_on_mdt0() in misc tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8ea0f5eb5fe0e9e08ed44a2e805c704e4f40d581

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43489
Subject: LU-13417 test: add mkdir_on_mdt0()
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b84ff3e11e3f3c064d5e257b13d21ce80053e4df

[Lai Siyao]

It's strange replay-dual 22d always failed, but can pass on autotest for other patches.

When I tried to test replay-dual alone on master code, it also failed on autotest.

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/40925
Subject: LU-13417 test: dump replay-dual 22d debug log
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: fedcbc7741ad5928b3c2f40a910c2126f37f060c