[LU-11213] DNE3: remote mkdir() in ROOT/ by default

Blocker — Mon, 7 Feb 2022 23:54:04 +0000

In order to start using multiple MDTs on a filesystem automatically, I wonder if it makes sense to start creating any directories under the filesystem ROOT/ directory remotely by default? This could be a client parameter that can be disabled if desired, but as can be seen in LU-11211 it is not very clear to users how to get multiple MDTs active by default. With OST object allocation we already do this by default for all objects.

While there is some overhead to creating remote directories, this top-level directories are best placed to allow maximal distribution across all of the MDTs. This would also work as new MDTs are added to the filesystem.

Using remote directories at the top level is a lot easier than trying to create a striped directory for ROOT/, since that would also need to change as new MDTs are added to the filesystem. It also allows more dynamic load balancing across MDTs.

As for implementation, the client tunable might be something like "llite.*.mdt_distribution_enabled" to turn it on/off, and "llite.*.mdt_distribution_seconds", which is how often the client will do obd_statfs() to the MDTs to get space/inode info, so that it doesn't need to do this for every create. Once per 5s would be the minimum, maybe as long as 60s?

The ROOT/ directory would get a flag like LUSTRE_TOPDIR_FL/LMAI_TOPDIR set when it is created that indicates the directory is a top-level directory and should have automatic remote directories created in it. This can be set manually by users on other directories with the "chattr +T" command, but it would not be inherited by subdirectories.

Using the same algorithms and tunables as LOV QOS for MDT selection would help leverage the same code, and allow us to improve the code together in the future.

Whamcloud Community JIRA

[LU-11213] DNE3: remote mkdir() in ROOT/ by default