[LU-14851] Enlarge OST_MAX_PRECREATE

Wed, 14 Jul 2021 04:19:52 +0000

There is the metadata performance limit due to hard-corded OST_MAX_PRECREATE=20000 in lustre_idl.h.
Here is test results.

# mdtest -n 1000000 -u -d /TC/dir -x /TC/mdtest-easy.stonewall -C -W 120 -a POSIX -F -w 0 -Y (0 byte file)
# mdtest -n 1000000 -u -d /TC/dir -x /TC/mdtest-easy.stonewall -C -W 120 -a POSIX -F -w 4096 -Y (4096 byte file)

	1xOST	2xOST
-w 0 (0 byte file)	39,741	64608
-w 4096 (4096 byte file)	20,864	43,824

It's not big problem if there are many OSTs in the Lustre. However, if the metadata server has good enough performance, but the number of OST are limited, the file creation rate hits a limit which is hard-corded max pre-creation rate per OST (OST_MAX_PRECREATE=20000).
According to "osp.*.create_count" stats, the rate is hitting upper limit (20000) and "osp.*.create_count" is not allowed to change.
Also, if the directory is striped and would create many files into it, OST objects need to be created into all OSTs and we will hit same performance limitation.

Whamcloud Community JIRA

[LU-14851] Enlarge OST_MAX_PRECREATE