Details
-
Improvement
-
Resolution: Unresolved
-
Minor
-
None
-
Lustre 2.15.0
-
None
-
9223372036854775807
Description
There is the metadata performance limit due to hard-corded OST_MAX_PRECREATE=20000 in lustre_idl.h.
Here is test results.
# mdtest -n 1000000 -u -d /TC/dir -x /TC/mdtest-easy.stonewall -C -W 120 -a POSIX -F -w 0 -Y (0 byte file) # mdtest -n 1000000 -u -d /TC/dir -x /TC/mdtest-easy.stonewall -C -W 120 -a POSIX -F -w 4096 -Y (4096 byte file)
| 1xOST | 2xOST | |
|---|---|---|
| -w 0 (0 byte file) | 39,741 | 64608 |
| -w 4096 (4096 byte file) | 20,864 | 43,824 |
It's not big problem if there are many OSTs in the Lustre. However, if the metadata server has good enough performance, but the number of OST are limited, the file creation rate hits a limit which is hard-corded max pre-creation rate per OST (OST_MAX_PRECREATE=20000).
According to "osp.*.create_count" stats, the rate is hitting upper limit (20000) and "osp.*.create_count" is not allowed to change.
Also, if the directory is striped and would create many files into it, OST objects need to be created into all OSTs and we will hit same performance limitation.