Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14851

Enlarge OST_MAX_PRECREATE

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • Lustre 2.15.0
    • None
    • 9223372036854775807

      There is the metadata performance limit due to hard-corded OST_MAX_PRECREATE=20000 in lustre_idl.h.
      Here is test results.

      # mdtest -n 1000000 -u -d /TC/dir -x /TC/mdtest-easy.stonewall -C -W 120 -a POSIX -F -w 0 -Y (0 byte file)
      # mdtest -n 1000000 -u -d /TC/dir -x /TC/mdtest-easy.stonewall -C -W 120 -a POSIX -F -w 4096 -Y (4096 byte file)
      
        1xOST 2xOST
      -w 0 (0 byte file) 39,741 64608
      -w 4096 (4096 byte file) 20,864 43,824

      It's not big problem if there are many OSTs in the Lustre. However, if the metadata server has good enough performance, but the number of OST are limited, the file creation rate hits a limit which is hard-corded max pre-creation rate per OST (OST_MAX_PRECREATE=20000).
      According to "osp.*.create_count" stats, the rate is hitting upper limit (20000) and "osp.*.create_count" is not allowed to change.
      Also, if the directory is striped and would create many files into it, OST objects need to be created into all OSTs and we will hit same performance limitation.

            wc-triage WC Triage
            sihara Shuichi Ihara
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: