Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-5603

Enable inline_data feature for Lustre

XMLWordPrintable

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • 15672

      Now, we found ldiskfs(Ext4) directory creation is much slower than file creation.

      Under RHEL7 with ldiskfs, we got following results.

      1. ./mdtest -d /mnt/test/ -n 100000 -i 3

      Directory creation: 52299.122 ops/second
      File creation: 106569 ops/second

      As we can see, directory creation performance is much slower than file creation, after doing some profiling, we found cost differences come from following calls:

      ->ext4_mkdir()
      ->ext4_init_new_dir()
      ->ext4_append()
      ->ext4_bread()
      ->ext4_getblk()
      ->ext4_map_blocks()
      So here an extra block allocation for '.' and '..' items will cost extra time, and we enable inline_data which will reduce block allocation for such case, things got better:

      Directory creation: 111276.454 ops/second
      File creation: 114920.338 ops/second

      As we can see, with inline_data enabled, directory creation performance is same as file creation.

      We also found inline_data feature not only help directory creation but also help
      directory reading, consider following case:

      1. creating 100M directories under test directory and then run 'time ls -R ./test'

      time cost reduce from 2m25s to 30s, huge differences! this is because inline_data will reduce an extra block allocation which also speed read performance as we don't have to do an extra block IO.

      In generally, Inline_data could improve performance and reduce space allocation which is also good.

      As now, inline_data is include in RHEL7 and there seems some conflicts with Lustre dirdata feature, but inline_data deserve us an eye, adding it to lustre will give us some improvement thought we still confirm it under lustre.

            dongyang Dongyang Li
            wangshilong Wang Shilong (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated: