Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-2476

poor OST file creation rate performance with zfs backend

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Won't Fix
    • Blocker
    • None
    • Lustre 2.4.0
    • RHEL 6.2
    • 2828

    Description

      We observe poor file creation rate performance with a zfs backend. The attached jpeg was generated using the createmany benchmark creating 1 million files in a single directory. Files created per second were reported for every 10,000 files created. The green line shows results for Lustre 2.1 and ldiskfs backend. The red line was with ZFS on Orion.

      The benchmark was run on the same hardware for both ldisks and ZFS:

      MDT: 24 GB RAM, 8 200GB SSD drives in two external SAS-2 enclosures
      (Linux MD-RAID10 for ldiskfs, 1 zpool with 8 mirrored pairs for zfs)
      OSS: 2 OSS nodes (3 8TB* OSTs each for ldiskfs, 1 72TB OST each for ZFS)
      OST: Netapp 60 drive enclosure, 6 24 TB RAID6 LUNs, 3 TB SAS drives, dual 4X QDR IB connections to each OST
      Network: 20 Gb/sec 4X DDR IB

      *LUNs were partitioned for ldiskfs for compatibility reasons

      Attachments

        1. creates.jpg
          creates.jpg
          64 kB
        2. zfs.jpg
          zfs.jpg
          45 kB
        3. llog-mds-zwicky-1-slow
          89 kB
        4. aplot.eps
          310 kB
        5. bplot.eps
          466 kB
        6. dmu-api-patch-createmany-results.ps
          23 kB
        7. osp_object_create.png
          osp_object_create.png
          128 kB
        8. clients.png
          clients.png
          11 kB
        9. mdstats.png
          mdstats.png
          7 kB
        10. ori434-run1.tar.bz2
          62 kB
        11. mdtest-arcdata.png
          mdtest-arcdata.png
          29 kB

        Issue Links

          Activity

            People

              bzzz Alex Zhuravlev
              nedbass Ned Bass (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: