Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-3321

2.x single thread/process throughput degraded from 1.8

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • Lustre 2.6.0
    • Lustre 2.4.0
    • Tested on 2.3.64 and 1.8.9 clients with 4 OSS x 3 - 32 GB OST ramdisks
    • 3
    • 8259

    Description

      Single thread/process throughput on tag 2.3.64 is degraded from 1.8.9 and significantly degraded when the client hits its caching limit (llite.*.max_cached_mb). Attached graph shows lnet stats sampled every second for a single dd writing 2 - 64 GB files followed by a dropping cache and reading the same two files. The tests were not done simultaenously but the graph has them starting from the same point. It also takes a significant amount of time to drop the cache on 2.3.64.

      Lustre 2.3.64
      Write (dd if=/dev/zero of=testfile bs=1M)
      68719476736 bytes (69 GB) copied, 110.459 s, 622 MB/s
      68719476736 bytes (69 GB) copied, 147.935 s, 465 MB/s

      Drop caches (echo 1 > /proc/sys/vm/drop_caches)
      real 0m43.075s

      Read (dd if=testfile of=/dev/null bs=1M)
      68719476736 bytes (69 GB) copied, 99.2963 s, 692 MB/s
      68719476736 bytes (69 GB) copied, 142.611 s, 482 MB/s

      Lustre 1.8.9
      Write (dd if=/dev/zero of=testfile bs=1M)
      68719476736 bytes (69 GB) copied, 63.3077 s, 1.1 GB/s
      68719476736 bytes (69 GB) copied, 67.4487 s, 1.0 GB/s

      Drop caches (echo 1 > /proc/sys/vm/drop_caches)
      real 0m9.189s

      Read (dd if=testfile of=/dev/null bs=1M)
      68719476736 bytes (69 GB) copied, 46.4591 s, 1.5 GB/s
      68719476736 bytes (69 GB) copied, 52.3635 s, 1.3 GB/s

      Attachments

        1. cpustat.scr
          0.5 kB
          Gregoire Pichon
        2. dd_throughput_comparison_with_change_5446.png
          7 kB
          Jeremy Filizetti
        3. dd_throughput_comparison.png
          6 kB
          Jeremy Filizetti
        4. lu-3321-singlethreadperf.tgz
          391 kB
          Gregoire Pichon
        5. lu-3321-singlethreadperf2.tgz
          564 kB
          Gregoire Pichon
        6. mcm8_wcd.png
          9 kB
          Jinshan Xiong
        7. perf3.png
          103 kB
          Jinshan Xiong

        Issue Links

          Activity

            People

              jay Jinshan Xiong (Inactive)
              jfilizetti Jeremy Filizetti
              Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: