Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-8964

use parallel I/O to improve performance on machines with slow single thread performance

Details

    • New Feature
    • Resolution: Duplicate
    • Major
    • None
    • None
    • None
    • 9223372036854775807

    Description

      On machines with slow single thread performance like KNL the bottleneck of I/O performance moved into code which just copy memory from one buffer to other (from user space to kernel or vice versa). In current Lustre implementation all I/O performs in single thread and this is become an issue for KNL. Significantly improve performance can be with solution which do parallel memory transfer of large buffers.

       

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. LU-1056 Single-client, single-thread and single-file is limited at 1.5GB/s

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-11069 ifort lseek returns wrong position on lustre 2.10.3

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-10367 FIO Fails to run with libaio

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-9618 Connect readahead to prep_partial_page to improve small (< 1 page) write performance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-11825 Remove LU-8964/pio feature & supporting framework

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-12043 improve Lustre single thread read performances

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. LU-6658 single stream write performance improvement with worker threads in llite

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-8709 parallel asynchronous readahead

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-8964] use parallel I/O to improve performance on machines with slow single thread performance
            spitzcor Cory Spitz added a comment -

            simmonsja, for the record, you mean LU-12043. LU-12403 is "add e2fsprog support for RHEL-8".

            spitzcor Cory Spitz added a comment - simmonsja , for the record, you mean LU-12043 . LU-12403 is "add e2fsprog support for RHEL-8".
            simmonsja James A Simmons made changes -
            Resolution New: Duplicate [ 3 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            simmonsja James A Simmons added a comment -

            LU-12403 will do this work correctly.

            simmonsja James A Simmons added a comment - LU-12403 will do this work correctly.
            simmonsja James A Simmons made changes -
            Link New: This issue is related to LU-12043 [ LU-12043 ]
            simmonsja James A Simmons made changes -
            Status Original: In Progress [ 3 ] New: Open [ 1 ]
            simmonsja James A Simmons made changes -
            Link New: This issue is related to LU-11825 [ LU-11825 ]
            simmonsja James A Simmons added a comment -

            Thanks Patrick for the heads up on ktask. I will be watching it closely and give it a spin under this ticket.

            simmonsja James A Simmons added a comment - Thanks Patrick for the heads up on ktask. I will be watching it closely and give it a spin under this ticket.
            simmonsja James A Simmons made changes -
            Assignee Original: Dmitry Eremin [ dmiter ] New: James A Simmons [ simmonsja ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-10367 [ LU-10367 ]
            jhammond John Hammond made changes -
            Link New: This issue is related to LU-11069 [ LU-11069 ]

            People

              simmonsja James A Simmons
              dmiter Dmitry Eremin (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              26 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: