Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-8709

parallel asynchronous readahead

Details

    • Improvement
    • Resolution: Duplicate
    • Minor
    • None
    • Lustre 2.10.0
    • 9223372036854775807

    Description

      Tracker for parallel async readahead improvement from DDN, as described in http://www.eofs.eu/_media/events/lad16/19_parallel_readahead_framework_li_xi.pdf

      Note that it would be very desirable in the single thread case if the copy_to_user was also handled in parallel, as this is a major CPU overhead on many-core systems and if it can be parallelized it may increase the peak read performance.

      As for lockahead integration with readahead, I agree that this is possible to do this, but it is only useful if the client doesn't get full-file extent locks. It would also be interesting if the write code detected sequential or strided writes and did lockahead at write time.

      Attachments

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. LU-8726 Do fake read page on OST to help read performance benchmarking

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. LU-6 Update the readahead logic in lustre 2.0

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-8413 sanity test_101f fails with 'misses too much pages!'

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. LU-8964 use parallel I/O to improve performance on machines with slow single thread performance

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-11416 Improve readahead for random read of small/medium files

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-12043 improve Lustre single thread read performances

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-8709] parallel asynchronous readahead
            adilger Andreas Dilger made changes -
            Resolution New: Duplicate [ 3 ]
            Status Original: Reopened [ 4 ] New: Resolved [ 5 ]
            adilger Andreas Dilger added a comment -

            Correct, with the caveat that LU-12043 patch is mainly improving single-threaded read performance. If there are many threads on the client the performance may not be very different, so it depends on your workload.

            adilger Andreas Dilger added a comment - Correct, with the caveat that LU-12043 patch is mainly improving single-threaded read performance. If there are many threads on the client the performance may not be very different, so it depends on your workload.
            lflis Lukasz Flis added a comment -

            Andreas, thank you for update, LU-12043 looks very promising.
            I had a quick look at changes and they are all on client side - can we assume that lustre-client 2.13 may have better performance on 2.10 servers ?

            lflis Lukasz Flis added a comment - Andreas, thank you for update, LU-12043 looks very promising. I had a quick look at changes and they are all on client side - can we assume that lustre-client 2.13 may have better performance on 2.10 servers ?
            adilger Andreas Dilger added a comment -

            Lukasz, the patch on this ticket is not currently being developed. The patch on LU-12043 is the one that will be landing on 2.13.

            adilger Andreas Dilger added a comment - Lukasz, the patch on this ticket is not currently being developed. The patch on LU-12043 is the one that will be landing on 2.13.
            lflis Lukasz Flis added a comment -

            We would like to give this patch a try in a mixed-workload environment on our test filesystem. Is it possible to get this patch for current b2_10 (2.10.7) ?

            lflis Lukasz Flis added a comment - We would like to give this patch a try in a mixed-workload environment on our test filesystem. Is it possible to get this patch for current b2_10 (2.10.7) ?
            pfarrell Patrick Farrell (Inactive) made changes -
            Link New: This issue is related to LU-12043 [ LU-12043 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-11416 [ LU-11416 ]
            sihara Shuichi Ihara made changes -
            Labels Original: Performance New: perf_optimization
            sihara Shuichi Ihara made changes -
            Labels New: Performance
            pjones Peter Jones made changes -
            Assignee Original: Li Xi [ lixi ] New: Li Xi [ lixi_wc ]

            People

              lixi_wc Li Xi
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: