Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-11775

buffered write single client improvements

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • Lustre 2.13.0
    • None
    • None
    • 9223372036854775807

    Description

      After landing LU-9906 the read performance improved to an acceptable level. Write performance also increased but still needs some improvements to reach the same level. This ticket tracks the work.

      Attachments

        1. master.svg
          275 kB
          Dongyang Li
        2. master+LU-11775.svg
          306 kB
          Dongyang Li
        3. master+LU-11775+LU-9920.svg
          320 kB
          Dongyang Li

        Issue Links

          Activity

            [LU-11775] buffered write single client improvements
            dongyang Dongyang Li added a comment -

            No more patches and I think this ticket can be closed now

            dongyang Dongyang Li added a comment - No more patches and I think this ticket can be closed now

            Dongyang, is there still work to be done for this ticket? It looks like the patches are included in 2.13.

            adilger Andreas Dilger added a comment - Dongyang, is there still work to be done for this ticket? It looks like the patches are included in 2.13.

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33861/
            Subject: LU-11775 obdclass: protect imp_sec using rwlock_t
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 8ed361345154138473707855b2030bad371e96dd

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33861/ Subject: LU-11775 obdclass: protect imp_sec using rwlock_t Project: fs/lustre-release Branch: master Current Patch Set: Commit: 8ed361345154138473707855b2030bad371e96dd

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33858/
            Subject: LU-11775 osc: reduce lock contention in osc_unreserve_grant
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 8a1ae45a3e4f828346c890bf4ccacbd68ea6752c

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33858/ Subject: LU-11775 osc: reduce lock contention in osc_unreserve_grant Project: fs/lustre-release Branch: master Current Patch Set: Commit: 8a1ae45a3e4f828346c890bf4ccacbd68ea6752c

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33860/
            Subject: LU-11775 osc: check if opg is in lru list without locking
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: b3af0798682b24d0f39424a7db12c9710a0e64b4

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33860/ Subject: LU-11775 osc: check if opg is in lru list without locking Project: fs/lustre-release Branch: master Current Patch Set: Commit: b3af0798682b24d0f39424a7db12c9710a0e64b4

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33859/
            Subject: LU-11775 osc: reduce atomic ops in osc_enter_cache_try
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 8b364fbd6bd9e0088440e6d6837861a641b923a0

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33859/ Subject: LU-11775 osc: reduce atomic ops in osc_enter_cache_try Project: fs/lustre-release Branch: master Current Patch Set: Commit: 8b364fbd6bd9e0088440e6d6837861a641b923a0

            master:
            Max Write: 15237.92 MiB/sec (15978.12 MB/sec)

            master + LU-11775:
            Max Write: 17045.11 MiB/sec (17873.09 MB/sec)

            master + LU-11775 + LU-9920:
            Max Write: 17476.77 MiB/sec (18325.72 MB/sec)

            This information should go into the commit comments of the various patches, like https://review.whamcloud.com/28667

            adilger Andreas Dilger added a comment - master: Max Write: 15237.92 MiB/sec (15978.12 MB/sec) master + LU-11775 : Max Write: 17045.11 MiB/sec (17873.09 MB/sec) master + LU-11775 + LU-9920 : Max Write: 17476.77 MiB/sec (18325.72 MB/sec) This information should go into the commit comments of the various patches, like https://review.whamcloud.com/28667
            dongyang Dongyang Li added a comment -

            Patrick, thanks for the nice tip!

            Yes I've been using noinline directive but apparently it did not work.

            I will give it a shot when I can get to the environment, and provide an update on LU-9920 as well.

            Cheers

            DY

            dongyang Dongyang Li added a comment - Patrick, thanks for the nice tip! Yes I've been using noinline directive but apparently it did not work. I will give it a shot when I can get to the environment, and provide an update on LU-9920 as well. Cheers DY

            Li,

            Looking at your graphics, I think maybe osc_lru_use has been inlined in to osc_page_cache_add.  I haven't dug in to "LU-11775 osc: check if opg is in lru list without locking" yet, but I don't think it would stop us from hitting that lock entirely.

            If you'd like to avoid inlining, the easiest way I've found is calling that function via a function pointer.  Then you could see it easily in your traces again.  (There are various "don't inline" directives, but I've never been able to get them to work when building Lustre.)

            pfarrell Patrick Farrell (Inactive) added a comment - Li, Looking at your graphics, I think maybe osc_lru_use has been inlined in to osc_page_cache_add.  I haven't dug in to " LU-11775 osc: check if opg is in lru list without locking" yet, but I don't think it would stop us from hitting that lock entirely. If you'd like to avoid inlining, the easiest way I've found is calling that function via a function pointer.  Then you could see it easily in your traces again.  (There are various "don't inline" directives, but I've never been able to get them to work when building Lustre.)
            dongyang Dongyang Li added a comment -

            As we can see LU-9920 can reduce the overhead in set_page_dirty/write_commit_callback.

            There is still __ _raw_spin_lock from osc_page_cache_add which I can not figure out the exact location.

            mpirun -np 32 ior -w -t 16m -b 16g -F -e -vv -o /cache1/file -k

            master:

            Max Write: 15237.92 MiB/sec (15978.12 MB/sec)

            master + LU-11775:

            Max Write: 17045.11 MiB/sec (17873.09 MB/sec)

            master + LU-11775 + LU-9920:

            Max Write: 17476.77 MiB/sec (18325.72 MB/sec)

            dongyang Dongyang Li added a comment - As we can see LU-9920 can reduce the overhead in set_page_dirty/write_commit_callback. There is still __ _raw_spin_lock from osc_page_cache_add which I can not figure out the exact location. mpirun -np 32 ior -w -t 16m -b 16g -F -e -vv -o /cache1/file -k master: Max Write: 15237.92 MiB/sec (15978.12 MB/sec) master + LU-11775 : Max Write: 17045.11 MiB/sec (17873.09 MB/sec) master + LU-11775 + LU-9920 : Max Write: 17476.77 MiB/sec (18325.72 MB/sec)

            People

              dongyang Dongyang Li
              dongyang Dongyang Li
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: