Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-56

Finish SMP scalability work

Details

    • Task
    • Resolution: Fixed
    • Major
    • Lustre 2.3.0
    • Lustre 2.3.0
    • None
    • 4564

    Description

      Please post the SMP patches to bugzilla and then push the changes to a branch on review.whamcloud.com/lustre. Keep the branch current as changes are made to the patches during the inspection/landing process, because we'll also be testing this branch at LLNL and possibly other places.

      Attachments

        Issue Links

          is related to

          Technical task - A technical task. LU-14676 Better hash distribution to different CPTs when LNET router is exist

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LUDOC-97 SMP Documentation

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          Trackbacks

          [Confluence: OpenSFS] SMP Node Affinity Scope Statement Introduction The following scope statement applies to the SMP Node Affinity Scope Statement project within the SFSDEV001 contract/SOW dates 08/01/2011. Problem Statement Followers of CPU design are observing steady increases in core count....

          [Confluence: Whamcloud Community Space] Lustre Community Development in Progress Features are being developed for future Lustre releases both at Whamcloud and by other organizations in the Lustre community. These will be eligible for inclusion in future Lustre releases as per our processes

          Activity

            [LU-56] Finish SMP scalability work
            liang Liang Zhen (Inactive) added a comment -

            patched mdtest which parses expression for multi-mount

            liang Liang Zhen (Inactive) added a comment - patched mdtest which parses expression for multi-mount
            jlevi Jodi Levi (Inactive) added a comment -

            Please let me know if more work is needed for this and I will reopen ticket.

            jlevi Jodi Levi (Inactive) added a comment - Please let me know if more work is needed for this and I will reopen ticket.
            liang Liang Zhen (Inactive) added a comment -

            SMP node affinity OpenSFS demonstration

            liang Liang Zhen (Inactive) added a comment - SMP node affinity OpenSFS demonstration
            liang Liang Zhen (Inactive) added a comment -

            SMP node affinity presentation for Lustre team Tech-call

            liang Liang Zhen (Inactive) added a comment - SMP node affinity presentation for Lustre team Tech-call
            spitzcor Cory Spitz added a comment -

            Cray has been testing (a little) for functionality only. No significant issues to report so far.

            spitzcor Cory Spitz added a comment - Cray has been testing (a little) for functionality only. No significant issues to report so far.
            spitzcor Cory Spitz added a comment -

            Re: the last two comments. Patches have landed. I'll copy an update of the patch list from Richard Henwood here:

            c3e985e7e98f41ebf5ecb78887dcd2554601f7ef LU-56 ptlrpc: post rqbd with flag LNET_INS_LOCAL
            8bbd62a7c0d2fc48d8f11e78d92bb42809968bba LU-56 ptlrpc: CPT affinity ptlrpc RS handlers
            d800fc41a1abdaf7aaf6c0e3e7ddcdec489985a8 LU-56 ptlrpc: partitioned ptlrpc service
            b43a6b1800265608cfa18159d4d0d006a1c23015 LU-56 o2iblnd: CPT affinity o2iblnd
            82e02a17c0c645a8d156e51b8d8da5eaa68b8f5b LU-56 lnet: re-finalize failed ACK or routed message
            1a73553d15b459208cbf7279ea6e5e5a110c632b LU-56 ksocklnd: CPT affinity socklnd
            6b4b780895dfdeaca316862fbf1696983608f96d LU-56 lnet: wrong assertion for optimized GET
            913c8e22cfc4fc5f52c4f0d6d3f0b4b86a7ac58c LU-56 lnet: tuning wildcard portals rotor
            c03783fce46ae0b40db0680388df6e2d6fca5008 LU-56 lnet: SMP improvements for LNet selftest
            7b2ab9beae02080797ff2da5105eaddadd67c151 LU-56 ldlm: SMP improvement for ldlm_lock_cancel
            07b8db220e48782369f48d86213c5d404a628ded LU-56 ptlrpc: Reduce at_lock dance
            c48a869557fe7663f4f3370b130d4c248958180e LU-56 libcfs: CPT affinity workitem scheduler
            8a5b8dbda960b155f669c13602504f1233a84c7e LU-56 obdclass: SMP improvement for lu_key
            e531dc437c56a08a65de9074a511faa55184712b LU-56 lnet: multiple cleanups for inspection
            e069296630240947f1815e505067fd48033909f7 LU-56 lnet: allow user to bind NI on CPTs
            a07e9d350b3e500c7be877f6dcf54380b86a9cbe LU-56 lnet: Partitioned LNet networks
            5e1957841df3e771f3d72d8ea59180213430bbb9 LU-56 lnet: cleanup for rtrpool and LNet counter
            279bbc81e03dc74d273ec12b4d9e703ca94404c4 LU-56 lnet: Partitioned LNet resources (ME/MD/EQ)
            582c110231cf06bcd7e5e0b3bdf4f2058e18ebe4 LU-56 ptlrpc: cleanup of ptlrpc_unregister_service
            ff0c89a73e141ce019ee2a94e5d01a8a37dd830a LU-56 ptlrpc: svc thread starting/stopping cleanup
            25766da50b627648b04549ff3fb55af12acbcb4b LU-56 lnet: reduce stack usage of "match" functions
            c7bff5640caff778d4cfca229672a2cc67b350d6 LU-56 lnet: Granulate LNet lock
            24564b398f53009521aeda5d653e57fe8b525775 LU-56 ptlrpc: partition data for ptlrpc service
            698d3088622b4610a84bd508f2b707a7a2dd1e3e LU-56 lnet: code cleanup for lib-move.c
            38fcdd3966da09517ca176b962230b7dae43514c LU-56 lnet: match-table for Portals
            f0aa1eef72e7438c2bd4b3eee821fefbc50d1f8e LU-56 lnet: code cleanup for lib-md.c
            75a8f4b4aa9ad6bf697aedece539e62111e9029a LU-56 lnet: split lnet_commit_md and cleanup
            06093c1f24da938418a0243259b5307c9fc338d5 LU-56 lnet: LNet message event cleanup
            2118a8b92cec2df85d1bdbe2e58b389d83fe06b2 LU-56 lnet: eliminate a few locking dance in LNet
            51a5b4df5bbbf5fd12c73d2722b230e93fe93327 LU-56 lnet: parse RC ping in event callback
            b9bad9bd7d1c3271df916ee62091106e3f3c98b7 LU-56 lnet: router-checker (RC) cleanup
            4fcc56be68c8c1667fbd91721d084874a2f05c3e LU-56 ptlrpc: common code to validate nthreads
            ed22093b2d569fd0e93f35504580171114bf212d LU-56 lnet: move "match" functions to lib-ptl.c
            a096d858b671f28fd4c5e6197b51643cd0780a50 LU-56 lnet: allow to create EQ with zero eq_size
            c1366da8f43ecfb98ef3bdcf629eec8a2fc9cd4c LU-56 lnet: cleanup for LNet Event Queue
            3211f6862cbbe96642db540e6593f3c614f9528c LU-56 lnet: new internal object lnet_peer_table
            7a51ad347960ef2b9d1dfad14644c0bca35b80b6 LU-56 ptlrpc: clean up ptlrpc svc initializing APIs
            facf5086667874c405c9ef6ce7f8f737868ffefd LU-56 lnet: container for LNet message
            c3a57ec36441c75df03cfbec8f718e053aaad12a LU-56 lnet: abstract container for EQ/ME/MD
            4bd9bf53728260d38efc74cac981318fe31280cd LU-56 lnet: add lnet_*_free_locked for LNet
            c8da7bfbe0505175869973b25281b152940774b0 LU-56 libcfs: more common APIs in libcfs
            b76f327de7836a854f204d28e61de52bc03011b1 LU-56 libcfs: export a few symbols from libcfs
            3a92c850b094019e556577ec6cab5907538dcbf5 LU-56 libcfs: NUMA allocator and code cleanup
            617e8e1229637908d4cce6725878dd5668960420 LU-56 libcfs: implementation of cpu partition
            19ec037c0a9427250b87a69c53beb153d533ab1c LU-56 libcfs: move range expression parser to libcfs

            spitzcor Cory Spitz added a comment - Re: the last two comments. Patches have landed. I'll copy an update of the patch list from Richard Henwood here: c3e985e7e98f41ebf5ecb78887dcd2554601f7ef LU-56 ptlrpc: post rqbd with flag LNET_INS_LOCAL 8bbd62a7c0d2fc48d8f11e78d92bb42809968bba LU-56 ptlrpc: CPT affinity ptlrpc RS handlers d800fc41a1abdaf7aaf6c0e3e7ddcdec489985a8 LU-56 ptlrpc: partitioned ptlrpc service b43a6b1800265608cfa18159d4d0d006a1c23015 LU-56 o2iblnd: CPT affinity o2iblnd 82e02a17c0c645a8d156e51b8d8da5eaa68b8f5b LU-56 lnet: re-finalize failed ACK or routed message 1a73553d15b459208cbf7279ea6e5e5a110c632b LU-56 ksocklnd: CPT affinity socklnd 6b4b780895dfdeaca316862fbf1696983608f96d LU-56 lnet: wrong assertion for optimized GET 913c8e22cfc4fc5f52c4f0d6d3f0b4b86a7ac58c LU-56 lnet: tuning wildcard portals rotor c03783fce46ae0b40db0680388df6e2d6fca5008 LU-56 lnet: SMP improvements for LNet selftest 7b2ab9beae02080797ff2da5105eaddadd67c151 LU-56 ldlm: SMP improvement for ldlm_lock_cancel 07b8db220e48782369f48d86213c5d404a628ded LU-56 ptlrpc: Reduce at_lock dance c48a869557fe7663f4f3370b130d4c248958180e LU-56 libcfs: CPT affinity workitem scheduler 8a5b8dbda960b155f669c13602504f1233a84c7e LU-56 obdclass: SMP improvement for lu_key e531dc437c56a08a65de9074a511faa55184712b LU-56 lnet: multiple cleanups for inspection e069296630240947f1815e505067fd48033909f7 LU-56 lnet: allow user to bind NI on CPTs a07e9d350b3e500c7be877f6dcf54380b86a9cbe LU-56 lnet: Partitioned LNet networks 5e1957841df3e771f3d72d8ea59180213430bbb9 LU-56 lnet: cleanup for rtrpool and LNet counter 279bbc81e03dc74d273ec12b4d9e703ca94404c4 LU-56 lnet: Partitioned LNet resources (ME/MD/EQ) 582c110231cf06bcd7e5e0b3bdf4f2058e18ebe4 LU-56 ptlrpc: cleanup of ptlrpc_unregister_service ff0c89a73e141ce019ee2a94e5d01a8a37dd830a LU-56 ptlrpc: svc thread starting/stopping cleanup 25766da50b627648b04549ff3fb55af12acbcb4b LU-56 lnet: reduce stack usage of "match" functions c7bff5640caff778d4cfca229672a2cc67b350d6 LU-56 lnet: Granulate LNet lock 24564b398f53009521aeda5d653e57fe8b525775 LU-56 ptlrpc: partition data for ptlrpc service 698d3088622b4610a84bd508f2b707a7a2dd1e3e LU-56 lnet: code cleanup for lib-move.c 38fcdd3966da09517ca176b962230b7dae43514c LU-56 lnet: match-table for Portals f0aa1eef72e7438c2bd4b3eee821fefbc50d1f8e LU-56 lnet: code cleanup for lib-md.c 75a8f4b4aa9ad6bf697aedece539e62111e9029a LU-56 lnet: split lnet_commit_md and cleanup 06093c1f24da938418a0243259b5307c9fc338d5 LU-56 lnet: LNet message event cleanup 2118a8b92cec2df85d1bdbe2e58b389d83fe06b2 LU-56 lnet: eliminate a few locking dance in LNet 51a5b4df5bbbf5fd12c73d2722b230e93fe93327 LU-56 lnet: parse RC ping in event callback b9bad9bd7d1c3271df916ee62091106e3f3c98b7 LU-56 lnet: router-checker (RC) cleanup 4fcc56be68c8c1667fbd91721d084874a2f05c3e LU-56 ptlrpc: common code to validate nthreads ed22093b2d569fd0e93f35504580171114bf212d LU-56 lnet: move "match" functions to lib-ptl.c a096d858b671f28fd4c5e6197b51643cd0780a50 LU-56 lnet: allow to create EQ with zero eq_size c1366da8f43ecfb98ef3bdcf629eec8a2fc9cd4c LU-56 lnet: cleanup for LNet Event Queue 3211f6862cbbe96642db540e6593f3c614f9528c LU-56 lnet: new internal object lnet_peer_table 7a51ad347960ef2b9d1dfad14644c0bca35b80b6 LU-56 ptlrpc: clean up ptlrpc svc initializing APIs facf5086667874c405c9ef6ce7f8f737868ffefd LU-56 lnet: container for LNet message c3a57ec36441c75df03cfbec8f718e053aaad12a LU-56 lnet: abstract container for EQ/ME/MD 4bd9bf53728260d38efc74cac981318fe31280cd LU-56 lnet: add lnet_*_free_locked for LNet c8da7bfbe0505175869973b25281b152940774b0 LU-56 libcfs: more common APIs in libcfs b76f327de7836a854f204d28e61de52bc03011b1 LU-56 libcfs: export a few symbols from libcfs 3a92c850b094019e556577ec6cab5907538dcbf5 LU-56 libcfs: NUMA allocator and code cleanup 617e8e1229637908d4cce6725878dd5668960420 LU-56 libcfs: implementation of cpu partition 19ec037c0a9427250b87a69c53beb153d533ab1c LU-56 libcfs: move range expression parser to libcfs

            People

              liang Liang Zhen (Inactive)
              rread Robert Read
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: