Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-8130

Migrate from libcfs hash to rhashtable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • Upstream
    • Lustre 2.9.0
    • None
    • 9223372036854775807

    Description

      The linux kernel has its own resizable hashtable which can be used in place of libcfs hash code. The linux rhashtable was developed for the networking layer and that layer has very high requirements for performance. Some of the benefits are low latency as well as lockless lookups. Migrating Lustre to rhashtable should mean HUGE PERFORMANCE gains!!!!!

      Attachments

        1. make.log
          31 kB
        2. rhashtable.h
          7 kB

        Issue Links

          is blocking

          Improvement - An improvement or enhancement to an existing feature or task. LU-14535 add option for LFS to get all quota information

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-14698 sanityn test_77b scheduling in_atomic()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-16365 cached 'ls -l' is slow

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-18031 soft lockup in cfs_hash_for_each_relax

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-6800 Significant performance regression with patch LU-5264

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-11624 BUG: unable to handle kernel NULL pointer at nid_hash()

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-14752 LustreError: 3594004:0:(lu_object.c:2472:lu_object_assign_fid()) ASSERTION( rc == 0 ) failed: failed hashtable insertion: rc = -16

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-10401 sanity test_133g: timeout during MDT mount

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-16253 sanityn: ASSERTION( orro->oo_ref == 0 ) in 77d

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-17191 sanity-quota test_1b, 1d, 1f, 1i: FAIL: user write success, but expect EDQUOT

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-17793 warning: objtool: __cfs_fail_check_set() falls through to next function __cfs_fail_timeout_set()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-18187 BUG: KASAN: vmalloc-out-of-bounds in ldlm_reclaim_lock_cb+0xa46/0xa50 [ptlrpc]

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. LU-17329 Relaxed POSIX Consistency for Lustre

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-12511 Prepare lustre for adoption into the linux kernel

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18192 TBF: generic combination TBF types with different granularities

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18351 Job stats scaling

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-8130] Migrate from libcfs hash to rhashtable
            simmonsja James A Simmons added a comment -

            With cfs hash moved to lustre out of libcfs we don't need this linked to LU-9859 anymore.

            simmonsja James A Simmons added a comment - With cfs hash moved to lustre out of libcfs we don't need this linked to LU-9859 anymore.
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56762/
            Subject: LU-8130 libcfs: don't use radix tree for xarray
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set:
            Commit: 45c624eb4656a6d25e3557e453a5e27c02ffc7f5

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56762/ Subject: LU-8130 libcfs: don't use radix tree for xarray Project: fs/lustre-release Branch: b2_15 Current Patch Set: Commit: 45c624eb4656a6d25e3557e453a5e27c02ffc7f5
            gerrit Gerrit Updater added a comment -

            "Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56762
            Subject: LU-8130 libcfs: don't use radix tree for xarray
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set: 1
            Commit: a4fa9ba262f74528f1c74b77985701115750d84f

            gerrit Gerrit Updater added a comment - "Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56762 Subject: LU-8130 libcfs: don't use radix tree for xarray Project: fs/lustre-release Branch: b2_15 Current Patch Set: 1 Commit: a4fa9ba262f74528f1c74b77985701115750d84f
            gerrit Gerrit Updater added a comment -

            "Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56289
            Subject: LU-8130 ldlm: ldlm_resource hash direct and rcu free
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: f6653e484e9ba267fca3ba5a9a76d8db7c5231b0

            gerrit Gerrit Updater added a comment - "Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56289 Subject: LU-8130 ldlm: ldlm_resource hash direct and rcu free Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: f6653e484e9ba267fca3ba5a9a76d8db7c5231b0
            gerrit Gerrit Updater added a comment -

            "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55305
            Subject: LU-8130 ldlm: test putref_final changes
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 1d211bb03666c38955f42c51ddbdf95790589bc1

            gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55305 Subject: LU-8130 ldlm: test putref_final changes Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 1d211bb03666c38955f42c51ddbdf95790589bc1
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54193/
            Subject: LU-8130 nrs: for TBF nid handling using rhashtables
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: e83051f5b1e6a10a11a5296fee47243ef01661dd

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54193/ Subject: LU-8130 nrs: for TBF nid handling using rhashtables Project: fs/lustre-release Branch: master Current Patch Set: Commit: e83051f5b1e6a10a11a5296fee47243ef01661dd
            gerrit Gerrit Updater added a comment -

            "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/54193
            Subject: LU-8130 nrs: for TBF nid handling using rhashtables
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 29594bf28737d76bf91f69f79c3b9a4116fb5d27

            gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/54193 Subject: LU-8130 nrs: for TBF nid handling using rhashtables Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 29594bf28737d76bf91f69f79c3b9a4116fb5d27
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/32038/
            Subject: LU-8130 osc: convert osc_quota hash to xarray
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: ac8c28f959d87c1ae852d9daccce3c2a3f7a6f39

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/32038/ Subject: LU-8130 osc: convert osc_quota hash to xarray Project: fs/lustre-release Branch: master Current Patch Set: Commit: ac8c28f959d87c1ae852d9daccce3c2a3f7a6f39
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40113/
            Subject: LU-8130 nrs: convert NRS ORR/TRR to rhashtable
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 42bf5f78ba23ed5a7a908c921a5d8ded7544a315

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40113/ Subject: LU-8130 nrs: convert NRS ORR/TRR to rhashtable Project: fs/lustre-release Branch: master Current Patch Set: Commit: 42bf5f78ba23ed5a7a908c921a5d8ded7544a315
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51840/
            Subject: LU-8130 libcfs: don't use radix tree for xarray
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 778791dd7da107710c2311935a24cfd7e7a5fd85

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51840/ Subject: LU-8130 libcfs: don't use radix tree for xarray Project: fs/lustre-release Branch: master Current Patch Set: Commit: 778791dd7da107710c2311935a24cfd7e7a5fd85
            gerrit Gerrit Updater added a comment -

            "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51840
            Subject: LU-8130 libcfs: free xa_node instead of radix_tree_node
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 7553c5aa770d0e8c61630e9cc1deff4a13e02b4c

            gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51840 Subject: LU-8130 libcfs: free xa_node instead of radix_tree_node Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 7553c5aa770d0e8c61630e9cc1deff4a13e02b4c

            People

              simmonsja James A Simmons
              simmonsja James A Simmons
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated: