Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-8130

Migrate from libcfs hash to rhashtable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • Upstream
    • Lustre 2.9.0
    • None
    • 9223372036854775807

    Description

      The linux kernel has its own resizable hashtable which can be used in place of libcfs hash code. The linux rhashtable was developed for the networking layer and that layer has very high requirements for performance. Some of the benefits are low latency as well as lockless lookups. Migrating Lustre to rhashtable should mean HUGE PERFORMANCE gains!!!!!

      Attachments

        1. make.log
          31 kB
          Jian Yu
        2. rhashtable.h
          7 kB
          Jian Yu

        Issue Links

          is blocking

          Improvement - An improvement or enhancement to an existing feature or task. LU-14535 add option for LFS to get all quota information

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-14698 sanityn test_77b scheduling in_atomic()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-16365 cached 'ls -l' is slow

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-18031 soft lockup in cfs_hash_for_each_relax

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. LU-6800 Significant performance regression with patch LU-5264

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-11624 BUG: unable to handle kernel NULL pointer at nid_hash()

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-14752 LustreError: 3594004:0:(lu_object.c:2472:lu_object_assign_fid()) ASSERTION( rc == 0 ) failed: failed hashtable insertion: rc = -16

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-10401 sanity test_133g: timeout during MDT mount

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-16253 sanityn: ASSERTION( orro->oo_ref == 0 ) in 77d

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-17191 sanity-quota test_1b, 1d, 1f, 1i: FAIL: user write success, but expect EDQUOT

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-17793 warning: objtool: __cfs_fail_check_set() falls through to next function __cfs_fail_timeout_set()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-18187 BUG: KASAN: vmalloc-out-of-bounds in ldlm_reclaim_lock_cb+0xa46/0xa50 [ptlrpc]

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. LU-17329 Relaxed POSIX Consistency for Lustre

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-12511 Prepare lustre for adoption into the linux kernel

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18192 TBF: generic combination TBF types with different granularities

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18351 Job stats scaling

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-8130] Migrate from libcfs hash to rhashtable
            gerrit Gerrit Updater added a comment -

            "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58592
            Subject: LU-8130 obd: replace gen hash with Xarray
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: c734d92562ca8f7db3e36bd3625079ab39f28243

            gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58592 Subject: LU-8130 obd: replace gen hash with Xarray Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: c734d92562ca8f7db3e36bd3625079ab39f28243
            simmonsja James A Simmons added a comment -

            With cfs hash moved to lustre out of libcfs we don't need this linked to LU-9859 anymore.

            simmonsja James A Simmons added a comment - With cfs hash moved to lustre out of libcfs we don't need this linked to LU-9859 anymore.
            simmonsja James A Simmons made changes -
            Link Original: This issue is related to LU-9859 [ LU-9859 ]
            simmonsja James A Simmons made changes -
            Link New: This issue is related to LU-18192 [ LU-18192 ]
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56762/
            Subject: LU-8130 libcfs: don't use radix tree for xarray
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set:
            Commit: 45c624eb4656a6d25e3557e453a5e27c02ffc7f5

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56762/ Subject: LU-8130 libcfs: don't use radix tree for xarray Project: fs/lustre-release Branch: b2_15 Current Patch Set: Commit: 45c624eb4656a6d25e3557e453a5e27c02ffc7f5
            gerrit Gerrit Updater added a comment -

            "Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56762
            Subject: LU-8130 libcfs: don't use radix tree for xarray
            Project: fs/lustre-release
            Branch: b2_15
            Current Patch Set: 1
            Commit: a4fa9ba262f74528f1c74b77985701115750d84f

            gerrit Gerrit Updater added a comment - "Jian Yu <yujian@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56762 Subject: LU-8130 libcfs: don't use radix tree for xarray Project: fs/lustre-release Branch: b2_15 Current Patch Set: 1 Commit: a4fa9ba262f74528f1c74b77985701115750d84f
            simmonsja James A Simmons made changes -
            Link New: This issue is related to LU-18351 [ LU-18351 ]
            simmonsja James A Simmons made changes -
            Link Original: This issue is related to LU-8802 [ LU-8802 ]
            simmonsja James A Simmons made changes -
            Link New: This issue is related to LU-18031 [ LU-18031 ]
            gerrit Gerrit Updater added a comment -

            "Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56289
            Subject: LU-8130 ldlm: ldlm_resource hash direct and rcu free
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: f6653e484e9ba267fca3ba5a9a76d8db7c5231b0

            gerrit Gerrit Updater added a comment - "Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56289 Subject: LU-8130 ldlm: ldlm_resource hash direct and rcu free Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: f6653e484e9ba267fca3ba5a9a76d8db7c5231b0

            People

              simmonsja James A Simmons
              simmonsja James A Simmons
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated: