Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Lustre 2.1.0
    • Fix Version/s: Lustre 2.2.0
    • Labels:
      None
    • Rank (Obsolete):
      4800

      Description

      Changing operations of shared directory are not good even with all SMP improvements, I made some efforts to localize changing operations on a few cores to decrease contention and they can somehow help to improvement performance of pdirops (see my comment on LU-55).

      At the same time, it's worth to try to increase pdirops performance in ldiskfs level. We had a dynlock based pdirops patch for ldiskfs several years ago, but we didn't maintain it. Actually, I found it's hard to maintain it because it's a complex patch and made a lot of changes to htree code of ext3/ext4. So I'm thinking about to have a more simpler implementation of pdirops patch.

      The idea is quite simple (just like how we implement pdirop in ldlm level):

      • design & implement a advanced lock with multiple modes (CR, PR, CW, PW, EX)
      • it's just a resource lock resides in osd-ldiskfs & ldiskfs, which will replace current osd_object::oo_ext_idx_sem
      • we can partially protect resource by locking N hash keys after holding CR & CW
        • for CR, sub locking is PR
        • for CW, sub locking is PW
      • for !is_dx(dir), change operations are always holding EX lock on dir, lookup & readdir hold PR lock on dir
      • for is_dx(dir)
        • change operations take CW lock, then take two sub-locks: name-hash as the first key, block number as the second key
        • lookup take CR lock, then take two sub-locks: name-hash as the first key, block number as the second key
        • readdir take PR lock (it's not necessary to take PR lock, but we already have PR lock in ldlm level, so it doesn't matter)
      • if we need to split any blocks (name entries block or indexing block) while have CW lock, we release CW lock and take EX lock and retry.
      • we can disable pdirops ldiskfs by turning osd-ldiskfs pdir_ldiskfs=0/1, if we disabled pdirops for osd-ldiskfs then osd_ldiskfs always EX lock dir on changing and PR lock dir on lookup/readdir.

      Although implementation of the lock is a little complex, but we just need very few changes to ext4/ldiskfs and osd-ldiskfs with this lock:

      • osd-ldiskfs: replace oo_ext_idx_sem with the new lock
      • ldiskfs: switch lock mode if we need to split/append (in ldiskfs).

      PS: I'm thinking about rename parl_lock to htree_lock, because it's really designed for htree operations, and Eric hates confusing naming like "parl_lock"

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                liang Liang Zhen (Inactive)
                Reporter:
                liang Liang Zhen (Inactive)
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: