[LU-50] pdirops patch for ldiskfs

• Type: New Feature
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.2.0
• Affects Version/s: Lustre 2.1.0
• Assignee: Liang Zhen (Inactive)
• Reporter: Liang Zhen (Inactive)

Description

Changing operations of shared directory are not good even with all SMP improvements, I made some efforts to localize changing operations on a few cores to decrease contention and they can somehow help to improvement performance of pdirops (see my comment on LU-55).

At the same time, it's worth to try to increase pdirops performance in ldiskfs level. We had a dynlock based pdirops patch for ldiskfs several years ago, but we didn't maintain it. Actually, I found it's hard to maintain it because it's a complex patch and made a lot of changes to htree code of ext3/ext4. So I'm thinking about to have a more simpler implementation of pdirops patch.

The idea is quite simple (just like how we implement pdirop in ldlm level):

• design & implement a advanced lock with multiple modes (CR, PR, CW, PW, EX)
• it's just a resource lock resides in osd-ldiskfs & ldiskfs, which will replace current osd_object::oo_ext_idx_sem
• we can partially protect resource by locking N hash keys after holding CR & CW
  • for CR, sub locking is PR
  • for CW, sub locking is PW
• for !is_dx(dir), change operations are always holding EX lock on dir, lookup & readdir hold PR lock on dir
• for is_dx(dir)
  • change operations take CW lock, then take two sub-locks: name-hash as the first key, block number as the second key
  • lookup take CR lock, then take two sub-locks: name-hash as the first key, block number as the second key
  • readdir take PR lock (it's not necessary to take PR lock, but we already have PR lock in ldlm level, so it doesn't matter)
• if we need to split any blocks (name entries block or indexing block) while have CW lock, we release CW lock and take EX lock and retry.
• we can disable pdirops ldiskfs by turning osd-ldiskfs pdir_ldiskfs=0/1, if we disabled pdirops for osd-ldiskfs then osd_ldiskfs always EX lock dir on changing and PR lock dir on lookup/readdir.

Although implementation of the lock is a little complex, but we just need very few changes to ext4/ldiskfs and osd-ldiskfs with this lock:

• osd-ldiskfs: replace oo_ext_idx_sem with the new lock
• ldiskfs: switch lock mode if we need to split/append (in ldiskfs).

PS: I'm thinking about rename parl_lock to htree_lock, because it's really designed for htree operations, and Eric hates confusing naming like "parl_lock"

Issue Links

• is related to: LU-13054 MDS kernel BUG at ldiskfs/htree_lock.c:429!
• is related to: LU-1365 Implement ldiskfs LARGEDIR support for e2fsprogs
• is related to: LU-896 change e2fsprogs to make it allow directories over 2GB
• Trackbacks: Demonstration Milestone for Parallel Directory Operations Overview This document describes the work required to demonstrate that the Parallel Directory Operations code meets the agreed acceptability criteria. The Parallel Directory Operations code is functionally complete....
• Trackbacks: Demonstration Milestone for Parallel Directory Operations Overview This document describes the work required to demonstrate that the Parallel Directory Operations code meets the agreed acceptability criteria. The Parallel Directory Operations code is functionally complete....
• Trackbacks: Changelog 2.2 version 2.2.0 Support for networks: o2iblnd OFED 1.5.4 Server support for kernels: 2.6.32220.4.2.el6 (RHEL6) Client support for unpatched kernels: 2.6.18274.18.1.el5 (RHEL5) 2.6.32220.4.2.el6 (RHEL6) 2.6.32.360....
• Trackbacks: Lustre 2.2 Lustre 2.2 is scheduled for release at the end of Q1 2012. This is its anticipated support matrix. Server Support \\ Client Support \\ RHEL 6, \\ CentOS 6 \\ RHEL 6, \\ CentOS 6, \\ RHEL 5, \\ CentOS 5, \\ SLES 11 ,...