[LU-14679] parallel e2fsck for pass2 directory scanning Created: 11/May/21  Updated: 02/Dec/21  Resolved: 02/Dec/21

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.14.0
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Andreas Dilger Assignee: WC Triage
Resolution: Duplicate Votes: 0
Labels: e2fsck

Issue Links:
Duplicate
duplicates LU-14894 Parallel pass2 support for e2fsck Open
Related
is related to LU-8465 parallel e2fsck performance at scale Resolved
Rank (Obsolete): 9223372036854775807

 Description   

LU-8465 implemented parallel inode table scanning for e2fsck pass1, and parallel bitmap loading for e2fsck pass5.

After these improvements, e2fsck of a 50% full 1PB filesystem running with 256 threads improved pass1 from 7777s to 191s (40x faster), and pass5 from 10523s to 286s (36x faster).

However, pass2 only improved marginally, from 1363s to 1265s (7%) and this now dominates the total e2fsck runtime (70% in this test) on large OSTs, and limits the aggregate speedup of e2fsck to "only" 10x faster despite the other improvements.

Processing directories in parallel should also be possible, in order to further speed up e2fsck times on large OSTs, and expecially on large MDT filesystems.


Generated at Sat Feb 10 03:11:51 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.