[LU-12988] improve mount time on huge ldiskfs filesystem

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Labels: LTS12 scalability
• Assignee: Alex Zhuravlev
• Reporter: Alex Zhuravlev

Description

during Lustre server startup few small files need to be updated (e.g. config backup).
at this point buddy/bitmap cache is empty but mballoc wants to find a big chunk of free space for group preallocation and reads bitmaps one by one.
sometimes this can take a very long time.
one possisble workaround is to disable preallocation during mount.
a long term plan is to limit scanning and prefetch bitmaps, but this needs more efforts and will be tracked separately (LU-12970)

Issue Links

• is related to: LU-14453 LDISKFS-fs error (device xxx) in ldiskfs_do_update_inode:4840: Journal has aborted
• is related to: LU-13290 Write performance regression in ldiskfs patches
• is related to: LU-12970 improve mballoc for huge filesystems
• is related to: LU-12103 Improve block allocation for large partitions

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37539/
Subject: LU-12988 ldiskfs: skip non-loaded groups at cr=0/1
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: de994667dda925109e862edadb4aa4feaecd0e6b

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37811/
Subject: LU-12988 ldiskfs: port ext4-mballoc-prefetch.patch to RHEL 8.1
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 896e12c2e4fc98cbc15c675ec2894e9511aa92a7

[Gerrit Updater]

Jian Yu ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37811
Subject: LU-12988 ldiskfs: port ext4-mballoc-prefetch.patch to RHEL 8.1
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 89fd17997924aafb78ab0cc24debaf12b0e17b87

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37633/
Subject: LU-12988 ldiskfs: mballoc to prefetch groups
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b7cd65a3d1d665f1bee5eb8ad3b989b12be7de08

[Shuichi Ihara]

Alex, please see LU-13290, there is another perforamnce regression in ldiksfs. I think there are two major performance regressions in ldiskfs. Although patch https://review.whamcloud.com/#/c/37619 was odd behaviors, we also need to fix another regression is caused by LU-12988.

[Alex Zhuravlev]

thanks for the report. have you got performance back to normal with mballoc-prefetch patch reverted? or it's still below expectation?