[LU-13241] mount.lustre for large filesystem runs slow debugfs commands

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Labels: scalability
• Assignee: Dongyang Li
• Reporter: Andreas Dilger

Description

Running strace of mount.lustre shows an interesting reason why it spent a huge mount of time on a large filesystem. mount.lustre invokes several 'debugfs' command internally, but they took a huge amount of time here. Here is example.

10056 10:35:21.590086 execve("/sbin/debugfs", ["debugfs", "-c", "-R", "stat CONFIGS/mountdata", "/dev/sda"]
10056 10:39:00.257586 +++ exited with 0 +++

10134 10:39:00.259343 execve("/bin/sh", ["sh", "-c", "debugfs -c -R 'dump /CONFIGS/mou"...]
10134 10:42:39.116052 +++ exited with 0 +++

Total mount time was 982 sec in this time. But, It took 419 sec for just those two debugfs calls against 982 sec.

These two calls are just reading the CONFIGS/mountdata file from the filesystem, but that could be done directly via libext2fs, along with many of the other operations in libmount_utuls_ldiskfs.c, rather than launching external binaries to do the work. We should take care not to load the whole filesystem metadata, if that can be avoided.

Issue Links

• is related to: LU-1083 tunefs.lustre generates incorrect message when PATH doesn't include /sbin
• is related to: LU-18818 remove mount+open+write+close from ldiskfs_write_ldd()

Comments

[Gerrit Updater]

Andreas Dilger ([e-mail]) merged in patch https://review.whamcloud.com/38027/
Subject: LU-13241 libext2fs: add quota to libext2fs
Project: tools/e2fsprogs
Branch: master-lustre-test
Current Patch Set:
Commit: ab3ebc4782c04ae25568a17e456c29441a0ce044

[Gerrit Updater]

Li Dongyang ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38028
Subject: LU-13241 utils: covert to libext2fs for rest of ldiskfs ops
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d1a62adc3449fe9b16c0adc22f77e99393698afe

[Gerrit Updater]

Li Dongyang ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38027
Subject: LU-13241 libext2fs: add quota to libext2fs
Project: tools/e2fsprogs
Branch: master-lustre-test
Current Patch Set: 1
Commit: 38122fbe0d40670efbf787141a549875b4186d4f

[Peter Jones]

Landed for 2.14

[Shuichi Ihara]

Although patch was alraedy landed, here is test resutls without/with patch https://review.whamcloud.com/37656/

1 x 1.2PB OST and 'fake' filled up 50% fileystem by patch https://review.whamcloud.com/#/c/37329/.

Average mount time (Tried 3 times)
Without patch: 72.3 sec (77s,70s,70s)
With    patch: 48.6 sec (51s,48s,47s)

So, patch contributed 33% speedup of mount time.