[LU-16155] allow importing inode/block allocation maps to new ldisks filesystem Created: 14/Sep/22  Updated: 18/Jul/23

Status: Open
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Andreas Dilger Assignee: Emoly Liu
Resolution: Unresolved Votes: 0
Labels: e2fsprogs, ldiskfs, performance

Issue Links:
Related
is related to LU-12970 improve mballoc for huge filesystems Open
is related to LU-14438 backport ldiskfs mballoc patches Open
is related to LU-13014 script to generate and fill ldiskfs f... Resolved
Rank (Obsolete): 9223372036854775807

 Description   

In order to more efficiently test mballoc performance in the face of full and fragmented filesystems, it would be useful to be able to efficiently fill a newly-formatted filesystem with a realistic block and inode allocation pattern, without having to write a large amount of data, or create millions of files. This should be done using an actual allocation pattern quickly, rather than just using a synthetic ratio of free blocks.

It should be possible to take the output from "dumpe2fs <dev>" as the input to "debugfs --import-bitmaps <input_file>" to have it set the block and inode bitmaps in the new target filesystem that are set in the source filesystem. This process shouldn't clear any bits in the bitmap, to avoid corrupting existing files and directories by double-allocating the blocks and inodes during testing. The kernel would skip those blocks and inodes for allocation until e2fsck is run on the filesystem. This would normally be used only for test filesystems.

For source filesystems smaller than the target filesystem, it can repeat the allocation pattern for the block groups as needed until all of the groups in the target filesystem has been "filled".

Rather than setting each bit in the bitmap separately, which would be billions of operations for a large filesystem, the code should use the free bitmap ranges provided by debugfs to set contiguous ranges of bits in the bitmap.



 Comments   
Comment by Gerrit Updater [ 10/Jul/23 ]

"Emoly Liu <emoly@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/tools/e2fsprogs/+/51614
Subject: LU-16155 e2image: add -d/-i options to dump/import bitmaps
Project: tools/e2fsprogs
Branch: master-lustre
Current Patch Set: 1
Commit: beb6ed1d1a98da1a70d9dc4d83d096607d7b96f2

Generated at Sat Feb 10 03:24:29 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.