[LU-17756] tunable limit for MDT overstripe factor

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Affects Version/s: Lustre 2.16.0
• Labels: dne3
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Andreas Dilger

Description

The patch https://review.whamcloud.com/35034 "LU-12273 lod: metadata overstriping" allows an MDT to hold more than one stripe of a directory, in order to raise the maximum directory size limit, as well as increase parallelism in directory locking at both the LDLM and OSD level.

However, this introduces a potential incompatibility with older clients that don't understand overstriped directories. It would be good to add a tunable parameter to limit the usage of this feature. It would be named like "lod.*.max_stripes_per_mdt" (or "mdt_max_overstripe_factor" or similar), and replace most uses of the hard-coded constant LMV_MAX_STRIPES_PER_MDT in the code. On master for 2.16, lod_max_stripes_per_mdt would be initialized to LMV_MAX_STRIPES_PER_MDT=5 (the hard upper limit due to transaction sizes), and if this is backported to older branches for interoperability it would initialized to "1".

In lod_prep_md_striped_create() if a new directory was requested with stripes_per_mdt > lod_max_stripes_per_mdt, then IMHO it would be better to silently clamp this value to lod_max_stripes_per_mdt to avoid application failure. If lod_max_stripes_per_mdt == 1 then the existing check in lod_mdt_alloc_rr() will strip the LMV_HASH_FLAG_OVERSTRIPED flag from the layout during creation, to maintain compatibility with older clients.

Issue Links

• is related to: LU-12926 lfs mkdir -c can cause repeated MDT indexes
• is related to: LU-12273 DNE3: Metadata overstriping

Comments

[Peter Jones]

Merged for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54945/
Subject: LU-17756 lod: add tunable lod.*.max_stripes_per_mdt
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 70539f31f95f6116335ed4a4c5a278f436e37d80

[Gerrit Updater]

"Lai Siyao <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/54945
Subject: LU-17756 lod: add tunable lod.*.max_stripes_per_mdt
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e8d4842a6acc5348281ad0e987f6a8a8f3bac186