[LU-15011] implement lod pool spilling

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.15.0
• Assignee: Alex Zhuravlev
• Reporter: Andreas Dilger

Description

If an OST pool becomes full, allow configuring a global default spill target to hold files that could not be written due to the pool running out of space. This is similar to SEL, but implemented at the pool selection level, rather than on a per-write basis. While SEL is probably flexible (essentially it implements pool spill on a per-component basis), very few users know enough about SEL to use it effectively, and any changes to the SEL configuration would require updating every file that is using a particualr SEL layout. In contrast, pool spill is a tunable on the MDS (for each source pool) and affects all existing layouts using that pool if there is a spill target pool and used space threshold specified.

Issue Links

• is related to: LU-15072 Pool spill is activated just by changing threshold
• is related to: LU-15043 OST spill pools should not allow spill pool loops
• is related to: LU-13058 Intermediate component removal (PFL/SEL)
• is related to: LUDOC-517 add pool_spill parameters to manual

Comments

[Andreas Dilger]

I filed LUDOC-517 for the "pool_spill" documentation.

[Andreas Dilger]

spitzcor, I don't see it in the manual. However, I found your patch to add this documentation https://review.whamcloud.com/49725 "LUDOC-512 manual: add append_pool and append_stripe_count" and will land it now.

[Cory Spitz]

Just curious, but was a LUDOC ever opened to track this Improvement?

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/45074/
Subject: LU-15011 tests: additional checks for pool spilling
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 26e765e0d23ba37577775331ac77ec857affd845

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/44947/
Subject: LU-15011 lod: count all spilling events
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4bd9b524537bce27fb7f53cf481b78290cd55a1d

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/45120/
Subject: LU-15011 tests: ost-pools to remove big files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ebed0531abed9512682574e44dd17f465fa9b04c

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/45056/
Subject: LU-15011 tests: pool spill test modifications
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d8ff73097be78cc3e603ab49b6aaa03a1c766fcb

[Gerrit Updater]

"Alex Zhuravlev <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/45120
Subject: LU-15011 tests: ost-pools to remove big files
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ad32cbc4f0654c3ae3c96b4bd1d1526445d94913

[Gerrit Updater]

"James Nunez <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/45056
Subject: LU-15011 tests: pool spill test modifications
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a097f6ea7e5a9514c08995d6ed721cba05bb1fb0

[Andreas Dilger]

Originally submitted as LU-14825, but that issue was lost due to a database error:

"Alex Zhuravlev <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/43989
Subject: LU-14825 lod: pool spilling
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0a998f4723f58e68d0fe75934df5c282eb529337