[LU-16724] generic page pool

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Assignee: Patrick Farrell
• Reporter: Patrick Farrell

Description

Looking at the unaligned DIO work in LU-13805, Andreas pointed out we should consider how we're getting pages for our unaligned DIO, the already existing client side encryption, and our future compression feature (LU-10092).

All three features need to allocate individual pages of memory for what are essentially bounce buffers.  The encryption feature uses a page pool, which seems like a good idea for the others.

This ticket is to try to settle on one way to do this and ideally have only a single page pool (if that's our solution).

sebastien , can you talk about why the page pool was chosen?  Was it performance or functionality?

I assume page pool performance is similar to or better than getting pages from the kernel allocator.  My only concern is wasting memory by having too large of a pool re-allocated pages.  In theory, we could need several GiB of bounce buffers if there is several GiB of outstanding IO.

However, it looks like there are shrinkers for the page pools, so if the pages are unused, the pool can be responsive to memory pressure.

So my first guess at a solution is this:
Have a single shared page pool ("lustre bounce buffers" or similar), with some fairly small pre-allocated size, then it can grow and shrink as needed.  Because page pools respond to kernel memory pressure, we shouldn't have the problem of holding on to unneeded memory.

So, we have one page pool for all our bounce buffers, and we don't worry too much about starting size except to keep it small, since the required size will be highly variable.  If a customer is not using any of these features, the total need is zero (so we should keep the cost small in this case).  If they are doing heavy IO while using one or more of these features, they could easily require several GiB at once.  So we pay the cost of allocating the necessary memory the first time it's used, then we don't have to pay that cost again unless the kernel asks us to shrink it.

How does this sound?  sebastien , adilger , ablagodarenko 

Issue Links

• is related to: LU-17422 unaligned DIO: use page pools

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/52634/
Subject: LU-16724 ptlrpc: refactor page pools patch 1
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1ec1858f64efbcf40d603eec4781ac29a3bf40eb

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/52335/
Subject: LU-16724 ptlrc: ptlrpc: extend sec bulk functionality
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 93af01f966a1222cf9bdbd32823b68e0a1d241bd

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52667
Subject: LU-16724 ptlrpc: rename 'pools' to 'ptr_pages'
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 31283ac63a783f82f5469bf8a01bf193d687836b

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52666
Subject: LU-16724 ptlrpc: rename npools to nptr_pages
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a24d26a514c73766eaeb3d43b66b5e4dea0363b4

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52665
Subject: LU-16724 ptlrpc: rename max_pools to max_ptr_pages
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ddfbc141aa299ff12ebb6d3de3a369b55692495b

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52664
Subject: LU-16724 ptlrpc: change "pool" to "page_ptrs"
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 6baf8186dfd1f196999a6593053266a320572c24

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52663
Subject: LU-16724 ptlrpc: remove PAGES_PER_POOL macro
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 459ce6decc1425cf5cc541cf8ce910959a662cff

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52662
Subject: LU-16724 ptlrpc: remove PAGES_POOL macro
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f2618ea7bc85183fda56e4e31552e483c4592652

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52652
Subject: LU-16724 ptlrpc: rename INDEX macros
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: cb5364cdbef5dcb1bb121d3f91620fa058fff3eb

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52651
Subject: LU-16724 ptlrpc: rename ppp_index to ppp_order
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 04fce416bb07af06d159752f7fdc3ef5625b9076

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52650
Subject: LU-16724 ptlrpc: begin renaming pool_index to order
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 70303af1034f0db94b62bc6c147b1426dd78a1e9