[LU-6708] osd-zfs: multiple OSTs in a same zpool breaks grants

• Status: Open
• Project: Lustre
• Type: Improvement
• Priority: Minor
• Reporter: Isaac Huang (Inactive)
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: zfs

Description

As Johann just explained to me: the OSS grants clients certain space due to client side write caching, to guarantee space would be available when the cache is flushed. With multiple OSTs in a same zpool, the free space of the pool is reported as available to all OSTs. Then the OSTs would believe they each have available the free space of the pool, while they actually share that space. Therefore grants would be broken when multiple OSTs reside in a same zpool.

If there's no use case for multiple OSTs in a same zpool, we should improve our documents to point out that it's not supported. Otherwise, we'd modify osd-zfs to make use of space reservation mechanisms to guarantee grant space availability when multiple OSTs share a same pool:
http://docs.oracle.com/cd/E19253-01/819-5461/gazvb/index.html

For example: zfs set reservation=10G pool/ost1

Comments

[Alex Zhuravlev]

I think we decided long ago do not put few OSTs into a single ZFS pool. for the number of reasons: space management, ro capability, etc.

[Isaac Huang (Inactive)]

Then we'd make it clear in the documentation. Maybe I missed something but I couldn't find any doc on the fact that multiple OSTs per pool is not supported.