[LU-8874] Adapt ko2iblnd to latest RDMA changes

• Status: In Progress
• Project: Lustre
• Affects Version/s: Upstream
• Fix Version/s: Upstream
• Type: Epic
• Priority: Critical
• Reporter: Doug Oucharek (Inactive)
• Assignee: Amir Shehata (Inactive)
• Resolution: Unresolved
• Labels: lnet

Sub-Tasks

Key	Summary	Type	Status	Assignee
LU-8875	Change to new RDMA done callback mech...	Technical task	In Progress	James A Simmons
LU-9026	Adapt to the removal of ib_get_dma_mr()	Technical task	Resolved	Doug Oucharek
LU-9164	Incorporate RDMA map/unmap APIs into ...	Technical task	Open	Amir Shehata
LU-10914	Incorporate new FastReg API	Technical task	Open	Amir Shehata
LU-10915	Draining QP	Technical task	Open	Amir Shehata

• Key: Summary
• Summary: Type
• Type: Status
• Status: Assignee

Description

A series of changes were made to the Linux kernel RDMA code which we need to adapt ko2iblnd to. Seems those changes will trigger deprecation of older mechanisms we are using.

The changes fall into two areas (and I will be creating two sub-tasks to track each):

1. Shift from ib_create_cq() to ib_alloc_cq() (this includes a change to how done callbacks are invoked).
2. Change to IB_WR_REG_MR/IB_WR_LOCAL_INV from FMR's and MR
that allow access to all of memory,

For details on item 1, see Linux commits: 765d67748 from Steve Wise and 14d3a3b2 from Christoph.

For details on item 2, see Linux commits: f7f7aab1a (srp adaptation) and 1dc7b1f10 (cq change).

Issue Links

• is related to: LU-12511 Prepare lustre for adoption into the ...
• is related to: LU-12511 Prepare lustre for adoption into the ...

Comments

[Doug Oucharek (Inactive)]

A third subtask was created to adapt ko2iblnd to the RDMA memory region support code.

[James A Simmons]

Thank you Doug for helping out.

[James A Simmons]

The GPU RDMA changes being pushed are incompatible with the request upstream to move ko2iblnd to the generic RDMA API. Do we just update the native linux client ko2iblnd driver to this API alone and just drop the idea of moving to the Generic RDMA API for the OpenSFS tree?