[LU-5783] o2iblnd: investigate new memory registration mechanisms

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.9.0
• Assignee: Dmitry Eremin (Inactive)
• Reporter: Isaac Huang (Inactive)

Description

The OFED has support of new memory registration mechanisms that the o2iblnd currently don't use.

• IB_WR_FAST_REG_MR: work request based asynchronous memory registration. It removes memory registration out of the critical path and has the potential of improving throughput.
• IB_WR_REG_INDIR_MR: allows registration of non-page aligned memory fragments. See:
  https://www.mail-archive.com/[e-mail]/msg21430.html

The 1st one may improve throughput and the 2nd one may solve the long-standing issue of "too many RDMA frags" failures we had (forgot which ticket, Liang?).

We should do some investigation and decide whether to add support of these OFED features to the o2iblnd.

Issue Links

• is related to: LU-3322 ko2iblnd support for different map_on_demand and peer_credits between systems
• is related to: LU-7124 MLX5: Limit hit in cap.max_send_wr
• is related to: LU-6850 Remove use of ib_reg_phys_mr() from o2iblnd