-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
Behind a translating IOMMU, every dma_map_sg() allocates one IOVA
range for the whole scatterlist. The IOMMU layer only caches ranges
up to dma_opt_mapping_size() (128 KiB on current kernels) per CPU.
o2iblnd maps a whole LNet bulk (up to 1 MiB, 256 pages) in one call,
so every bulk map and unmap goes to the rbtree allocator under the
IOMMU domain's iova_rbtree_lock, first trying the 32-bit window. On
RoCE clients with the IOMMU in translated mode, Lustre bulk throughput
drops to 1/2 or 1/3 of line rate on about a third of boots. With
iommu=pt, 0 of 21 boots degraded, versus 8 of 21 without it.