[rdma-next,0/8] Introducing Multi-Path DMA Support for mlx5 RDMA Driver

From: Leon Romanovsky <leonro@nvidia.com>

From Yishai,

Overview
--------
This patch series aims to enable multi-path DMA support, allowing an
mlx5 RDMA device to issue DMA commands through multiple paths. This
feature is critical for improving performance and reaching line rate
in certain environments where issuing PCI transactions over one path
may be significantly faster than over another. These differences can
arise from various PCI generations in the system or the specific system
topology.

To achieve this functionality, we introduced a data direct DMA device
that can serve the RDMA device by issuing DMA transactions on its behalf.

The main key features and changes are described below.

Multi-Path Discovery
--------------------
API Implementation:
 * Introduced an API to discover multiple paths for a given mlx5 RDMA device.
IOCTL Command: 
 * Added a new IOCTL command, MLX5_IB_METHOD_GET_DATA_DIRECT_SYSFS_PATH, to
   the DEVICE object. When an affiliated Data-Direct/DMA device is present,
   its sysfs path is returned.

Feature Activation by mlx5 RDMA Application
-------------------------------------------
UVERBS Extension:
 * Extended UVERBS_METHOD_REG_DMABUF_MR over UVERBS_OBJECT_MR to include
   mlx5 extended flags.
Access Flag: 
 * Introduced the MLX5_IB_UAPI_REG_DMABUF_ACCESS_DATA_DIRECT flag, allowing
   applications to request the use of the affiliated DMA device for DMABUF
   registration.

Data-Direct/DMA Device
----------------------
New Driver:
 * Introduced a new driver to manage the new DMA PF device ID (0x2100).
   Its registration/un-registration is handled as part of the mlx5_ib init/exit
   flows, with mlx5 IB devices as its clients.
Functionality: 
 * The driver does not interface directly with the firmware (no command interface,
   no caps, etc.) but works over PCI to activate its DMA functionality. It serves
   as the DMA device for efficiently accessing other PCI devices (e.g., GPU PF) and
   reads its VUID over PCI to handle NICs registrations with the same VUID.

mlx5 IB RDMA Device
---------------------------
VUID Query: 
 * Reads its affiliated DMA PF VUID via the QUERY_VUID command with the data_direct
   bit set.
Driver Registration:
 * Registers with the DMA PF driver to be notified upon bind/unbind.
Application Request Handling: 
 * Uses the DMA PF device upon application request as described above.

DMABUF over Umem
----------------
Introduced an option to obtain a DMABUF UMEM using a different DMA
device instead of the IB device, allowing the device to register over
IOMMU with the expected DMA device for a given buffer registration.

Further details are provided in the commit logs of the patches in this
series.

Thanks

Yishai Hadas (8):
  net/mlx5: Add IFC related stuff for data direct
  RDMA/mlx5: Introduce the 'data direct' driver
  RDMA/mlx5: Add the initialization flow to utilize the 'data direct'
    device
  RDMA/umem: Add support for creating pinned DMABUF umem with a given
    dma device
  RDMA/umem: Introduce an option to revoke DMABUF umem
  RDMA: Pass uverbs_attr_bundle as part of '.reg_user_mr_dmabuf' API
  RDMA/mlx5: Add support for DMABUF MR registrations with Data-direct
  RDMA/mlx5: Introduce GET_DATA_DIRECT_SYSFS_PATH ioctl

 drivers/infiniband/core/umem_dmabuf.c         |  66 +++-
 drivers/infiniband/core/uverbs_std_types_mr.c |   2 +-
 drivers/infiniband/hw/bnxt_re/ib_verbs.c      |   3 +-
 drivers/infiniband/hw/bnxt_re/ib_verbs.h      |   2 +-
 drivers/infiniband/hw/efa/efa.h               |   2 +-
 drivers/infiniband/hw/efa/efa_verbs.c         |   4 +-
 drivers/infiniband/hw/irdma/verbs.c           |   2 +-
 drivers/infiniband/hw/mlx5/Makefile           |   1 +
 drivers/infiniband/hw/mlx5/cmd.c              |  21 ++
 drivers/infiniband/hw/mlx5/cmd.h              |   2 +
 drivers/infiniband/hw/mlx5/data_direct.c      | 227 +++++++++++++
 drivers/infiniband/hw/mlx5/data_direct.h      |  23 ++
 drivers/infiniband/hw/mlx5/main.c             | 125 +++++++
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  22 +-
 drivers/infiniband/hw/mlx5/mr.c               | 304 +++++++++++++++---
 drivers/infiniband/hw/mlx5/odp.c              |   5 +-
 drivers/infiniband/hw/mlx5/std_types.c        |  55 +++-
 drivers/infiniband/hw/mlx5/umr.c              |  93 ++++--
 drivers/infiniband/hw/mlx5/umr.h              |   1 +
 include/linux/mlx5/mlx5_ifc.h                 |  51 ++-
 include/rdma/ib_umem.h                        |  18 ++
 include/rdma/ib_verbs.h                       |   2 +-
 include/uapi/rdma/mlx5_user_ioctl_cmds.h      |   9 +
 include/uapi/rdma/mlx5_user_ioctl_verbs.h     |   4 +
 24 files changed, 944 insertions(+), 100 deletions(-)
 create mode 100644 drivers/infiniband/hw/mlx5/data_direct.c
 create mode 100644 drivers/infiniband/hw/mlx5/data_direct.h
  

On Thu, 01 Aug 2024 15:05:09 +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
> 
> From Yishai,
> 
> Overview
> --------
> This patch series aims to enable multi-path DMA support, allowing an
> mlx5 RDMA device to issue DMA commands through multiple paths. This
> feature is critical for improving performance and reaching line rate
> in certain environments where issuing PCI transactions over one path
> may be significantly faster than over another. These differences can
> arise from various PCI generations in the system or the specific system
> topology.
> 
> [...]

Applied, thanks!

[2/8] RDMA/mlx5: Introduce the 'data direct' driver
      https://git.kernel.org/rdma/rdma/c/281658bd04e7b9
[3/8] RDMA/mlx5: Add the initialization flow to utilize the 'data direct' device
      https://git.kernel.org/rdma/rdma/c/302b01afc28b1e
[4/8] RDMA/umem: Add support for creating pinned DMABUF umem with a given dma device
      https://git.kernel.org/rdma/rdma/c/b047ecbd7672d2
[5/8] RDMA/umem: Introduce an option to revoke DMABUF umem
      https://git.kernel.org/rdma/rdma/c/bc9be75e01373c
[6/8] RDMA: Pass uverbs_attr_bundle as part of '.reg_user_mr_dmabuf' API
      https://git.kernel.org/rdma/rdma/c/83f44068da564d
[7/8] RDMA/mlx5: Add support for DMABUF MR registrations with Data-direct
      https://git.kernel.org/rdma/rdma/c/19ae08911f8be1
[8/8] RDMA/mlx5: Introduce GET_DATA_DIRECT_SYSFS_PATH ioctl
      https://git.kernel.org/rdma/rdma/c/d222b19c595f63

Best regards,