[v6,1/2] habanalabs: define uAPI to export FD for DMA-BUF
Commit Message
User process might want to share the device memory with another
driver/device, and to allow it to access it over PCIe (P2P).
To enable this, we utilize the dma-buf mechanism and add a dma-buf
exporter support, so the other driver can import the device memory and
access it.
The device memory is allocated using our existing allocation uAPI,
where the user will get a handle that represents the allocation.
The user will then need to call the new
uAPI (HL_MEM_OP_EXPORT_DMABUF_FD) and give the handle as a parameter.
The driver will return a FD that represents the DMA-BUF object that
was created to match that allocation.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
Reviewed-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
include/uapi/misc/habanalabs.h | 28 +++++++++++++++++++++++++++-
1 file changed, 27 insertions(+), 1 deletion(-)
Comments
On Sun, Sep 12, 2021 at 07:53:08PM +0300, Oded Gabbay wrote:
> /* HL_MEM_OP_* */
> __u32 op;
> - /* HL_MEM_* flags */
> + /* HL_MEM_* flags.
> + * For the HL_MEM_OP_EXPORT_DMABUF_FD opcode, this field holds the
> + * DMA-BUF file/FD flags.
> + */
> __u32 flags;
> /* Context ID - Currently not in use */
> __u32 ctx_id;
> @@ -1072,6 +1091,13 @@ struct hl_mem_out {
>
> __u32 pad;
> };
> +
> + /* Returned in HL_MEM_OP_EXPORT_DMABUF_FD. Represents the
> + * DMA-BUF object that was created to describe a memory
> + * allocation on the device's memory space. The FD should be
> + * passed to the importer driver
> + */
> + __u64 fd;
fd's should be a s32 type in a fixed width uapi.
I usually expect to see the uapi changes inside the commit that
consumes them, splitting the patch like this seems strange but
harmless.
Jason
On Tue, Sep 28, 2021 at 8:13 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Sun, Sep 12, 2021 at 07:53:08PM +0300, Oded Gabbay wrote:
> > /* HL_MEM_OP_* */
> > __u32 op;
> > - /* HL_MEM_* flags */
> > + /* HL_MEM_* flags.
> > + * For the HL_MEM_OP_EXPORT_DMABUF_FD opcode, this field holds the
> > + * DMA-BUF file/FD flags.
> > + */
> > __u32 flags;
> > /* Context ID - Currently not in use */
> > __u32 ctx_id;
> > @@ -1072,6 +1091,13 @@ struct hl_mem_out {
> >
> > __u32 pad;
> > };
> > +
> > + /* Returned in HL_MEM_OP_EXPORT_DMABUF_FD. Represents the
> > + * DMA-BUF object that was created to describe a memory
> > + * allocation on the device's memory space. The FD should be
> > + * passed to the importer driver
> > + */
> > + __u64 fd;
>
> fd's should be a s32 type in a fixed width uapi.
Yep, will correct this.
>
> I usually expect to see the uapi changes inside the commit that
> consumes them, splitting the patch like this seems strange but
> harmless.
I'll remember that when I send the RDMA patches down the road :)
Thanks,
Oded
>
> Jason
@@ -960,6 +960,10 @@ union hl_wait_cs_args {
#define HL_MEM_OP_UNMAP 3
/* Opcode to map a hw block */
#define HL_MEM_OP_MAP_BLOCK 4
+/* Opcode to create DMA-BUF object for an existing device memory allocation
+ * and to export an FD of that DMA-BUF back to the caller
+ */
+#define HL_MEM_OP_EXPORT_DMABUF_FD 5
/* Memory flags */
#define HL_MEM_CONTIGUOUS 0x1
@@ -1031,11 +1035,26 @@ struct hl_mem_in {
/* Virtual address returned from HL_MEM_OP_MAP */
__u64 device_virt_addr;
} unmap;
+
+ /* HL_MEM_OP_EXPORT_DMABUF_FD */
+ struct {
+ /* Handle returned from HL_MEM_OP_ALLOC. In Gaudi,
+ * where we don't have MMU for the device memory, the
+ * driver expects a physical address (instead of
+ * a handle) in the device memory space.
+ */
+ __u64 handle;
+ /* Size of memory allocation. Relevant only for GAUDI */
+ __u64 mem_size;
+ } export_dmabuf_fd;
};
/* HL_MEM_OP_* */
__u32 op;
- /* HL_MEM_* flags */
+ /* HL_MEM_* flags.
+ * For the HL_MEM_OP_EXPORT_DMABUF_FD opcode, this field holds the
+ * DMA-BUF file/FD flags.
+ */
__u32 flags;
/* Context ID - Currently not in use */
__u32 ctx_id;
@@ -1072,6 +1091,13 @@ struct hl_mem_out {
__u32 pad;
};
+
+ /* Returned in HL_MEM_OP_EXPORT_DMABUF_FD. Represents the
+ * DMA-BUF object that was created to describe a memory
+ * allocation on the device's memory space. The FD should be
+ * passed to the importer driver
+ */
+ __u64 fd;
};
};