Class RdmaConnection

Class Documentation

class switchml::RdmaConnection

The RdmaConnection represents the connection to both the controller and the switch.

It is used by the backend to setup the connection by exchanging the needed information with the controller via the GrpcClient, sets up and brings up queue pairs, and finally worker threads then use it to send and receive messages.

Public Functions

RdmaConnection(Config &config)

Initialize all members and allocate buffer memory region.

Parameters

config – [in] a reference to the switchml configuration.

~RdmaConnection()
RdmaConnection(RdmaConnection const&) = delete
void operator=(RdmaConnection const&) = delete
RdmaConnection(RdmaConnection&&) = default
RdmaConnection &operator=(RdmaConnection&&) = default
void Connect()

Performs all needed setup and bringup to establish the RDMA connection.

This should be the first function to be called after creating the object. After calling this function, you can go ahead and use the getters to access the created queue pairs, memory region and so on. You can also then use the PostSend() and PostRecv() functions to send and receive messages.

ibv_cq *GetWorkerThreadCompletionQueue(WorkerTid worker_thread_id)
std::vector<ibv_qp*> GetWorkerThreadQueuePairs(WorkerTid worker_thread_id)

Get the range of queue pairs corresponding to a worker thread.

Parameters

worker_thread_id – [in]

Returns

std::vector<ibv_qp*>

std::vector<uint32_t> GetWorkerThreadRkeys(WorkerTid worker_thread_id)

Get the range of rkeys corresponding to a worker thread.

Parameters

worker_thread_id –

Returns

std::vector<uint32_t>

std::pair<void*, uint32_t> GetWorkerThreadMemoryRegion(WorkerTid worker_thread_id)

Get the memory region information corresponding to a worker thread.

Parameters

worker_thread_id –

Returns

std::pair<void*, uint32_t> first element is the first address in the memory region that the worker thread can access. Second element is the lkey of the memory region.

RdmaEndpoint &GetEndpoint()

Get the underlying used endpoint.

Returns

RdmaEndpoint&

Public Static Functions

static void PostRecv(ibv_qp *qp, ibv_recv_wr *wr)

Post receive work request and check its success.

Parameters

  • qp – [in] The queue pair to use.

  • wr – [in] The receive work request to post

static void PostSend(ibv_qp *qp, ibv_send_wr *wr)

Post send work request and check its success.

Parameters

  • qp – [in] The queue pair to use

  • wr – [in] The send work request to post.