Class PrePostProcessor

Inheritance Relationships

Derived Types

Class Documentation

class switchml::PrePostProcessor

A PrePostProcessor (PPP) is an object that handles loading and unloading of the data between the client and the network.

Depending on the implementation, the PPP may convert the representation of the data (maybe even compress it) and may require extra information or metadata to be sent so that it can undo its representation changes.

In the prepostprocessor we use ‘LTU’ to refer to the logical unit of transmission that the backend will use. But the prepostprocessor itself does not care what that “logical transmission unit” really is. Its just dealing with a series of blocks of data that is being sent and received. Call it a packet (for dpdk), a block, a message (in rdma).

Subclassed by switchml::BypassPPP, switchml::CpuExponentQuantizerPPP

Public Functions

~PrePostProcessor() = default
PrePostProcessor(PrePostProcessor const&) = delete
void operator=(PrePostProcessor const&) = delete
PrePostProcessor(PrePostProcessor&&) = default
PrePostProcessor &operator=(PrePostProcessor&&) = default
virtual uint64_t SetupJobSlice(JobSlice *job_slice) = 0

Setup the PPP’s internal structures and prepare to start processing the passed job slice.

Parameters

job_slice – [in] A pointer to the job slice currently being worked on by the worker thread.

Returns

uint64_t the number of transmission units that prepostprocessor will need to be sent and received by the backend so that the whole tensor is processed. (This does not include LTUs from the extra batch that might be needed). We let the PPP return this information so that the backend is aware in case the PPP reduces the size of the data and thus needs a smaller number of LTUs to be transmitted.

virtual bool NeedsExtraBatch() = 0

Check whether this prepostprocessor needs to send an extra batch for the current job slice or not.

Some prepostprocessor’s need extra info / metadata to be sent along the payload so that they convert between representations correctly. And they usually need that extra info to be present before the first real batch is sent. In that case the backend sends an extra first batch to make this information available for the first real batch later.

Returns

true If the prepostprocessor needs an extra batch

Returns

false If it doesn’t

virtual void PreprocessSingle(uint64_t ltu_id, void *entries_ptr, void *extra_info = nullptr) = 0

Preprocess an LTU converting its representation if needed and moving its payload into the backend’s buffers.

See

PostprocessSingle()

Parameters

  • ltu_id – [in] The id of the logical transmission unit to be preprocessed within the current job slice. ltu_id will be used to compute the offset into the job slice ltu_id * ltu_size.

  • entries_ptr – [out] A pointer to where we will store the payload to be ready for transmission.

  • extra_info – [out] A pointer to where we will store the extra info if we need it.

virtual void PostprocessSingle(uint64_t ltu_id, void *entries_ptr, void *extra_info = nullptr) = 0

Postprocess an LTU converting it to the original representation if needed and moving its payload into the client’s buffers.

See

PreprocessSingle()

Parameters

  • ltu_id – [in] The id of the logical transmission unit to be postprocessed within the current job slice. ltu_id will be used to compute the offset into the job slice ltu_id * ltu_size.

  • entries_ptr – [in] A pointer to where we will read the received payload from.

  • extra_info – [in] A pointer to where we will read the extra info from if we need it.

virtual void CleanupJobSlice() = 0

Cleans up all internal structures and released any dynamically allocated memory associated with the job slice.

See

SetupJobSlice()

Public Static Functions

static std::shared_ptr<PrePostProcessor> CreateInstance(Config &config, WorkerTid worker_tid, Numel ltu_size, Numel batch_num_ltus)

Create an instance of the prepostprocessor specified in the configuration passed.

Parameters

  • config – [in] A reference to the context’s configuration.

  • worker_tid – [in] The worker thread that this prepostprocessor belongs to.

  • ltu_size – [in] The size in bytes of the logical transmission unit used by the backend.

  • batch_num_ltus – [in] How many LTUs constitute a batch.

Returns

std::shared_ptr<PrePostProcessor> a shared pointer to the prepostprocessor’s created instance.

Protected Functions

PrePostProcessor(Config &config, WorkerTid worker_tid, Numel ltu_size, Numel batch_num_ltus)

Construct a new PrePostProcessor object.

Parameters

  • config – [in] A reference to the context configuration

  • worker_tid – [in] The worker thread that this prepostprocessor belongs to

  • ltu_size – [in] The size in bytes of the logical transmission unit used by the backend.

  • batch_num_ltus – [in] How many LTUs constitute a batch.

Protected Attributes

Config &config_

A reference to the context configuration

WorkerTid worker_tid_

The worker thread that this prepostprocessor belongs to

Numel ltu_size_

The size in bytes of the logical transmission unit used by the backend.

Numel batch_max_num_ltus_

What’s the maximum number of LTUs that can constitute a batch.