network#

Module Contents#

DistNetwork

Manage torch.distributed network.

type2byte

type2byte#
class DistNetwork(address, world_size, rank, ethernet=None, dist_backend='gloo')#

Bases: object

Manage torch.distributed network.

Parameters

  • address (tuple) – Address of this server in form of (SERVER_ADDR, SERVER_IP)

  • world_size (int) – the size of this distributed group (including server).

  • rank (int) – the rank of process in distributed group.

  • ethernet (str) –

  • dist_backend (str or torch.distributed.Backend) – backend of torch.distributed. Valid values include mpi, gloo, and nccl. Default: "gloo".

init_network_connection(self)#

Initialize torch.distributed communication group

close_network_connection(self)#

Destroy current torch.distributed process group

send(self, content=None, message_code=None, dst=0, count=True)#

Send tensor to process rank=dst

recv(self, src=None, count=True)#

Receive tensor from process rank=src

__str__(self)#

Return str(self).