hierarchical#

Package Contents#

ClientConnector

Connect with clients.

ServerConnector

Connect with server.

Scheduler

Middle Topology for hierarchical communication pattern
class ClientConnector(network, write_queue, read_queue)#

Bases: Connector

Connect with clients.

This class is a part of middle server which used in hierarchical structure.

Parameters

  • network (DistNetwork) – Manage torch.distributed network communication.

  • write_queue (torch.multiprocessing.Queue) – Message queue to write.

  • read_queue (torch.multiprocessing.Queue) – Message queue to read.

run(self)#

Main Process:

  1. Initialization stage.

  2. FL communication stage.

  3. Shutdown stage. Close network connection.

on_receive(self, sender, message_code, payload)#

Define the reaction of receiving message.

Parameters

  • sender (int) – rank of sender in dist group.

  • message_code (MessageCode) – Message code.

  • payload (list(torch.Tensor)) – A list of tensors received from other process.

deal_queue(self)#

Process message queue

Strategy of processing message from server.

class ServerConnector(network, write_queue, read_queue)#

Bases: Connector

Connect with server.

This class is a part of middle server which used in hierarchical structure.

Parameters

  • network (DistNetwork) – object to manage torch.distributed network communication.

  • write_queue (torch.multiprocessing.Queue) – message queue

  • read_queue (torch.multiprocessing.Queue) – message queue

run(self)#

Main Process:

  1. Initialization stage.

  2. FL communication stage.

  3. Shutdown stage. Close network connection.

on_receive(self, sender, message_code, payload)#

Define the reaction of receiving message.

Parameters

  • sender (int) – rank of sender in dist group.

  • message_code (MessageCode) – Message code.

  • payload (list(torch.Tensor)) – A list of tensors received from other process.

deal_queue(self)#

Define the procedure of dealing with message queue.

class Scheduler(net_upper, net_lower)#

Bases: torch.multiprocessing.Process

Middle Topology for hierarchical communication pattern

Scheduler uses message queues to decouple connector modules

Parameters

  • net_upper (DistNetwork) – Distributed network manager of server from upper level.

  • net_lower (DistNetwork) – Distributed network manager of clients from lower level.

run(self)#