Robust Aggregator#

The RobustAggregator class provides a robust mechanism for aggregating gradients in federated learning setups. It combines pre-aggregation techniques with robust aggregation methods to mitigate the effects of adversarial inputs and outliers, ensuring reliable updates for the global model.

Key Features#

Pre-Aggregation: Supports applying multiple pre-aggregation techniques sequentially, such as Static Clipping and Nearest Neighbor Mixing (NNM), to refine the input gradients.
Robust Aggregation: Implements aggregation strategies like Trimmed Mean to handle Byzantine gradients and ensure robustness against adversarial attacks.
Flexible Input Handling: Compatible with various input formats, including NumPy arrays, PyTorch tensors, and lists of these data types.

class byzfl.RobustAggregator(aggregator_info, pre_agg_list=[])[source]#

Bases: object

Initialization Parameters:

aggregator_info (dict) – A dictionary specifying the aggregation method and its parameters.
- Keys:
  - “name”: str
    Name of the aggregation method (e.g., “TrMean”).
  - “parameters”: dict
    A dictionary of parameters required by the specified aggregation method.
pre_agg_list (list, optional (default: [])) – A list of dictionaries, each specifying a pre-aggregation method and its parameters.
- Keys:
  - “name”: str
    Name of the pre-aggregation method (e.g., “NNM”).
  - “parameters”: dict
    A dictionary of parameters required by the specified pre-aggregation method.

aggregate_vectors(vectors)[source]#: Applies the specified pre-aggregation and aggregation methods to the input vectors, returning the aggregated result.

Calling the Instance

Input Parameters:: vectors (numpy.ndarray, torch.Tensor, list of numpy.ndarray, or list of torch.Tensor) – A collection of input vectors, matrices, or tensors to process. These vectors conceptually correspond to gradients submitted by honest and Byzantine participants during a training iteration.
Returns:: numpy.ndarray or torch.Tensor – The aggregated output vector with the same data type as the input.

Examples

Initialize the RobustAggregator with both pre-aggregation and aggregation methods:

>>> from byzfl import RobustAggregator
>>> # Define pre-aggregation methods
>>> pre_aggregators = [
>>>     {"name": "Clipping", "parameters": {"c": 2.0}},
>>>     {"name": "NNM", "parameters": {"f": 1}},
>>> ]
>>> # Define an aggregation method
>>> aggregator_info = {"name": "TrMean", "parameters": {"f": 1}}
>>> # Create the RobustAggregator instance
>>> rob_agg = RobustAggregator(aggregator_info, pre_agg_list=pre_aggregators)

Apply the RobustAggregator to various types of input data:

Using NumPy arrays:

>>> import numpy as np
>>> vectors = np.array([[1., 2., 3.], [4., 5., 6.], [7., 8., 9.]])
>>> rob_agg.aggregate_vectors(vectors)
array([0.95841302, 1.14416941, 1.3299258])

Using PyTorch tensors:

>>> import torch
>>> vectors = torch.tensor([[1., 2., 3.], [4., 5., 6.], [7., 8., 9.]])
>>> rob_agg.aggregate_vectors(vectors)
tensor([0.9584, 1.1442, 1.3299])

Using a list of NumPy arrays:

>>> import numpy as np
>>> vectors = [np.array([1., 2., 3.]), np.array([4., 5., 6.]), np.array([7., 8., 9.])]
>>> rob_agg.aggregate_vectors(vectors)
array([0.95841302, 1.14416941, 1.3299258])

Using a list of PyTorch tensors:

>>> import torch
>>> vectors = [torch.tensor([1., 2., 3.]), torch.tensor([4., 5., 6.]), torch.tensor([7., 8., 9.])]
>>> rob_agg.aggregate_vectors(vectors)
tensor([0.9584, 1.1442, 1.3299])

aggregate_vectors(vectors)[source]#

Applies the configured pre-aggregations and robust aggregation method to the input vectors.

Parameters:: vectors (numpy.ndarray, torch.Tensor, list of numpy.ndarray, or list of torch.Tensor) – A collection of input vectors to process.
Returns:: numpy.ndarray or torch.Tensor – The aggregated output vector with the same data type as the input.

Notes#

The pre-aggregation methods are applied in the order they are listed in pre_agg_list.
The robust aggregation method is applied after all pre-aggregation techniques are completed.