MoNNA#

class byzfl.MoNNA(f=0, idx=0)[source]#

Description#

Apply the MoNNA aggregator [1]:

\[\mathrm{MoNNA}_{f, \mathrm{idx}} \ (x_1, \dots, x_n) = \frac{1}{n-f} \sum_{i \in \mathit{N}_{\mathrm{idx}+1}} x_{i}\]

where

  • \(x_1, \dots, x_n\) are the input vectors, which conceptually correspond to gradients submitted by honest and Byzantine participants during a training iteration.

  • \(f\) conceptually represents the expected number of Byzantine vectors.

  • \(\mathit{N}_{i}\) is the set of the \(n − f\) nearest neighbors of \(x_{i}\) in \(\{x_1, \dots , x_n\}\).

  • \(\mathrm{idx} \in \{0, \dots, n-1\}\) is the ID of the chosen worker/vector for which the neighborhood is computed. In other words, \(x_{\mathrm{idx}+1}\) is the vector sent by the worker with ID \(\mathrm{idx}\).

Therefore, MoNNA computes the average of the \(n − f\) nearest neighbors of the chosen vector with ID \(\mathrm{idx}\).

Initialization parameters:

  • f (int, optional) – Number of faulty vectors. Set to 0 by default.

  • idx (int, optional) – Index of the vector for which the neighborhood is computed. Set to 0 by default.

Calling the instance

Input parameters:

vectors (numpy.ndarray, torch.Tensor, list of numpy.ndarray or list of torch.Tensor) – A set of vectors, matrix or tensors.

Returns:

numpy.ndarray or torch.Tensor – The data type of the output will be the same as the input.

Note

MoNNA is used in peer-to-peer settings where \(\mathrm{idx}\) corresponds to the ID of a vector that is trusted to be correct (i.e., not faulty).

Examples

>>> import byzfl
>>> agg = byzfl.MoNNA(1, 1)

Using numpy arrays

>>> import numpy as np
>>> x = np.array([[1., 2., 3.],       # np.ndarray
>>>               [4., 5., 6.],
>>>               [7., 8., 9.]])
>>> agg(x)
array([2.5, 3.5, 4.5])

Using torch tensors

>>> import torch
>>> x = torch.tensor([[1., 2., 3.],   # torch.tensor
>>>                   [4., 5., 6.],
>>>                   [7., 8., 9.]])
>>> agg(x)
tensor([2.5000, 3.5000, 4.5000])

Using list of numpy arrays

>>> import numpy as np
>>> x = [np.array([1., 2., 3.]),      # list of np.ndarray
>>>      np.array([4., 5., 6.]),
>>>      np.array([7., 8., 9.])]
>>> agg(x)
array([2.5, 3.5, 4.5])

Using list of torch tensors

>>> import torch
>>> x = [torch.tensor([1., 2., 3.]),  # list of torch.tensor
>>>      torch.tensor([4., 5., 6.]),
>>>      torch.tensor([7., 8., 9.])]
>>> agg(x)
tensor([2.5000, 3.5000, 4.5000])

References

On this page