Centered Clipping#

class byzfl.CenteredClipping(m=None, L=1, tau=100.0)[source]#

Description#

Apply the Centered Clipping aggregator [1]:

\[\mathrm{CenteredClipping}_{m, L, \tau} \ (x_1, \dots, x_n) = v_{L}\]

where

  • \(x_1, \dots, x_n\) are the input vectors, which conceptually correspond to gradients submitted by honest and Byzantine participants during a training iteration.

  • \(\big|\big|.\big|\big|_2\) denotes the \(\ell_2\)-norm.

  • \(v_0 = m\).

  • \(v_{l+1} = v_{l} + \frac{1}{n}\sum_{i=1}^{n}(x_i - v_l)\min\left(1, \frac{\tau}{\big|\big|x_i - v_l\big|\big|_2}\right) \ \ ; \ \forall l \in \{0,\dots, L-1\}\).

Initialization parameters:

  • m (numpy.ndarray, torch.Tensor, optional) – Initial value of the CenteredClipping aggregator. Default (None) makes it start from zero, a vector with all its coordinates equal to 0.

  • L (int, optional) – Number of iterations. Default is set to 1.

  • tau (float, optional) – Clipping threshold. Default is set to 100.0.

Calling the instance

Input parameters:

vectors (numpy.ndarray, torch.Tensor, list of numpy.ndarray or list of torch.Tensor) – A set of vectors, matrix or tensors.

Returns:

numpy.ndarray or torch.Tensor – The data type of the output will be the same as the input.

Note

If the instance is called more than once, the value of \(m\) used in the next call is equal to the output vector of the previous call.

Note

In case the optional parameter \(m\) is specified when initializing the instance, \(m\) has to be of the same type and shape as the input vectors \(\{x_1, \dots, x_n\}\) used when calling the instance.

Examples

>>> import byzfl
>>> agg = byzfl.CenteredClipping()

Using numpy arrays

>>> import numpy as np
>>> x = np.array([[1., 2., 3.],       # np.ndarray
>>>               [4., 5., 6.],
>>>               [7., 8., 9.]])
>>> agg(x)
array([4., 5., 6.])

Using torch tensors

>>> import torch
>>> x = torch.tensor([[1., 2., 3.],   # torch.tensor
>>>                   [4., 5., 6.],
>>>                   [7., 8., 9.]])
>>> agg(x)
tensor([4., 5., 6.])

Using list of numpy arrays

>>> import numpy as np
>>> x = [np.array([1., 2., 3.]),      # list of np.ndarray
>>>      np.array([4., 5., 6.]),
>>>      np.array([7., 8., 9.])]
>>> agg(x)
array([4., 5., 6.])

Using list of torch tensors

>>> import torch
>>> x = [torch.tensor([1., 2., 3.]),  # list of torch.tensor
>>>      torch.tensor([4., 5., 6.]),
>>>      torch.tensor([7., 8., 9.])]
>>> agg(x)
tensor([4., 5., 6.])

References

On this page