Optimal Inner Product Manipulation (Opt-IPM)#

class byzfl.Optimal_InnerProductManipulation(agg=<byzfl.aggregators.aggregators.Average object>, pre_agg_list=[<byzfl.aggregators.preaggregators.Clipping object>], f=1, evals=20, start=0.0, delta=10.0, ratio=0.8)[source]#

Bases: object

Description#

Generalization of the Inner Product Manipulation (IPM) attack [1] by optimizing the attack factor \(\tau\).

\[\text{Opt-IPM}_{\textit{agg}, \textit{pre_agg_list}, f}(x_1, \dots, x_n) = - \tau_{opt} \cdot \frac{1}{n} \sum_{i=1}^{n} x_i\]

where

  • \(x_1, \dots, x_n\) are the input vectors, which conceptually correspond to correct gradients submitted by honest participants during a training iteration.

  • \(\textit{agg}\) is the robust aggregator to be used to aggregate the vectors during the training.

  • \(\textit{pre_agg_list}\) is the list of robust pre-aggregators to be used to transform the vectors during the training.

  • \(f\) conceptually represents the expected number of Byzantine vectors.

  • \(\tau_{opt} \in \mathbb{R}\) is the optimal attack factor found using a line-search optimization method.

This attack is designed to optimize the attack factor \(\tau\) of the IPM attack by maximizing a specific function. The function quantifies the effect of the attack, in particular the \(\ell_2\)-norm of the distance between the aggregated vectors (including Byzantine vectors) and the average of honest vectors. The goal is to find the attack factor that results in the maximum disruption.

This attack, developed by the ByzFL team, draws inspiration from the IPM attack and has been utilized in [2].

Initialization parameters:

  • agg (object, optional (default: Average)) – An instance of a robust aggregator that will be used to aggregate the vectors during the optimization process.

  • pre_agg_list (list, optional (default: [Clipping])) – A list of pre-aggregation functions, where each element is an object representing a pre-aggregation method.

  • f (int, optional (default: 1)) – The number of Byzantine participants. Must be a non-negative integer.

  • evals (int, optional (default: 20)) – The maximum number of evaluations during the optimization process. Must be a positive integer.

  • start (float, optional (default: 0.0)) – The initial attack factor to evaluate. Must be a float.

  • delta (float, optional (default: 10.0)) – The initial step size for the optimization process. Must be a non-zero float.

  • ratio (float, optional (default: 0.8)) – The contraction ratio used to reduce the step size during the contraction phase. Must be between 0.5 and 1 (both excluded).

Calling the instance

Input parameters:

vectors (numpy.ndarray, torch.Tensor, list of numpy.ndarray or list of torch.Tensor) – A set of vectors, matrix or tensors.

Returns:

numpy.ndarray or torch.Tensor – The data type of the output is the same as the input.

Examples

>>> import byzfl
>>> # Instantiate the robust aggregator
>>> agg = byzfl.TrMean(f=1)
>>> # Instantiate the list of pre-aggregators
>>> pre_agg_list = [byzfl.NNM(f=1), byzfl.Clipping()]
>>> # Instantiate the attack
>>> attack = byzfl.Optimal_InnerProductManipulation(agg, pre_agg_list=pre_agg_list, f=1)

Using numpy arrays

>>> import numpy as np
>>> x = np.array([[1., 2., 3.],       # np.ndarray
>>>               [4., 5., 6.],
>>>               [7., 8., 9.]])
>>> attack(x)
array([-2.98949673 -3.73687091 -4.48424509])

Using torch tensors

>>> import torch
>>> x = torch.tensor([[1., 2., 3.],   # torch.tensor
>>>                   [4., 5., 6.],
>>>                   [7., 8., 9.]])
>>> attack(x)
tensor([-2.9895, -3.7369, -4.4842])

Using list of numpy arrays

>>> import numpy as np
>>> x = [np.array([1., 2., 3.]),      # list of np.ndarray
>>>      np.array([4., 5., 6.]),
>>>      np.array([7., 8., 9.])]
>>> attack(x)
array([-2.98949673 -3.73687091 -4.48424509])

Using list of torch tensors

>>> import torch
>>> x = [torch.tensor([1., 2., 3.]),  # list of torch.tensor
>>>      torch.tensor([4., 5., 6.]),
>>>      torch.tensor([7., 8., 9.])]
>>> attack(x)
tensor([-2.9895, -3.7369, -4.4842])

References