# Pytorch Participant Example¶

This is an example of a PyTorch implementation of a `Participant`

class for Federated Learning.

We cover the requirements of the Participant Abstract Base Class, give ideas on how to handle a Pytorch Model and Pytorch Data in the `Participant`

, and show how to implement a federated learning Pytorch Training Round. You can find the complete source code here. The example code makes use of typing to be precise about the expected data types.

## Participant Abstract Base Class¶

The SDK provides an abstract base class for `Participant`

s which can be imported as

```
from xain_sdk.participant import Participant as ABCParticipant
```

A custom `Participant`

should inherit from the abstract base class, like

```
class Participant(ABCParticipant):
```

and must implement the `init_weights()`

and `train_round()`

methods in order to be able to execute a round of federated learning, where each round consists of a certain number of epochs. These methods adhere to the function signatures

```
init_weights(self) -> ndarray
train_round(self, weights: ndarray, epochs: int, epoch_base: int) -> Tuple[ndarray, int]
```

The expected arguments are:

`weights (ndarray)`

: A Numpy array containing the flattened weights of the global model.`epochs (int)`

: The number of epochs to be trained during the federated learning round. Can be any non-negative number including zero.`epoch_base (int)`

: A global training epoch number (e.g. for epoch dependent learning rate schedules and metrics logging).

The expected return values are:

`ndarray`

: The flattened weights of the local model which result from initialization or the global model after certain`epochs`

of training on local data.`int`

: The number of samples in the train dataset used for aggregation strategies.

The `Participant`

’s base class provides utility methods to set the weights of the local model according to the given flat weights vector, by

```
set_pytorch_weights(weights: ndarray, shapes: List[Tuple[int, ...]], model: Module) -> None
```

and to get a flattened weights vector from the local model, by

```
get_pytorch_weights(model: Module) -> ndarray
```

as well as the original shapes of the weights of the local model, by

```
get_pytorch_shapes(model: Module) -> List[Tuple[int, ...]]
```

Also, metrics of the current training epoch can be send to a time series data base via the coordinator by

```
update_metrics(epoch, epoch_base, MetricName=metric_value, ...)
```

for any number of metrics.

## Pytorch Model¶

A Pytorch model definition might either be loaded from a file, generated during the initialization of the `Participant`

, or even generated on the fly. Here, we present a simple dense neural network for classification generated during the `Participant`

’s initialization, which is wrapped in the `init_model()`

helper function.

The following attributes are only used to make the model configurable, via

```
self.features: int
self.units: int
self.categories: int
```

The example model inherits from

```
class DummyModel(Module):
```

and consists of an input layer holding `features`

parameters per sample. Next, it has a fully connected hidden layer with `units`

relu-activated units, as

```
self.hidden_layer: Linear = Linear(in_features=features, out_features=units, bias=True)
init.xavier_uniform_(self.hidden_layer.weight)
init.zeros_(self.hidden_layer.bias)
```

Finally, it has a fully connected output layer with `categories`

softmax-activated units, as

```
self.output_layer: Linear = Linear(in_features=units, out_features=categories, bias=True)
init.xavier_uniform_(self.output_layer.weight)
init.zeros_(self.output_layer.bias)
```

The layers are connected in the `forward()`

function, by

```
def forward(self, x: Tensor) -> Tensor:
x = functional.relu(self.hidden_layer(x))
x = functional.softmax(self.output_layer(x), dim=1)
return x
```

The model gets equiped with an Adam optimizer and the categorical crossentropy loss function, like

```
self.optimizer: Adam = Adam(params=self.parameters())
self.loss: CrossEntropyLoss = CrossEntropyLoss()
```

The utility method for setting the model weights require the original shapes of the weights, obtainable as

```
self.model_shapes: List[Tuple[int, ...]] = self.get_tensorflow_shapes(model=self.model)
```

## Pytorch Data¶

The data on which the model will be trained, can either be loaded from a data source (e.g. file, bucket, database) during the initialization of the `Participant`

or on the fly in a `train_round()`

. Here, we employ randomly generated placeholder data as an example, which is wrapped in the `init_datasets()`

helper function. This is by no means a meaningful dataset, but it should be sufficient to convey the overall idea.

The following attributes are only used to make the dataset configurable, via

```
self.train_samples: int
self.val_samples: int
self.test_samples: int
self.batch_size: int
```

The dataset for training gets shuffled and batched, like

```
self.trainset: DataLoader = DataLoader(
dataset=TensorDataset(
torch.from_numpy(np.ones(shape=(self.train_samples, self.features), dtype=np.float32)),
torch.from_numpy(
np.tile(
np.eye(self.categories, dtype=np.float32), reps=(int(np.ceil(self.train_samples / self.categories)), 1)
)[0 : self.train_samples, :]
).argmax(dim=1),
),
batch_size=self.batch_size,
shuffle=True,
)
```

while the datasets for validation and testing only get batched, like

```
self.valset: DataLoader = DataLoader(
dataset=TensorDataset(
torch.from_numpy(np.ones(shape=(self.val_samples, self.features), dtype=np.float32)),
torch.from_numpy(
np.tile(
np.eye(self.categories, dtype=np.float32), reps=(int(np.ceil(self.val_samples / self.categories)), 1)
)[0 : self.val_samples, :]
).argmax(dim=1),
),
batch_size=self.batch_size,
)
self.testset: DataLoader = DataLoader(
dataset=TensorDataset(
torch.from_numpy(
np.ones(shape=(self.test_samples, self.features), dtype=np.float32)
),
torch.from_numpy(
np.tile(
np.eye(self.categories, dtype=np.float32), reps=(int(np.ceil(self.test_samples / self.categories)), 1)
)[0 : self.test_samples, :]
).argmax(dim=1),
),
batch_size=self.batch_size,
)
```

## Pytorch Training Round¶

Whenever the coordinator needs to get freshly initialized model weights, e.g. in the 0-th round of the training, the `init_weights()`

method is called, which consists of two main steps. First, new model weights are initialized according to the model definition, and finally, these weights are returned without further training, as

```
self.init_model()
return self.get_pytorch_weights(model=self.model)
```

The implementation of the actual `train_round()`

method consists of three main steps. First, the provided `weights`

of the global model are loaded into the local model, as

```
self.set_pytorch_weights(weights=weights, shapes=self.model_shapes, model=self.model)
```

Next, the local model is trained for a certain number of `epochs`

on the local data, whereby the metrics are gathered in each epoch, as

```
for epoch in range(epochs):
for data, label in self.trainset:
self.model.optimizer.zero_grad()
self.model.loss(self.model(data), label).backward()
self.model.optimizer.step()
loss: float = 0.0
accuracy: float = 0.0
for data, label in self.valset:
prediction: Tensor = self.model(data)
loss += self.model.loss(prediction, label).item()
accuracy += (prediction.argmax(dim=1) == label).sum().float()
accuracy /= self.val_samples
self.update_metrics(epoch=epoch, epoch_base=epoch_base, Loss=loss, Accuracy=accuracy)
```

Finally, the updated weights of the local model and the number of samples of the train dataset are returned, as

```
return self.get_pytorch_weights(model=self.model), self.train_samples
```