Speaker：Prof. Huang, Su-Yun (Institute of Statistical Science, Academia Sinica)
Topic：Robust aggregation for federated learning by minimum gamma-divergence estimation
Speaker：Prof. Huang, Su-Yun (Institute of Statistical Science, Academia Sinica)
Date Time：Fri. Jun 10, 2022, 10:40 AM - 11:30 AM
Place：Online Seminars
Online Seminars- Google Meet
Abstract
Federated learning is a framework for multiple devices or institutions, called local clients, to collaboratively train a global model without sharing their data. For federated learning with a central server, an aggregation algorithm integrates model information sent from local clients to update the parameters for a global model. Sample mean is the simplest and most commonly used aggregation method. However, it is not robust for data with outliers or under the Byzantine problem, where Byzantine clients send malicious messages to interfere with the learning process. Some robust aggregation methods were introduced in literature including marginal median, geometric median and trimmed-mean. In this talk, we will introduce an alternative robust aggregation method, named
Gamma-mean, which is the minimum divergence estimation based on a robust density power divergence. This gamma-mean aggregation mitigates the influence of Byzantine clients by assigning fewer weights. This weighting scheme is data-driven and controlled by the gamma value. Robustness from the viewpoint of the influence function is discussed and some numerical results are presented.
Gamma-mean, which is the minimum divergence estimation based on a robust density power divergence. This gamma-mean aggregation mitigates the influence of Byzantine clients by assigning fewer weights. This weighting scheme is data-driven and controlled by the gamma value. Robustness from the viewpoint of the influence function is discussed and some numerical results are presented.