Tsukada Lab. Computer Network Group

Limitations and Future Aspects of Communication Costs in Federated Learning: A Survey

Muhammad Asad, Saima Shaukat, Dou Hu, Zekun Wang, Ehsan Javanmardi, Jin Nakazato, Manabu Tsukada

Sensors, vol. 23, no. 17, Aug 23, 2023

Summary in three lines

  • This survey examines communication costs in federated learning across distributed clients.
  • It groups approaches into model updates, compression, edge and cloud resource management, and client selection.
  • It reviews open problems and future directions for communication-efficient training.

Figures from the paper

Figure 1: Centralized, distributed, and federated learning compared
Figure 1. Centralized, distributed, and federated learning comparedSource: Figure 1 from Asad et al., Sensors, 2023 (© 2023 The Authors, CC BY 4.0, publisher version)The diagram shows where data and training are located and what information moves between clients and servers.

Abstract

This paper explores the potential for communication-efficient federated learning (FL) in modern distributed systems. FL is an emerging distributed machine learning technique that allows for distributed training of a single machine learning model across multiple geographically distributed clients. This paper surveys the various approaches to communication-efficient FL, including model updates, compression techniques, resource management for edge and cloud, and client selection. We also review the various optimization techniques associated with communication-efficient FL, such as compression schemes and structured updates. Finally, we highlight the current research challenges and discuss the potential future directions for communication-efficient FL.

Related research and software

How to cite

M. Asad, S. Shaukat, D. Hu, Z. Wang, E. Javanmardi, J. Nakazato, and M. Tsukada, “Limitations and Future Aspects of Communication Costs in Federated Learning: A Survey,” Sensors, vol. 23, no. 17, 2023, doi: 10.3390/s23177358.

BibTeX

@article{Asad2023b,
  title         = {Limitations and Future Aspects of Communication Costs in Federated Learning: A Survey},
  author        = {Muhammad Asad and Saima Shaukat and Dou Hu and Zekun Wang and Ehsan Javanmardi and Jin Nakazato and Manabu Tsukada},
  doi           = {10.3390/s23177358},
  issn          = {1424-8220},
  year          = {2023},
  date          = {2023-08-23},
  urldate       = {2023-08-23},
  journal       = {Sensors},
  volume        = {23},
  number        = {17},
  abstract      = {This paper explores the potential for communication-efficient federated learning (FL) in modern distributed systems. FL is an emerging distributed machine learning technique that allows for distributed training of a single machine learning model across multiple geographically distributed clients. This paper surveys the various approaches to communication-efficient FL, including model updates, compression techniques, resource management for edge and cloud, and client selection. We also review the various optimization techniques associated with communication-efficient FL, such as compression schemes and structured updates. Finally, we highlight the current research challenges and discuss the potential future directions for communication-efficient FL.},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {article}
}