The Changing Landscape of Machine Learning: A Comparative Analysis of Centralized Machine Learning, Distributed Machine Learning and Federated Machine Learning

Abstract

The landscape of machine learning is changing rapidly due to the ever-evolving nature of data and devices. The large centralized data is replaced by the distributed data and a central server is replaced with a large number of geographically distributed, loosely connected devices, such as smartphones, laptops, and other IoT devices. Therefore, the centralized machine learning (CML) which involves centralized data training on a central server is no longer an effective solution when the data is inherently distributed or too big to process on a central server, or data privacy is paramount; and the quest for a suitable machine learning to resolve these issues led to the evolution of distributed machine learning (DML). For large-scale learning tasks, DML has evolved to effectively handle enormous data within big data and distributed computing environment. Resolving most limitations faced by CML with the implementation of parallel learning on a large number of nodes to optimise time, learning resources and performance. However, DML may not necessarily ensure strict data privacy leading to further development and innovation of federated machine learning (FML) which is a type of DML that further decentralizes learning operations using local data on each participating node incorporating data privacy adherence. This paper analyses the transformation journey of machine learning whilst explaining its evolution from centralized, distributed to federated machine learning. Examining these three variants of machine learning exemplifies their coherent and comparative analysis. Which helps grasp a better understanding of each machine learning type as well as presenting the reason for the changing landscape. Additionally, the paper will address each type of machine learning alongside their different types, strengths and limitations.