Centralized datasets raise concerns about data privacy, security, and fairness, especially when handling large volumes of sensitive or heterogeneous data, and they demand significant computational resources that may be inaccessible to individual organizations or researchers. Collaborative approaches such as federated learning are presented as a promising paradigm to address these challenges.

According to the chapter, advances in machine learning, such as deep neural networks and large language models, frequently rely on centralized datasets and significant computational resources. This reliance creates barriers because such resources may be inaccessible to individual organizations or researchers. More importantly, using centralized datasets introduces critical concerns about data privacy, security, and fairness, particularly when the data is sensitive or highly heterogeneous. To address these challenges, the chapter points to collaborative machine learning approaches, including federated learning, split learning, and swarm learning, as an emerging and promising paradigm. Federated learning, in particular, fits within this collaborative, privacy-preserving direction, though the text does not detail its internal mechanisms.

Key points

Centralized datasets are common in advanced ML but can require computational resources beyond the reach of many organizations.
Key concerns include data privacy, data security, and fairness.
Concerns heighten when datasets are large, sensitive, or heterogeneous.
Federated learning, split learning, and swarm learning are collaborative approaches proposed to address these challenges.
These approaches are framed within a privacy-preserving collaborative machine learning paradigm.

What are the main concerns associated with centralized datasets in machine learning, and how do collaborative approaches like federated learning address them?