What are the limitations of Byzantine-resilient defenses against adaptive model-poisoning attacks?

Byzantine-resilient defenses generally assume that attackers are naive and form a minority, so adaptive model-poisoning attackers can undermine them by adjusting updates to evade detection. Even a small fraction of collaborating adaptive attackers can bypass Byzantine-robust aggregation and anomaly detection, especially through backdoor attacks that preserve overall accuracy. Stronger countermeasures often require impractical assumptions such as server access to a root dataset or client update histories, creating privacy and efficiency trade-offs.

Byzantine-resilient aggregation methods are designed to suppress outlier updates and are effective against data-poisoning and nonadaptive model-poisoning attacks. However, they are only effective under the assumption that attackers are naive and constitute a minority. Adaptive attackers can strategically modify their updates during the learning process to evade detection, and scenarios with a higher proportion of malicious participants can significantly undermine the robustness of these methods. Adaptive strategies include incorporating a detection matrix into local training, applying small but continuously accumulating malicious updates, and scaling model updates to remain undetected. These tactics allow even a small fraction of collaborating attackers to embed backdoors into global models while maintaining high overall accuracy, which neutralizes many Byzantine-resilient defenses. Moreover, detecting such backdoors is inherently difficult, and robustness to backdoors implies robustness to adversarial examples, a major open challenge. Defenses that attempt to counter adaptive attacks often rely on stringent measures such as server access to a representative root dataset, careful scrutiny of training data, holdout datasets for validation, or complete server control, which create significant trade-offs in privacy, efficiency, or practical feasibility. Anomaly detection that uses client update histories can also be eroded over time by an arms race between attacker and defender, making robust defenses difficult to sustain.