Preprint
Computer Vision

Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

Antonio Emanuele Ciná(Ca' Foscari University of Venice), Kathrin Grosse(École Polytechnique Fédérale de Lausanne), Ambra Demontis(University of Cagliari), Sebastiano Vascon(Ca' Foscari University of Venice), Werner Zellinger(Software Competence Center Hagenberg (Austria)), Bernhard A. Moser(Software Competence Center Hagenberg (Austria)), Alina Oprea(Northeastern University), Battista Biggio(University of Cagliari), Marcello Pelillo(Ca' Foscari University of Venice), Fabio Roli(University of Genoa)
March 1, 2023ACM Computing Surveys205 citations
arXivPDFDOI

205

Citations

12

Influential Citations

ACM Computing Surveys

Venue

2023

Year

Abstract

The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model’s performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the past 15 years. We start by categorizing the current threat models and attacks and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning and shed light on the current limitations and open research questions in this research field.

Analysis

Why This Paper Matters

This survey addresses a critical gap in the machine learning security literature: the lack of a unified, comprehensive systematization of poisoning attacks and defenses. As ML models are increasingly deployed in safety-critical applications (e.g., autonomous driving, medical diagnosis), understanding how adversaries can corrupt training data to compromise model behavior is paramount. The paper consolidates over 100 papers from the past 15 years, providing a structured view that helps researchers and practitioners navigate a fragmented field.

The timing is particularly relevant given the rise of large-scale training datasets scraped from the internet, which are inherently vulnerable to poisoning. By categorizing threat models and defenses, the survey enables the community to identify common patterns, evaluate the maturity of different defense strategies, and prioritize future research directions.

Technical Contributions

  • Unified Taxonomy: The paper introduces a clear categorization of poisoning attacks based on threat model (e.g., availability vs. integrity attacks, targeted vs. indiscriminate) and attack vector (e.g., label flipping, backdoor insertion, data manipulation).
  • Defense Mapping: Defenses are systematically organized according to the attack categories they address, covering techniques such as data sanitization, robust aggregation, and anomaly detection.
  • Cross-Modal Applicability: While focused on computer vision, the authors argue that the taxonomy generalizes to other data modalities (e.g., text, audio), providing a framework for future work.
  • Resource Compilation: The survey catalogs existing datasets, benchmarks, and open-source tools for poisoning research, lowering the barrier to entry for new researchers.

Results

The survey does not present new experimental results but synthesizes findings from the literature. Key takeaways include: (1) most defenses are evaluated against specific attack types and often fail under stronger threat models; (2) backdoor attacks remain a particularly challenging threat due to their stealthiness; (3) there is a lack of standardized evaluation benchmarks, making comparison across studies difficult. The paper reports that the field has seen a surge in publications, with over 100 papers reviewed, indicating growing interest but also fragmentation.

Significance

This survey is likely to become a standard reference for researchers entering the field of adversarial machine learning. By providing a clear taxonomy and identifying open problems (e.g., defenses against adaptive attackers, scalability to large models), it sets the agenda for future research. For practitioners, it offers a practical guide to understanding and mitigating poisoning risks in real-world deployments. The work underscores that poisoning is not a solved problem and that continued investment in robust training pipelines is essential.

machine-learningsecuritypoisoning-attacksdefensescomputer-visionsurveyadversarial-machine-learning
Computer Vision

Related Papers