Preprint
Computer Vision

The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

Paul Bergmann(Software (Germany)), Kilian Batzner(Software (Germany)), Michael Fauser(Software (Germany)), David Sattlegger(Software (Germany)), Carsten Steger(Software (Germany))
January 6, 2021International Journal of Computer Vision574 citations
PDFDOI

574

Citations

75

Influential Citations

International Journal of Computer Vision

Venue

2021

Year

Abstract

Abstract The detection of anomalous structures in natural image data is of utmost importance for numerous tasks in the field of computer vision. The development of methods for unsupervised anomaly detection requires data on which to train and evaluate new approaches and ideas. We introduce the MVTec anomaly detection dataset containing 5354 high-resolution color images of different object and texture categories. It contains normal, i.e., defect-free images intended for training and images with anomalies intended for testing. The anomalies manifest themselves in the form of over 70 different types of defects such as scratches, dents, contaminations, and various structural changes. In addition, we provide pixel-precise ground truth annotations for all anomalies. We conduct a thorough evaluation of current state-of-the-art unsupervised anomaly detection methods based on deep architectures such as convolutional autoencoders, generative adversarial networks, and feature descriptors using pretrained convolutional neural networks, as well as classical computer vision methods. We highlight the advantages and disadvantages of multiple performance metrics as well as threshold estimation techniques. This benchmark indicates that methods that leverage descriptors of pretrained networks outperform all other approaches and deep-learning-based generative models show considerable room for improvement.

Analysis

Why This Paper Matters

The MVTec Anomaly Detection (MVTec AD) dataset fills a critical gap in computer vision: the lack of a comprehensive, real-world benchmark for unsupervised anomaly detection. Prior datasets were either synthetic, small-scale, or lacked pixel-level annotations, making it difficult to evaluate and compare methods fairly. By providing 5354 high-resolution images across 15 diverse categories (textures and objects) with over 70 defect types and pixel-precise ground truth, MVTec AD enables rigorous benchmarking. Its release in 2021 has already garnered 574 citations, reflecting its rapid adoption as a de facto standard in the field.

Technical Contributions

The paper's primary contribution is the dataset itself, but the thorough evaluation of existing methods is equally valuable. Key innovations include:

  • Dataset design: Covers both structured objects (e.g., bottles, cables) and textures (e.g., carpet, leather), capturing a wide range of real-world defects like scratches, dents, and contaminations.
  • Pixel-level annotations: Enables evaluation of localization performance, not just image-level detection.
  • Comprehensive benchmark: Tests deep architectures (autoencoders, GANs), feature descriptors from pretrained CNNs (e.g., ResNet, VGG), and classical methods (e.g., SSIM, LBP).
  • Performance metrics analysis: Compares AUC, F1-score, and other metrics, and discusses threshold estimation techniques.

Results

The benchmark reveals clear performance hierarchies: methods using pretrained CNN features (e.g., SPADE, PaDiM) achieve the highest AUC scores, often exceeding 95% on several categories. In contrast, generative models like autoencoders and GANs lag behind, with AUCs around 70-85% depending on the category. Classical methods perform worst, typically below 70%. The paper also highlights that pixel-level localization remains challenging, with many methods struggling to precisely delineate defect boundaries.

Significance

MVTec AD has become the go-to dataset for unsupervised anomaly detection research, enabling fair comparisons and driving methodological advances. Its impact extends beyond academia to industrial applications such as automated visual inspection in manufacturing. The finding that pretrained feature extractors outperform end-to-end deep generative models has shifted research focus toward leveraging transfer learning. Future work may expand to video anomaly detection, 3D data, or more complex defect types, but MVTec AD remains a foundational benchmark.

anomaly-detectionunsupervised-learningcomputer-visiondatasetdeep-learningpretrained-networksdefect-detection
BenchmarksComputer VisionNeural Networks

Related Papers