Attention Is All You Need
Ashish Vaswani, Noam Shazeer et al.
139
Citations
5
Influential Citations
Scientific Reports
Venue
2021
Year
Abstract Atom segmentation and localization, noise reduction and deblurring of atomic-resolution scanning transmission electron microscopy (STEM) images with high precision and robustness is a challenging task. Although several conventional algorithms, such has thresholding, edge detection and clustering, can achieve reasonable performance in some predefined sceneries, they tend to fail when interferences from the background are strong and unpredictable. Particularly, for atomic-resolution STEM images, so far there is no well-established algorithm that is robust enough to segment or detect all atomic columns when there is large thickness variation in a recorded image. Herein, we report the development of a training library and a deep learning method that can perform robust and precise atom segmentation, localization, denoising, and super-resolution processing of experimental images. Despite using simulated images as training datasets, the deep-learning model can self-adapt to experimental STEM images and shows outstanding performance in atom detection and localization in challenging contrast conditions and the precision consistently outperforms the state-of-the-art two-dimensional Gaussian fit method. Taking a step further, we have deployed our deep-learning models to a desktop app with a graphical user interface and the app is free and open-source. We have also built a TEM ImageNet project website for easy browsing and downloading of the training data.
This paper addresses a critical bottleneck in materials science: the robust and precise analysis of atomic-resolution scanning transmission electron microscopy (STEM) images. Traditional algorithms like thresholding and edge detection fail under strong background interference or large thickness variations, which are common in real experimental data. By introducing a deep-learning approach trained on simulated images, the authors demonstrate that models can self-adapt to experimental conditions, achieving higher precision than the established 2D Gaussian fitting method. This is significant because it opens the door to fully automated, high-throughput atom-by-atom analysis, which is essential for understanding structure-property relationships in advanced materials.
Moreover, the release of a free, open-source desktop app with a graphical user interface lowers the barrier for non-experts to adopt deep learning in their microscopy workflows. The TEMImageNet project website also facilitates community access to training data, promoting reproducibility and further innovation.
The key quantitative result is that AtomSegNet consistently outperforms the state-of-the-art two-dimensional Gaussian fit method in atom detection and localization precision, especially under challenging contrast conditions (e.g., large thickness variation). The paper reports that the deep-learning model achieves higher precision and robustness, though specific numerical metrics (e.g., precision, recall, localization error) are not detailed in the abstract. The models also demonstrate denoising and deblurring capabilities, enhancing image quality for subsequent analysis.
This work bridges the gap between computer vision and materials characterization, providing a practical deep-learning solution for a longstanding problem in electron microscopy. By making the tools open-source and user-friendly, it accelerates the adoption of AI in experimental science. The approach of training on simulated data and transferring to real-world images is a valuable paradigm for domains where labeled experimental data is scarce. Future work could extend these models to other imaging modalities or incorporate additional physical priors for even higher accuracy.
Ashish Vaswani, Noam Shazeer et al.
Jakubův, Jan, Chvalovský, Karel et al.
Pauli Virtanen, Ralf Gommers et al.
Tom B. Brown, Benjamin Mann et al.