Preprint
Knowledge Graphs

A joint deep learning model to recover information and reduce artifacts in missing-wedge sinograms for electron tomography and beyond

Guanglei Ding, Yitong Liu, Rui Zhang, Huolin L. Xin
September 5, 2019Scientific Reports72 citations
DOI

72

Citations

1

Influential Citations

Scientific Reports

Venue

2019

Year

Abstract

Abstract We present a joint model based on deep learning that is designed to inpaint the missing-wedge sinogram of electron tomography and reduce the residual artifacts in the reconstructed tomograms. Traditional methods, such as weighted back projection (WBP) and simultaneous algebraic reconstruction technique (SART), lack the ability to recover the unacquired project information as a result of the limited tilt range; consequently, the tomograms reconstructed using these methods are distorted and contaminated with the elongation, streaking, and ghost tail artifacts. To tackle this problem, we first design a sinogram filling model based on the use of Residual-in-Residual Dense Blocks in a Generative Adversarial Network (GAN). Then, we use a U-net structured Generative Adversarial Network to reduce the residual artifacts. We build a two-step model to perform information recovery and artifacts removal in their respective suitable domain. Compared with the traditional methods, our method offers superior Peak Signal to Noise Ratio (PSNR) and the Structural Similarity Index (SSIM) to WBP and SART; even with a missing wedge of 45°, our method offers reconstructed images that closely resemble the ground truth with nearly no artifacts. In addition, our model has the advantage of not needing inputs from human operators or setting hyperparameters such as iteration steps and relaxation coefficient used in TV-based methods, which highly relies on human experience and parameter fine turning.

Analysis

Why This Paper Matters

Electron tomography is a critical technique for 3D imaging at nanoscale, but it suffers from the missing-wedge problem due to limited tilt range, causing elongation, streaking, and ghost tail artifacts. Traditional methods like WBP and SART cannot recover unacquired projection information, and TV-based methods require extensive human expertise for parameter tuning. This paper introduces a deep learning approach that not only inpaints missing sinogram data but also cleans residual artifacts, achieving near-ground-truth quality even with a 45° missing wedge. This is significant because it automates a previously labor-intensive process and pushes the boundary of what is recoverable from limited-angle data.

Technical Contributions

The key innovations are: (1) A two-step joint model that separates information recovery (sinogram inpainting) from artifact reduction (tomogram refinement), each performed in its most suitable domain. (2) Use of Residual-in-Residual Dense Blocks within a GAN for sinogram filling, which captures fine details and long-range dependencies. (3) A U-net structured GAN for artifact removal, leveraging its encoder-decoder architecture for effective denoising. (4) The model requires no human input or hyperparameter tuning, unlike TV-based methods, making it more practical for routine use.

Results

The authors report that their method achieves superior Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) compared to WBP and SART. Specifically, even with a missing wedge of 45°, the reconstructed images closely resemble the ground truth with nearly no artifacts. This is a substantial improvement over traditional methods, which produce distorted and artifact-laden reconstructions under the same conditions. The paper does not provide exact numerical PSNR/SSIM values in the abstract, but the qualitative results are compelling.

Significance

This work demonstrates the power of deep learning to solve a long-standing inverse problem in electron microscopy. By eliminating the need for manual parameter tuning, it makes high-quality tomography accessible to non-experts. The joint model framework could be adapted to other limited-angle imaging modalities (e.g., X-ray CT, MRI) and may inspire further research into domain-specific GAN architectures for scientific imaging. The use of Residual-in-Residual Dense Blocks is a notable architectural contribution that could benefit other image restoration tasks.

deep-learninggenerative-adversarial-networkelectron-tomographysinogram-inpaintingartifact-reductionimage-reconstructionresidual-in-residual-dense-blocksu-net

Related Papers