High-Resolution Historical Images: A Massive New Dataset and Super-Resolution Model

Discover a groundbreaking 2-million-image dataset of historical photos at 512x512 resolution, paired with HiResNet—a diffusion-based super-resolution model that revives blurry archives for modern AI research.

Reviving the Past: The Quest for Crystal-Clear Historical Imagery

Historical photographs offer invaluable glimpses into bygone eras, but many suffer from low resolution due to aging film, poor scanning, or limited original quality. Researchers at NYU's Adelphi Lab have tackled this head-on with a massive new dataset and a cutting-edge super-resolution model. This initiative transforms pixelated relics into high-definition treasures, enabling deeper analysis in fields like genealogy, urban planning, and cultural studies.

Super-resolution (SR) techniques use AI to upscale images while preserving or enhancing details. Traditional methods like bicubic interpolation often introduce artifacts, but modern deep learning approaches—especially diffusion models—generate photorealistic results. The new work builds on this, focusing specifically on the unique characteristics of historical photos: sepia tones, grain, scratches, and period-specific artifacts.

A Monumental Dataset: History in High Resolution

At the core is the History in High Resolution (HiHR) dataset, comprising 2 million images at 512x512 pixels. Sourced from public-domain archives like the New York Public Library and Library of Congress, these span from the 1840s to the 1970s, covering portraits, street scenes, architecture, and events.

Key Dataset Features

• Scale and Diversity: Over 2M images dwarf previous historical SR datasets (e.g., DIV2K has only 800 training pairs). It includes varied subjects: people (45%), buildings (30%), landscapes (15%), and objects.
• High-Quality Ground Truth: Images were meticulously scanned at high DPI and cropped to avoid distortions, providing true high-res references for training.
• Metadata Richness: Each image comes with timestamps, locations, captions, and categories, facilitating targeted research.
• Licensing: Fully public domain or CC0, ready for unrestricted use.

You can access the full dataset via GitHub, including download scripts, previews, and evaluation splits (train: 1.8M, val: 100K, test: 100K).

Practical Example: For urban historians, query images by city (e.g., 50K+ NYC photos) to track architectural evolution. Load a sample:

git clone https://github.com/nyu-adelphi/history-in-high-resolution
cd history-in-high-resolution
python download.py --split train --num 1000

This dataset addresses a critical gap: most SR benchmarks (Set5, BSD100) use modern photos, failing on historical quirks like emulsion noise or handwritten annotations.

HiResNet: A Diffusion Powerhouse for Historical SR

Trained on HiHR, HiResNet is a diffusion-based model excelling at 4x and 8x upscaling from low-res inputs (128x128 or 64x64). Diffusion models iteratively denoise random noise into coherent images, outperforming GANs (e.g., ESRGAN) on fidelity metrics.

Model Architecture Deep Dive

• Backbone: U-Net with attention blocks, conditioned on low-res input via FiLM layers.
• Diffusion Process: 50 denoising steps; uses DDIM sampler for speed.
• Loss Function: Combines L1 reconstruction, perceptual (VGG), and adversarial losses tailored for historical textures.
• Training Details: 100K A100-GPU hours; batch size 32; LR 1e-4 with cosine annealing.

HiResNet crushes baselines:

On 8x, gains are even larger. Visually, it recovers faded faces, sharpens signage, and minimizes hallucinations.

Implement it easily:

from hiresnet import HiResNet
from PIL import Image

model = HiResNet.from_pretrained('nyu-adelphi/hiresnet-4x')
img = Image.open('lowres_historical.jpg')
hires = model(img, scale=4)
hires.save('superres.jpg')

Repo: HiResNet on GitHub with inference code, weights (Hugging Face), and training scripts.

Real-World Applications and Extensions

1. Academic Research

Pair with segmentation models (e.g., SAM) for object detection in crowdsourced history projects. Example: Analyze 19th-century fashion trends by upscaling 10K portraits, then applying CLIP for attribute classification.

2. Cultural Heritage

Museums can upscale archives for VR exhibits. A pilot with the Smithsonian restored 5K Civil War photos, revealing hidden details like uniform insignias.

3. Genealogy and AI Tools

Integrate into apps like Ancestry.com: Users upload blurry heirlooms; HiResNet enhances for facial recognition matches.

Code Snippet for Batch Processing:

import torch
from pathlib import Path

dataset_path = Path('historical_lowres')
model.eval()
with torch.no_grad():
    for img_path in dataset_path.glob('*.jpg'):
        img = load_image(img_path)
        sr_img = model(img)
        sr_img.save(img_path.parent / f'sr_{img_path.name}')

4. Benchmarking and Fine-Tuning

HiHR serves as a new standard. Fine-tune on domain-specific subsets (e.g., aerial photos for GIS).

Challenges Overcome and Future Directions

Challenges included:

• Domain Shift: Modern SR models blur historical grain; HiResNet learns to preserve it.
• Data Curation: Automated filtering removed duplicates/low-quality scans using CLIP similarity.
• Compute Scale: Distributed training via DeepSpeed.

Future: Video SR for historical footage, multilingual captions via mCLIP, and integration with foundation models like Stable Diffusion for inpainting scratches.

Getting Started: Step-by-Step

1. Clone repos: Dataset here, model here.
2. Install: pip install hiresnet torch torchvision.
3. Download subset: Use provided scripts.
4. Run inference: As shown above.
5. Evaluate: PSNR/SSIM scripts included.

This resource democratizes high-res history, fueling AI-driven discoveries. Dive in and resurrect the past in stunning detail.