How AI Photo Restoration Actually Works: What Happens When You Upload That Faded 1970s Print

Last week, a client sent me a photo that stopped me in my tracks. Her grandmother's 1947 wedding portrait—barely visible through decades of silver mirroring and emulsion cracking. The kind of damage that would've had me billing 8–12 hours of manual restoration work five years ago.

I ran it through our AI pipeline. Forty-seven seconds later, her grandmother's face emerged from the chemical haze, sharp enough to count individual lace flowers on the veil.

My coffee took longer to brew than that restoration. (And yes, I still don't fully trust it sometimes. More on that in a bit.)

If you've ever wondered what's actually happening in those 47 seconds—beyond "AI magic"—here's what's going on under the hood.

The Problem: Why Old Photos Are Genuinely Hard to Fix

Before we get into neural networks, let's talk about why this is difficult in the first place.

Chemical Warfare Against Your Memories

Pre-1990s photographs weren't digital files. They were physical chemical reactions suspended in gelatin. And chemistry doesn't age gracefully:

• Silver mirroring: Metallic silver particles migrate to the surface, creating that mirror-like sheen you see on old prints
• Emulsion delamination: The image layer literally peels away from the paper base
• Vinegar syndrome: Acetate film breaks down and releases acetic acid (smells like vinegar, hence the name)
• Dye coupling failure: Color prints from the 1970s–80s lose cyan first, leaving everything reddish-orange

I've seen prints where the gelatin layer cracked into a spiderweb pattern—reticulation, for the film nerds out there—because someone's grandfather stored them in a humid Florida garage for 40 years.

The Information Loss Problem

Here's the brutal truth: sometimes the information just isn't there anymore.

A 2x3 inch Kodachrome slide from 1965 might've captured incredible detail originally. But by the time it's been stored in a cardboard box, exposed to temperature swings, and scanned on a consumer flatbed, you're working with maybe 30% of the original tonal range.

Traditional restoration approach: A skilled artist (like I used to be, before AI stole my job—kidding, mostly) would spend hours:

• Hand-painting missing areas using period-appropriate color references
• Clone-stamping individual scratches
• Rebuilding facial features based on what probably looked right

Time investment: 4–15 hours per image Cost: $150–$600 Consistency: Depends entirely on the artist's skill and how much coffee they've had

AI doesn't get tired or need coffee breaks. But it has its own quirks.

The Technology: How Modern AI Restoration Actually Works

Layer 1: Convolutional Neural Networks—The Pattern Recognition Engine

At the foundation, we're using Convolutional Neural Networks (CNNs)—a type of deep learning architecture that's been around since the late 1980s (Yann LeCun's original work on handwritten digit recognition).

Think of CNNs as learning hierarchical feature detectors:

• Early layers detect edges, gradients, textures
• Middle layers recognize shapes, patterns, facial components
• Deep layers understand context: "This is a face," "This is a scratch across a face," "This is what an unscratchedface should look like"

The breakthrough came when researchers realized you could train these networks on paired datasets: damaged photos next to their pristine originals. Show the network 50,000 examples of scratched faces paired with clean versions, and it learns what "remove scratch" means without anyone explicitly programming scratch-removal logic.

Layer 2: Generative Adversarial Networks—The Realism Engine

Here's where it gets interesting. GANs (Generative Adversarial Networks, introduced by Ian Goodfellow in 2014) consist of two neural networks locked in competition:

The Generator tries to create realistic restored images. The Discriminator tries to distinguish "real" restored photos from AI-generated ones.

The generator gets better at fooling the discriminator. The discriminator gets better at detecting fakes. They push each other toward photorealism.

Why this matters: Early AI restoration (pre-GAN era, ~2018) would denoise images but leave them looking weirdly smooth—what we call over-smoothing or the "plastic skin" effect. GANs learned to synthesize realistic texture instead of just blurring everything.

Layer 3: Face-Specific Models (GFPGAN, CodeFormer)

Humans are absurdly sensitive to facial features. We can detect tiny asymmetries, unnatural skin textures, or "off" expressions instantly. This is called the uncanny valley effect, and it's why face restoration needed specialized models.

GFPGAN (Generative Facial Prior GAN, from Tencent, 2021) was trained on 70,000 high-quality face images. It doesn't just enhance faces—it understands facial structure well enough to:

• Reconstruct occluded features (e.g., if half the face is damaged)
• Maintain identity (critical—you don't want Grandma looking like a different person)
• Add realistic detail (pores, fine wrinkles, hair strands) without hallucinating features that weren't there

CodeFormer (2022) takes a different approach using vector quantization—basically storing a "codebook" of realistic facial features and learning which combinations look natural. It's better at handling extreme degradation but sometimes struggles with ethnic diversity in training data (a known limitation we're still working on industry-wide).

The Restoration Pipeline: What Happens in Those 47 Seconds

When you upload a photo to a modern AI restoration service, here's the actual sequence:

Stage 1: Preprocessing & Damage Assessment (0–5 seconds)

• Format normalization: Convert to working color space (usually sRGB or Adobe RGB)
• Resolution analysis: Determine if the image is large enough for face detection (minimum ~256x256 pixels)
• Damage classification: AI scans for scratch patterns, stain regions, noise characteristics
• Face detection: MTCNN or RetinaFace models locate facial regions

Why this matters: Not all damage is equal. A uniform fade across the whole image is trivial to fix. A complex overlay of scratches, stains, and emulsion damage requires different model parameters.

Stage 2: Global Restoration (5–20 seconds)

• Denoising: Remove film grain or digital sensor noise
• Scratch removal: Inpainting algorithms fill in linear damage patterns
• Tone curve adjustment: Restore contrast lost to fading
• Color balance: Neutralize yellowing or color shifts

The tricky part: Vintage photos often have intentional grain (especially 1960s–70s Tri-X film). Aggressive denoising can make them look "wrong" even if technically cleaner. Modern models are trained to preserve period-appropriate grain structure.

Stage 3: Face Enhancement (15–35 seconds)

This is where GFPGAN or CodeFormer come in:

1. Extract facial regions (with padding to include hair, neck, shoulders)
2. Normalize face alignment (rotate and scale to a standard orientation)
3. Run through face restoration model
4. Blend enhanced face back into the original image using feathered masks

Critical parameter: Restoration strength (typically 0.5–0.9). Higher values give sharper results but risk the "plastic face" effect. We usually default to 0.7—sharp enough to see eyelashes, subtle enough to preserve natural skin texture.

Limitation alert: If the original face is smaller than ~128x128 pixels in the source image, results get unpredictable. The model hasn't seen enough training examples at that scale.

Stage 4: Super-Resolution (20–45 seconds)

Upscaling using Real-ESRGAN (Enhanced Super-Resolution GAN):

• Takes a 500x500 pixel image and outputs 2000x2000 (4x scale)
• Synthesizes realistic detail instead of just interpolating pixels
• Adds edge sharpness, texture patterns, fine details

What it's doing: Real-ESRGAN was trained on hundreds of thousands of image pairs (low-res input, high-res target). It learned what "plausible high-frequency detail" looks like for faces, fabrics, foliage, architecture.

What it's NOT doing: Magically recovering information that was never captured. If the original photo was out of focus, the AI adds plausible detail, not the actual missing detail. This is important.

Stage 5: Colorization (Optional, 30–60 seconds)

For black-and-white photos, models like DeOldify or DDColor predict colors:

• Semantic understanding: "Sky is probably blue," "Grass is probably green"
• Contextual clues: Wedding dress = white, vintage car = period-appropriate colors
• Skin tone estimation: Based on facial features and lighting

Honest limitation: The AI is guessing. It doesn't know your grandmother's dress was actually lavender, not pink. If color accuracy matters, manual correction is still necessary.

Training the Models: Where This All Comes From

Modern restoration models are trained on:

1. Synthetic degradation datasets: Take 100,000 high-quality photos, artificially add scratches/noise/fading using Photoshop scripts, now you have paired training data
2. Historical archives: Partnerships with libraries (e.g., Library of Congress, George Eastman Museum) provide real degraded photos + professionally restored versions
3. Crowdsourced contributions: Some projects (like GFPGAN) asked the community to submit before/after restoration pairs
4. Film simulation: Digitally simulate the chemical processes of Kodachrome, Ektachrome, Tri-X, Plus-X to understand era-specific degradation patterns

Training time: 2–6 weeks on 8x NVIDIA A100 GPUs (roughly $15,000–$40,000 in compute costs) Inference time: 30–90 seconds on consumer hardware (or 5–15 seconds on server-grade GPUs)

This is why most restoration services run on cloud infrastructure—processing happens on remote servers, not your laptop.

Limitations: Where AI Still Falls Short

I'd be lying if I said AI restoration was perfect. Here's where it struggles:

Problem 1: Hallucination

The AI sometimes invents details that weren't there. I've seen it:

• Add wrinkles to a young person's face
• Generate earrings that didn't exist in the original
• "Improve" blurry backgrounds into unnaturally sharp landscapes

Why this happens: GANs are trained to generate realistic images, not necessarily accurate reconstructions. The model fills missing information with what's statistically plausible.

Our approach: We always provide both the AI-restored version AND the original, so clients can verify nothing critical changed.

Problem 2: Ethnic Bias in Training Data

Most face datasets skew heavily toward lighter skin tones and Western facial features. CodeFormer and GFPGAN have improved significantly since 2021, but you'll still occasionally see:

• Over-lightening of darker skin tones
• Misinterpretation of ethnic facial structures
• Westernization of non-Western features

Industry response: Researchers are actively building more diverse training datasets (e.g., Fairface, DiverseFace). It's getting better, but we're not there yet.

Problem 3: Identity Drift

Sometimes enhanced faces don't quite look like the original person—subtle changes in eye shape, nose width, jawline.

Mitigation: Modern models include identity loss in their training (comparing facial embeddings between input and output). It's better than 2020-era restoration, but manual review is still wise for important photos.

Problem 4: Can't Fix Everything

If the photo is:

• More than 70% destroyed (large missing sections)
• Completely out of focus (motion blur, defocus)
• Extremely low resolution (<100x100 pixels for faces)

...AI will struggle. At that point, you're back to needing a human restoration artist who can make educated guesses based on historical context.

The Future: Where This Is Heading

Short-term (2026–2027):

• Video restoration (already shipping—check out DAIN, RIFE for frame interpolation)
• Real-time mobile processing (Apple's Neural Engine, Google's Tensor chips are getting there)
• Multi-image fusion (combining multiple damaged photos of the same scene to reconstruct a better composite)

Medium-term (2028–2030):

• 3D reconstruction from 2D photos (NeRF-style approaches for photogrammetry)
• Historical accuracy tuning (AI trained on period-specific photography—e.g., "restore this as a 1940s Kodachrome would've looked")
• Guided restoration (text prompts: "enhance the face but preserve original film grain")

Long-term speculation:

• AI-assisted archival research (cross-referencing restored faces with genealogy databases)
• Temporal super-resolution (reconstructing intermediate frames from old home movies)

Why This Matters Beyond Nostalgia

Yes, restoring Grandma's wedding photo is emotionally valuable. But AI photo restoration also serves:

Historical preservation: The National Archives, Smithsonian, and Library of Congress are using these techniques to digitize millions of deteriorating photos before they're completely destroyed.

Forensic analysis: Law enforcement can enhance degraded surveillance footage or old case photos.

Cultural heritage: Indigenous communities, immigrant families, and marginalized groups can preserve visual history that was never professionally archived.

Medical imaging: Similar techniques restore degraded X-rays, MRIs, and pathology slides (yes, really—the same GANs used for faces work on medical scans).

Try It (With Realistic Expectations)

If you've got old photos to restore, here's my honest recommendation:

1. Scan at 600 DPI (minimum—higher if the original print is small)
2. Save as TIFF or PNG (not JPEG—compression artifacts confuse the AI)
3. Try AI restoration first (ArtImageHub, Remini, MyHeritage—we're all using similar underlying models)
4. Review results carefully (zoom in, check faces, verify nothing critical changed)
5. If it's a once-in-a-lifetime photo, consider hiring a human expert for final touch-ups

AI won't replace professional restoration artists entirely (at least not yet). But for 80% of family photos, it delivers professional-quality results in seconds instead of hours.

And my coffee? Still takes longer than the restoration process.

---

Technical References:

• Wang et al. (2021). "Towards Real-World Blind Face Restoration with Generative Facial Prior" (GFPGAN paper)
• Zhou et al. (2022). "Towards Robust Blind Face Restoration with Codebook Lookup Transformer" (CodeFormer paper)
• Wang et al. (2021). "Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data"
• Antic, J. (2019). "DeOldify: A Deep Learning based project for colorizing and restoring old images"

Related Reading:

• Old Photo Restoration: A Practical Workflow for Family Historians
• Comparing AI Photo Tools: Real Tests with Actual Damaged Photos
• Photo Preservation 101: Scanning, Storage, and Digital Archiving