HyMPOR — Hybrid Multi-Stage Pipeline for Restoring Damaged and Occluded Old Photographs

Old photographs suffer from complex, co-occurring degradations — scratches, stains, fading, occlusions, and loss of fine detail — that no single restoration model can adequately address. We present HyMPOR, a hybrid multi-stage pipeline that sequentially applies state-of-the-art models for scratch detection and removal, face enhancement, object removal via segmentation-guided inpainting, and colorization. Evaluated on both synthetic and real damaged photographs, HyMPOR achieves superior quantitative scores and perceptually compelling results across all degradation types.

Everything runs in Google Colab — no manual setup, no separate weight downloads.

1. Click the Open in Colab badge above.
2. Set the runtime to GPU: Runtime ▸ Change runtime type ▸ T4 GPU.
3. Run the first cell. It automatically downloads the complete project (code + all pre-trained weights) from the latest Release and extracts it.
4. Run the remaining cells in order to restore your photo.

The full project (~10 GB) is split into parts in the Release and reassembled automatically by the first cell. The first run takes 10–20 minutes depending on connection speed; afterwards everything is cached for the session.

Input (Damaged Old Photo)
        │
        ▼
┌─────────────────────────────────────┐
│  Stage 1 — Scratch & Damage Removal │  ← Bringing Old Photos Back to Life
│  Global restoration + face detection│
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│  Stage 2 — Object / Occlusion       │  ← SAM2 + AOT-GAN
│  Segmentation → Inpainting          │
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│  Stage 3 — Face Enhancement         │  ← GFPGAN
│  Blind face restoration via GAN     │
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│  Stage 4 — Colorization             │  ← DeOldify
│  Self-attention GAN colorization    │
└─────────────────┬───────────────────┘
                  │
                  ▼
        Output (Restored Photo)

The project bundles both GFPGAN face-enhancement weights — v1.4 (default) and v1.3. You can switch between them by editing a single line in the GFPGAN cell (Step 3):

# Default — v1.4
GFPGAN_WEIGHTS = f'{GFPGAN_DIR}/experiments/pretrained_models/GFPGANv1.4.pth'

# To use v1.3 instead, change 1.4 → 1.3:
GFPGAN_WEIGHTS = f'{GFPGAN_DIR}/experiments/pretrained_models/GFPGANv1.3.pth'

Both weights are included in the project, so no extra download is needed — just change the number and re-run the cell.

✅ All pre-trained weights are bundled in the Release and downloaded automatically by the notebook. There is nothing to download or place manually.

HyMPOR/
├── HyMPOR_GitHub.ipynb         # Main Colab notebook — start here
├── pipeline/                   # Core pipeline code
│   ├── run.py                  # Orchestrator (SAM2 → selector → AOT-GAN)
│   ├── config.py               # Central configuration
│   ├── segmentation.py         # SAM2 segmentation
│   ├── inpainting.py           # AOT-GAN inpainting
│   └── selector.py             # Content-aware weight selector
│
└── modules/
    ├── Bringing_Old_Photos_Back_to_Life/   # Stage 1
    ├── SAM2/                               # Stage 2a
    ├── AOT_GAN/                            # Stage 2b
    ├── GFPGAN/                             # Stage 3
    └── DeOldify/                           # Stage 4

If you find this work useful, please cite:

@article{HyMPOR2026,
  title   = {A Hybrid Multi-Stage Pipeline for Restoring Damaged and Occluded Old Photographs},
  author  = {Anas Hameed Ali},
  journal = {Under Review},
  year    = {2026}
}

This work builds upon several outstanding open-source projects:

• Bringing Old Photos Back to Life — Microsoft Research
• SAM2 — Meta AI Research
• AOT-GAN — Zeng, Yanhong and Fu, Jianlong and Chao, Hongyang and Guo, Baining
• GFPGAN — Tencent ARC
• DeOldify — Jason Antic

For questions or collaborations, please open an issue.

Released under the MIT License.