📷 Advanced Camera ISP Pipeline

End-to-End RAW Image Processing Pipeline with Research-Grade ISP Components

---

📖 Project Overview

The Advanced Camera ISP Pipeline is a complete RAW image processing system that converts Bayer sensor RAW data into visually enhanced final RGB images.

This project recreates the major stages used inside real smartphone and DSLR camera pipelines.

The system processes .dng / RAW images through multiple ISP stages including:

• RAW linearization
• Dead Pixel Correction
• Lens Shading Correction
• Advanced Demosaicing
• White Balance
• Color Correction Matrix (CCM)
• Exposure Correction
• Gamma Rendering
• Adaptive Final Rendering

Goal: Design and implement a complete software-based camera ISP pipeline from scratch using image science and research methods.

---

📷 Data Acquisition

📱 Smartphone RAW Capture

RAW images were captured using the Open Camera application:

👉 https://play.google.com/store/apps/details?id=net.sourceforge.opencamera

Features used:

• RAW (DNG) capture enabled
• Manual exposure control
• Auto white balance disabled (for ISP evaluation)

Device: Samsung Galaxy S24 Ultra

---

📸 DSLR RAW Dataset

DSLR RAW images were sourced from the MIT-Adobe FiveK Dataset:

👉 https://data.csail.mit.edu/graphics/fivek/

---

⚙️ ISP Pipeline Architecture

---

✨ Core Features

🔹 1. RAW Linearization

• Reads Bayer RAW .dng
• Per-channel black level subtraction
• White level normalization
• CFA pattern detection

Supported Bayer patterns:

• RGGB
• BGGR
• GBRG
• GRBG

---

🔹 2. Dead Pixel Correction

Smart CFA-aware hot/dead pixel removal using same-color neighborhood analysis.

Features:

• Hot pixel correction
• Dead pixel correction
• Highlight protection
• Adaptive thresholds

---

🔹 3. Lens Shading Correction

Corrects corner vignetting and sensor brightness falloff.

Features:

• Radial gain compensation
• Adaptive corner brightening

---

🔹 4. Advanced Demosaicing

Implemented multiple demosaicing algorithms:

• Bilinear
• OpenCV Edge Aware
• Malvar-He-Cutler (MHC)
• Hamilton-Adams
• Menon
• ARI
• CNN-based Demosaic

---

🔹 5. White Balance Research Lab

Multiple white balance algorithms:

• Camera Metadata WB
• Gray World
• Shades of Gray
• Grey Edge
• White Patch
• Gamut Mapping
• Bayesian WB

---

🔹 6. Color Correction Matrix (CCM)

Transforms sensor RGB into display color space.

Includes:

• Adaptive CCM
• Warm CCM
• Cool CCM
• Neutral CCM

---

🔹 7. Exposure Engine

Scene brightness correction:

• Global Gain
• Mean Target
• Percentile Exposure
• Highlight Safe Exposure
• Adaptive Exposure
• Shadow Lift

---

🔹 8. Gamma / Tone Mapping

Implemented rendering curves:

• Gamma 2.2
• Gamma 2.4
• sRGB Gamma
• BT.709
• Adaptive Gamma
• Filmic Curve

---

🔹 9. Adaptive Final Renderer

Final aesthetic stage:

• Scene-aware brightness
• Vibrance
• Saturation
• Contrast
• Black Point
• Warmth control
• Orientation correction

---

🖼️ Final Results

📸 DSLR Pipeline Output

---

📱 Samsung S24 Ultra RAW Pipeline

---

🔍 Comparison Insight

• DSLR pipeline tuned for natural tones and controlled contrast
• Smartphone pipeline optimized for vibrant colors and dynamic scenes
• Custom rendering outperforms default RAW processing in highlight handling

---

📊 Highlights

• Better highlight preservation in bright skies
• Full custom RAW pipeline from scratch
• Modular ISP architecture
• Multiple research algorithms per stage
• Real camera imaging science implementation

---

📂 Project Structure

Project/
│
├── Core/
│   ├── methods/
│   ├── calibration/
│   ├── data/
│   ├── docs/
│   ├── models/
│   └── notebooks/
│
├── run_phone_pipeline.ipynb
├── run_dslr_pipeline.ipynb
│
├── README.md
└── .gitignore

---

🚀 Run Project

Install Dependencies

pip install -r requirements.txt

Run Notebook

jupyter notebook

Open:

run_phone_pipeline.ipynb  
run_dslr_pipeline.ipynb

---

🔬 Future Improvements

• AI Auto White Balance
• AI Demosaicing
• Noise Reduction
• Sharpening Engine
• HDR Merge
• Portrait Skin Tuning
• Real-time Mobile ISP

---

👨‍💻 Author

Keshav B.Tech CSE Bennett University

---

⭐ Support

If you like this project, give it a star ⭐ on GitHub.