DirectInversion+P2P with Alpha/Beta Scheduling

📖 Overview

This repository extends the DirectInversion (PnP Inversion) project with dynamic alpha/beta scheduling for fine-grained control over diffusion-based image editing.

What's New?

We introduce three key enhancements to DirectInversion+P2P:

1. Alpha Parameter (α): Controls the coupling strength between source and target diffusion branches

   • Negative values: More aggressive edits, prioritize target prompt
   • Zero: Baseline DirectInversion behavior
   • Positive values: More conservative edits, preserve source image structure

2. Beta Parameter (β): Controls the decay/evolution rate of alpha over diffusion timesteps

   • Higher β (e.g., 0.99): Slower evolution, gradual changes
   • Lower β (e.g., 0.9): Faster evolution, more pronounced changes

3. Alpha Schedulers: Define how alpha evolves during the diffusion process

   • exp: Exponential decay α(t) = α₀ × β^t
   • linear: Linear interpolation α(t) = α₀ × (1 - progress × (1 - β))
   • cosine: Cosine-based smooth interpolation
   • constant: No scheduling, constant α throughout

These parameters enable systematic exploration of the trade-off space between edit fidelity (CLIP similarity to target) and structure preservation (PSNR with source).

---

🚀 Quick Start

Running a Single Configuration

Edit an image with specific alpha/beta/scheduler settings:

python run_editing_p2p.py \
    --edit_category_list 0 \
    --edit_method_list "directinversion+p2p" \
    --alpha 0.5 \
    --beta 0.9 \
    --alpha_sch exp \
    --output_path outputs_alpha_beta/alpha_0.5_beta_0.9_sch_exp

Parameters:

• --alpha: Coupling strength (e.g., -1.0, -0.5, 0.0, 0.5, 1.0)
• --beta: Decay rate (e.g., 0.9, 0.99, 1.0)
• --alpha_sch: Scheduler type (exp, linear, cosine, constant)
• --edit_category_list: Editing categories from PIE-Bench (0-9)
• --output_path: Directory for output images

Example - Conservative Edit (Preserve Structure):

python run_editing_p2p.py \
    --edit_category_list 0 \
    --edit_method_list "directinversion+p2p" \
    --alpha 1.0 \
    --beta 0.99 \
    --alpha_sch cosine \
    --output_path outputs_alpha_beta/alpha_1.0_beta_0.99_sch_cosine

Example - Aggressive Edit (Prioritize Target Prompt):

python run_editing_p2p.py \
    --edit_category_list 0 \
    --edit_method_list "directinversion+p2p" \
    --alpha -1.0 \
    --beta 0.9 \
    --alpha_sch exp \
    --output_path outputs_alpha_beta/alpha_-1.0_beta_0.9_sch_exp

---

🔬 Grid Search (Hyperparameter Sweep)

Running the Complete Grid Search

Systematically explore the alpha/beta/scheduler space using the provided script:

bash alpha_search.sh

This script sweeps over predefined combinations. You can customize it by editing alpha_search.sh:

#!/bin/bash

mkdir -p outputs_alpha_beta

# Customize these ranges for your experiment
for beta in 0.9 0.99 1.0
do
    for alpha in -1.0 -0.5 -0.25 0.0 0.25 0.5 1.0
    do
        for alpha_sch in exp linear cosine
        do
            outdir="outputs_alpha_beta/alpha_${alpha}_beta_${beta}_sch_${alpha_sch}"
            mkdir -p "$outdir"

            echo "Running α = ${alpha}, β = ${beta}, scheduler = ${alpha_sch} ..."

            python run_editing_p2p.py \
                --edit_category_list 0 \
                --edit_method_list "directinversion+p2p" \
                --alpha "${alpha}" \
                --beta "${beta}" \
                --alpha_sch "${alpha_sch}" \
                --output_path "${outdir}"

            echo "Finished α = ${alpha}, β = ${beta}, scheduler = ${alpha_sch}"
        done
    done
done

Expected Runtime:

• Single configuration (~140 images): 15-30 minutes (depending on GPU)
• Full grid (7 alphas × 3 betas × 3 schedulers = 63 configs): ~16-32 hours

Output Structure:

outputs_alpha_beta/
├── alpha_-1.0_beta_0.9_sch_exp/
│   └── directinversion+p2p/
│       └── annotation_images/
│           └── 0_random_140/
│               ├── 000000000000/
│               │   └── image.jpg
│               └── ...
├── alpha_0.0_beta_0.99_sch_linear/
└── ...

---

📊 Evaluation

Evaluate All Configurations

Compute quantitative metrics (PSNR, LPIPS, SSIM, CLIP similarity, etc.) for all runs:

bash run_all_evaluations.sh

This script:

1. Scans all directories in outputs_alpha_beta/
2. Extracts alpha/beta/scheduler from folder names
3. Runs evaluation and saves results to evaluation_results/
4. Automatically skips existing evaluations (resumes on interruption)

Metrics Computed:

• Structure Preservation: PSNR, SSIM, MSE, LPIPS
• Edit Fidelity: CLIP similarity to target prompt
• Partial Metrics: Separate metrics for edited and unedited regions

Output:

evaluation_results/
├── alpha_-1.0_beta_0.9_sch_exp_evaluation.csv
├── alpha_0.0_beta_0.99_sch_linear_evaluation.csv
├── ...
├── direct_inversion.csv        # Baseline benchmark
└── null_text.csv                # Null-text inversion benchmark

Evaluate a Single Configuration

python evaluation/evaluate.py \
    --metrics "psnr" "lpips" "ssim" "clip_similarity_target_image" \
    --result_path evaluation_results/my_config_evaluation.csv \
    --edit_category_list 0 \
    --tgt_folder outputs_alpha_beta/alpha_0.5_beta_0.9_sch_exp/directinversion+p2p

---

📈 Visualization & Analysis

Using view_annotations.ipynb

This Jupyter notebook provides comprehensive analysis and visualization of all evaluation results.

Launch the notebook:

jupyter notebook view_annotations.ipynb

What You Can See:

1. Summary Table

• Mean metrics (PSNR, LPIPS, SSIM, CLIP similarity) for all (α, β, scheduler) combinations
• Best configurations for each metric
• Statistical comparison across schedulers

2. Pareto Frontier Analysis ⭐

• Minimalistic scatter plot showing the trade-off between:
  • CLIP Target Similarity (edit fidelity)
  • PSNR (structure preservation)
• Color-coded by scheduler:
  • 🔴 Exponential (exp)
  • 🔵 Linear (linear)
  • 🟢 Cosine (cosine)
  • 🟣 Constant (constant)
• Solid circles: Pareto-optimal configurations (best trade-offs)
• Transparent circles: Non-optimal configurations
• Yellow/orange crosses: Benchmark methods (Dir Inv, NT)
• Smooth curve: Pareto frontier sketch showing optimal boundary

Key Insights from Pareto Plot:

• Identify configurations that maximize both edit quality and structure preservation
• Compare your custom alpha/beta combinations against baseline methods
• Visualize the trade-off space to select optimal hyperparameters for your use case

3. Per-Image Comparison

• View specific images across different alpha/beta/scheduler settings
• Side-by-side comparison of original, edited images
• Interactive prompts display

4. Metric Heatmaps

• Visualize how metrics vary across alpha/beta grids for each scheduler
• Identify sweet spots for specific editing types

5. Improvement Analysis

• Rank images by improvement from baseline (α=0)
• Discover which images benefit most from different configurations

6. Full Dataset Results

• If you ran experiments on the complete PIE-Bench (700 images)
• Separate analysis section for all_* evaluation results

Workflow:

1. Run all cells (Kernel → Restart & Run All)
2. Navigate sections using markdown headers
3. Adjust parameters in cells to explore specific configurations
4. Export figures for your paper/presentation