Interactive All-in-One Image Restoration and Fusion

Cao, Bing; Zhang, Qiang; Xu, Xingxin; Zhu, Pengfei

doi:10.24963/ijcai.2026/105

IJCAI 2026 · Research project

AIR-Fusion

Interactive All-in-One
Image Restoration and Fusion

Bing Cao · Qiang Zhang^† · Xingxin Xu · Pengfei Zhu

† First student author. Author order follows the published paper.

Abstract

Supervised infrared-visible image fusion (IVIF) often overfits limited training distributions, creating a critical generalization gap under open-world degradations (rain, haze, low light, noise, blur). To address this issue, we propose AIR-Fusion, a parameter-efficient adaptation of a frozen, restoration-capable latent diffusion backbone for degraded IVIF without full fine-tuning, transferring restoration priors for robust fusion.

A Cross-Modal Bridging Adapter (CMBA) aligns infrared cues and textual instructions with the frozen diffusion conditioning space and injects them into multi-scale denoising features to steer instruction-guided restoration-aware fusion.

In addition, a Trajectory-Constrained Rectifier (TCR) regularizes stochastic sampling via a pixel-latent closed loop, rectifying intermediate predictions with source-referenced structures and re-encoding them to stabilize the denoising trajectory and recover fine details suppressed by latent compression. Experiments across multiple datasets and degradation settings show consistent improvements in restoration quality and fusion fidelity, with strong generalization under complex and compounded degradations.

Read the paper ↗GitHub ↗IJCAI proceedings ↗Cite this work 中文解读 →

Paper and project materials available. Implementation and model weights have not yet been released.

Two real haze scenes with infrared and visible inputs, Text-IF, ControlFusion and AIR-Fusion results, in the paper's original order — **Real-world haze, beyond the training dataset.** Original Figure 4 from the paper: qualitative comparisons on AWMM-100k. AIR-Fusion was trained on LLVIP. Click any figure to inspect the full-resolution extraction.

01 / Approach

Method

A restoration model knows how to remove degradations, but does not natively combine infrared and visible inputs. AIR-Fusion keeps the AutoDIR restoration backbone frozen, learns multimodal conditioning, and constrains sampling with the source images.

Preserve the backbone

Retain pretrained restoration priors instead of fully fine-tuning the diffusion backbone on limited fusion data.

CMBA · Align conditions

The Cross-Modal Bridging Adapter injects infrared-conditioned features into the frozen denoising network under text guidance.

TCR · Preserve structure

The Trajectory-Constrained Rectifier uses source images to rectify intermediate predictions and re-encodes them into the sampling loop.

Original Figure 2: text interaction, frozen backbone with CMBA, and the TCR pixel-latent sampling loop — **Figure 2 · Overall framework.** Extracted directly from page 3 of the published paper. Training uses proxy fused priors; inference starts from Gaussian noise with infrared, visible and text conditioning.

02 / Evidence

Results across five degradations

Trained only on LLVIP and evaluated on LLVIP plus unseen MSRS and RoadScene data. The table below reproduces the PSNR columns for every method in Table 1; higher is better.

PSNR (dB) · Published Table 1
Method	BlurLLVIP	HazeLLVIP	Low lightLLVIP	RainMSRS	NoiseRoadScene
U2Fusion*	27.98	27.66	28.01	28.33	28.23
TarDAL*	28.15	27.99	27.92	27.79	27.83
SwinFusion*	28.39	28.46	28.00	28.62	28.82
CDDFuse*	28.54	27.90	28.00	28.62	28.04
DDFM*	28.11	28.47	28.15	28.62	27.85
EMMA*	28.08	27.85	27.86	28.71	27.87
Text-IF	29.34	29.16	28.64	29.18	28.78
ControlFusion	29.46	29.28	28.97	29.33	28.81
AIR-Fusion	29.56	29.53	29.21	30.54	29.08

* AutoDIR restoration is applied before fusion for these baselines. Interactive methods use degradation-matched prompts. These are reported paper results, not new runs.

AIR-Fusion has the highest PSNR in all five listed settings. This is not a claim of leading every metric: haze VIF and noise SSIM, VIF and LPIPS favor other methods. See the full table for the trade-offs.

View the full original table: PSNR, SSIM, VIF and LPIPS

Original Table 1 with all four metrics and nine methods across five degradation settings — Table 1 · Paper page 5. Higher PSNR, SSIM and VIF; lower LPIPS.

Explore original visual comparisons across all six settings

Original Figure 3 showing all comparison methods for blur, haze, low light, rain, noise and a clean case — Figure 3 · Rows: blur, haze, low light, rain, noise and a clean case. All method columns and original labels are preserved. Paper page 6.

03 / Text interaction

Text instructions

Dehazing and deblurring

In the paper's compound haze-and-blur example, a dehazing instruction lifts the haze while largely retaining blur. A deblurring instruction sharpens structure while leaving much of the haze.

Figure 6 also illustrates low-resolution recovery and a natural-language dehazing prompt. These are qualitative examples, not a comprehensive instruction-following benchmark.

Read Section 4.5 ↗

Original Figure 6: low-resolution recovery, natural-language dehazing, and selective deblurring versus dehazing — Figure 6 · Original examples from the paper.

04 / Scope

Ablation and limitations

Restoration and source fidelity

In Table 2, adding TCR raises VIF from 0.87 to 1.04 and Q^AB/F from 0.57 to 0.76 compared with the No TCR variant. PSNR rises from 29.01 to 29.99. The full model does not maximize every ablation metric: removing color loss yields a higher PSNR.

Remaining limits

Evaluation mainly targets degraded visible inputs. Degraded infrared inputs, degradation in both modalities, and misregistration remain future work. Real-world haze and detection examples are qualitative.

The paper reports 11.397 seconds at 100 denoising steps on an RTX A6000, with 0.361B trainable parameters. This is not real-time inference.

05 / Reference

Citation

Published in the Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (IJCAI 2026), pp. 935–943. Official BibTeX ↗

@inproceedings{ijcai2026p105,
  title     = {Interactive All-in-One Image Restoration and Fusion},
  author    = {Cao, Bing and Zhang, Qiang and Xu, Xingxin and Zhu, Pengfei},
  booktitle = {Proceedings of the Thirty-Fifth International Joint
               Conference on Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {935--943},
  year      = {2026},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2026/105},
  url       = {https://doi.org/10.24963/ijcai.2026/105}
}