Authors:
Bowen Xin、Tony Young、Claire E Wainwright、Tamara Blake、Leo Lebrat、Thomas Gaass、Thomas Benkert、Alto Stemmer、David Coman、Jason Dowling
Paper:
https://arxiv.org/abs/2408.09432
Introduction
Medical image synthesis is a crucial technique in modern healthcare, providing additional imaging modalities that are often costly, invasive, or harmful to acquire. This technology is particularly beneficial in scenarios such as MRI-only radiotherapy dose planning or children’s airway assessment, where additional scans can be avoided. Generative Adversarial Networks (GANs) have been widely adopted for this purpose, leveraging either well-aligned imaging pairs (supervised methods) or randomly unpaired data (unsupervised methods). However, substantial misalignment between image pairs, such as lung MRI-CT pairs affected by respiratory motion, poses significant challenges. This paper introduces a novel Deformation-aware GAN (DA-GAN) designed to dynamically correct misalignment during image synthesis, thereby improving image fidelity and correspondence.
Related Work
Supervised GANs
Supervised GANs, such as Pix2pix and its variants, rely on pixel-wise loss from well-aligned imaging pairs to learn optimal mappings. However, obtaining perfectly aligned pairs is often impractical due to patient motion or organ movement, leading to accumulated errors and suboptimal results.
Unsupervised GANs
Unsupervised GANs, like CycleGAN, enable training on randomly unpaired data by leveraging constraints such as cycle consistency or mutual information. Despite their flexibility, these models are not designed to utilize pairing information to uncover optimal mappings, which is essential for tasks like radiotherapy planning.
Diffusion Models
Diffusion models have shown promise in capturing distributions in computer vision applications. However, their computational expense and data requirements limit their applicability in the medical domain.
Registration-based GANs
Recent works like RegGAN have explored directional registration in image synthesis. However, real-life settings often involve large deformations, leading to correspondence ambiguity and degraded image fidelity.
Research Methodology
Problem Formulation
The objective is to accurately synthesize target imaging that spatially corresponds to the source image for tasks such as radiotherapy treatment planning. Given a training dataset with misaligned imaging pairs, the goal is to generate images that preserve spatial correspondence while differing in intensity characteristics.
DA-GAN Overview
The proposed DA-GAN consists of three major components:
1. Modality Generators (G and F): Translate the source image to the target appearance while preserving spatial correspondence.
2. Symmetric Spatial Aligners (Ay and Ax): Exploit symmetric correspondence during image-to-image translation to optimize unique and optimal mapping.
3. Deformation-aware Discriminators (Dy and Dx): Disentangle mismatched spatial morphology from the judgment of image fidelity.
DA-GAN Objective
DA-GAN is constrained by three loss functions:
1. Symmetric Registration Loss (Lsr): Punishes dissimilarity between misaligned imaging pairs and encourages local smoothness on the deformation field.
2. Multi-objective Inverse-consistency Loss (Lmic): Improves image alignment during symmetric registration and synthesis correspondence to the source image.
3. Deformation-aware Adversarial Loss (Ladv_da): Disentangles the influence of spatial morphology from intensity characteristic learning.
Experimental Design
Simulation Dataset
The public brain T1-T2 MRI dataset (BraTS 2018) was used for simulation experiments. The dataset was augmented with six levels of non-affine misalignments to simulate real-world conditions.
Clinical Lung MRI/CT Dataset
A private lung MRI-CT dataset was used to validate the proposed algorithm in a real-world radiotherapy treatment planning setting. The dataset included pairs of ultrashort-echo time MRI and CT imaging, with preliminary registration but observed alignment errors.
Experiment Settings
Experiments were implemented in Pytorch on a 64-bit Ubuntu Linux system with a 16 GB Nvidia P100 GPU. The training included 50 epochs for both datasets, with evaluation metrics including Normalized Mean Absolute Error (NMAE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity (SSIM).
Results and Analysis
Simulation Experiments
DA-GAN consistently outperformed comparison methods across all metrics and levels of non-affine misalignment. The results remained stable with increased levels of misalignment, demonstrating the robustness of the proposed method.
Lung MRI-CT Dataset
DA-GAN achieved superior performance compared to eight state-of-the-art methods, particularly in challenging regions such as the spine, bones, and heart. The results highlight the potential of DA-GAN for synthetic CT in MRI-only radiotherapy.
Ablation Study
The ablation study confirmed the contributions of the proposed loss functions (Ladv_da and Lmic) to the overall performance improvement. The paired t-test showed statistically significant improvements in all metrics.
Overall Conclusion
This study introduces DA-GAN, a novel approach for medical image synthesis with substantially misaligned imaging pairs. By incorporating symmetric registration and deformation-aware adversarial loss, DA-GAN achieves high-fidelity image synthesis while dynamically correcting misalignment. The method was validated on both simulated and real-world datasets, demonstrating its potential for clinical applications such as early diagnosis and radiotherapy planning.
