RegC2P: A registration-enhanced GAN for 3D CT-to-PET translation
6 viewsDOI:
https://doi.org/10.54939/1859-1043.j.mst.102.2025.125-132Keywords:
3D Image-to-Image translation; 3D PET/CT imaging; CT-to-PET; GAN; U-Net.Abstract
The fusion of Positron Emission Tomography (PET) and Computed Tomography (CT) has significantly advanced cancer imaging by combining metabolic and anatomical information, enhancing diagnosis, staging, and treatment monitoring. However, the widespread use of PET/CT systems is limited by the scarcity of PET scanners and the reliance on radioactive tracers. To address this, 3D image-to-image translation has emerged as a promising solution for generating synthetic PET images from CT scans. Existing generative methods based on Generative Adversarial Networks (GANs) face challenges such as instability and stochastic outputs that lack precision for reliable 3D CT-to-PET translation. We propose RegC2P, a novel approach that integrates a registration module into a GAN-based framework to generate accurately aligned 3D PET images from CT scans. RegC2P transforms the problem into a slice-by-slice 2D image-to-image translation task, where individual 2D CT slices are translated into PET slices and then stacked into a 3D PET volume. Misaligned 2D PET slices are treated as noisy labels, and the generator is trained with an additional registration network to adaptively correct the misalignment, optimizing both the translation and registration tasks simultaneously. To ensure smoothness and consistency across generated PET slices throughout the entire volume, we introduce a 3D U-Net refinement network. Extensive experiments on large datasets demonstrate that RegC2P outperforms state-of-the-art methods, achieving a 10.16% reduction in MAE, a 0.96% improvement in SSIM, and a 3.6% increase in PSNR, setting a new benchmark for the quality of synthesized 3D PET images.
References
[1]. L. Wei and I. El Naqa, “Artificial intelligence for response evaluation with PET/CT,” in Seminars in Nuclear Medicine, vol. 51, pp. 157–169, Elsevier, (2021).
[2]. S. R. Cherry, T. Jones, J. S. Karp, J. Qi, W. W. Moses, and R. D. Badawi, “Total-body PET: maximizing sensitivity to create new opportunities for clinical research and patient care,” Journal of Nuclear Medicine, vol. 59, no. 1, pp. 3–12, (2018).
[3]. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in Neural Information Processing Systems, vol. 27, (2014).
[4]. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125– 1134, (2017).
[5]. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the 2017 IEEE International Conference on Computer Vision, pp. 2223–2232, (2017).
[6]. H. Tang, D. Xu, N. Sebe, Y. Wang, J. J. Corso, and Y. Yan, “Multichannel attention selection GAN with cascaded semantic guidance for cross-view image translation,” in Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2417– 2426, (2019).
[7]. K. Armanious, C. Jiang, M. Fischer, T. Kustner, T. Hepp, K. Nikolaou, S. Gatidis, and B. Yang, “Medgan: Medical image translation using GANs,” Computerized Medical Imaging and Graphics, vol. 79, p. 101684, (2020).
[8]. Z. Liang, H. Wei, G. Liu, M. Cheng, J. Gao, S. Li, and X. Tian, “Leveraging GAN-based CBCT-to-CT translation models for enhanced image quality and accurate photon and proton dose calculation in adaptive radiotherapy,” Journal of Radiation Research and Applied Sciences, vol. 17, no. 1, p. 100809, (2024).
[9]. Y. Zhu, Y. Tang, Y. Tang, D. C. Elton, S. Lee, P. J. Pickhardt, and R. M. Summers, “Cross-domain medical image translation by shared latent gaussian mixture model,” in Proceedings of the 23rd International Conference Medical Image Computing and Computer Assisted Intervention, pp. 379–389, Springer, (2020).
[10]. L. Kong, C. Lian, D. Huang, Y. Hu, Q. Zhou, et al., “Breaking the dilemma of medical image-to-image translation,” Advances in Neural Information Processing Systems, vol. 34, pp. 1964–1978, (2021).
[11]. D. T. Nguyen, T. T. Nguyen, H. T. Nguyen, T. T. Nguyen, H.-H. Pham, T. H. Nguyen, T. N. Truong, and P. L. Nguyen, “CT to PET Translation: A Large-scale Dataset and Domain-Knowledge-Guided Diffusion Approach,” Accepted for publication at the 2025 IEEE/CVF Winter Conference on Applications of Computer Vision, (2024).
[12]. Xie, Huidong, et al. “DDPET-3D: Dose-aware Diffusion Model for 3D Ultra Low-dose PET Imaging.” arXiv preprint arXiv:2311.04248 (2023).
[13]. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241, (2015).
