Image-to-image translation of label-free molecular vibrational images for a histopathological review using the UNet+/seg-cGAN model

Yunjie He, Jiasong Li, Steven Shen, Kai Liu, KELVIN K. WONG, Tiancheng He, STEPHEN T.C. WONG

Research output: Contribution to journalArticlepeer-review

Abstract

Translating images generated by label-free microscopy imaging, such as Coherent Anti-Stokes Raman Scattering (CARS), into more familiar clinical presentations of histopathological images will help the adoption of real-time, spectrally resolved label-free imaging in clinical diagnosis. Generative adversarial networks (GAN) have made great progress in image generation and translation, but have been criticized for lacking precision. In particular, GAN has often misinterpreted image information and identified incorrect content categories during image translation of microscopy scans. To alleviate this problem, we developed a new Pix2pix GAN model that simultaneously learns classifying contents in the images from a segmentation dataset during the image translation training. Our model integrates UNet+ with seg-cGAN, conditional generative adversarial networks with partial regularization of segmentation. Technical innovations of the UNet+/seg-cGAN model include: (1) replacing UNet with UNet+ as the Pix2pix cGAN's generator to enhance pattern extraction and richness of the gradient, and (2) applying the partial regularization strategy to train a part of the generator network as the segmentation sub-model on a separate segmentation dataset, thus enabling the model to identify correct content categories during image translation. The quality of histopathological-like images generated based on label-free CARS images has been improved significantly.

Original languageEnglish (US)
Pages (from-to)1924-1938
Number of pages15
JournalBiomedical Optics Express
Volume13
Issue number4
DOIs
StatePublished - Apr 1 2022

ASJC Scopus subject areas

  • Biotechnology
  • Atomic and Molecular Physics, and Optics

Fingerprint

Dive into the research topics of 'Image-to-image translation of label-free molecular vibrational images for a histopathological review using the UNet+/seg-cGAN model'. Together they form a unique fingerprint.

Cite this