24 February 2024 UCGAN: a contrastive learning-based unaligned image translation model with importance reweighting
Shibin Wang, Shiying Zhang, Dong Liu, Jinghui Zhong
Author Affiliations +
Abstract

Image-to-image translation with unaligned domains is a seminal work that is capable of automatically selecting suitable image of unaligned dataset for translation. Unaligned datasets often contain irrelevant images that disrupt translation. However, the accuracy of selecting appropriate images is an issue that needs to be urgently improved. To improve the grasp of image features and enhance the performance of unaligned image-to-image translation without changing much semantic information, we propose a novel framework unaligned image-to-image translation model based on contrastive learning and generative adversarial networks (UCGAN), which can focus on both local and global features of the image. Inspired by contrastive learning, here we propose the patch-by-patch feature contrastive (PPFC). PPFC reuses the encoder of the generator to extract multi-layer features of different image patches. We then normalize the various features using L1-norm instead of L2-norm. To further improve translation performance, we also set up paired feature comparisons. We compare our proposed method UCGAN with state-of-the-art algorithms and the experimental results have demonstrated the superiority and practicality of our method.

© 2024 SPIE and IS&T
Shibin Wang, Shiying Zhang, Dong Liu, and Jinghui Zhong "UCGAN: a contrastive learning-based unaligned image translation model with importance reweighting," Journal of Electronic Imaging 33(1), 013055 (24 February 2024). https://doi.org/10.1117/1.JEI.33.1.013055
Received: 27 November 2023; Accepted: 7 February 2024; Published: 24 February 2024
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Education and training

Gallium nitride

Feature extraction

Image processing

Image enhancement

Semantics

Design

Back to Top