Unaligned Image-to-Image Translation with Grad-CAM

Abstract

Unsupervised image-to-image translation has been succeeded in changing an a certain style of image while preserving the original content. However, previous methods still have difficulty when domains are not aligned. To address these issues, we propose a class activation map loss based on pretrained gradient-weighted class activation mapping (Grad-CAM) network. The pretrained Grad-CAM highlights critical regions in the image using distinct class activation. Additionally, based on the class-activation map, a generator can focus on more critical regions while ignoring insignificant regions of the image. We also propose an alternative U-Net based discriminator architecture based on insights from the U-Net-based generative adversarial network. The proposed U-Net discriminator architecture allows the generator to focus on more discriminative image regions between domains.