SMILE: Semantically-guided Multi-attribute Image and Layout Editing
ICCVW 2021
AndrĂ©s Romero    Luc Van Gool    Radu Timofte   
     
[Paper 📑]    
[Github 💻]    
[Poster 📦]    
[Slides 📽]    


Abstract

Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs). Exploring the disentangled attribute space within a transformation is a very challenging task due to the multiple and mutually-inclusive nature of the facial images, where different labels (eyeglasses, hats, hair, identity, etc.) can co-exist at the same time. Several works address this issue either by exploiting the modality of each domain/attribute using a conditional random vector noise, or extracting the modality from an exemplary image. However, existing methods cannot handle both random and reference transformations for multiple attributes, which limits the generality of the solutions. In this paper, we successfully exploit a multimodal representation that handles all attributes, be it guided by random noise or exemplar images, while only using the underlying domain information of the target domain. We present extensive qualitative and quantitative results for facial datasets and several different attributes that show the superiority of our method. Additionally, our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space, and it can be easily extended to head-swapping and face-reenactment applications without being trained on videos.


Video


Method



Exemplar Attribute Manipulation

We show the results of manipulating an input image with respect to reference attributes.



More examples on Lip Syncing



It also works for videos!


Paper

Romero, A., Van Gool, L., Timofte, R. SMILE: Semantically-guided Multi-attribute Image and Layout Editing
ICCVW, 2021.
(camera ready)


[Bibtex]


Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.