Text Style Transfer based on Multi-factor Disentanglement and Mixture

Anna Zhu, Zhanhui Yin, Brian Kenji Iwana, Xinyu Zhou, Shengwu Xiong
2022 Proceedings of the 30th ACM International Conference on Multimedia  
Text style transfer aims to transfer the reference style of one text image to another text image. Previous works have only been able to transfer the style to a binary text image. In this paper, we propose a framework to disentangle the text images into three factors: text content, font, and style features, and then remix the factors of different images to transfer a new style. Both the reference and input text images have no style restrictions. Adversarial training through multi-factor cross
more » ... ognition is adopted in the network for better feature disentanglement and representation. To decompose the input text images into a disentangled representation with swappable factors, the network is trained using similarity mining within pairs of exemplars. To train our model, we synthesized a new dataset with various text styles in both English and Chinese. Several ablation studies and extensive experiments on our designed and public datasets demonstrate the effectiveness of our approach for text style transfer. CCS CONCEPTS • Computing methodologies → Computer vision; Image-based rendering.
doi:10.1145/3503161.3548239 fatcat:7e2nkumntjcxrdrxcirdy2irmy