Application of Wasserstein GAN

When it was proposed that GAN uses Wasserstein distance as the training metric, GAN is usually seen as a transportation problem. Previously, it was mentioned in a previous post that GAN can be seen as a transportation problem, and because of that, some computation can be simplified by relating a kernel in the discriminator and the generator.

GAN can be used in word translation problem too. In a recent preprint in arXiv (refer to arXiv:1710.04087), Wasserstein GAN has been used to train a machine translation machine, given that there are no parallel data between the word embeddings between two languages. The translation mapping is seen as a generator, and the mapping is described using Wasserstein distance. The training objective is cross-domain similarity local scaling (CSLS). Their work has been performed in English-Russian and English-Chinese mappings.

It seems to work. Given GAN sometimes does not work for unknown reasons, it is an excitement that it works.

• “Generative Adversarial Networks,” Everything About Data Analytics, WordPress (2017). [WordPress]
• Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, “Generative Adversarial Networks,” arXiv:1406.2661 (2014). [arXiv]
• Ian Goodfellow, “NIPS 2016 Tutorial: Generative Adversarial Networks,” arXiv:1701.00160 (2017). [arXiv]
• Na Lei, Kehua Su, Li Cui, Shing-Tung Yau, David Xianfeng Gu, “A Geometric View of Optimal Transportation and Generative Model,” arXiv:1710.05488 (2017). [arXiv]
• “On Wasserstein GAN,” Everything About Data Analytics, WordPress (2017). [WordPress]
• “Interpretability of Neural Networks,”Everything About Data Analytics, WordPress (2017).  [WordPress]
• Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou, “Word Translation Without Parallel Data,” arXiv:1710.04087 (2017). [arXiv]
• “Word Mover’s Distance as a Linear Programming Problem,”Everything About Data Analytics, WordPress (2017). [WordPress]
• 罗若天, “论文笔记：Word translation without parallel data无监督单词翻译” RT的论文笔记以及其他乱七八糟的东西. (2017) [Zhihu]
• “Word Embedding Algorithms,” Everything About Data Analytics, WordPress (2016). [WordPress]