This week we’ll have two talks in the seminar: one by Stephan Alaniz in the regular Thursday slot (announcement will appear soon), and an extra one on Wednesday October 31 at 16:00 in C3.163, where Giorgio Patrini will give a talk titled “Sinkhorn AutoEncoders”. Afterwards there are the usual drinks and snacks!
Abstract: Optimal Transport offers an alternative to maximum likelihood for learning generative autoencoding models. We show how this principle dictates the minimization of the Wasserstein distance between the encoder aggregated posterior and the prior, plus a reconstruction error. We prove that in the non-parametric limit the autoencoder generates the data distribution if and only if the two distributions match exactly, and that the optimum can be obtained by deterministic autoencoders. We then introduce the Sinkhorn AutoEncoder (SAE), which casts the problem into Optimal Transport on the latent space. The resulting Wasserstein distance is minimized by backpropagating through the Sinkhorn algorithm. SAE models the aggregated posterior as an implicit distribution and therefore does not need a reparameterization trick for gradients estimation. Moreover, it requires virtually no adaptation to different prior distributions. We demonstrate its flexibility by considering models with hyperspherical and Dirichlet priors, as well as a simple case of probabilistic programming. SAE matches or outperforms other autoencoding models in visual quality and FID scores.
Joint work with Marcello Carioni (KFU Graz), Patrick Forré, Samarth Bhargav, Max Welling, Rianne van den Berg, Tim Genewein (Bosch Centre for AI), Frank Nielsen (Ecole Polytecnique)