Talk by Giorgio Patrini

This week we’ll have two talks in the seminar: one by Stephan Alaniz in the regular Thursday slot (announcement will appear soon), and an extra one on Wednesday October 31 at 16:00 in C3.163, where Giorgio Patrini will give a talk titled “Sinkhorn AutoEncoders”. Afterwards there are the usual drinks and snacks!

Abstract: Optimal Transport offers an alternative to maximum likelihood for learning generative autoencoding models. We show how this principle dictates the minimization of the Wasserstein distance between the encoder aggregated posterior and the prior, plus a reconstruction error. We prove that in the non-parametric limit the autoencoder generates the data distribution if and only if the two distributions match exactly, and that the optimum can be obtained by deterministic autoencoders. We then introduce the Sinkhorn AutoEncoder (SAE), which casts the problem into Optimal Transport on the latent space. The resulting Wasserstein distance is minimized by backpropagating through the Sinkhorn algorithm. SAE models the aggregated posterior as an implicit distribution and therefore does not need a reparameterization trick for gradients estimation. Moreover, it requires virtually no adaptation to different prior distributions. We demonstrate its flexibility by considering models with hyperspherical and Dirichlet priors, as well as a simple case of probabilistic programming. SAE matches or outperforms other autoencoding models in visual quality and FID scores.

Joint work with Marcello Carioni (KFU Graz), Patrick Forré, Samarth Bhargav, Max Welling, Rianne van den Berg, Tim Genewein (Bosch Centre for AI), Frank Nielsen (Ecole Polytecnique)

Talk by Kihyuk Sohn

You are all cordially invited to the AMLab seminar on Thursday October 25 at 16:00 in C3.163 (FNWI, Amsterdam Science Park), where Kihyuk Sohn (NEC) will give a talk titled “Deep Domain Adaptation in the Wild”. Afterwards there are the usual drinks and snacks.

Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain. However, it is not well studied at which levels of representations the adaptation should happen and their complementary properties. Furthermore, the theory of domain adaptation is limited to classification problems whose source and target domains share the same task. In this talk, we address these challenges in deep domain adaptation. Firstly, we argue that the adaptation may happen at various levels of representations, such as input pixels, intermediate features, or output labels, with an injection of different insights at different levels. Secondly, we generalize the theory of domain adaptation to the case where source and target domains do not necessarily share the same classification task. We demonstrate the effectiveness of the proposed methods in several vision applications, namely, car recognition in the surveillance domain, face recognition across various ethnicity, and semantic segmentation.

Kihyuk Sohn is a researcher in the Media Analytics group of NEC Laboratories America. His research interest lies in machine learning and computer vision, with a focus on deep representation learning from large-scale, structured and multimodal data for robust visual perception. He obtained his Ph.D (2015) from the department of Electrical Engineering and Computer Science, University of Michigan.

PhD student Noud de Kroon joined AMLab

Noud de Kroon has joined the UvA in October 2018 as a PhD student of AMLab, under the joint supervision of dr. Joris Mooij and dr. Danielle Belgrave (Microsoft Research Cambridge). Previously, he obtained a bachelor’s degree in software science at Eindhoven University of Technology and a master’s degree in computer science at the University of Oxford. His research focus is on combining causality and reinforcement learning in order to make better
decisions and improve data efficiency, with applications for example in the medical domain.

Talk by Shihan Wang

You are all cordially invited to the AMLab seminar on Thursday October 18 at 16:00 in C3.163, where Shihan Wang will give a talk titled “Apply Machine Learning and Data Mining to Promote Physical Activity”. Afterwards there are the usual drinks and snacks!

Abstract: In this talk, we will introduce our research work in the “playful data-driven active urban living” (PAUL) project. Targeting on physical inactivity issue in modern society, we aim to motive less active people to participate in more physical activity. Given an overview of this project, we will mainly present recent papers.
Driven by a large-scale dataset of Dutch people’s running records (over 10K people in about 4 years), we start with characterizing runners based on their different temporal activity patterns. Then, in respect of diverse users, we studied how environmental situations (time, weather, geographical and social information) at the start time of a run affect the running distance. A rule-based machine learning method is applied to capture combined situations frequently associated with relevant long-distance runs. These environmental situations are going to be used in a mobile system, to identify the ‘right timing’ for motivating people to start longer-distance runs via message interventions.

Talk by Patrick Forré

You are all cordially invited to the AMLab seminar on Thursday October 11 at 16:00 in C3.163, where Patrick Forré will give a talk titled “Non-linear structural causal models with cycles and latent confounders”. Afterwards there are the usual drinks and snacks!

Abstract: In this talk we will present main results of two of our recent papers.
We will introduce a flexible class of general structural causal models that allow for non-/linear functional relations (like neural networks, etc.), arbitrary probability distributions (like discrete, continuous, mixtures, etc.), causal cycles (like feedback, etc.) and latent variables (aka confounders). For such models we will demonstrate several desirable properties, how to do causal reasoning, the rules of do-calculus and graphical criteria for conditional independence relations. We will also show how the latter can be exploited for causal discovery algorithms in such general context.

Talk by Bela Mulder

You are all cordially invited to the AMLab seminar on Thursday October 4 at 16:00 in C3.163, where Bela Mulder (AMOLF) will give a talk titled “Pitting man against machine in the arena of bottom-up design of crystal structures”. Afterwards there are the usual drinks and snacks!

Abstract: In this highly informal seminar I would like to pitch the question “Can a machine learning system develop a theory?” One of the much-touted properties of deep learning networks is that their deeper levels develop higher order generalization representations of their inputs. This begs the question whether they are able to hit upon the type of hidden structures in physical problem that are the cornerstone of effective physical theories. I would like to propose to test this idea in a concrete setting related to the highly relevant question of inverse design of self-assembling matter. I have recently formulated a novel approach towards inferring the specific short range isotropic interactions between particles of multiple types on lattices of given geometry in order that they spontaneously form specified periodic states of essentially arbitrary complexity. This approach rests upon the subtle intertwining between the group of transformations that leave the lattice structure invariant, with the group of permutations in the set of particle types induced by these same transformations on the target ordered structure. The upshot of this approach is that the number of independent coupling constants in the lattice can be systematically reduced from O(N2), where N is the number of distinct species, to O(N). The idea would be to see whether a machine learning approach which uses the space of possible patterns and their trivial transforms under symmetry operations as input, the set of possible constants as outputs, and feedback based on the degree to which the target structure is realized with these coupling constants is able to “learn” the symmetry-based rules, in a way that also generalizes to similar patterns not included in the training set.

Talk by Jakub Tomczak

You are all cordially invited to the AMLab seminar on Thursday September 27 at 16:00 in C3.163 (FNWI, Amsterdam Science Park), where Jakub Tomczak will give a talk titled “Deep Learning and Bayesian Inference for Medical Imaging”. Afterwards there are the usual drinks and snacks.

The aim of the DeeBMED project was to develop a powerful automatic medical imaging tool that can cope with main problems associated with complex images like medical scans, namely, multimodality of data distribution, large number of dimension and small number of examples, small amount of labeled data, multi-source learning, and robustness to transformations. In order to counteract these issues I have proposed to use a probabilistic framework, namely, the Variational Auto-Encoder (VAE), that combines deep learning and Bayesian inference. Within the project I have followed two lines of research:

– Development of the VAE by:
* enriching the encoder (Householder flow, Sylvester flow, Hyperspherical VAE);
* enriching the prior (VampPrior);
* enriching the decoder (an ongoing work with Rianne van den Berg & Christos Louizos);
* learning fair representations (Hierarchical VampPrior VFAE);
* learning disentangled representation (ongoing work with Maximilian Ilse).

– Development of deep neural networks by:
* learning from large images, i.e., ~10,000×10,000 pixels, (Deep MIL, and an ongoing work with Nathan Ing, Arkadiusz Gertych, Beatrice Knudsen);
* learning from multiple sources, e.g., different views (an ongoing work with Henk van Voorst).

During the talk I will outline assumptions of the DeeBMED project and its successes. At the end, a possible direction for future work will be presented.

During the project I have a great pleasure to publish with the following people (in alphabetical order):
* the University of Amsterdam: Rianne van den Berg, Philip Botros, Nicola de Cao, Tim Davidson, Luca Falorsi, Shi Hu, Maximilian Ilse, Thomas Kipf, Max Welling;
* the University of Oxford: Leonard Hasenclever;
* the Academic Medical Center in Amsterdam: Onno de Boer, Sybren Meijer;
* the Cedars-Sinai Medical Center in Los Angeles: Arkadiusz Gertych, Nathan Ing, Beatrice Knudsen.

Last but not least, all former and current members of AMLAB, QUVA Lab, Delta Lab and Philips Lab made my project successful through multiple discussions, meetings and seminars.

Slides (pdf)

Talk by Patrick van der Smagt

You are all cordially invited to the AMLab seminar on Thursday September 20 at 16:00 in C3.163 (FNWI, Amsterdam Science Park), where Patrick van der Smagt (Volkswagen AI Research) will give a talk titled “Latent optimal control”. Afterwards there are the usual drinks and snacks.

Abstract: Control of multidimensional systems typically relies on accurately engineered models. Breaking this requirement is problematic with neural networks, as their Gaussian data assumptions typically do not hold. In my talk I will demonstrate how this problem can be efficiently solved by combining latent variable models with specific type of optimal control. The theory is demonstrated on various simulated closed-loop control systems as well as on real hardware.

Talk by Cédric Archambeau

You are all cordially invited to the AMLab seminar on Tuesday September 11 at 15:00(!) in C0.110 (FNWI, Amsterdam Science Park), where Cédric Archambeau (Amazon Core AI) will give a talk titled “Learning Representations for Hyperparameter Transfer Learning”. Afterwards there are the usual drinks and snacks.

Abstract: Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization, critical in deep learning. Typically, BO relies on conventional Gaussian process regression, whose algorithmic complexity is cubic in the number of evaluations. As a result, Gaussian process-based BO cannot leverage large numbers of past function evaluations, for example, to warm-start related BO runs. After a brief intro to BO and an overview of several use cases at Amazon, I will discuss a multi-task adaptive Bayesian linear regression model, whose computational complexity is attractive (linear) in the number of function evaluations and able to leverage information of related black-box functions through a shared deep neural net. Experimental results show that the neural net learns a representation suitable for warm-starting related BO runs and that they can be accelerated when the target black-box function (e.g., validation loss) is learned together with other related signals (e.g., training loss). The proposed method was found to be at least one order of magnitude faster than competing neural net-based methods recently published in the literature.

This is joint work with Valerio Perrone, Rodolphe Jenatton, and Matthias Seeger. It will be presented at NIPS 2018.

Bio: Cedric is the science lead of Amazon Core AI, with teams in Berlin, Barcelona, Tuebingen, and Seattle. His work on democratizing machine learning enables teams at Amazon deliver a wide range of machine learning-based products, including customer facing services part of Amazon SageMaker ( Currently, he is interested in algorithms that learn representations, algorithms that learn to learn, and algorithms that avoid catastrophic forgetting (in deep learning). Prior to joining Amazon, he led the Machine Learning group at Xerox Research Centre Europe (now Naver Labs Europe). His team conducted applied research in machine learning, computational statistics and mechanism design, with applications in customer care, transportation and governmental services. He joined Amazon, Berlin, as an Applied Science Manager in October 2013, where he was in charge of delivering zero-parameter machine learning algorithms.

Talk by Joris Mooij

Now that the summer months are over, weekly AMLab seminars are starting up again. With a change: from now on, the talks will be held on Thursdays instead of Tuesdays.

You are all cordially invited to the AMLab seminar on Thursday September 6 at 16:00 in C3.163 (FNWI, Amsterdam Science Park), where Joris Mooij will give a talk titled “Validating Causal Discovery Methods”. Afterwards there are the usual drinks and snacks.

Since the pioneering work by Peirce and Fisher, the gold standard for causal discovery is a randomized experiment. An intriguing alternative approach to causal discovery was proposed in the nineties, based on conditional independence patterns in the data. Over the past decades, dozens of causal discovery methods based on that idea have been proposed. These methods clearly work on simulated data when all their assumptions are satisfied. However, demonstrating their usefulness on real data has been a challenge. In this talk, I will discuss some of our recent attempts at validating causal discovery methods on large-scale interventional data sets from molecular biology. I will discuss a micro-array gene expression data set and a mass cytometry data set that seem perfectly suited for validation of causal discovery methods at first sight. As it turns out, however, both causal discovery on these data and the validation of such methods is more challenging than one might think initially. We find that even sophisticated modern causal discovery algorithms are outperformed by simple (non-causal) baselines on these data sets.

(joint work with Philip Versteeg and Tineke Blom)