AMLab | Amsterdam Machine Learning Lab

Jan-Willem van de Meent

Associate professor (UHD)
AMLab and Delta Lab
Informatics Institute
University of Amsterdam
Science Park, Lab 42, L4.13

Personal page Google scholar Github Mastodon Twitter

Dr. Jan-Willem van de Meent is an Associate Professor (Universitair Hoofddocent) at the University of Amsterdam. He co-directs the AMLab with Max Welling and co-directs the Uva Bosch Delta Lab with Theo Gevers. He also holds a position as an Assistant Professor at Northeastern University, where he is currently on leave. Prior to becoming faculty at Northeastern, he held a postdoctoral position with Frank Wood at Oxford, as well as a postdoctoral position with Chris Wiggins and Ruben Gonzalez at Columbia University. He carried out his PhD research in biophysics at Leiden and Cambridge with Wim van Saarloos and Ray Goldstein.

Jan-Willem van de Meent’s group develops models for artificial intelligence by combining probabilistic programming and deep learning. A major theme in this work is understanding how we can develop data-efficient models in machine learning by incorporating knowledge of an underlying physical system, causal structure, or symmetries of the underlying domain. At a technical level, his group develops inference methods for probabilistic programming systems. He is one of the creators of Anglican, a probabilistic language based on Clojure and of Probabilistic Torch, a library for deep generative models that extends PyTorch. He is also an author on a forthcoming book on probabilistic programming, a draft of which is available on arXiv. To ground methodological work in practice, his group collaborates with researchers in neuroscience, NLP, healthcare, robotics, physics, and chemistry.

Jan-Willem van de Meent served as a founding co-chair of the international conference on probabilistic programming (PROBPROG). He is currently serving as a program chair for the international conference on artificial intelligence and statistics (AISTATS). He was the recipient of an NWO Rubicon Fellowship and is a current recipient of the NSF CAREER award.

Recent Publications

2023

TMLR

A Variational Perspective on Generative Flow Networks

Zimmermann, Heiko, Lindsten, Fredrik, Meent, Jan-Willem, and Naesseth, Christian A

Transactions on Machine Learning Research 2023

HTML PDF Code
LAFI

Verified Reversible Programming for Verified Lossless Compression

Townsend, James, and van de Meent, Jan-Willem

In POPL Workshop on Languages for Inference (LAFI) 2023

HTML PDF
LAFI

A convenient category of tracing measure kernels

Sennesh, Eli, and van de Meent, Jan-Willem

In POPL Workshop on Languages for Inference (LAFI) 2023

HTML PDF
LAFI

Semantics of Probabilistic Program Traces

Lew, Alexander, Sennesh, Eli, van de Meent, Jan-Willem, and Mansinghka, Vikash

In POPL Workshop on Languages for Inference (LAFI) 2023

HTML PDF

2022

PLOS Comp Bio

Probabilistic Program Inference in Network-Based Epidemiological Simulations

Smedemark-Margulies, Niklas, Walters, Robin, Zimmermann, Heiko, Laird, Lucas, Loo, Christian, Kaushik, Neela, Caceres, Rajmonda, and Meent, Jan-Willem

PLOS Computational Biology Nov 2022

Abs HTML PDF

Accurate epidemiological models require parameter estimates that account for mobility patterns and social network structure. We demonstrate the effectiveness of probabilistic programming for parameter inference in these models. We consider an agent-based simulation that represents mobility networks as degree-corrected stochastic block models, whose parameters we estimate from cell phone co-location data. We then use probabilistic program inference methods to approximate the distribution over disease transmission parameters conditioned on reported cases and deaths. Our experiments demonstrate that the resulting models improve the quality of fit in multiple geographies relative to baselines that do not model network topology.
Bio. Pysch.

Interoception as Modeling, Allostasis as Control

Sennesh, Eli, Theriault, Jordan, Brooks, Dana, Meent, Jan-Willem, Barrett, Lisa Feldman, and Quigley, Karen S.

Biological Psychology Nov 2022

Abs HTML PDF

The brain regulates the body by anticipating its needs and attempting to meet them before they arise – a process called allostasis. Allostasis requires a model of the changing sensory conditions within the body, a process called interoception. In this paper, we examine how interoception may provide performance feedback for allostasis. We suggest studying allostasis in terms of control theory, reviewing control theory’s applications to related issues in physiology, motor control, and decision making. We synthesize these by relating them to the important properties of allostatic regulation as a control problem. We then sketch a novel formalism for how the brain might perform allostatic control of the viscera by analogy to skeletomotor control, including a mathematical view on how interoception acts as performance feedback for allostasis. Finally, we suggest ways to test implications of our hypotheses.
Neuroinf.

A Computational Neural Model for Mapping Degenerate Neural Architectures

Khan, Zulqarnain, Wang, Yiyu, Sennesh, Eli, Dy, Jennifer, Ostadabbas, Sarah, van de Meent, Jan-Willem, Hutchinson, J. Benjamin, and Satpute, Ajay B.

Neuroinformatics Mar 2022

Abs HTML PDF

Degeneracy in biological systems refers to a many-to-one mapping between physical structures and their functional (including psychological) outcomes. Despite the ubiquity of the phenomenon, traditional analytical tools for modeling degeneracy in neuroscience are extremely limited. In this study, we generated synthetic datasets to describe three situations of degeneracy in fMRI data to demonstrate the limitations of the current univariate approach. We describe a novel computational approach for the analysis referred to as neural topographic factor analysis (NTFA). NTFA is designed to capture variations in neural activity across task conditions and participants. The advantage of this discovery-oriented approach is to reveal whether and how experimental trials and participants cluster into task conditions and participant groups. We applied NTFA on simulated data, revealing the appropriate degeneracy assumption in all three situations and demonstrating NTFA’s utility in uncovering degeneracy. Lastly, we discussed the importance of testing degeneracy in fMRI data and the implications of applying NTFA to do so.
WACV

Enhancing Few-Shot Image Classification With Unlabelled Examples

Bateni, Peyman, Barber, Jarred, Meent, Jan-Willem, and Wood, Frank

In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Jan 2022

Abs HTML PDF Code

We develop a transductive meta-learning method that uses unlabelled instances to improve few-shot image classification performance. Our approach combines a regularized Mahalanobis-distance-based soft k-means clustering procedure with a modified state of the art neural adaptive feature extractor to achieve improved test-time classification accuracy using unlabelled data. We evaluate our method on transductive few-shot learning tasks, in which the goal is to jointly predict labels for query (test) examples given a set of support (training) examples. We achieve state of the art performance on the Meta-Dataset, mini-ImageNet and tiered-ImageNet benchmarks. All trained models and code have been made publicly available

2021

AISTATS

Rate-Regularization and Generalization in Variational Autoencoders

Bozkurt, Alican, Esmaeili, Babak, Tristan, Jean-Baptiste, Brooks, Dana, Dy, Jennifer, and Meent, Jan-Willem

In International Conference on Artificial Intelligence and Statistics Mar 2021

Abs HTML PDF

Variational autoencoders (VAEs) optimize an objective that comprises a reconstruction loss (the distortion) and a KL term (the rate). The rate is an upper bound on the mutual information, which is...
UAI

Learning proposals for probabilistic programs with inference combinators

Stites, Sam, Zimmermann, Heiko, Wu, Hao, Sennesh, Eli, and Meent, Jan-Willem

In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence 27–30 jul 2021

Abs HTML PDF Code

We develop operators for construction of proposals in probabilistic programs, which we refer to as inference combinators. Inference combinators define a grammar over importance samplers that compose primitive operations such as application of a transition kernel and importance resampling. Proposals in these samplers can be parameterized using neural networks, which in turn can be trained by optimizing variational objectives. The result is a framework for user-programmable variational methods that are correct by construction and can be tailored to specific models. We demonstrate the flexibility of this framework by implementing advanced variational methods based on amortized Gibbs sampling and annealing.
EMNLP

Disentangling Representations of Text by Masking Transformers

Zhang, Xiongyi, van de Meent, Jan-Willem, and Wallace, Byron

In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing Nov 2021

Abs HTML PDF

Representations from large pretrained models such as BERT encode a range of features into monolithic vectors, affording strong predictive accuracy across a range of downstream tasks. In this paper we explore whether it is possible to learn disentangled representations by identifying existing subnetworks within pretrained models that encode distinct, complementary aspects. Concretely, we learn binary masks over transformer weights or hidden units to uncover subsets of features that correlate with a specific factor of variation; this eliminates the need to train a disentangled model from scratch for a particular task. We evaluate this method with respect to its ability to disentangle representations of sentiment from genre in movie reviews, toxicity from dialect in Tweets, and syntax from semantics. By combining masking with magnitude pruning we find that we can identify sparse subnetworks within BERT that strongly encode particular aspects (e.g., semantics) while only weakly encoding others (e.g., syntax). Moreover, despite only learning masks, disentanglement-via-masking performs as well as — and often better than —previously proposed methods based on variational autoencoders and adversarial training.
NAACL

On the Impact of Random Seeds on the Fairness of Clinical Classifiers

Amir, Silvio, van de Meent, Jan-Willem, and Wallace, Byron

In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Jun 2021

Abs HTML PDF

Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). We explore the implications of this phenomenon for model fairness across demographic groups in clinical prediction tasks over electronic health records (EHR) in MIMIC-III —— the standard dataset in clinical NLP research. Apparent subgroup performance varies substantially for seeds that yield similar overall performance, although there is no evidence of a trade-off between overall and subgroup performance. However, we also find that the small sample sizes inherent to looking at intersections of minority groups and somewhat rare conditions limit our ability to accurately estimate disparities. Further, we find that jointly optimizing for high overall performance and low disparities does not yield statistically significant improvements. Our results suggest that fairness work using MIMIC-III should carefully account for variations in apparent differences that may arise from stochasticity and small sample sizes.
NeurIPS

Nested Variational Inference

Zimmermann, Heiko, Wu, Hao, Esmaeili, Babak, and Meent, Jan-Willem

In Advances in Neural Information Processing Systems Jun 2021

Abs HTML PDF

We develop nested variational inference (NVI), a family of methods that learn proposals for nested importance samplers by minimizing an forward or reverse KL divergence at each level of nesting. NVI is applicable to many commonly-used importance sampling strategies and provides a mechanism for learning intermediate densities, which can serve as heuristics to guide the sampler. Our experiments apply NVI to (a) sample from a multimodal distribution using a learned annealing path (b) learn heuristics that approximate the likelihood of future observations in a hidden Markov model and (c) to perform amortized inference in hierarchical deep generative models. We observe that optimizing nested objectives leads to improved sample quality in terms of log average weight and effective sample size.
AAMAS

Action Priors for Large Action Spaces in Robotics

Biza, Ondrej, Wang, Dian, Platt, Robert, Meent, Jan-Willem, and Wong, Lawson L.S.

In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems Jun 2021

Abs HTML PDF Code

In robotics, it is often not possible to learn useful policies using pure model-free reinforcement learning without significant reward shaping or curriculum learning. As a consequence, many researchers rely on expert demonstrations to guide learning. However, acquiring expert demonstrations can be expensive. This paper proposes an alternative approach where the solutions of previously solved tasks are used to produce an action prior that can facilitate exploration in future tasks. The action prior is a probability distribution over actions that summarizes the set of policies found solving previous tasks. Our results indicate that this approach can be used to solve robotic manipulation problems that would otherwise be infeasible without expert demonstrations. Source code is available at https://github.com/ondrejba/action_priors.
ICML

Conjugate Energy-Based Models

Wu*, Hao, Esmaeili*, Babak, Wick, Michael L, Tristan, Jean-Baptiste, and van de Meent, Jan-Willem

In International Conference on Machine Learning Jun 2021

Abs

In this paper, we propose conjugate energy-based models (CEBMs), a new class of energy-based models that define a joint density over data and latent variables. The joint density of a CEBM decomposes into an intractable distribution over data and a tractable posterior over latent variables. CEBMs have similar use cases as variational autoencoders, in the sense that they learn an unsupervised mapping from data to latent variables. However, these models omit a generator network, which allows them to learn more flexible notions of similarity between data points. Our experiments demonstrate that conjugate EBMs achieve competitive results in terms of image modelling, predictive power of latent space, and out-of-domain detection on a variety of datasets.

2020

NeurIPS

Neural Topographic Factor Analysis for fMRI Data

Sennesh*, Eli, Khan*, Zulqarnain, Wang, Yiyu, Hutchinson, J. Benjamin, Satpute, Ajay, Dy, Jennifer, and van de Meent, Jan-Willem

Advances in Neural Information Processing Systems 2020

HTML PDF Code
ICML

Amortized Population Gibbs Samplers with Neural Sufficient Statistics

Wu, Hao, Zimmermann, Heiko, Sennesh, Eli, Le, Tuan Anh, and van de Meent, Jan-Willem

Advances in Neural Information Processing Systems 2020

Abs HTML PDF

Amortized variational methods have proven difficult to scale to structured problems, such as inferring positions of multiple objects from video images. We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frame structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can be used to train highly-structured deep generative models in an unsupervised manner, and achieve substantial improvements in inference accuracy relative to standard autoencoding variational methods.
MLHC

Query-Focused EHR Summarization to Aid Imaging Diagnosis

McInerney, Denis Jered, Dabiri, Borna, Touret, Anne-Sophie, Young, Geoffrey, van de Meent, Jan-Willem, and Wallace, Byron C.

Machine Learning for Healthcare Apr 2020

Abs HTML PDF

Electronic Health Records (EHRs) provide vital contextual information to radiologists and other physicians when making a diagnosis. Unfortunately, because a given patient’s record may contain hundreds of notes and reports, identifying relevant information within these in the short time typically allotted to a case is very difficult. We propose and evaluate models that extract relevant text snippets from patient records to provide a rough case summary intended to aid physicians considering one or more diagnoses. This is hard because direct supervision (i.e., physician annotations of snippets relevant to specific diagnoses in medical records) is prohibitively expensive to collect at scale. We propose a distantly supervised strategy in which we use groups of International Classification of Diseases (ICD) codes observed in ’future’ records as noisy proxies for ’downstream’ diagnoses. Using this we train a transformer-based neural model to perform extractive summarization conditioned on potential diagnoses. This model defines an attention mechanism that is conditioned on potential diagnoses (queries) provided by the diagnosing physician. We train (via distant supervision) and evaluate variants of this model on EHR data from a local hospital and MIMIC-III (the latter to facilitate reproducibility). Evaluations performed by radiologists demonstrate that these distantly supervised models yield better extractive summaries than do unsupervised approaches. Such models may aid diagnosis by identifying sentences in past patient reports that are clinically relevant to a potential diagnoses.