Exploiting Inferential Structure in Neural Processes
Tailor, Dharmesh, Khan, Mohammad Emtiyaz, and Nalisnick, Eric
In 5th Workshop on Tractable Probabilistic Modeling at UAI 2022 Aug 2022
Neural processes (NPs) can be extremely fast at test time, but their training requires a wide range of context sets to generalize well. We propose to address this issue by incorporating the structure of graphical models into NPs. This leads to aggregation strategies in which context points are appropriately weighted, generalizing a recent proposal by Volpp et al., . The weighting further reveals an interpretation of each point, which we refer to as the neural sufficient statistics. It is expected that by exploiting information in structured priors, the data inefficiency of NPs can be alleviated.