You are all cordially invited to the AMLab seminar on Tuesday March 13 at 16:00 in C3.163 (FNWI, Amsterdam Science Park), where prof. Max Welling will give a talk titled “Stochastic Deep Learning”. Afterwards there are the usual drinks and snacks.
Abstract: Deep learning has been very successful in many applications, but there are a number challenges that still need to be addressed:
1) DL does not provide reliable confidence intervals
2) DL is susceptible to small adversarial input perturbations
3) DL easily overfits
4) DL uses too much energy and memory
In this talk I will argue that we should be looking at stochastic DL models where the hidden units are noisy. We can train these models with variational methods.
A number of interesting connections emerge in such models:
1) The noisy hidden units form an information bottleneck
2) Through local reparameterization we can interpret these models as Bayesian
3) The noise can be used to create privacy preserving models
4) Stochastic quantization to low bit-width can make DL more power and memory efficient.
This talk will not go in great depth in these topics but rather paint the larger picture.