You are all cordially invited to the AMLab seminar on Thursday 12th September at 14:00 in C4.174, where Marco Federici will give a talk titled “Towards Robust Representations by Exploiting Multiple Data Views”. There are the usual drinks and snacks!
Abstract: The problem of creating data representations can be formulated as the definition of an encoding function which maps observations into a predefined code space. Whenever the encoding is used as an intermediate step for a predictive task, among the possible encodings, we are generally interested in the ones that retain the desired target information. Furthermore, recent literature has shown that discarding irrelevant factors of variation in the data (minimality) yield robustness and invariance to nuisances of the task. Following these two general guidelines, in this work, we introduce an information-theoretical method that exploits some known properties of the predictive task to create robust data representations without requiring direct supervision signals. By exploiting pairs of joint observations, our model learns representations that are as discriminative as the original data for the predictive task while being more robust than the raw-signal. The proposed theory builds upon well-known self-supervised algorithms (such as Contrastive Predictive Coding and the InfoMax principle), bridging the gap between information bottleneck and probabilistic invariance. Empirical evidence shows the applicability of our model for both multi-view and single-view datasets.