The strange case of out-of-distribution generalization
Machine learning systems are unable to generalize out-of-distribution, crumbling when deployed in conditions different to those of training. In this talk, I will investigate this phenomenon from multiple perspectives. In particular, I will:
* Present some interesting links between the causal structure of data and its invariant correlations;
* Review the Invariant Risk Minimization (IRM) principle;
* Share with you the largest empirical study on domain generalization algorithms, revealing a frustrating result;
* Propose the use of simple "unit-tests" to debug out-of-distribution generalization algorithms. In particular, I will share a set of challenging low-dimensional linear problems that you can try at home;
* Offer my thoughts on where to go next.
David is a research scientist at Facebook AI Research (FAIR), where he develops theory and algorithms to discover causation from data, in order to create robust learning machines. Previously he has worked at Google, the European Space Agency, and the Red Bull Formula 1 team. He graduated with a PhD from the Max Planck Institute for Intelligent Systems (in Tübingen, with Bernhard Schölkpof) and the University of Cambridge (with Zoubin Ghahramani).