[SAIF 2019] Day 1: Towards Compositional Understanding of the World by Deep Learning - Yoshua Bengio
[SAIF 2019] Day 1: Towards Compositional Understanding of the World by Deep Learning — Yoshua Bengio
Humans are much better than current AI systems at generalizing out-of-distribution. What ingredients can bring us closer to that level of competence? We propose 4 ingredients combined: (a) meta-learning (to learn end-to-end to generalize to modified distributions, sampled from a distribution over distributions), (b) designing modular architectures with the property that modules are fairly independent of each other and interacting sparsely while made to be composed in new ways easily, © capturing causal structure decomposed into independent mechanisms so as to correctly infer the effect of interventions by agents which modify the data distribution, and (d) building better and more stable models of the invariant properties of possibly changing environments by taking advantage of the interactions between the learner and its environment to learn semantic high-level variables and their interactions, i.e., adopting an agent perspective on learning to benefit deep learning of abstract representations. The last ingredient implies that learning purely from text is not sufficient and we need to strive for learning agents which build a model of the world, to which linguistic labels can be associated, i.e., performing grounded language learning. Whereas this agent perspective is reminiscent of deep reinforcement learning, the focus is not on how deep learning can help reinforcement learning (as a function approximation black box) but rather how the agent perspective common in reinforcement learning can help deep learning discover better representations of knowledge.
#SAIF #SamsungAIForum
For more info, visit our page:
#SAIT(Samsung Advanced Institute of Technology): smsng.co/sait
Humans are much better than current AI systems at generalizing out-of-distribution. What ingredients can bring us closer to that level of competence? We propose 4 ingredients combined: (a) meta-learning (to learn end-to-end to generalize to modified distributions, sampled from a distribution over distributions), (b) designing modular architectures with the property that modules are fairly independent of each other and interacting sparsely while made to be composed in new ways easily, © capturing causal structure decomposed into independent mechanisms so as to correctly infer the effect of interventions by agents which modify the data distribution, and (d) building better and more stable models of the invariant properties of possibly changing environments by taking advantage of the interactions between the learner and its environment to learn semantic high-level variables and their interactions, i.e., adopting an agent perspective on learning to benefit deep learning of abstract representations. The last ingredient implies that learning purely from text is not sufficient and we need to strive for learning agents which build a model of the world, to which linguistic labels can be associated, i.e., performing grounded language learning. Whereas this agent perspective is reminiscent of deep reinforcement learning, the focus is not on how deep learning can help reinforcement learning (as a function approximation black box) but rather how the agent perspective common in reinforcement learning can help deep learning discover better representations of knowledge.
#SAIF #SamsungAIForum
For more info, visit our page:
#SAIT(Samsung Advanced Institute of Technology): smsng.co/sait