On Question Answering on Images and Databases

Dzmitry Bahdanau / McGill University

Mar 15, 2021

Abstract: The AI's ability to answer questions that are grounded in context is interesting from both academic and practical perspectives. In this talk, I will present two projects that study question answering (QA) in visual and symbolic database context respectively. In the first project my collaborators and I show that a neuro-symbolic visual QA system can learn variable bindings from the top-down QA training signal only. The system successfully learns both conventional bindings (e.g. objects) and less trivial ones (e.g. groups). In our second project my colleagues at Element AI and I explore text2sql systems for answering questions about databases. We perform extensive ablation testing to understand what is really necessary for such systems to achieve best performance. I will end the talk with a quick preview of our on-going few-shot text2sql research efforts.

Bio: Dzmitry Bahdanau is an Adjunct Professor at McGill University and a research scientist at ServiceNow Element AI. Prior to that, he obtained his PhD at Mila and Université de Montréal working with Yoshua Bengio. He is interested in fundamental and applied questions concerning natural language understanding. His main research areas include semantic parsing, language user interfaces, systematic generalization and hybrid neural-symbolic systems. He invented the content-based neural attention that is now a core tool in deep-learning-based natural language processing.