Title: Integrating Logical and Vector Representations and Using Plan-Based Understanding for Complex QA, slides
Abstract: Both logical and vector representations of natural language semantics have strengths and weaknesses. Integrating both representations can improve reasoning for complex QA. We will review our previous work on using Markov Logic for such integration and discuss more recent neuro-symbolic approaches. Plan-based narrative understanding, on the other hand, is an approach introduced in the 1970's that tried to produce “causal explanations” of characters actions in a story in terms of their plans and goals, allowing comprehension of narrative text that effectively supports answering “why” questions. We will review early work on this approach and its use in Explanation Based Learning, and discuss our on-going work on using deep-learning to modernize plan-based understanding and make it more robust.
Bio: Raymond J. Mooney is a Professor in the Department of Computer Science at the University of Texas at Austin. He received his Ph.D. in 1988 from the University of Illinois at Urbana/Champaign. He is an author of over 170 published research papers, primarily in the areas of machine learning and natural language processing. He was the President of the International Machine Learning Society from 2008-2011, program co-chair for AAAI 2006, general chair for EMNLP 2005, and co-chair for ICML 1990. He is a Fellow of AAAI, ACM, and ACL and the recipient of the Classic Paper award from AAAI-19 and best paper awards from AAAI-96, KDD-04, ICML-05 and ACL-07.
Title: Towards AI systems that can build coherent causal models of what they read!, slides
Abstract: There has been a tremendous progress in machine comprehension in the past couple of years, measured through various AI benchmarks. But are these models capable of building a coherent mental model of what they read? Humans make countless implicit commonsense inferences about everyday situations. When even young children read or listen to a narrative, they construct an elaborate mental model of what happened and why, combining the presented information with relevant background knowledge to construct the causal chain that describes how the events unfolded. Though humans build such mental models of situations with ease, AI systems for tasks such as reading comprehension remain far from exhibiting similar commonsense reasoning capability. Two major bottlenecks have been acquiring commonsense knowledge and finding ways to incorporate it into the state-of-the-art AI systems. In this talk, I will introduce a brand new commonsense reasoning framework and benchmark that helps solve both problems at scale!
Bio: Nasrin is a senior research scientist at Elemental Cognition, focusing on commonsense reasoning and story understanding. Nasrin received her PhD at the University of Rochester working at the conversational interaction and dialogue research group. Her research focus has been on language understanding and commonsense reasoning in the context of stories, mainly through the lens of events and their causal and temporal relations. She has started various lines of research that push AI toward deeper language understanding with applications ranging from story generation to vision & language. Prior to joining Elemental Cognition, Nasrin held research positions at BenevolentAI, Microsoft, and Google. She has various publications in top AI conferences and has been a keynote speaker, chair, organizer, and program committee member at different AI venues. Nasrin was named to Forbes’ 30 Under 30 in Science 2019.
Title: Why (and how) should we study Question Answering?, slides
Abstract: Recent work has shown exceptional performance on some QA datasets, along with lack of generalization and embarrassing performance in many situations. It seems clear that supporting high level decisions that depend on natural language understanding is still beyond our capabilities, partly since most of these tasks are sparse and our current methodology of generating supervision signals for it does not scale. What should we expect from research on QA then? And how should we think about the role of the various datasets and methodologies developed in this area? I will discuss these issues in the context of some recent results exhibiting the role of QA in improving latent semantic representations, and some positive and negative results on our ability to support some levels of reasoning over text.
Bio: Dan Roth is the Eduardo D. Glandt Distinguished Professor at the Department of Computer and Information Science, University of Pennsylvania, and a Fellow of the AAAS, the ACM, AAAI, and the ACL. In 2017 Roth was awarded the John McCarthy Award, the highest award the AI community gives to mid-career AI researchers. Roth was recognized “for major conceptual and theoretical advances in the modeling of natural language understanding, machine learning, and reasoning.” Roth has published broadly in machine learning, natural language processing, knowledge representation and reasoning, and learning theory, was the Editor-in-Chief of the Journal of Artificial Intelligence Research (JAIR) and a program co-chair of AAAI, ACL and CoNLL. Roth is a co-founder and the chief scientist of NexLP, Inc., a startup that leverages the latest advances in Natural Language Processing (NLP), Cognitive Analytics, and Machine Learning in the legal and compliance domains. Prof. Roth received his B.A Summa cum laude in Mathematics from the Technion, Israel, and his Ph.D. in Computer Science from Harvard University in 1995.
Title: Evaluating and Testing Question Answering Capabilities, slides
Abstract: Current evaluation of question answering primarily consists of measuring the accuracy of identifying the answer of a question from a set of choices (explicitly multi-choice or all possible spans) on held-out QA pairs. Since the held-out pairs are often gathered using similar annotation process as the training data, they include the same biases that act as shortcuts for reasoning, allowing machine learning models to achieve accurate results, without requiring actual comprehension. Thus accuracy is a poor proxy for generalization for reading comprehension tasks.
In this talk, I will introduce a number of approaches we are investigating to perform a more thorough evaluation of QA systems. I will first describe automated techniques for perturbing instances in the dataset that identify loopholes and shortcuts in the question-answering process, including semantic adversaries and universal triggers. I will then describe recent work in creating comprehensive and thorough tests and evaluation benchmarks for question answering, that aim to evaluate reading comprehension capabilities of question answering systems.
Bio: Dr. Sameer Singh is an Assistant Professor of Computer Science at the University of California, Irvine (UCI). He is working primarily on robustness and interpretability of machine learning algorithms, along with models that reason with text and structure for natural language processing. Sameer was a postdoctoral researcher at the University of Washington and received his PhD from the University of Massachusetts, Amherst, during which he also worked at Microsoft Research, Google Research, and Yahoo! Labs. His group has received funding from Allen Institute for AI, NSF, DARPA, Adobe Research, and FICO.
Abstract: ConceptNet is a knowledge graph that aims to represent the basic things about word meanings that people know and computers inherently don’t. The ways in which it is used have changed a lot over its 20-year history. In our present context of powerful neural language models that are used for question answering, text generation, story understanding, and many other tasks, we can still see mistakes that would be improved by computational common sense, and sometimes the knowledge in ConceptNet can help. I’ll discuss how ConceptNet is integrated with various recent machine learning systems, how this relationship is evolving, and some best practices for using it.
Bio: Robyn Speer is the lead maintainer of ConceptNet, the multilingual common sense knowledge graph, and a founder of Luminoso, a Boston startup that implements transfer learning for domain-specific NLP. She is an alumna of MIT. She writes about developments in ConceptNet, related open-source tools, and fairness in machine learning, on her blog at https://blog.conceptnet.io.
Title: Knowledge-aware Machine Reading for Question Answering, slides
Abstract: Recent years have seen a lot of progress in the development of machine reading systems for question answering. In order to correctly answer questions, a system needs to not only understand the question, but also gather relevant information, and synthesize it to produce a coherent answer. In practice, the information that is required for answering questions is often not provided to the system in a well-organized form. Instead, it can be embedded in disparate sources, such as the web, books, domain expertise, and personal experience. How to effectively distill knowledge from these sources and use it for question answering? In this talk, I will discuss machine reading models that are capable of leveraging internal/external knowledge for question answering and describe the remaining challenges of these models for real-world question answering problems.
Bio: Bishan Yang’s interests are in building machine learning algorithms that can extract, organize, and analyze actionable information from large volumes of data. Most recently she was a Postdoc at Carnegie Mellon Machine Learning department working with Tom Mitchell on problems of machine reading and knowledge base construction. Prior to that she obtained her PhD from Cornell University where she was advised by Claire Cardie. In June 2018, together with Igor Labutov, she co-founded LAER AI., a startup based in New York City, focusing on developing next-generation semantic search technologies for enterprises.
