Main Conference – November 17

The spontaneous gestures that people produce when they talk can index cognitive instability and reflect thoughts not yet found in speech. But gesture can go beyond reflecting thought to play a role in changing thought.  I consider whether gesture brings about change because it is itself an action and thus brings action into our mental representations.  I provide evidence for this hypothesis but suggest that it’s not the whole story.  Gesture is a special kind of action––it is representational and thus more abstract than direct action on objects, which may be what allows gesture to play a role in learning.

Susan Goldin-Meadow is the Beardsley Ruml Distinguished Service Professor in the Departments of Psychology and Comparative Human Development at the University of Chicago. Her research focuses on the most basic building blocks of language and thought as they are developed in early childhood. Specifically, she is interested in uncovering linguistic components that are so basic they will arise in a child’s communication system even if the child has limited access to outside linguistic input. Professor Goldin-Meadow’s research has also generated more broadly applicable insights into how the spontaneous gestures that learners produce can reveal their readiness to learn language, math, and scientific concepts. Professor Goldin-Meadow is the founding Editor of Language Learning and Development, and has been president of the International Society for Gesture Studies, president of the Cognitive Development Society, chair of the Cognitive Science Society, and president of the Association for Psychological Society.  She is the recipient of a Guggenheim Fellowship, received the William James award for lifetime achievement in basic research from APS, was elected to the American Academy for Arts and Science in 2005, and the National Academy of Sciences in 2020.  She was awarded the David E. Rumelhart Prize in 2020 for significant contributions to the theoretical foundations of human cognition.

If we want to improve the user’s experience, we need algorithms that can understand their emotions, intents, and interests. Furthermore, to determine how to best help a user, we need systems that can answer hypothetical questions, i.e., what if questions. In short, we need to equip AI systems with a theory of mind. In this talk, I will present a number of projects my research group worked on to address this general goal. Specifically, I will first present the work we have completed on the interpretation of emotion from faces, bodies, and context. I will present a number of medical applications, the interpretation of user’s interactions with a device, and the automatic recognition of sign languages. Following these example applications, I will show how the biomechanics of agents determine whether their actions are performed intentionally or not. And, we will define the first attempt to designing algorithms that can answer hypothetical questions. Throughout, we will derive supervised and unsupervised methods as well as a new approach that allows developers to know whether their deep neural networks are learning to generalize or simply learning to memorize. I will conclude with a discussion of how these efforts are bringing us closer to our goal of designing a theory of mind for AI and how this will improve the user’s experience with the devices of the future.

Aleix M. Martinez is a Sr. Manager of Applied Sciences at Amazon and a Professor in the Department of Electrical and Computer Engineering at The Ohio State University (OSU). Prior to joining Amazon and OSU, he was with the Department of Electrical and Computer Engineering at Purdue University, and a Research Scientist at the Sony Computer Science Lab. Aleix has served as Associate Editor of several journals, including IEEE Transactions on PAMI, been an Area Chair for many top conferences, and, in 2014, was Program Chair of CVPR. Aleix is most known for being the first to define many problems and solutions in face recognition (e.g., recognition under occlusions, expression, imprecise landmark detection), discriminant analysis (e.g., Bayes optimal solutions, subclass-approaches, optimal kernels), structure from motion (e.g., using kernel mappings to better model non-rigid deformations, noise invariance), demonstrating the existence of a much larger set of cross-cultural facial expressions of emotion than previously known (i.e., compound emotion) as well as the transmission of emotion through changes in facial color, and defining a new algebraic topology approach to explaining how deep networks learn to generalize. He has received best paper awards at CVPR and ECCV, a Google Faculty Research award, a Lumely Research Award, and, from 2012-2018, he served as a member of NIH’s Cognition and Perception study section. His research has been featured by the media, including NYTimes, CNN, NPR, Washington Post, Time, The Guardian, Spiegel, El Pais, and Le Monde, among others.