Emilie Rusch

Tom Williams wins NSF CAREER Award to improve robotic communication skills

What if robots had working memory? Williams aims to find out.
Tom Williams looks at Pepper the robot

Tom Williams, assistant professor of computer science at Colorado School of Mines, has received a National Science Foundation CAREER Award for work aimed at improving the ability of robots to communicate in humanlike ways — inspired by how the actual human mind processes information.

Cognitive psychology is the field of scientific study that explores the operation of human mental processes related to perceiving, thinking, language, memory and more.

Williams will receive $549,999 over five years for his project, “Cognitively-Informed Memory Models for Language-Capable Robots,” which will apply lessons from cognitive psychology in hopes of developing working memory processes in robots.

“Working memory refers to the small amount of information that humans are able to keep on hand and manipulate in the context of cognitive processing,” Williams said.

Here, he answers a few questions about his research and why the constrained size of working memory is actually a good thing, for humans and potentially robots.

Q: What is your latest research focused on?

A: Our lab’s research applies theories, methods, and techniques from fields of study like psychology to understand how we can and should design robots that communicate with users in humanlike ways, such as through spoken language and gesture. In our new work, we are studying how theories from cognitive psychology surrounding working memory can be used to improve robots’ language generation capabilities.

Working memory is really fascinating because it’s an extremely resource-limited system, which would seem like a limitation. But its constrained size is actually a blessing in disguise because it means that high-level cognitive processes only need to consider the very small amount of information in working memory rather than constantly searching through all of long-term memory.

So, during conversation, for example, if I’m trying to decide how to refer to something, like a particular location, instead of carefully considering all of the different properties that location has and all the different words that could be used to describe it, I might just rely on the properties I currently have associated with that location in Working Memory (if any). If these properties are in Working Memory because they’re the properties you just used to describe the location to me, then by relying on them I’ll coincidentally be generating a description that is easy for you to understand. And because properties don’t tend to stick around for very long in Working Memory, I can be pretty confident that the information held in working memory isn’t out of date. These benefits, however, can only be realized due to the effectiveness of the human mind’s mechanisms for ensuring that the information stored in working memory is actually relevant and useful.

In this project, we’re studying how Working Memory systems can be developed for interactive robots, allowing robots to keep small amounts of relevant and useful information on-hand in order to improve those robots’ cognitive processes.

Q: What do you find most exciting about your research?

A: This research is particularly exciting from a cognitive systems perspective because there are so many different angles to consider. Developing robotic models of working memory requires answering a number of interesting design questions, such as how robots’ working memory buffers are structured, how they’re distributed throughout a robot’s cognitive system, how they’re coordinated, what their size limitations are, how information is “forgotten” out of Working Memory, how the robot decides what is relevant and useful, and so forth.

But what’s really interesting is that the answers to these questions will be different depending on your research goal. If your goal is to perfectly model human cognition, your answers to these questions might be different from if your goal is to optimize robots’ computational efficiency. So part of what we’re looking at is in fact the space of possible designs, and which designs are best suited for different types of scientific and engineering objectives.

Q: What is the potential impact of this work?

A: By improving general purpose robotic cognitive reasoning mechanisms, we stand to improve the capabilities of intelligent robots in any domain where robots will need to communicate and make decisions. This includes domains like eldercare, education, space robotics, and search and rescue robotics. However, this project also stands to feed insights back into the field of psychology by providing a new depth of understanding of these different algorithmic choices from a cognitive modeling perspective.

Q: How does this research agenda inform your teaching and outreach activities?

A: The research we’ll be conducting in this project falls into the area of cognitive systems, an area at the intersection of computer science and psychology that is surprisingly small, despite being very closely aligned with the traditional goals of the field of Artificial Intelligence. To help train the next generation of cognitive systems researchers, we’ll be running a series of workshops with other cognitive systems researchers from around the country to develop curricular modules that can be integrated into graduate and undergraduate computer science and psychology classrooms, as well as into outreach activities with K-12 students. We’ll also be organizing public outreach events to share the research of Colorado cognitive science and robotics researchers with our local communities.

Emilie Rusch

Emilie Rusch

Director of Communications
About Mines
Colorado School of Mines is a public R1 research university focused on applied science and engineering, producing the talent, knowledge and innovations to serve industry and benefit society – all to create a more prosperous future.