MIT system makes human-like predictions about how objects move through the world

The result of some Galileo experiments. Heat maps are user predictions, orange crosses are the computer system, and white crosses are "ground truth."
The result of some Galileo experiments. Heat maps are user predictions, orange crosses are the computer system, and white crosses are "ground truth."

MIT researchers say they’ve built an artificial intelligence system that’s as good as humans in guessing how physical objects move through the world.

Some day, the scientists say, their research could be used to help build robots that can assist people in complex natural environments.

But for now, they plan to teach the computer model — nicknamed Galileo — about the movement patterns of things like springs and fluids.

The research, from MIT’s Computer Science and Artificial Intelligence Lab, simulates the human brain’s ability to decide whether a given object will roll down a hill, glide to a stop, or smash through an obstruction.

They’re the kind of highly detailed calculations that people make almost moment by moment as they move through the world. For example, a child would easily be able to tell you how fast a basketball will roll down a hill.

But making that same judgment can be much harder for a typical computer simulation because it has to weigh friction, gravity, mass, and a long list of other factors, said Ilker Yildirim, a CSAIL researcher who co-authored a recent paper on the Galileo system.

“Where humans learn to make such judgments intuitively, we essentially had to teach the system each of these properties and how they impact each other collectively,” Yildirim said.

The MIT team used several methods to build the Galileo system. First, the system analyzed 150 videos showing how a bunch of different objects moved and reacted with their environment, including items made of rubber, metal, and cardboard.

Next, the software was given data from Bullet, a “3-D physics engine” that is typically employed in creating video games and movie special effects. After that, the MIT researchers developed “deep learning” algorithms, which can repeatedly analyze a set of data in order to teach a software system to spot patterns.

They tested the system by having it predict, alongside humans, what would happen in a series of videos showing objects sliding down pitched surfaces or colliding with each other. The Galileo predictions got so human-like, the researchers said, that even its incorrect guesses were similar to the wrong answers given by people in the experiment.

You can test out your own physics prediction skills in this online simulation. And don’t worry if you get a few wrong — you’re only human.