Affectiva teams up with developers to make video games that know your feelings

A level in the game Nevermind.
A level in the game Nevermind.

Affectiva, the Waltham tech startup that is making software that can watch your face and track your feelings, is staking its claim to new market: Video games.

A psychological thriller game, “Nevermind,” will be the first to deploy Affectiva’s technology to measure players’ reaction and adjust the level of play accordingly; the more frightened, or more anxious players get, the harder the game becomes to play.

The game from California studio Flying Mollusk, already allows players to use standard-issue heart rate monitors to track their pulse during play, and increase the level of difficulty as their rate increases.

Starting Tuesday, an updated version of “Nevermind” will use a standard webcam to watch the fright on players’ faces as they tackle levels, and respond based on the emotion they display. For example, the game can unexpectedly introduce a stressful challenge, such as trapping players in an threatening environment, forcing them to maintain calm as they play their way to safety.

“It’s a stress management tool disguised as a game,” said Flying Mollusk founder Erin Reynolds. Here’s the twist: the more frightened the player gets, the harder — and scarier — the game becomes.

Reynolds hopes that mastering the game–and your emotions during it– will have practical use. “When you’re in traffic or about to go into a stressful meeting, you can manage [your emotions] in the real world,” she said.

“Gaming has always been on our radar,” said Rana el Kaliouby, co-founder of Affectiva. Games to tend to bring out very strong emotions, she said, so “we decided that as a company we would enter that in a big way.”

Using cameras installed in computers or mobile devices, the company’s software can read a person’s face and tell whether she is happy, sad, anxious or some other emotion. Drawing on a database that analyzed millions of facial expressions, the software recognizes common expressions such as smiling or smirking, but also subtler moods like anxiety.

Advertising and market research companies already license Affectiva’s software, using it, for example, to test consumers’ reactions to advertisements. But other applications are around the corner, the company said: car makers, lawyers, and robotics companies building social robots are testing the software. Gaming will be the company’s next big commercial bet.

To allow broader access to its tech, Affectiva has made a version of the software available on Unity, a game engine used by nearly half of developers in the business. Affectiva also inked a patent for applying emotion-reading algorithms as inputs in video games, that would change characters and game environments based on feelings you display.

“So far we’ve all been using the joystick or the keyboard — and with Kinect, it’s your body movements. Now emotion can be another input in the game dynamic,” el Kaliouby said.

Affectiva worked with  Flying Mollusk to weave in the emotion-sensing capability into the game’s structure. While the concept is still new, Affectiva intends to collaborate with other developers to make games that know your feelings.

Nidhi Subbaraman writes about science and research. Email her at [email protected]
Follow Nidhi on Twitter - Facebook