Amazon’s Mass. challenge: How hard is it to pick up a box of crayons?

WPI is developing robots for an Amazon-sponsored competition. Their goal is to design robots that can pick products off of shelves and put them into boxes. Photo Joanne Rathe/Globe Staff
WPI is developing robots for an Amazon-sponsored competition. Their goal is to design robots that can pick products off of shelves and put them into boxes. Photo Joanne Rathe/Globe Staff

I never knew just how hard it could be to pluck a box of crayons from a shelf and drop it into a bin.

On Tuesday, I visited two teams participating in a competition organized by’s warehouse robotics division, based in North Reading. The teams have been designing hardware and writing software to take a fictional customer order, find the items on a shelving unit, grab them, and put them into a box without damaging them. The best bot will win $20,000 this month at the inaugural Amazon Picking Challenge in Seattle.

Let me put it this way: If you entered a 5-year-old who could read the order list, she’d be stiff competition for the robots.

The Amazon challenge is a big deal in the “small world of robotics research,” says Brian Benoit, a senior manager at Boston-based Rethink Robotics, which is supplying dual-armed manufacturing robots to several of the 31 teams competing in the challenge.

The professors I visited at MIT and WPI see the challenge the same way a pole vaulter sees the Olympics — it provides a focal point for their work. Alberto Rodriguez, an assistant professor at MIT, calls it an Amazon-sponsored “brainstorming session” to get lots of smart people thinking about how to build more capable robots.

But it’s also a not-so-veiled threat to the thousands of hourly workers who do these tasks in Amazon warehouses around the world. In April, Amazon said it had about 50,000 employees working in what it calls “fulfillment centers”— but that figure excludes the 80,000 seasonal workers who last year filled orders in the run-up to the holidays. Amazon pays full-time warehouse workers about $26,000 a year, according to the career site Glassdoor.

A Massachusetts robotics company, Kiva Systems, helped Amazon introduce the first significant wave of robots to its warehouses. Amazon paid $775 million to acquire Kiva in 2012, and in December, the company revealed that 15,000 of Kiva’s bright orange bots were already on the job.

But Kiva’s robots, which look like rolling ottomans, are designed to fetch a tall metal rack full of merchandise and move it to a “pick station,” where a person is responsible for grabbing the right items and putting them into a box. The picking challenge seeks to automate that second stage of filling an order.

BetaBoston Player. Player build from player.

And it ain’t easy — even if you have a crew of doctoral students and top-of-the-line industrial robots to work with. Amazon tried to make things simple for the robots. For example, there are only 24 possible items that the robots need to pick up, from bags of cat treats to a box holding a single spark plug.

In Rodriguez’s Manipulation Lab on the MIT campus, a one-armed robot typically used for painting and welding is hunting for a box of Crayolas. But instead of grasping it with its two-fingered hand, it pushes the box further away, and comes up empty. MIT’s robot uses three cameras to see the world, and Rodriguez explains that some of the objects — like a wire mesh pencil cup — are incredibly difficult for it to perceive.

WPI’s entrant in the challenge is a two-armed bot made by Motoman of Ohio. I watched it, too, fail to grab some Crayolas, due to an error that caused the robot to lose track of where its arm was in space. Its long fingers opened and closed, and while they held nothing, the bot dropped the invisible object into the bin.

It’s always hard to predict how quickly technology will advance, says Dmitry Berenson, an assistant professor at WPI, but “we’re a long way off from where the robot can do real order fulfillment” at a reasonable speed.

Amazon has always had the drive to dominate retail. It’s no surprise the company is pursuing a dual-path strategy: build the most efficient warehouses today, and staff them with people, while supporting the development of robots that might one day do these jobs faster, with fewer errors, and without lunch breaks.

“Just look at the tasks these people are doing,” says Frank Tobe, editor of the trade publication The Robot Report. “A laser points to the item and a red light blinks to show you which item to pick, then you put it into the box. What you see is a person acting as a robot.”

Tobe suggests — and I agree — that those workers would be wise to take advantage of Amazon’s 95 percent tuition reimbursement to advance their education and skills. “Those low-skilled jobs are going to be falling off the grid,” Tobe predicts. The unanswered question is whether Massachusetts has a dual-path strategy of its own.

State and local governments gave Amazon almost $15 million in tax breaks to build a 1-million-square-foot fulfillment center in Fall River and Freetown. It’s set to open next year. Amazon guaranteed it would create 500 jobs that will last a minimum of five years.

Does that suggest optimism about how much robots will improve by 2021? Amazon didn’t respond to calls or e-mails requesting comment.

Amazon is putting up about $300,000 to run its high-profile robotics challenge. I’d like to see the state and industry groups launch similar initiatives to foster more robotics action in Massachusetts. We need to make sure that, by 2021, we have lots of $126,000-a-year jobs designing robots and writing software for them — not the $26,000 warehouse jobs.

Because robots will eventually get good at grabbing the Crayolas.

Scott Kirsner writes the Innovation Economy column every Sunday in the Boston Globe, in which he tracks entrepreneurship, investment, and big company activities around New England.
Follow Scott on Twitter - Facebook - Google+