First-responders gearTeaching robots to pick up oddly shaped objects

The use of robots in military and first response missions is growing. In some of these missions, however, robots need more flexibility and dexterity than is currently available. Researchers offer encouraging news on this front.

When Cornell engineers developed a new type of robot hand that could pick up oddly shaped objects, it presented a challenge: It was easy for a human operator to choose the best place to take hold of an object, but an autonomous robot, like the ones we may someday have helping around the home, office, or in the field, would need a new kind of programming. So the engineers have developed a procedure — an algorithm — that allows a robot to learn grasping skills from experience and apply them in new situations.

A Cornell University release reports that the new method, although inspired by the “universal jamming gripper” created in the lab of Hod Lipson, associate professor of mechanical engineering and computer science, it is hardware agnostic, and will work with any type of robot gripper.

The work by Lipson and Ashutosh Saxena, assistant professor of computer science, a specialist in machine learning, will be presented 16 May at the International Conference on Robotics and Automation in St. Paul, Minnesota. Co-authors of their paper are graduate students Yun Jiang and John Amend.

Lipson’s gripper consists of a flexible bag filled with a granular material. As the bag settles on an object it deforms to fit, then air is sucked out of the bag, causing the granules to pull together and tighten the grip. Previous grasping algorithms have been based on 3-D models of the object and the robot’s gripping mechanism. A robot’s computer brain creates an image of how its hand will look when attached to, say, a cup handle or pencil, and computes the motions needed to arrive in that position. But modeling how a deformable bag shapes around irregular objects is too hard to compute, so the researchers adopted a learning approach.

In a 3-D image of the object, the robot examines a series of rectangles that match the size of the gripper and tests each one on a variety of features. The robot is trained on images of many different objects until it has built a library of features common to good grasping rectangles.

Presented with a new object, it chooses the rectangle with the highest score based on the rules it has discovered. For example, if a rectangle is divided into three strips and the center strip is higher than the other two, that might be a good place to grab.

The robot also considers the overall size and shape of the object to choose a stable grasping point. You don’t want to pick up a heavy, irregular object by one end; but arbitrarily choosing the center may not work either. The “center” of a computer mouse is halfway along the cord.

To test the method the researchers fitted an industrial robot arm with the jamming gripper and a Microsoft Kinect 3-D camera. In trying to pick up twenty-three objects, including tools, toys and dishes, the robot succeeded an average of 90 percent to 100 percent of the time, depending on the type of object. In most cases, the robot was able to successfully grasp new objects that had not been in the training set. Deformable objects like a shoe or a purse were harder, with the robot averaging only 67 percent success.

The release notes that the engineers ran the same tests with a simple “pick it up at the center” directive, scoring only 30 percent to 50 percent, except on flat objects, where both approaches tied at 89 percent.

The algorithm also was tested with the standard parallel jaws most modern robots use, with about the same results, except that the jaws were 100 percent successful in picking up soft, deformable objects. A future robot may need interchangeable hands for different jobs. An extension of their work, the researchers said, might be to include a gripper selection feature.

— Read more in John Amend et al., “A Positive Pressure Universal Gripper Based on the Jamming of Granular Material,” IEEE Transactions on Robotics 28 (3 April 2012): 341-50 (doi: 10.1109/TRO.2011.2171093)