Teaching robots to successfully perform daily tasks and adapt to changing environments

Imagine a surgeon who has watched and learned from millions of surgeries. Now imagine he is able to recall exactly which of those scenarios are most similar to what he is currently experiencing in the operating room and how it will play out. With tele-operated robots like Intuitive Surgical's da Vinci being used for over half a million surgeries each year, such data sets will soon exist.  While building up such extensive experience is not possible within the lifetime of a human surgeon, computers could watch and process videos of all surgeries ever recorded.

The challenge, of course, is for computers to make sense of all this data and put it to good use.   If this were possible, such a system could inform surgeons in the operating room about similar cases, how they were handled, and how that played out for patients in the short and the long run.  Beyond that, it might even be able to directly, robotically control some of the surgical instruments and perform parts of the surgery autonomously.

Such a system would make widely available the most advanced surgical decisions rather than being limited to those patients who happen to have access to the top surgeons. It would also enable surgeries to be performed remotely in areas where medical attention is sparse. Similar technology would enable basic medical functions to be performed without the presence of a doctor. Patients would be able to receive at-home-care with robotic nurses.  If the patient is disabled the robot would be able to fold laundry, help bathe the patient, clean the dishes, and complete a myriad of other necessary chores. While this appears to be next millennium technology, Dr. Pieter Abbeel of University of California, Berkeley, is currently developing the robot learning algorithms that are targeting these kinds of applications.

  • Abbeel's group studies two types of robot learning approaches: In a first line of work they study how robots can learn from watching humans perform the task of interest. In a second line of work they study how robots can learn from trial and error. A key differentiator with traditional robotics is the need for robots in medical and household environments to be able to adapt to ever-changing environments, making hard-coding of robot motions, as is prevalent and successful in manufacturing, not a viable approach.
  •  A key aspect of their work is the underlying representation used during learning: their work allows the robot to take a RGB-D image of a scene and then finds a warping from demonstration scene and motion onto the new environment.  This allows the robot to replicate an appropriately adopted version of the demonstrated action.
  •  His group has enabled robots to reliably pick up a crumpled laundry article and fold it.  His group has also successfully taught the robot to perform activities such as applying sutures and tying knots.
  • His group has also studied application of their algorithms to treatment of prostate cancer and cervical cancer.  Key in those treatments is to deliver targeted radiation therapy. Their advanced motion planning techniques can generate plans for steerable needles as well as 3-D printed channel layouts that enable delivering radiation pellets to the targeted tissue areas.

Whether the robot needs to apply sutures, perform surgical cuts, 3-D printing medical treatment molds, or operate to remove an appendix, Dr. Pieter Abbeel's work has established an expectation that these are medical functions that, with further research and testing, could be suitable for a robot to perform.


Dr. Pieter Abbeel received a B.S./M.S. in Electrical Engineering from KU Leuven, Belgium and received his Ph.D. degree in Computer Science from Stanford University in 2008. He joined the faculty at University of California Berkeley in Fall 2008, with an appointment in the Department of Electrical Engineering and Computer Sciences.

Dr. Abbeel has developed apprenticeship learning algorithms which have enabled advanced helicopter aerobatics, including maneuvers such as tic-tocs, chaos and auto-rotation, which only exceptional human pilots can perform.

Dr. Abbeel's current research focuses on robotics and machine learning with a particular focus on challenges in personal robotics, surgical robotics and connectomics.

In the News

In Search of a Robot More Like Us

Investing in advanced robotic technologies.

Towel-Folding Robot Could Fix Laundry Woes

Folding clothes is the worst part of doing laundry. But if you can be patient enough, a small robot might be able to do the job for you.

Robots May be Part of Future Surgery Staff

New research in the Bay Area is helping machines learn some of the most delicate tasks now performed by humans.