Providing the average individual with tools that train computers to perceive complex visual scenes

Dr. Nuno Vasconcelos and his research team at University of California, San Diego are integrating machine learning with computer vision to provide machines the visual capabilities of a human -- allowing it to make smart inferences based on what they "see." With Dr. Vasconcelos' technology, computers will be able to reach conclusions about photographs and videos. Imagine that you are walking in the woods and stumble upon a mysterious fruit. By simply taking a photo of the red berry you can quickly gain access to vital information such as,"Okay to eat" or "Place back down unless upset bowels desired."

Although significant progress has been achieved in object recognition over the last decades, the "red berry scenario" is only hypothetical at this time. While a "red berry" classifier could be designed with current technology, this design requires significant expertise in computer vision. Hence, it is only cost-effective for a few objects of interest for applications of large commercial interest (such as faces or people). There is very little incentive to design a "red berry" detector, when the majority of the population does not walk the woods. On the other hand, the average person can identify up to 70,000-80,000 objects. Dr. Vasconcelos believes that recognition at this scale, involving many thousands of objects, will only be possible if non-expert individuals become involved in the design process.  

  • Dr. Vasconcelos is currently developing technologies to enable non-computer vision experts to design computer vision algorithms through the use of a downloadable application. Current emphasis is on object detection.

  • How does it work? Two steps: A data collection module first collects images of the desired object class (e.g. fruits) through the web and assembles a collection of representative examples. After which a machine learning algorithm uses these examples to automatically design a detector of the desired objects.

  • Dr. Vasconcelos is redefining the way typical computer vision programming occurs, which currently takes place in a primarily university setting. Dr. Vasconcelos believes that many of the opportunities for advancing computer vision technologies are vastly limited by the lack of training the mass public has in utilizing such technologies. The proposed research aims to make the design of computer vision algorithms accessible to the public, at the technical level of the high-school curriculum. To reiterate, the potential application for these technologies is vast but currently limited by their limited exposure.

Today, most computer vision research and development occurs at universities, which isolates the public from accessing and developing a pioneering technology which could shift the pendulum in the ways in which information is sourced and discovered. Dr. Vasconcelos' application has the opportunity to impact areas such as health-care, entertainment, and robotics. Dr. Vasconcelos states, "The areas of impact are endless; however, we must first roll out an application that properly equips the average individual to successfully navigate the computer vision realm - and this can be done."

Dr. Vasconcelos, Ph.D., is a Professor in the Department of Electrical and Computer Engineering at University of California, San Diego. He received his Ph.D. in media arts and sciences from M.I.T. in 2000. He obtained a bachelor's degree in electrical and computer engineering from the Universidad do Porto, Portugal in 1988. Prior to coming to University of California, San Diego, Vasconcelos was a researcher at Compaq Computer Corporation Cambridge Research Laboratory.

From 1991 to 2000, he was a research assistant at M.I.T.'s Media Laboratory where he worked on such things as image compression, image retrieval, and computer vision. And from 1988 to 1991 he was a researcher at the Instituto do Engenharia de Sistemas e Computadores in Portugal.

He served as a research staff member for the Compaq Computer Corporation at the Cambridge Research Lab in Boston from 2000 to 2002. He also served as a research staff member for Hewlett Packard at the Cambridge Research Lab in Boston from 2002 to 2003.

He was an assistant and associate professor in the department of Electrical and Computer Engineering at University of California, San Diego from 2008 to 2011,  when in 2011 he became a full professor.

Dr. Vasconcelos was named a Hellman Fellow from 2005 to 2006, and was Awarded an NSF Career Award from 2005 to 2010. He is a Senior Member of the Institute of Electrical and Electronics Engineers (IEEE). His students have received several best paper awards from international conferences.

Hellman Fellowship, 2005

The purpose of the Hellman award is to support the research and creative activities of promising assistant professors who show capacity for great distinction in their work.

NSF CAREER Award

NSF Organization: IIS Division of Information & Intelligent Systems