Capturing and sharing visual knowledge

If one knows what something is called, a text-based search on websites such as Google and Wikipedia are tremendously useful. However, what if one only has an image or mental image of that "thing" - how can one find it? Using text-driven searches for identifying images has so far proven very difficult. The ultimate goal for Dr. Serve Belongie of Cornell University is to fill this gap in online-search engines and revolutionize the way we search for and identify specific images. Visipedia, an innovative combination of computer vision, machine learning and crowdsourcing, bridges this gap. His research uncovers novel ways of capturing and sharing visual knowledge. By applying the tools of computer vision and machine learning, he will help people visualize the world around them in ways once thought unimaginable

  • Dr. Belongie's research in computer vision technology draws upon the complementary strengths of humans and machines to solve an array of human needs.
  • More specifically, he trains computers to serve as human visual-assistants for a wide range of applications. For example, he started the "Grocery Shopping Assistant for the Blind" initiative which provides assistive technology for the visually impaired. This technology grants access to textual information located in grocery store settings - thus, comparing two identical cans of soup is no longer a privilege of the sighted!
  • Additional applications of Dr. Belongie's research include an electronic-eye to monitor sensitive coral reefs, video fire-detection systems, and facial recognition programs. At this time, the "Visipedia" project (in collaboration with Caltech) remains the top focus for his personal research.
  • Today, Visipedia enables bird watchers to identify hundreds of species of North American birds. Once a photo of a bird has been submitted to the Visipedia application, a few questions regarding the bird's physical characteristics are proposed to the application user. Once the user provides answers, Visipedia can uncover the bird species and provide background information.

Ultimately, the Visipedia project aims to augment Wikipedia with search-by-image functionality for visual categories including, but not limited to, species of animals and plants, styles of clothing, and architecture. The goal is to enhance our human experience by teaching computers to see. 

Dr. Belongie's background in start-up companies, Digital Persona (a world leader in fingerprint biometrics) and Anchovi Labs (a photo annotation and management company that was recently acquired by Dropbox) helped him understand how to concentrate his research in ways that are commercially viable; however, he also enjoys discovering innovation that benefits education. His research for the Visipedia project has the potential to improve the functionality of a highly used research database (Wikipedia) not only for students, but for millions of people around the world.

Dr. Belongie received a B.S. (with honors) in EE from Caltech in 1995 and a Ph.D. in EECS from Berkeley in 2000. 

While at Berkeley, his research was supported by an NSF Graduate Research Fellowship. From 2001-2013, he was a professor in the Department of Computer Science and Engineering at University of California, San Diego.  

He is currently a professor at Cornell NYC Tech and the Cornell Computer Science Department. His research interests include Computer Vision, Machine Learning, Crowdsourcing and Human-in-the-Loop Computing. 

He is also a co-founder of several companies including Digital Persona, Anchovi Labs (acquired by Dropbox) and Orpix. 

He is a recipient of the NSF CAREER Award, the Alfred P. Sloan Research Fellowship and the MIT Technology Review "Innovators Under 35" Award.

Meet Serge Belongie, a professor of computer science and engineering at the University of California, San Diego. Belongie, 36, sees the world more fully than most

Computer scientists from the University of California, San Diego are developing an algorithm that can determine what subculture you belong to by analyzing photos of you

As scientists work to develop intelligent machines, some are taking their cues from biology

As Technology makes life richer and easier, we leave a trail of information that is susceptible to prying eyes

There's a market for software that recognizes your face and fingerprints, but also increasing fear that Big Brother will be the one staring hard at your eyes and nose.

MIT Technology Review "Innovators Under 35" Award, 2004

Celebrating people who are driving the next generation of technological breakthroughs

NSF Career Award

NSF Selects Two UC San Diego Experts in Computer Vision to Receive Five-Year CAREER Grants