More than meets the eye: Cameras for Capturing Patterns

Using computational photography combined with machine learning to sense and interpret patterns

Computational Photography and Intelligent Cameras comprise a subfield of computer vision that leverages the physics of image formation to create advanced methods of sensing light and analyzing images in order to reveal scene properties. Another term for this field is computational sensing. Cameras are often thought of as simple consumer devices useful for taking a snapshot for viewing. However, a camera can quantitatively sense light and measure scenes in ways that extends human vision. The traditional camera is analogous to the human eye, but computational photography takes computer vision a step further by redesigning the camera in order to capture scene information that the human eye is not capable of seeing. The form and function of the camera is malleable and creative research transforms the camera to a collection of optical, mechanical and electrical components that send, sense, and interpret light.

Dr. Dana and her team innovate with a unique style and have introduced the first texture camera and reflectance sensor of its kind as well as novel material estimation work that models objects not only by color and geometry but also by complex material properties such as iridescence. Research in computational photography and computer vision augments human vision and allows interaction and understanding of our physical world and environment. One of the most exciting aspects of Dr. Dana’s work is its interdisciplinary nature. In her career, she has developed intelligent camera systems for medical diagnostics, civil engineering, microbiology and communications. Computational photography enables us to see in a scientific manner, creating diagnostic imagery, and using light and illumination as a tool of discovery. Dr. Dana envisions applications from her research aiding dermatologists, ecommerce, scientists exploring the microbiome in the future, and even individual’s yoga practice!

Current research in “Seeing Patterns” includes:

  • Reflectance Cameras: Dr. Dana has developed several novel camera systems and is starting a new research program that will realize the next step in camera evolution--the creation of the first material camera or MatCam that outputs a per-pixel label of object material properties that can be used in any visual computing task. This will propel technology toward in-hand devices that can differentiate between a vast number of materials with nearly unlimited applications. The camera will make material recognition a commodity usable by many in the broad scientific and engineering communities, in a similar way that depth cameras are currently used in many fields outside of computer vision.
  • Dermatology: Recent advances in microbiology research has led to the concept of personalized medicine based on the skin microbiome. The ultimate goal is to create dermatology treatments for disorders such as psoriasis and acne. One major challenge is the expense associated with genetic sequencing of bacteria. Dr. Dana’s new work in this area integrates her prior work on visual appearance modeling with microbiome data. Recent studies in dermatology have indicated the role of specific strains of bacteria associated with healthy skin appearance, suggesting the presence of good bacteria. By quantifying appearance in a more precise manner, the relationship between skin appearance and the skin microbiome can be explored.
  • Yoga: One of Dr. Dana’s newest projects involves studying the subtle effects of yoga practice. By capturing the three-dimensional structure of the human body, Dr. Dana believes she can help to encourage yoga participants to continue their practice by being able to see how much progress their body has made even if it is not visible to the naked eye.
  • Camera-Display Messaging: The goal of this project is to enable cameras and electronic displays (or any lights) to communicate. Our modern society has pervasive electronic displays such as billboards, computers, tablets, signage, kiosks and even vehicle lights. The prevalence of these displays provides opportunities to develop computational photographic methods where intentional messages are encoded in the display images that are undetectable to the eye and must be recovered by a camera. While watermarking methods can be used to hide information in images, the photometry of light emitted by a display and observed by the camera is modeled in this work to build messaging and decoding methods that work in real world scenarios.

Bio

Dr. Kristin Dana explains that as a college student she directed her general interest in technology towards robotics. After attending a tour at the NYU robotics lab, she persistently applied for an internship until there was funding for a position for her. This foot in the door propelled her towards rigorous research experiences that led her to an additional internship at  Bell Labs in computer vision and machine perception. After completing her undergraduate degree, Dr. Dana went to MIT where she completed her master’s degree and became more interested in biomedical applications.

After her master’s degree, Dr. Dana entered the job market and worked at a research center in Princeton, New Jersey. After three years there, she realized that by pursuing a Ph.D. she could be driving the questions rather than just trying to answer them. Therefore, she returned to school, at Columbia University where she completed her Ph.D. Her current research is a continuation of her Ph.D. studies.

Aside from research, Dr. Dana enjoys running and maintaining her yoga practice. Despite the winter chill in New Jersey, Dr. Dana still manages to fit in a 6 AM run with a record of running in 5 degree (F) weather. Her active lifestyle also includes keeping up with three children and dabbling in classic guitar.

Website: www.ece.rutgers.edu/~kdana

Awards

Charles Pankow National Award for Engineering Innovation- American Society of Civil Engineering (ASCE), 2014

Robotic Bridge Assessment Tool, lead role in computer vision component

Department Service Award, 2011

Rutgers University Electrical and Computer Engineering

NSF Faculty Early Career (CAREER) Surface Science for Vision and Graphics, 2001

Sarnoff Corporation Team Award, 1995

For algorithms to register infrared and visible

Sarnoff Corporation Technical Achievement Award, 1994

For algorithms applied to breast cancer detection