Your friend posts pictures of a gruesome scene in some far off country and urges you to act now and help. What you and your friend don't realize is you've both been duped - the photos are fake. Photographs can no longer be trusted. Forged images have appeared in tabloid magazines, mainstream media outlets, political attacks, scientific journals, and the hoaxes that land in our email inboxes. These doctored photographs are appearing with growing frequency and sophistication, and even experts often cannot rely on visual inspection to distinguish authentic images from forgeries.

In addition to the ethical, political, and legal implications raised by this lack of trust in photography, studies have shown that doctored photographs can alter our own memories of actual events. In one such study participants were shown original and doctored photographs of memorable public events at which they were present. The doctored photographs, showing either larger crowds or more violence, changed the way in which participants later recalled the events. This surprising finding is due to a number of factors including our natural acceptance of visual images and our general inability to easily detect doctored photographs.

The need for forensic techniques for exposing photo fakery is, therefore, critical. Professor James O'Brien and his research group at UC Berkeley are working on developing tools for forensic image analysis to expose forgeries in images. These methods focus on finding inconsistencies in the geometric relationships among objects depicted in a photograph - an approach called geometric content analysis. The geometric relationships in the 2D image correspond to the projection of the relations that exist in the 3D scene. If a scene is known to contain a given relationship but the projected relation does not hold in the photograph, then one may conclude that the photograph is not a true projective image of the scene. The goal is to build a set of testable hard constraints that must be satisfied or else the image must be fake. The tests should be robust to common phenomena such as lens distortion, lossy image coding, and minor acceptable photo editing.

• The approach to Geometric Image Analysis divides the task into two separate steps. The first is to identify relevant portions of the images that can be used to specify constraints, and the second is to check those constraints for mutual consistency.

• In general, specifying the constraints requires understanding the image content. Computer vision researchers have made substantial strides in image understanding over the last decade but tasks such as matching shadows to the casting objects remain outside the realm of what can be done automatically. Accordingly, Dr. O'Brien's approach is to let a human analyst annotate the image by selecting appropriate features. Once these have been specified, the computer algorithmically determines if they are mutually consistent.

• There are two key ideas that motivate this approach. First, letting the human do what humans do well: understanding image content, while letting the computer do what it does well: examining large sets of constraints for mutual consistency. In the future the human analyst might be replaced with algorithms for image understanding, but Dr. O'Brien's approach allows a practical solution now.

• Second, they are changing the question of image authenticity from a very subjective and unreliable discussion where one asks, "Do these features look right?", to instead asking, "Has the analyst correctly identified objects in the image?". Both formal studies and informal experience (e.g. visit one of the many "Photoshop or not" quiz websites) show that the first question is difficult to answer objectively and without bias, while the second question is relatively simple. Once an acceptable set of features is agreed upon, the question of proving an image fake is simply a matter of objective constraint satisfaction.

• A potential concern with their approach is how to account for potential error sources. For example the specific part of an object causing a shadow may be unclear, or lens distortion may compromise the assumption of linear perspective, or blurred images may make the selection of a specific image point uncertain. They deal with this by building conservative constraints.  If a feature's location is uncertain, then instead of specifying a single point, an enclosing region is specified around the uncertain location.

• While the approach uses a human analyst to select image features and specify constraints, the algorithms used for constraint satisfaction may be used to assist the analyst in finding useful features. For example, once a few shadow constraints have been specified in an image, the system may display the current solution region and draw a wedge subtending this region as the user scrubs over the image. Additional constraints that would either further constrain the region or that are incompatible with the region can be quickly identified. The published demonstration video illustrates this process.

If a picture is worth a thousand words, is a manipulated picture worth a thousand lies? As an abundance of hardware and software tools is dramatically decreasing the cost and effort required to manipulate digital images, the risks and dangers associated with malicious attackers easily routing doctored images through computer and social networks to purposefully influence opinions, attitudes, and actions have never been more severe. While there's a growing awareness that images no longer represent an authentic proof of reality, people remain easily convinced by what they see.

• An example in politics: A photograph of Senator John Kerry and Jane Fonda sharing the stage at an anti-war rally emerged during the 2004 presidential primaries as Senator Kerry was campaigning for the Democratic nomination. The photograph, however, was a fake. The picture of Senator Kerry was from a speech in June of 1971, and the unrelated picture of Jane Fonda was from August 1972. The two photographs were composited together to give the impression that Senator Kerry shared the controversial anti-war views of activist Jane Fonda.
   
• An example in scientific research: In 2004 Professor Hwang Woo-Suk of Seoul National University and colleagues published what appeared to be ground-breaking advances in stem cell research. Evidence slowly emerged that these results were manipulated and/or fabricated. After months of controversy, Hwang retracted the article and resigned his position at the University. An independent Korean panel investigating the accusations of fraud found that at least nine of the eleven customized stem cell colonies that Hwang had claimed to have made were fictitious. Much of the evidence for those nine colonies, the panel said, involved doctored photographs of two other, authentic, colonies.
   
• An example in the news media: The Economist was criticized when it published, in June 2010, a cover photo showing a solitary President Obama on the Louisiana beach inspecting the BP oil spill. The photo was accompanied with the headline "The damage beyond the spill," alluding to potential political problems facing President Obama as a result of the oil spill. This photograph, however, had been altered to remove two other people standing alongside the President. The altered image conveyed a significantly different perceived emotional tone than the original.

Software for image and video manipulation is becoming increasingly sophisticated and there is a real and growing need for tools to reliably differentiate between authentic and forged media. This work directly addresses this need by developing methods and tools for detecting forgeries. These tools will help reporters, law enforcement, scientists, and others perform their important tasks by restoring confidence in photographic evidence.

As with much of Dr. O'Brien's past work, the intellectual products of this project will be communicated via academic publications, and publicly released online source code. His goal in this work is to develop a suite of tools that will be freely available for use in objectivity testing images and video.