Natural and physical sounds are all around us: a piece of paper rubbing against the table or a glass of water being poured, the distant sound of a lawn mower or a jet plane tearing through the sky. Sound provides important auditory cues for perceiving and understanding our environment, the motions and interactions of objects, and their physical compositions. Dr. Doug James, Professor of Computer Science at Stanford University, develops computer simulations that can synthesize realistic sounds with complex physics-based animations such as splashing fluids or fracturing solids. Despite the importance of sound, most sounds generated in computer graphics and virtual environments unfortunately lack synchronization and predictive realism, and are usually pre-recorded “canned sounds” that do not match the underlying physical process. To this end, Dr. James hopes to make fundamental advances in physics-based audiovisual simulation so that humans can both see and hear simulated virtual models.

While realistic computer-animated imagery has revolutionized engineering insight and practice, the lack of realistic auditory feedback is a major impediment to the serious use of virtual reality for scientific and engineering applications. By enabling computer-simulated physical objects to make sound in virtual environments, Dr. James and his team will be able to synthesize sound and imagery all at once and improve the creative process in many areas, including architectural design and development, virtual prototyping in engineering design, models for acoustic remote sensing, and computational robotics. For example, we hope to devise methods which can improve future robots’ perceptual models of sound around them, to build computational models of mechanical processes that can estimate and reduce the noise of industrial machinery, to support more realistic training in immersive virtual environments, and to provide future animation directors with immediate auditory feedback instead of waiting for sound post-production.

Current projects include:

• Water: Dr. James and his team hope to compute highly realistic visual and auditory renderings of water. Currently, there are no methods for computing rich all-frequency sounds due to acoustic bubbles of widely varying shapes and sizes, from millimeter to decimeter scale, e.g., such as when a wave breaks on the shore. By properly resolving multi-scale bubbles and their associated vibrations and radiation, Dr. James plans to develop sound-enabled simulations of water (and other liquids) that can deliver rich multi-sensory experiences.
• Many-body Contact Sounds: Many current sound models for rigid bodies are fast but deficient in several ways. They are unable to reproduce typical contact sounds in laboratory experiments, such as the chattering produced by a spoon dropped into a cereal bowl, let alone be able to compute validated sound for animated mechanisms such as a robot or a whirring mechanical assembly. Dr. James hopes to develop parallel algorithms to efficiently resolve complex contact events, chattering, and inter-object coupling effects. In addition, many-body sound radiation is important for complex many-part assemblies, and must be addressed to capture the proper shadowing, scattering and diffraction effects which make many-body systems, such as a mechanical assembly, sound the way they do.
• Destructive Processes (i.e. Fracture): In the first physics-based sound model for brittle fracture, Dr. James showed how to synthesize familiar sounds of breaking wine glasses and piggy banks. However, significant challenges remain for synthesizing sounds of virtual destruction. For example, large deformation and plastic phenomena, such as twisted metal during a car crash or tearing paper, require improved time-domain methods to avoid the limitations of modal sound models, and high space-time costs of brute-force time-stepping approaches. Furthermore, simulating sound for objects very large relative to sound wavelengths, such as a collapsing architectural structure or calving glacier, are particularly difficult due to challenges of mid-frequency acoustics and vibration analysis. Dr. James considers practical methods to render these problems.