Imagine you are a musician, improvising music — while your computer is listening and suggesting harmonic accompaniments to complete your music-making. Or a hiker, using her smartphone to identify a bird species by its song. The development of robust listening machines has many potential applications that can have transformational impact in music, biology, urban studies and other areas of scholarly and professional practice. Dr. Juan Bello, Associate Professor of Music Technology and Electrical and Computer Engineering at New York University, studies the fundamentals of sound and develops cutting-edge computational methods in the arising field of machine listening. By examining the patterns of information in music and environmental sounds, Dr. Bello hopes to create systems that can intelligently interact with humans.

Further advancements in machine listening research will enhance the ability to automatically recognize the sources in an auditory scene, determine the pitch, rhythmic, structural, and emotional content in recorded music, and help characterize the similarities that exist between sounds or musical pieces. Dr. Bello’s research sits at the intersection of the fields of music, acoustics, electronic engineering, and computer science, while seeking to advance the state of the art in all of them. A fundamental step towards context awareness in machines, new machine listening technologies can enable novel applications in fields as diverse as robotics, human-computer interaction, hearing aids, and security.

Current research projects include:

• Foundations of Machine Listening: One of the key components of Dr. Bello’s research is how to improve existing audio representations by exploring the trade-off between feature design and feature learning strategies. Data-driven and deep feature learning methods have shown tremendous potential in a variety of domains including audio. Despite their versatility, however, the success of these methods still depends on a thorough adaptation to the specificities of the domain. This requires new paradigms for feature design that differ from how audio features are traditionally generated, but still benefit from knowledge of signal processing, acoustics, sound perception, and music, extending to the way data is collected, annotated and used. Dr. Bello’s lab is currently applying these insights to develop novel and robust solutions to a range of important machine listening problems.
• Large Scale Music Analysis and Interaction: Dr. Bello seeks richer descriptions of music content that go beyond standard methods of organizing music in terms of genre, artist, album, sales or play counts. His team is developing techniques for analyzing large collections of music recordings and automatically obtaining chord sequences, melodic lines, sections (e.g. chorus, verse, bridge), bar boundaries, repetitive motifs and instrument identities over time. They are also exploring computer applications to the retrieval of cover songs, identifying similarities between music from different traditions, and interactively accompanying solo performers.
• Understanding Sound Environments: Dr. Bello’s work holds the potential for transformational advances in the way we monitor and interact with the environment around us. His team’s machine listening solutions are soon to be deployed as part of intelligent sensing networks in both urban and natural environments to, for example, identify sources of noise pollution and their emission patterns across time and space, or characterize the composition of bird species in a migratory path. These applications can be used to empower city agencies tasked with monitoring noise pollution and enforcing its regulation, and thus have a direct impact on the quality of life of urban citizens. It can also advance research in fields such as ornithology, marine biology and entomology in understanding ecosystems and how they are affected by human and other factors.