Exploring Technological Means for Society's Challenges

How technology may help in aspects ranging from politics to speech processing

When you consider that the average presidential campaign costs close to one billion dollars, it is plain to see that money talks when it comes to politics. But what if there were means to reduce politicians' need to spend their time soliciting campaign funds, and instead were able to increase civic engagement? Dr. Nelson Morgan, of the International Computer Science Institute, has recently begun leading a project that hopes to provide just such means. Through technological approaches, he and his colleagues hope to reduce the influence of money on politics. In addition to his newest "Money Talks" project (referred to as Undermining Political "Scratch" Effects with Technology or UPSET), Dr. Morgan continues to retain interest in making speech recognition systems work much better under real world conditions, and the use of neural networks for the design of new features to help this to happen. Overall, these ongoing projects reflect his continued interest in making systems work in the real world, and his continuing work on speech, audio, hearing, and the brain is an important part of this. The potential for each of Dr. Morgan's projects is great. His work with UPSET is likely to increase the ability of elected officials to deal with the problems that confront our country rather than having to focus on funding a successful campaign, while his work with speech recognition will have an impact on increasing understanding of both human and machine mechanisms. By decreasing the limitations currently constraining speech recognition technology, Dr. Morgan will increase the utility of speech recognition systems, which are particularly important for those that need to use hands-free devices, especially those that use them for medical reasons.

One thing that unifies each of Dr. Morgan's projects is the successful collaborations that make it possible to substantially improve working systems. In the UPSET project, he and his colleagues are approaching campaign finance issues through technology rather than via direct political means, thereby creating an opportunity to create a substantial shift to the current system. His other projects also offer something new with the potential of affecting society as a whole. His speech recognition research is focused on gaining a systematic understanding of the relationship between deep neural networks and the signal representations that they interpret, rather than just incrementally improving the technology;  and on building machine algorithms for speech processing from human cortical neurophysiology rather than just from engineering considerations or psychophysical experiments. Therefore, his work will allow scientists to better understand the capabilities of such technologies and how the brain works to process sound in order to help the millions of people that rely on these technologies. As the author of over 200 publications, and three books, Dr. Morgan has proven his ability to conduct innovative and diligent research that has a lasting impact on society; previous innovations have been incorporated in millions of cell phones, for instance. The current generation of "deep neural networks" that are now used for speech recognition by major companies has made significant use of research that he and his colleagues conducted at ICSI. With the addition of his newest project and the continuation of his work within speech recognition, it is no surprise that he relentlessly continues to work for substantial progress, and with an increasing focus on the greater good of his community.

Current research includes:

  • UPSET: As money in politics is a new project, Dr. Morgan has begun compiling a team of advisors that includes George Lakoff, a famous cognitive linguist, and Scott Shenker, one of the most cited researchers in computer science. After building a robust team, Dr. Morgan hopes to develop a portal and associated tools in order to reduce the influence of money on politics using technological means.

  • Brain Function and Speech Recognition: Dr. Morgan hopes to learn more about brain function for audio signal processing both for its own sake and to learn how to provide machine processing of speech with some of the useful properties of human hearing.

  • Signal processing vs. machine learning: Dr. Morgan hopes to systematically chart and ultimately understand the tradeoffs between feature engineering and deep neural network learning for automatic speech recognition (ASR).


Dr. Nelson Morgan is the former Director of the International Computer Science Institute (ICSI), an independent not-for-profit research laboratory that is closely affiliated with UC Berkeley. He also led the Speech Group at ICSI from 1988 to 2012. He is a Professor-in-residence Emeritus in the EECS Department at the University of California, Berkeley, where he received his Ph.D. as an NSF Fellow in 1980. He has been working on problems in signal processing and pattern recognition since 1974, with a primary emphasis on speech processing. He is a former Editor-in-chief of Speech Communication, and has been a member of the IEEE Speech Technical Committee, the IEEE Neural Networks Committee, and the ISCA Advisory Council. He is also a Fellow of the IEEE and of the International Speech Communication Association (ISCA). In 1997, he received the Signal Processing Magazine best paper award. He was the Principal Investigator for the multi-site coalition, funded by the DARPA EARS Novel Approaches project, which was the 2002-5 US government program focusing on long-term progress in speech recognition.

Dr. Morgan was a naturally curious child. He has always gotten a lot of satisfaction out of making tangible progress on problems. For much of his career, his reaction to these motivations has been to personally dive into the problem. But increasingly, he learned how his ideas could be leveraged by collaborating with others. His most productive efforts have therefore been with the help of others, leading to his most recent collaborations that have the promise of being his most successful yet! Aside from his rigorous research, Dr. Morgan also enjoys writing parodies that connect technical advances to mainstream culture. In fact, his most recent recording is a parody from the Frozen soundtrack (you can listen to it here) that was written to illustrate the UPSET project's goals.

Website: http://www.icsi.berkeley.edu/~morgan/

In the News

Nelson Morgan

ICSI Berkeley


Deep vs. Wide: Depth on a Budget for Robust Speech Recognition


Hearing is Believing: Biologically-Inspired Feature Extraction for Robust Automatic Speech Recognition


Deep and Wide: Multiple Layers in Automatic Speech Recognition


Speech and Audio Signal Processing: Processing and Perception of Speech and Music


Continuous Speech Recognition: An Introduction to the Hybrid HMM/Connectionist Approach


RASTA Processing of Speech



Signal Processing Magazine Award for May 1995 paper, 1997

with H. Bourlard

Fellow of the IEEE

Fellow of International Speech Communication Association (ISCA)

Plenary speaker ICASSP, 2012

Plenary speaker ASRU, 2013

Member of Phi Beta Kappa, Eta Kappa Nu, and Tau Beta Pi

Member of Editorial Board for Speech Communication (former co-Editor-in-Chief)

Member of Scientific Advisory Board for IDIAP (Switzerland)

Member, Board of Trustees for the Toyota Technological Institute in Chicago


U.S. Patent No. 6,308,155: "Feature extraction for automatic speech recognition"

B. E. D. Kingsbury, S. Greenberg, and N. H. Morgan. Oct. 2001.

U.S. Patent No. 5,537,647: "Noise resistant auditory model for parametrization of speech"

H. Hermansky and N. H. Morgan. July 1996.

U.S. Patent No. 5,450,522: "Auditory model for parametrization of speech,"

H. Hermansky, N. H. Morgan, and P. D. Kohn. Sep. 1995.

U.S. Patent No. 5,020,538: "Low noise magnetoencephalogram system and method"

N. H. Morgan and A. S. Gevins. June 1991.

U.S. Patent 4,736,751: "Brain wave source network location scanning method and system"

A. S. Gevins, N. H. Morgan, and D. S. Greer. April 1988.