Lee Miller Vividly Recalls The Day in 2021 when he met a woman who had lost the function of her vocal cords. In Horese, Whispering Tones, She explained how her voice has been instrumental to her vocation. Losing it, she said, undercut her life’s purpose. Her Words Were Fant, but the Lesson was powerful.

“Our voice is so important to our sense of identity and empowerment,” said Miller, A Professor of Neurobiology, Physiology and Behavior in the University of California, Davisalnia of California of Biological Sciences, A Professor of otolaryngology and Head and Neck Surgery at the UC Davis School of Medicine and Technical Director of the Center for Mind and Brain.

Now, Miller is Working to Restore Original Voices to those who have lost them- Based partly on adapting Technology for Interpreting GESTIONS and Controlling Robotic Limbs.

Every year, Nearly 1 Million People Worldwide Are Diagnosed with Head and Neck Cancer. Many of them lose their ability to spendak intelligibly due to surgical removal of -or radiation damage to -s Larynx, Mouth and Tongue. These people can learn to spendak against using devices that emit artificial sounds, which they can shape into words. But their new voices are often weak, mechanical or distressingly unfamiliar.

Miller and his collaborators are developing a system that could one day restore a person’s unique, Original Voice.

Decoding the mind

Miller is working on a project to record electromyographic (EMG) Signals on a Person’s Skin, Generated by the Muscle Contractions Created by Reaching Out OR CLANCHING A FISTO Digital instruments that can be used to control a robotic Arm. Doing this might one day allows Astronauts to Repair Equipment Outside A Space Station without Undertaking A Potanly Risky Spacewalk.

In a 2024 study published in the Journal of Neural EngineeringMiller Worked With the Company Meta to Use emg signals to recognize and interpreet a person’s gestures So they can interact with computers using natural body language – Carther than a mouse and keyboard.

The deficulty is that emg signals often vary from person to person, depending on their age, skin characteristics, body weight and other factors. These biological Signals also produce mountains of data per second – which computers will need to be able to be able to get process.

“We have only a limited Amount of time,” said Miller, “Perhaps only 50 milliseconds, before the computer causes a delay, which would make make real-time interpretations is impossible.”

Miller and Graduate Student Harsha Gowda (in the Electrical and Computer Engineering Graduate Group), Solved this by using only tiny bits of the ignals who iGNOing EVYYTING Elseing Else. Rather than tracing the chaotic ups and downs of each emg electrode on a person’s arm, gowda employed a strategy that simply measuresures signal reational relations

These simplified Signal representations turns out to be “very well-behaved,” and don’t vary from one person to another, Miller Said. “So now we have a gesture decoder that works for everybody.”

Restoring Voice

Miller Became Inteced in Applying these Lessons to speech during a visit in 2021 with Peter Belafsky, Professor of otolaryngology at the school of medicine and director of the uc Davis Center For Voice and Swallowing. It was at belafsky’s clinic that he met the woman whose voice had ben part of her vocation and others who had lost their Voice. Hearing their stories “was profoundly motivating,” said Miller.

Miller Embarked on the Silent Speech Project in 2022, Collaborating With Belafsky, Sergey Stive, Assistant Professor of Neurological Surgery, and David Brandman, A Professor of NeurOSURGERGERI ATHE School of Medicine.

Miller and Gowda Began the Project by Working With Healthy Volunteers, Using Emg Electrodes to record the movements of their mouth and face Muscles during speech. Then, they used the simplified emg signals with simultaneously recorded speech to train a computer to match different emg patterns with different speech sounds for each percent. The result is tailored, computer-generated speech that is created using the unique tones of the person’s voice.

“We don’t need that much data to clone the person’s voice,” said Miller. In his experience, it requires only about five minutes of speech combined with that person’s emg signals.

Miller and Colleagues are now trying to use this system to restore the Voice of people who no longer have functional larynxes. For there individuals, it is no longer possible to record natural voices, so miller and his colleagues try to stitch togethr meaning meaningful Samples from Other Sources like Family videos. One Patient Had Recorded An Audio Diary to Capture a record of his voice in the weeks before his larynx was surgically removed.

“It was a very personal choice that he made, preserving a memento of his voice that he knew he was about to lose,” said Miller. “It was very special that he shared those recording with us.” They turned out to be a perfect trove of raw material for digitally recreating his voice.

Miller’s Team is now Pairing theoses with emg and video of the man’s face, which they recorded as he spoke the same words Silently, without his larynx. Here is An example of restored “Silent speech” From an individual who no longer has a voice due to laryngectomy.

Miller envisions that this system might one day run on a smartphone. The Person would move their mouth to spendak Silently Into their phone as thought a video call. The phone would simultaneously record emg signs and video of their face –combining these with a sample of the person’s voice to create natural-following speech.

Engineering this system so that it works outside of the laboratory for a wide array of people could take take take seveal years, Said Miller. Even so, “Ultimately, we want this to work easily for anybody.”