Voice to text translation for the deaf

Discussions around miscellaneous technologies and projects for the general membership.
Church Employee
Church Employee
Posts: 112
Joined: Tue Oct 19, 2010 10:10 am
Location: Utah, USA

Re: Voice to text translation for the deaf

Postby StevePoulsen » Tue Apr 04, 2017 11:11 am

russellhltn wrote:
lajackson wrote:I am amazed at the accuracy with which Google Voice turns my voice into text in emails, notes, and other documents. It is not perfect, and the original voice needs to be clear and accurate.

Perhaps someone can figure out how to wire a hearing impaired receiver to Google Voice - that way the sound is "direct" and as clear as you can get it.

I have heard of members doing this, with good (not perfect) results.

Also, depnding ont he hearing aids you have (they need a T-coil) you can add a neck loop to the receiver so that your aids will pickup the signal from the ASL system.
Steve Poulsen - Meetinghouse Facilities Technology Engineer

New Member
Posts: 5
Joined: Mon Jan 03, 2011 2:59 pm

Re: Voice to text translation for the deaf

Postby JonSevy » Fri Mar 02, 2018 9:56 am

...one dear sister sits beside me in all the meetings and writes quickly brief phrases about what the speaker is saying. She is adept at summarizing. It is a life-saver! I feel so grateful to her! Feel a part of the lessons and talks.

Tweet for deaf AND other translations.

Great discussion. We recently had a couple move in, both totally deaf. Active in Arizona, fell away in their last ward d/t no translation. I pulled out my Android cell phone and started dictating into "Memo". Very accurate, and large enough to read, but they had severely limited comprehension. That puzzled me.

My sister in law is very skilled in ASL. She said signing is Pidgeon: simple, Tweet-like phrases, basic words. Normal grammar is to hard them to follow, and too hard to type or read. So a key seems to be . . . talk like to foreigner who no speak English. Type like Tweet. No points grammar spelling.

Chilean sister, too. New our ward. Need translate Spanish. Me think full-screen lap-top or cell phone app, show summary AND 2nd language, serve both deaf and no - [English] speakers. Slow type OK. Quiet mic OK. Big screen front of room OK.

New Member
Posts: 1
Joined: Tue Jan 24, 2017 3:12 pm

Re: Voice to text translation for the deaf

Postby daveyjones » Thu May 17, 2018 9:16 pm

I'm a software developer currently working on an automated LDS captioning system. I have approval from our stake to test the software in our ward and I'm hoping to have the system up and running in the next few weeks. If it goes well, I'll certainly share with those interested.

There's actually a very good chance that automated captioning could work well in a Sacrament Meeting setting—the sound could be piped directly into the captioning device (e.g. a tablet), so there would be very little noise, and a custom language model could be developed around LDS language.

Speech recognition relies on statistical language models, and those models are developed using a corpus—a body of texts intended to represent normal speech patterns. The problem is, LDS language is quite unique; not only do we have a massive set of unique pronouns in the Book of Mormon, but we also have unique speech patterns (e.g. the words "bear" and "testimony" aren't used together very often in everyday speech, so a generic language model would find it statistically unlikely for those words to be near each other, but an LDS-specific language model would find it very likely). If only there was a massive corpus of LDS language... ;)

Using the thousands of General Conference talks that are freely available, along with the standard works, I believe a very good LDS language model can be developed. The plan is to develop on iOS, so you would plug the house sound directly into the tablet and then hook a tablet up to a TV, which in most chapels could be placed opposite the sacrament table (down low so that only the first couple rows can see the captions, so they don't become a distraction for everyone else).

I'll keep you updated!

Senior Member
Posts: 1651
Joined: Sun Jul 01, 2007 5:06 pm
Location: Plattsmouth, NE

Re: Voice to text translation for the deaf

Postby drepouille » Fri May 18, 2018 5:35 am

It is easy to use the ComTek receivers to inject audio from the house sound system into a laptop/tablet. I tested it, and it does work. However, the free speech recognition web site I was using was too slow, and our DSL connection was too slow, and I was sharing the Wi-Fi with everyone else in the meetinghouse. When you add all those problems to the unique vocabulary you noted, and the fast and rambling speech we often hear, it all makes the task quite daunting.

In reality, people can't read as fast as they can listen. So if your speech-to-text conversion was absolutely perfect, the deaf reader could not possibly keep up. That is why most captions are simplified, and omit a lot of "verbal noise" that is not necessary for the conversion. That is why we just have a real person type a simplified version of what he hears on a laptop, so the deaf person can read and keep up with the gist of the talk or lesson.
Dana Repouille, Plattsmouth, Nebraska

Community Administrator
Posts: 23752
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Voice to text translation for the deaf

Postby russellhltn » Sat May 19, 2018 4:32 am

drepouille wrote:In reality, people can't read as fast as they can listen.

Not quite true. The average person reads about twice as fast as the average speaker. In some cases, three times as fast. However, for some deaf people, their first language is ASL (which like many languages has a different word order), so they are more like English as a second language. It all depends on when they lost their hearing.
Have you searched the Wiki?
Try using a Google search by adding "site:tech.lds.org/wiki" to the search criteria.

Return to “Other Member Technologies”

Who is online

Users browsing this forum: No registered users and 0 guests