Voice to text translation for the deaf

Discussions around miscellaneous technologies and projects for the general membership.
GGTTLJB
New Member
Posts: 1
Joined: Mon Aug 27, 2018 4:25 pm

Re: Voice to text translation for the deaf

Postby GGTTLJB » Mon Aug 27, 2018 4:30 pm

hi my name is Terry and I just read your article about you needing some kind of program for transcribing typist I didn't quite understand it but maybe this'll help you out Google has a free program that's used for translating two different languages from English to to any language you want but what it does is as you speak it puts the words down in print. maybe somehow this could help you or put you in the right direction or I don't know but I thought I'd give it a shot good luck to you and I hope it helps have a great day Terri Holmes Sacramento California User I'd: GGTTLJB. END

User avatar
lfanelli
New Member
Posts: 4
Joined: Fri Jan 13, 2017 6:50 pm
Location: Fredericksburg, VA, USA

Re: Voice to text translation for the deaf

Postby lfanelli » Mon Oct 15, 2018 8:24 am

Creating a Speech to Text ‘Server’ on MacOS 10.12

All credit goes to @daveyjones for his code and help. I just made the tutorial. I use the Focusrite Scarlett 2i2 USB Audio Interface for the connection from the Aux line out of the chapel to the MacBook Pro.

SERVER-
Get an IBM Dev Account - Standard – https://console.bluemix.net/catalog/ser ... ch-to-text
In Terminal install Homebrew -

Code: Select all

 usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install node and npm2 via homebrew -

Code: Select all

$ brew install node

Install the ‘mic module’ on MacOS using

Code: Select all

brew install sox
or Linux

Code: Select all

 brew install arecord


Using the following directory scheme, create the following:

/usr/local/bin/
|----public
|----|+-- index.html
|----|+-- server.js
|----|+-- package.json
|----|---- scripts
|----|----+-- main.js
|----|---- styles
|----|----+-- main.css


Create the text file named ‘Server.js’
[Note: you will need to use YOUR username and password from IBM’s dev center below]

Code: Select all

var app = require("express")();
var events = require("events");
var server = require("http").Server(app);
var mic = require("mic");
var os = require("os");
var io = require("socket.io")(server);
var watson = require("watson-developer-cloud/speech-to-text/v1");
var wav = require("wav");
 
events.EventEmitter.prototype._maxListeners = 100;
 
//====================================================================================================
// Watson
//====================================================================================================
 
var speechToText = new watson({
           url: "https://stream.watsonplatform.net/speech-to-text/api",
           username: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx",
           password: "xxxxxxxx"
});
 
var micInstance = mic({rate: "44100", channels: "1", debug: false});
var micInputStream = micInstance.getAudioStream();
var wavStream = new wav.Writer({sampleRate: 44100, channels: 1});
 
var recognizeStream = speechToText.recognizeUsingWebSocket({
           content_type: "audio/wav",
           //customization_id: "##########",
           interim_results: true,
           objectMode: true,
           inactivity_timeout: 3600
});
 
 
var cleanTranscript = (data) => {
           transcript = data.results[0].alternatives[0].transcript;
           // Remove hesitation indicators
           transcript = transcript.replace(/\s*\%HESITATION\s*/g, " ");
           // Remove supposed profanity
           transcript = transcript.replace(/\*/g, "");
           // Remove extra spaces
           transcript = transcript.replace(/ +/g, " ").trim();
           // Return the cleaned transcript
           return transcript;
}
 
micInputStream.pipe(wavStream);
wavStream.pipe(recognizeStream);
micInstance.start();
 
//====================================================================================================
// Server
//====================================================================================================
 
app.get("/", function (req, res) {
           res.sendFile(__dirname + "/public/index.html");
});
 
app.get("/scripts/main.js", function (req, res) {
           res.sendFile(__dirname + "/public/scripts/main.js");
});
 
app.get("/styles/main.css", function (req, res) {
           res.sendFile(__dirname + "/public/styles/main.css");
});
 
io.on("connection", function (socket) {
           recognizeStream.on("data", function(data) {
                       var transcript = cleanTranscript(data);
                       if (transcript.length > 0) {
                                   socket.emit("data", {
                                               final: data.results[0].final,
                                               transcript: cleanTranscript(data)
                                   });
                       }
           });
});
 
server.listen(8000);


Create the text file called package.json


Code: Select all

{  "name": "local-server",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.16.3",
    "mic": "^2.1.2",
    "socket.io": "^2.1.1",
    "watson-developer-cloud": "^3.5.0",
    "wav": "^1.0.2"
  }
}


Create file called index.html

Code: Select all

<!DOCTYPE html>
<html>
<head>
           <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
           <meta name="apple-mobile-web-app-capable" content="yes">
           <title>LDS Captions</title>
           <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Libre+Baskerville">
           <link rel="stylesheet" href="/styles/main.css">
           <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.1.1/socket.io.js"></script>
           <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
           <script src="/scripts/main.js?v=2"></script>
</head>
<body>
           <div id="content">
                       <span id="t0"></span>
           </div>
</body>
</html>


Create a folder ‘scripts’ and text file called main.js
[Note: change ‘localhost’ with the IP of the server]

Code: Select all

var socket = io("http://localhost:8000");
var i = 0;
socket.on("data", function (data) {
           $("#t" + i).text(data.transcript);
           if (data.final) {
                       var words = $("#t" + i).text().split(" ");
                       if (words.length < 2) {
                                   $("#t" + i).prev(".dot").remove();
                                   $("#t" + i).remove();
                       }
                       i++;
                       $("#content").append(` <span class="dot">&bull;</span> <span id="t${i}"></span>`);
           }
});
 
setInterval(function() {
           $("#content").scrollTop($("#content").scrollTop() + 1);
}, 10);


Create a folder 'styles’ and text file called main.css

Code: Select all

body {
           background: black;
}
 
#content {
           position: fixed;
           top: 48px;
           right: 48px;
           bottom: 48px;
           left: 48px;
           line-height: 1.75em;
           font-family: "Libre Baskerville";
           font-size: 48px;
           color: white;
           overflow: hidden;
}
 
.dot {
           font-size: 48px;
           opacity: 0.4;
}
 
.dot:first-child {
           display: none;
}


Now to change the default location of apache’s Document Root Directory location from /Library/WebServer/Documents to our new location of /usr/local/bin/public

Open Terminal and type

Code: Select all

sudo nano /etc/apache2/httpd.conf
and press enter.
Press Ctrl+W which will bring up a search.
Search for /Library/WebServer/Documents and press enter.
Change the two (2) instances of /Library/WebServer/Documents to /usr/local/bin/public
Press Ctrl+O followed by Enter to save the change you just made.
Press Ctrl+X to exit nano.

Type

Code: Select all

sudo apachectl restart
and press enter.

Type

Code: Select all

cd /usr/local/bin/public/
and press enter.

Type

Code: Select all

npm install
and press enter. [This installs all dependencies for the server.js]

Let’s test by running node server.js
Type

Code: Select all

node server.js
and press enter.

On the server go to http://localhost (Note: you may need to enter the full IP of the machine.)
You should see a black screen and if you talk, you should get speech to text!

If all works out, lets run the server.js in pm2 so if it crashes the pm2 app will restart it automatically and keep it up and running.
Type

Code: Select all

pm2 start server.js
and press enter.


CLIENT-
Open a web browser and go to the http://IPofServer:8000

rmrichesjr
Community Moderators
Posts: 1882
Joined: Thu Jan 25, 2007 11:32 am
Location: Dundee, Oregon

Re: Voice to text translation for the deaf

Postby rmrichesjr » Tue Oct 16, 2018 5:39 pm

lfanelli wrote:Creating a Speech to Text ‘Server’ on MacOS 10.12

All credit goes to @daveyjones for his code and help. I just made the tutorial. I use the Focusrite Scarlett 2i2 USB Audio Interface for the connection from the Aux line out of the chapel to the MacBook Pro.

SERVER-
Get an IBM Dev Account - Standard – https://console.bluemix.net/catalog/ser ... ch-to-text
In Terminal install Homebrew -
...


A brief discussion among moderators resulted in approval of the above post. During that discussion a few concerns were expressed:

    Caution is advisable regarding sending Church meeting audio to third-party computing infrastructure.

    The moderators cannot guarantee the safety of the posted code.

    Moderators do not guarantee whether the use of an IBM developer account for this purpose or the use of the Watson resource are within the permissions for such an account.

laengland
New Member
Posts: 1
Joined: Tue Sep 05, 2017 12:49 pm

Re: Voice to text translation for the deaf

Postby laengland » Mon Oct 29, 2018 9:50 am

Don't count on our church to provide any help for the hearing impaired.

russellhltn
Community Administrator
Posts: 26077
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Voice to text translation for the deaf

Postby russellhltn » Mon Oct 29, 2018 12:40 pm

laengland wrote:Don't count on our church to provide any help for the hearing impaired.

They might not help as much as you want, but considering that a transmitter is part of every stock installation for a chapel, they commitment is fairly significant.
Have you searched the Help Center? Try doing a Google search and adding "site:churchofjesuschrist.org/help" to the search criteria.

So we can better help you, please edit your Profile to include your general location.

rmrichesjr
Community Moderators
Posts: 1882
Joined: Thu Jan 25, 2007 11:32 am
Location: Dundee, Oregon

Re: Voice to text translation for the deaf

Postby rmrichesjr » Mon Oct 29, 2018 7:57 pm

Also, along with the cost of the transmitter for each meetinghouse, last time I checked each of the receivers is pretty spendy.

russellhltn
Community Administrator
Posts: 26077
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Voice to text translation for the deaf

Postby russellhltn » Mon Feb 04, 2019 9:20 pm

Google has announced a new app "Live Transcribe" which is designed specifically for helping the deaf.

Making audio more accessible with two new apps

I found it in Google Play on my Pixel 1. Dirt simple. It works, but I do see some errors when I play back some YouTube videos with speech. But then I doubt of any technology is perfect at this point. It is interesting to see it change it's word choice as more words come in. You don't want to be reading at the very edge but about a line or so back.
Have you searched the Help Center? Try doing a Google search and adding "site:churchofjesuschrist.org/help" to the search criteria.

So we can better help you, please edit your Profile to include your general location.


Return to “Other Member Technologies”

Who is online

Users browsing this forum: No registered users and 0 guests