Using Siri, Google Now or Cortana to talk with your smartphone is pretty normal these days. But have you ever considered creating your own Voice Dialog? And would you think this is possible within a hybrid app using Cordova and the Ionic Framework? Let me prove it to you.
In this article I will show you how to build text to speech voice output and speech recognition input using Cordova plugins with the Ionic Framework.
Setting up the App
We start with a blank Ionic app and install our two Cordova plugins, so go ahead and run:
ionic start cordova-speech blank cd cordova-speech cordova plugin add cordova-plugin-tts cordova plugin add http://ift.tt/1j3OWL1
This creates a new folder cordova-speech with a clean Ionic Framework app and adds two Cordova plugins we need for our example:
The first plugin is a cordova wrapper for the native text to speech output functions. This plugin uses the AVSpeechSynthesizer on iOS and the android.speech.tts.TextToSpeech package on Android, so it’s really a simple mapper of the best native functions for giving out text. This plugin also seems to work for Windows phone, but I had no chance to test this part. If one of you has a Windows Phone why not give it a try and leave a comment with your findings below?
The second plugin is a cordova wrapper for speech recognition. On Android this plugin uses the native android.speech.SpeechRecognizer package, on iOS it’s a bit harder to achieve this task so the author used the iSpeech SDK which you can test for free for now. If you plan to use this SDK in production additional costs may arise.
Besides those two plugins we don’t need any more tools or fancy components inside our app!
Crafting our Speech View
The view of our app will be quite simple. First of all we need an input field for some text which our app can speak later on.
Below this field we need two buttons, the first is to call our text to speech action so the text we have inserted get’s read to our user. The second button triggers our speech recognition dialog and listens for whatever we say.
The recorded text (or what the app thinks we have said) will be printed below our other components, just to see how precise the recognition is. This can vary quite a bit, depending on surrounding sounds, how clear you speak, how difficult the words and sentences are and so on. You can play around with the results later and leave your opinions about this plugin below!
For now go ahead and open your www/index.html and replace the current body with:
<body ng-app="starter">
<ion-pane ng-controller="AppCtrl">
<ion-header-bar class="bar-stable">
<h1 class="title">Cordova Text-to-Speech</h1>
</ion-header-bar>
<ion-content class="padding">
<div class="list list-inset">
<label class="item item-input">
<i class="icon ion-speakerphone placeholder-icon"></i>
<input type="text" placeholder="Let me speak..." ng-model="data.speechText">
</label>
</div>
<button class="button button-full button-positive" ng-click="speakText()">
Speak!
</button>
<button class="button button-full button-positive"ng-click="record()">
Record
</button>
<div class="card">
<div class="item item-text-wrap">
{{recognizedText}}
</div>
</div>
</ion-content>
</ion-pane>
</body>
There is really no magic in this view so let’s head to the real bread and butter of this article!
Using the Cordova Speech Plugins
We have already installed all the needed plugins and don’t have to import any headers or anything else besides. Therefore, open your www/js/app.js and replace everything inside with:
angular.module('starter', ['ionic'])
.controller('AppCtrl', function($scope) {
$scope.data = {
speechText: ''
};
$scope.recognizedText = '';
$scope.speakText = function() {
TTS.speak({
text: $scope.data.speechText,
locale: 'en-GB',
rate: 1.5
}, function () {
// Do Something after success
}, function (reason) {
// Handle the error case
});
};
$scope.record = function() {
var recognition = new SpeechRecognition();
recognition.onresult = function(event) {
if (event.results.length > 0) {
$scope.recognizedText = event.results[0][0].transcript;
$scope.$apply()
}
};
recognition.start();
};
});
As you can see there is not that much code needed for quite cool features! Let’s take a closer look at our controller implementation.
We create a data object which will is connected to our input field. We need to make it an object so the value is updated correctly using ng-model.
If we want our inserted text to be spoken, the speakText() function get’s called. We use the Cordova plugin by calling TTS.speak() and also pass an object with some properties. In this example we pass the text (which you should always do) and also a language and a rate which defines how fast the text is spoken. You can create quite funny outputs playing around with these values, trust me I had my fun creating this article!
Finally we can also specify a success and error callback, which I left empty for now because handling those errors is very custom for every app.
Our voice recognition starts on calling record(). Here we create a new instance of a SpeechRecognition object. After creating it, we assign a function to the onresult property of the object. This function will be called once the recognizer thinks he is done listening to you talking.
We check if we actually have any result, and if that’s the case we apply the text our recognizer understood to our scope variable which appears below our other view elements.
After setting the result block we only have to start our recognition and the app starts listening, that’s all!
Conclusion
In this step-by-step tutorial I explained how to use two Cordova plugins for text to speech and voice recognition inside an Ionic app. You might think that these two features are hard to implement, but given the great wrappers they are super cheap and easy to include in your hybrid app.
If you want to learn more about building great mobile Apps make sure to check out my Blog Devdactic and follow me on Twitter @schlimmson!
Cheers,
Simon
by Simon Reimler via Devgirl's Weblog
No comments:
Post a Comment