On Day 2, we’ll enhance our AI assistant by adding voice recognition so users can interact with the chatbot hands-free. We’ll use React Native Voice for speech-to-text processing, allowing users to speak commands instead of typing.
1. Why Add Voice Input?
✅ Hands-Free Conversations – Users can talk to the assistant naturally.
✅ Faster Interactions – No need to type long messages.
✅ Accessibility – Helps users with limited mobility.
We’ll use:
🔹 react-native-voice – Converts speech to text.
🔹 Expo Speech API – Allows the assistant to speak back (optional).
2. Installing Speech-to-Text Dependencies
Step 1: Install React Native Voice for Speech Recognition
npm install react-native-voice
Step 2: Install Expo Speech for Text-to-Speech (Optional)
expo install expo-speech
3. Setting Up Voice Recognition
Step 1: Create VoiceInput.js
Inside src/components/, create a new file:
import React, { useState, useEffect } from 'react';
import { View, Button, Text, StyleSheet } from 'react-native';
import Voice from 'react-native-voice';
export default function VoiceInput({ onRecognize }) {
const [isListening, setIsListening] = useState(false);
const [spokenText, setSpokenText] = useState('');
useEffect(() => {
Voice.onSpeechResults = (event) => {
setSpokenText(event.value[0]);
onRecognize(event.value[0]); // Send recognized text to chatbot
};
return () => {
Voice.destroy().then(Voice.removeAllListeners);
};
}, []);
const startListening = () => {
setIsListening(true);
Voice.start('en-US'); // Set language
};
const stopListening = () => {
setIsListening(false);
Voice.stop();
};
return (
<View style={styles.container}>
<Button title={isListening ? "Listening..." : "Start Talking"} onPress={startListening} />
{spokenText ? <Text style={styles.text}>You said: {spokenText}</Text> : null}
{isListening && <Button title="Stop" onPress={stopListening} />}
</View>
);
}
const styles = StyleSheet.create({
container: { alignItems: 'center', padding: 10 },
text: { fontSize: 16, marginTop: 10 },
});
4. Integrating Voice Input with the Chatbot
Modify ChatBot.js to use voice input:
import React, { useState } from 'react';
import { View, Text, ScrollView, Button, StyleSheet } from 'react-native';
import axios from 'axios';
import VoiceInput from './VoiceInput';
const OPENAI_API_KEY = 'YOUR_OPENAI_API_KEY';
export default function ChatBot() {
const [messages, setMessages] = useState([]);
const sendMessage = async (input) => {
if (!input.trim()) return;
const newMessages = [...messages, { text: input, sender: 'user' }];
setMessages(newMessages);
try {
const response = await axios.post(
'https://api.openai.com/v1/chat/completions',
{
model: 'gpt-4',
messages: [{ role: 'user', content: input }],
},
{
headers: { Authorization: `Bearer ${OPENAI_API_KEY}` },
}
);
const botReply = response.data.choices[0].message.content;
setMessages([...newMessages, { text: botReply, sender: 'bot' }]);
} catch (error) {
console.error('Error:', error);
}
};
return (
<View style={styles.container}>
<ScrollView style={styles.chatContainer}>
{messages.map((msg, index) => (
<Text key={index} style={msg.sender === 'user' ? styles.userText : styles.botText}>
{msg.text}
</Text>
))}
</ScrollView>
<VoiceInput onRecognize={sendMessage} />
</View>
);
}
const styles = StyleSheet.create({
container: { flex: 1, padding: 10 },
chatContainer: { flex: 1, marginBottom: 10 },
userText: { alignSelf: 'flex-end', backgroundColor: '#d1e7dd', padding: 10, borderRadius: 10, margin: 5 },
botText: { alignSelf: 'flex-start', backgroundColor: '#f1f1f1', padding: 10, borderRadius: 10, margin: 5 },
});
5. Adding Text-to-Speech (Optional)
Modify ChatBot.js to make the assistant speak its responses:
import * as Speech from 'expo-speech';
const speakResponse = (text) => {
Speech.speak(text, { pitch: 1.0, rate: 1.0 });
};
// Modify the API call to trigger speech
const sendMessage = async (input) => {
// (Existing API request logic...)
const botReply = response.data.choices[0].message.content;
speakResponse(botReply);
};
6. Testing the AI Voice Assistant
Step 1: Start the App
expo start
Step 2: Test Voice Input
- Tap “Start Talking” and say:
“Tell me a joke.” - The chatbot should recognize your voice and respond.
Step 3: Test Text-to-Speech
- The chatbot should speak its response back to you.
7. Optimizing Voice Recognition Performance
✅ Improve Speech Accuracy with a Language Selector
Modify VoiceInput.js:
<Picker selectedValue={language} onValueChange={setLanguage}>
<Picker.Item label="English" value="en-US" />
<Picker.Item label="Spanish" value="es-ES" />
<Picker.Item label="French" value="fr-FR" />
</Picker>
✅ Reduce Background Noise Interference
Set speech threshold settings:
Voice.start('en-US', { partialResults: false });
✅ Add a “Listening…” Indicator
Modify VoiceInput.js:
{isListening && <Text>Listening...</Text>}
8. Key Concepts Covered
✅ Integrated speech-to-text using React Native Voice.
✅ Linked voice input to the chatbot for hands-free interaction.
✅ Added text-to-speech so the chatbot speaks back.
9. Next Steps: Mapping Facial Expressions to Chat Responses
Tomorrow, we’ll:
🔹 Detect user emotions from speech.
🔹 Map chatbot emotions to facial expressions.
10. References & Learning Resources
11. SEO Keywords:
React Native voice assistant, AI speech-to-text chatbot, text-to-speech in mobile apps, OpenAI GPT voice commands, voice-controlled AI chatbots.
