On Day 5, we’ll enhance our AI assistant by adding real-time avatar animations. This allows the 3D avatar to nod, shake its head, wave, and react naturally to user interactions.
1. Why Add Avatar Animations?
✅ Brings the AI assistant to life – Users feel like they’re talking to a real digital being.
✅ Enhances engagement – The avatar reacts with realistic movements.
✅ Syncs with voice & facial emotions – Making interactions more immersive.
We’ll use:
🔹 React Three Fiber – For rendering 3D avatar animations.
🔹 @react-three/drei – For pre-built animation utilities.
🔹 Framer Motion – For smoother movement transitions.
2. Installing Animation Dependencies
Step 1: Install React Three Fiber for 3D Animations
npm install @react-three/fiber three
Step 2: Install Drei for Pre-Built Three.js Animations
npm install @react-three/drei
Step 3: Install Framer Motion for Smooth Transitions
npm install framer-motion
3. Setting Up Basic Avatar Animations
Step 1: Create AvatarAnimation.js
Inside src/components/, create a new file:
import React, { useRef } from 'react';
import { useFrame } from '@react-three/fiber';
export default function AvatarAnimation({ emotion }) {
const headRef = useRef();
const handRef = useRef();
useFrame(() => {
if (headRef.current) {
if (emotion === 'happy') {
headRef.current.rotation.x = 0.1; // Slight head tilt forward
} else if (emotion === 'sad') {
headRef.current.rotation.x = -0.1; // Slight head down
} else if (emotion === 'angry') {
headRef.current.rotation.y = 0.1; // Head shake left-right
} else {
headRef.current.rotation.set(0, 0, 0); // Neutral pose
}
}
if (handRef.current && emotion === 'waving') {
handRef.current.rotation.x += 0.02; // Wave motion
}
});
return (
<group>
{/* Head */}
<mesh ref={headRef} position={[0, 1, 0]}>
<sphereGeometry args={[0.5, 32, 32]} />
<meshStandardMaterial color="orange" />
</mesh>
{/* Hand for Waving */}
<mesh ref={handRef} position={[0.6, 0.5, 0]}>
<boxGeometry args={[0.2, 0.5, 0.2]} />
<meshStandardMaterial color="orange" />
</mesh>
</group>
);
}
4. Integrating Avatar Animations with Voice & Expressions
Modify AvatarRenderer.js:
import React, { useState } from 'react';
import { Canvas } from '@react-three/fiber';
import AvatarAnimation from './AvatarAnimation';
import VoiceEmotionAnalyzer from './VoiceEmotionAnalyzer';
export default function AvatarRenderer() {
const [emotion, setEmotion] = useState('neutral');
return (
<>
<VoiceEmotionAnalyzer onEmotionDetect={setEmotion} />
<Canvas>
<ambientLight intensity={0.5} />
<directionalLight position={[0, 5, 5]} intensity={1} />
<AvatarAnimation emotion={emotion} />
</Canvas>
</>
);
}
Now, when a user speaks happily, the avatar will nod; when a user sounds angry, it will shake its head.
5. Adding More Avatar Reactions
Modify AvatarAnimation.js to include: ✅ Waving – If the user says “Hello”.
✅ Head Tilting – If the chatbot is thinking.
✅ Fist Clench – If the user sounds frustrated.
useFrame(() => {
if (handRef.current && emotion === 'waving') {
handRef.current.rotation.x += 0.02; // Wave motion
}
if (headRef.current) {
if (emotion === 'thinking') {
headRef.current.rotation.z = Math.sin(Date.now() * 0.002) * 0.1;
}
if (emotion === 'frustrated') {
handRef.current.scale.y = 1.2; // Fist clench
}
}
});
6. Testing Avatar Reactions
Step 1: Start the App
expo start
Step 2: Say the Following Phrases
| Phrase | Expected Animation |
|---|---|
| “Hello!” | Avatar waves |
| “I’m happy” | Avatar nods slightly |
| “I’m tired” | Avatar tilts head |
| “This is frustrating” | Avatar clenches fist |
| “Are you listening?” | Avatar shakes head |
7. Optimizing Avatar Movements
✅ Use Animation Interpolation for Smoother Motion
Modify AvatarAnimation.js:
import { useSpring, a } from '@react-spring/three';
const headProps = useSpring({ rotation: [0, emotion === 'happy' ? 0.1 : 0, 0] });
<a.mesh rotation={headProps.rotation}>
<sphereGeometry args={[0.5, 32, 32]} />
<meshStandardMaterial color="orange" />
</a.mesh>;
✅ Reduce Animation Processing Load
Update animations only every 2nd frame:
if (frameCount % 2 === 0) updateAnimation();
✅ Use GPU-Optimized Shaders for Avatar Rendering
<meshStandardMaterial attach="material" color="orange" roughness={0.5} metalness={0.3} />
8. Key Concepts Covered
✅ Added real-time animations for AI avatars.
✅ Mapped voice and facial emotions to avatar movements.
✅ Implemented waving, nodding, and head tilts for a lifelike AI assistant.
9. Next Steps: Adding Interactive Gestures & Controls
Tomorrow, we’ll:
🔹 Allow users to control the avatar with hand gestures.
🔹 Implement gesture-based commands (e.g., nod to confirm, wave to dismiss).
10. References & Learning Resources
11. SEO Keywords:
Real-time AI avatar animations, voice-controlled 3D avatars, interactive chatbot animation, building VTuber animations, AI avatars with gesture control.
