How Our AI Tutor Works Internally
A complete breakdown of the architecture, animation pipeline, and intelligence system behind our AI Tutor experience.
How Our AI Tutor Works Internally
Our AI Tutor system is designed to feel natural, responsive, and human-like while still being fully automated. The internal architecture combines real-time rendering, animation blending, speech generation, and backend intelligence into one unified workflow. The goal is simple: when a student asks a question, the tutor should think, respond, speak, and animate as if it were alive.
The Core Architecture
The system is built using three major layers that work together continuously:
-
Frontend Layer (Babylon.js Avatar)
This is responsible for rendering the 3D tutor, triggering animations like talking, explaining, reacting emotionally, and pointing at virtual objects. -
AI Reasoning Layer (OpenAI Models)
This layer interprets the student's question, forms a response, and generates the ideal tone — whether helpful, strict, humorous, or encouraging. -
Backend Layer (FastAPI Services)
The backend manages speech synthesis, animation mapping, student analytics, and state tracking.
Together, these layers create an interactive teaching experience.
How the Tutor Responds to Students
Whenever a student types or speaks:
- The input is sent to the backend.
- The backend sends the text to the AI model.
- The AI model generates a structured response:
- natural language answer
- tone/emotion tag
- animation cue
- speech text
- The frontend avatar receives this payload and performs:
- animation changes
- mouth-movement syncing
- visual gestures
- spoken audio playback
This entire cycle typically completes in under a second.
Architecture Diagram
Below is a reference diagram for the system:
The pipeline shows how the AI model, animation engine, and backend services interact in real time.
Why Animation Matters
A typical chatbot gives answers — but a tutor must teach.
Animation allows the tutor to:
- point at imaginary boards
- nod to signal understanding
- use hand gestures while explaining
- change facial expressions based on student performance
This greatly improves attention, especially for long learning sessions.
How Code Execution Works Internally
Some lessons require the tutor to run Python or JavaScript code.
To support this, we use a sandboxed execution environment.
Here’s a simplified example used in development:
def execute(code: str):
try:
exec_globals = {}
exec(code, exec_globals)
return exec_globals
except Exception as e:
return {"error": str(e)}
result = execute("x = 5; y = 10; z = x + y")
print(result)