HuiShan Lai
← Research archive

Master's Thesis · University of Aizu · Completed

The Role of Embodied Avatars and Generative AI in Self Learning VR Classroom

A master's thesis examining how VR, generative AI, and an embodied teaching avatar shape immersion, enjoyment, comprehension, and cognitive load in self-directed calculus learning.

HuiShan Lai

Graduate School of Computer Science and Engineering, University of Aizu · Aizuwakamatsu, Japan

Thesis PDF GitHub Repository
Side-by-side views of the text-panel and embodied-avatar modes in the self-learning VR classroom
The two VR learning modes: a text-based generative AI panel and a speaking embodied AI assistant.

Abstract

This thesis investigates how embodied avatars and generative AI can support self-directed mathematics learning inside a virtual classroom.

The work combines short calculus videos, quizzes, speech interaction, a generative AI assistant, and an optional animated avatar in one Meta Quest 3 experience. The central question is not only whether VR feels more immersive, but whether visual embodiment changes how learners experience and evaluate AI-supported study.

A two-participant pilot informed the final interaction design, followed by a within-subject study with 18 participants. Both VR conditions produced significantly higher immersion and enjoyment than desktop video learning. The study found no statistically significant differences in comprehension, cognitive load, effort, or frustration, and no reliable rating-scale advantage of the avatar over the text-based VR assistant.

18
study participants
3
learning conditions
7
shared experience measures
125
pages in the thesis

Research framing

What the thesis asks

The study separates the effect of immersive delivery from the added effect of giving a conversational AI assistant a visible, speaking body.

RQ1

Does embodiment change the AI learning experience?

How does adding an embodied avatar to a generative AI assistant affect engagement and enjoyment in a self-learning VR classroom?

RQ2

How does VR compare with desktop learning?

How do VR-based conditions compare with PC video learning in immersion, enjoyment, and quiz performance?

Three contributions

  1. A working VR self-learning system that combines calculus videos, quizzes, voice dialogue, generative AI, and an embodied teaching avatar.
  2. A counterbalanced within-subject study that isolates desktop, text-based VR assistant, and embodied-avatar conditions.
  3. Empirical evidence that distinguishes affective benefits from learning outcomes, including the limitations of embodiment in a short educational session.

System

A complete self-learning classroom

The room is divided into an AI interaction area and a video learning area so that explanation, practice, and assessment remain available within one spatial environment.

Annotated VR classroom showing the AI interaction area on the left and the video learning and quiz area on the right
The learner-facing environment combines a conversational AI area, a central video player, playback controls, video selection, and three follow-up quizzes.

Video learning

Three approximately three-minute Khan Academy derivatives videos, playback timing, play/pause, five-second skip controls, and matching quizzes.

AI assistant

Voice input is transcribed with Whisper and sent through a local Flask backend to a GPT-4o-mini assistant grounded in the lesson transcripts and quiz material.

Embodied avatar

A Ready Player Me teacher uses Mixamo animation, Amazon Polly speech, and Oculus LipSync to present the same AI response through voice and facial motion.

Interaction logging

Local logs retain quiz answers, timestamps, questions, AI responses, response time, and avatar visibility for later analysis.

Four-module architecture for video learning, the AI assistant, embodied avatar, and local interaction logging
The implemented architecture separates the learning, assistant, embodiment, and logging responsibilities while connecting them through Unity and a local Python service.
VR application
Unity 2022.3.62f1, URP, OpenXR
Hardware
Meta Quest 3 via Quest Link
Speech input
Whisper base.en in Unity
AI backend
Python 3.13, Flask, OpenAI SDK
Response model
GPT-4o-mini assistant
Avatar output
Amazon Polly, Oculus LipSync, Mixamo

Interaction design

From lesson to dialogue

The prototype supports two connected learning loops: watch and answer a quiz, or pause and ask the assistant for a spoken explanation.

Flow diagram for selecting a calculus video, controlling playback, opening a quiz, submitting an answer, and saving a local log
Video learning flow: select, watch, navigate, answer, and save.
Flow diagram for recording speech, Whisper transcription, the GPT-4o-mini assistant, optional avatar speech, and interaction logging
AI interaction flow: record, transcribe, generate, present, and log.

Shared dialogue core

C2 and C3 use the same speech recognition, prompt grounding, backend, and response generation. The experimental difference is how the response is presented: text only or text plus a speaking animated avatar.

Controller-first operation

The A button starts and stops recording, while the X button sends the transcript. A visual voice-activity indicator changes between idle and recording states so the learner can confirm when the system is listening.

Design iteration

The pilot changed the final system

A two-participant pilot was used as an engineering and usability checkpoint. It showed that the prototype was stable, but also exposed passive-viewing fatigue, missing navigation controls, unclear recording feedback, and AI response delays of approximately 3-5 seconds.

  1. Shorter lessonsVideo segments were reduced from roughly five minutes to approximately three minutes to limit fatigue and loss of attention.
  2. Playback controlFive-second forward and backward controls were added so learners could revisit difficult moments or skip familiar material.
  3. Measured latencyAI response timing was added to the logs and assistant instructions were refined, while cloud-processing delay remained an acknowledged constraint.

Evaluation

A counterbalanced three-condition study

Every participant experienced every condition. A balanced Latin square varied the condition order, reducing sequence effects without changing the fixed lesson and quiz sequence.

A study participant wearing a Meta Quest 3 headset while seated in front of the desktop study station
A participant completing the study in a controlled lab environment with Meta Quest 3, controllers, and the connected Unity workstation.
  1. C1 · Desktop videoComputer-based calculus video and quiz without VR or AI. Participants could use paper for notes and calculations.
  2. C2 · VR + text AIVR video learning with voice input and a generative AI response shown on a floating text panel.
  3. C3 · VR + embodied AIThe same VR and AI pipeline, with responses also spoken by a lip-synced animated teacher avatar.
01

Learn

Watch one fixed three-minute differential calculus video.

02

Ask

In C2 and C3, ask at least one lesson-related question through speech.

03

Answer

Complete the matching comprehension quiz for that session.

04

Reflect

Submit the questionnaire, then rank all three conditions at the end.

Measures

Seven shared 7-point items covered immersion, enjoyment, comprehension, mental demand, physical demand, effort, and frustration. The questionnaire adapted relevant concepts from IPQ and NASA-TLX and was presented in English, Japanese, and Chinese.

C3 added four avatar-specific ratings inspired by AttrakDiff: human-likeness, usefulness, pleasantness, and appearance. A typical full study lasted 25-45 minutes, with most participants finishing within 35 minutes.

Quantitative results

Experience improved more than performance

Cumulative link mixed models accounted for the ordinal ratings and repeated observations from each participant. Condition affected immersion and enjoyment, but not the other five shared measures.

Mean ratings for immersion, enjoyment, comprehension, mental demand, physical demand, effort, and frustration across C1, C2, and C3
Ratings across the seven shared measures. The statistically reliable condition effects were limited to immersion and enjoyment.
MeasureChi-squarep-valueInterpretation
Immersion18.25< .001Significant
Enjoyment14.57< .001Significant
Comprehension2.50.287No condition effect
Mental demand1.77.413No condition effect
Physical demand0.66.717No condition effect
Effort2.19.334No condition effect
Frustration0.11.948No condition effect
Immersion

C1 was lower than both VR conditions

C1 vs C2: estimate -2.64, p=.002. C1 vs C3: estimate -3.79, p<.001. C2 and C3 were not significantly different (p=.188).

Enjoyment

The same VR advantage appeared again

C1 vs C2: estimate -2.35, p=.006. C1 vs C3: estimate -3.19, p<.001. C2 and C3 were not significantly different (p=.431).

Quiz performance

A numerical rise, not a demonstrated gain

Mean scores were approximately 78 in C1 and 83 in C2/C3, but the thesis found no statistically significant performance difference.

Stacked bar chart showing C3 most often ranked first, C2 most often ranked second, and C1 most often ranked third
Preference rankings favored the embodied-avatar condition: about 73% of participants ranked C3 first, while C2 was most often ranked second and C1 third.

Avatar perception

Pleasant and useful, but not fully human-like

Pleasantness had the highest median rating at approximately 6.5/7. Appearance and usefulness were both around 6, while human-likeness was lower at around 5 and varied most between participants.

Boxplots for avatar appearance, human-likeness, pleasantness, and usefulness ratings on a seven-point scale
Avatar ratings show strong pleasantness alongside greater disagreement about human-likeness.

Discussion

What this thesis establishes

The strongest contribution is a practical, end-to-end framework showing that conversational AI and embodiment can be integrated into a coherent VR learning workflow and evaluated without conflating preference with learning effectiveness.

RQ1 received partial support: C3 was the most preferred condition and was rated slightly higher in enjoyment, but the measured difference between C2 and C3 was not statistically significant. RQ2 was supported for subjective experience: both VR conditions improved immersion and enjoyment relative to desktop learning, but not short-term comprehension or quiz performance.

Next research directions

  • Context-aware tutoring: connect the assistant directly to the current video, quiz, and learner progress.
  • Multilingual interaction: support multiple input/output languages, subtitles, and adjustable speaking speed.
  • More expressive embodiment: improve gaze, gesture, affect, timing, and proactive classroom behavior.
  • Active learning tools: add spatial notes, handwriting, whiteboards, scaffolding, and adaptive questions.
  • Longer and broader studies: test retention, repeated use, more participants, and subjects beyond calculus.

Evidence boundary: this was a short within-subject study with 18 participants. It supports an affective and experiential benefit for VR, but not a demonstrated learning gain or a statistically reliable advantage of embodiment over the text-based VR assistant.

System constraints included delayed responses, imperfect speech recognition, English-only interaction, limited avatar behavior, and the absence of in-VR note-taking. These limitations are part of the research contribution because they identify where an embodied AI tutor must improve before longer-term educational claims are justified.

Authors

HuiShan LaiUniversity of Aizu

Thesis citation

Lai, Hui Shan. “The Role of Embodied Avatars and Generative AI in Self Learning VR Classroom.” Master's thesis, Graduate School of Computer Science and Engineering, University of Aizu, September 2025.

@mastersthesis{lai2025embodied,
  author  = {Lai, Hui Shan},
  title   = {The Role of Embodied Avatars and Generative AI in
             Self Learning VR Classroom},
  school  = {University of Aizu},
  type    = {Master's thesis},
  address = {Aizuwakamatsu, Japan},
  month   = sep,
  year    = {2025}
}