SocialEyes: Adaptive VR Social Skills Training for Autism

SocialEyes investigates adaptive virtual agents for structured social skills practice with children with autism spectrum disorder. Developed for IAT 813 at Simon Fraser University, the course research project includes sixteen agents with distinct behavioral profiles. Agents respond to a child's gaze, using contingent feedback to structure interactions within a virtual social environment.

System Architecture

A Bayesian network determines each agent's response to the child's gaze and to nearby agents' behavior. The network supports variation within the constraints of each assigned personality. Agents communicate through six validated facial expressions: happiness, sadness, surprise, anger, fear, and neutral. The implementation uses eye tracking on the Oculus Quest 3 to identify the child's focus and activates only the nearest relevant agent.

Evaluation and Key Findings

The activity involves identifying five social friends among thirty-two agents. I assessed the internal consistency of the agent model through a Monte Carlo simulation of thousands of gaze and gesture sequences. Simulated response rates closely matched probabilities calculated from the Bayesian network; for one gesture, the simulated rate was 24 percent compared with an expected rate of 25 percent.

Current Status

SocialEyes remains a proof of concept, with interaction limited to gaze within a classroom environment. The reported evaluation concerns the computational model and does not establish clinical effectiveness. A proposed extension is facial expression recognition in real time, enabling agents to respond to the child's expressive cues as well as gaze.