Skip to project
Roman Jahandideh ← All projects
Accepted, ACM ISS 2026 Companion

Reading a Remote Teammate's Body

ACM ISS 2026 Companion · doi:10.1145/3845796.3859311 · Apple Watch HR/HRV · Unity cooperative play · two studies

Experimental multiplayer game environment used in the remote teammate study, showing Prep, Runner, and Cook roles and teammate state icons
Experimental multiplayer game environment used in the study. The interface shows the Prep, Runner, and Cook roles alongside icons representing teammates' states.

Reading a Remote Teammate's Body asks how a remote teammate's bodily state can remain socially readable without reducing it to an emotion label or an intrusive stream of numbers. The project translates heart rate (HR) into pulse speed, brightness, and glow, and heart rate variability (HRV) into the coherence of a surrounding ring. Together, these signals form one animated halo per teammate: a graded cue that players interpret in the context of a shared task rather than a diagnosis presented as fact.

Two-Study Methodology

Study 1 used a within-subjects, research-through-design methodology to test whether the icon could be read without numbers or emotion words. Thirty-six participants viewed sixteen combinations of four HR states and four HRV states in an online forced-choice study, followed by clarity, urgency, stability, support-intention, and workload measures. HR recognition reached 66.3% and HRV recognition 58.2%, both well above the 25% chance level. Joint recognition of both channels reached 44.6% against a 6.25% chance level.

Study 2 carried the validated icon into a live Unity cooperative-cooking prototype. Thirty-six new participants formed twelve three-player teams and completed three counterbalanced seven-minute missions under No Cue, Numeric Cue, and Icon Cue conditions. Each participant used an Apple Watch and a local HealthKit-to-Unity bridge; the system replicated classified HR/HRV states rather than raw physiological time series. Scripted pressure events tested whether teammates noticed overload and offered timely support.

Research Figures

Figures from the ACM paper and study materials. Select any figure to view the full-resolution version.

Experimental Environment

The live study used a custom Overcooked-style cooperative cooking task with Prep, Cook, and Runner roles. Three scripted events created measurable coordination pressure: an ingredient bottleneck, a pot overload, and a service rush. The primary behavioral measure was support response time, defined as the time from the pressure event to the first relevant support action from a teammate.

Findings and Design Trade-offs

Teams responded to pressure in 15.3 seconds under No Cue, 12.5 seconds under Numeric Cue, and 11.4 seconds under Icon Cue. The Icon Cue was 3.9 seconds faster than No Cue and matched the numeric display statistically on response time, while reducing workload and perceived invasiveness. Teammate awareness rose from 4.21 under No Cue to 5.34 under Icon Cue. The result is deliberately mixed: Numeric Cue scored higher on trust, coordination usefulness, and mission score, while Icon Cue offered a lower-friction and less invasive way to notice a teammate's condition.

The central contribution is therefore not a claim that ambiguity always outperforms precision. It is a demonstrated design trade-off: a label-free, ambient cue can preserve much of the coordination benefit of explicit physiological disclosure while reducing attention and privacy costs. Participants specifically requested opt-in visibility, separate permissions for icon and numeric displays, and an immediate off switch.

Publication

Jahandideh, R., Jahandideh, A., & DiPaola, S. (2026). Reading a Remote Teammate's Body: A Body-Worn Sensing Interface for Group Coordination in a Distributed Interactive Space. In Companion Proceedings of the Conference on Interactive Surfaces and Spaces (ISS Companion '26), Turin, Italy. ACM. doi:10.1145/3845796.3859311