Utilizing XR Technology to Deliver Information to Participants Through Visual, Auditory, and Haptic Sensory Channels

Abstract

In complex, high-stakes tasks such as remote drone and vehicle operation, effective Human-Machine interfaces are critical for ensuring operators comprehend task-critical information and maintain high situation awareness. However, flooding operators with information can overwhelm their senses, causing critical details to be lost. This raises the question: how should designers determine which sensory channel best suits each information cue? A 3×3×3 study was conducted, with information-to-sensory-channel mapping as a between-subjects factor and task type as a within-subjects factor. Tasks were performed in an Unreal Engine simulation. Preliminary analysis of 18 participants revealed that the visual channel consistently outperformed auditory and haptic channels in both accuracy and reaction time. Additionally, the nature of the information cue had an effect on comprehension. These findings demonstrate that effective multimodal interface design requires strategically mapping information cues to appropriate sensory channels rather than distributing them arbitrarily.