A Multi-sensory Approach to Present Phonemes as Language through a Wearable Haptic Device

Nathan Dunkelberger, Jennifer L. Sullivan, Joshua Bradley, Indu Manickam, Gautam Dasarathy, Richard G. Baraniuk, Marcia K. Omalley

Research output: Contribution to journalArticle

Abstract

Communication is an important part of our daily interactions; however, communication can be hindered, either through visual or auditory impairment, or because usual communication channels are overloaded. When standard communication channels are not available, our sense of touch offers an alternative sensory modality for transmitting messages. Multi-sensory haptic cues that combine multiple types of haptic sensations have shown promise for applications, such as haptic communication, that require large discrete cue sets while maintaining a small, wearable form factor. This paper presents language transmission using a multi-sensory haptic device that occupies a small footprint on the upper arm. In our approach, phonemes are encoded as multi-sensory haptic cues consisting of vibration, radial squeeze, and lateral skin stretch components. Participants learned to identify haptically transmitted phonemes and words after training across a four day training period. A subset of our participants continued their training to extend word recognition free response. Participants were able to identify words after four days using multiple choice with an accuracy of 89% and after eight days using free response with an accuracy of 70%. These results show promise for the use of multi-sensory haptics for haptic communication, demonstrating high word recognition performance with a small, wearable device.

Original languageEnglish (US)
JournalIEEE Transactions on Haptics
DOIs
StatePublished - Jan 1 2020

Keywords

  • language communication
  • multi-sensory haptics
  • phoneme coding
  • tactile device
  • wearable haptics

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Science Applications

Fingerprint Dive into the research topics of 'A Multi-sensory Approach to Present Phonemes as Language through a Wearable Haptic Device'. Together they form a unique fingerprint.

Cite this