Sound Descriptions of Haptic Experiences of Art Work by Deafblind Cochlear Implant Users

Riitta Lahtinen, Camilla Groth, Russ Palmer
2018 Multimodal Technologies and Interaction  
Deafblind persons' perception and experiences are based on their residual auditive and visual senses, and touch. Their haptic exploration, through movements and orientation towards objects give blind persons direct, independent experience. Few studies explore the aesthetic experiences and appreciation of artefacts of deafblind people using cochlear implant (CI) technology, and how they interpret and express their perceived aesthetic experience through another sensory modality. While speech
more » ... nition is studied extensively in this area, the aspect of auditive descriptions made by CI users are a less-studied domain. This present research intervention describes and analyses five different deafblind people sharing their interpretation of five statues vocally, using sounds and written descriptions based on their haptic explorations. The participants found new and multimodal ways of expressing their experiences, as well as re-experiencing them through technological aids. We also found that the CI users modify technology to better suit their personal needs. We conclude that CI technology in combination with self-made sound descriptions enhance memorization of haptic art experiences that can be re-called by the recording of the sound descriptions. This research expands the idea of auditive descriptions, and encourages user-produced descriptions as artistic supports to traditional linguistic, audio descriptions. These can be used to create personal auditive-haptic memory collections similar to how sighted create photo albums. Multimodal Technologies and Interact. 2018, 2, 24 2 of 15 the world adjacent to his body by the use of his body" [17] (p. 97). The concept includes the person's deliberate and active movements, balance and orientation as well as proprioception [17] (pp. 36-37). The CI is a hearing aid device that partly operates underneath the skin of the user. CI surgery started in the early 1970s, when the focus was mainly on speech recognition. Since then, CI processor technology has been developed and expanded to a level where even musical perception has become identifiable. For the past 10 years, research has been concentrating on deaf children's speech, music and pitch perception, singing and playing of different instruments [18, 19] and also the perception of music by deafened postlingual adults [20, 21] . Recordings of these data were aimed at sighted deaf or deafened people. There appears to be very little, if any, research on deafblind cochlear implant users and their sound exploration and production. As the tactual environment of the deafblind person is so important, the CI users' haptic senses are heightened and sounds may also be perceived through vibrations in the air or through different media, therefore music can be one means for enjoyment and self-expression. Haptic hobbies such as ceramic crafts, linoleum cutting and printing as well as cooking classes that concentrate on taste and smell perceptions are also popular with deafblind CI users [8] . This user group's love of arts and culture is sustained through environmental guiding and technology that support their activities, such as audio description pages, portable induction loops, neck loops and radio receivers. Similarly, human guiding and audio descriptions aid the deafblind CI user in their explorations of artistic objects and events. When it comes to more fundamental aspects of experience, human touch and showing through movements might get closer to the natural experience of events. For example, statues in a museum that cannot be touched because of restrictions, can be experienced by re-enacting the postures of the sculptures together with a sighted person [22] . A deafblind person may experience art forms through their hands and body and sensing vibrations which can be felt from the art work through touching. When exploring the artefact using different tapping motions this may produce different vibrations and sounds depending upon whether the artefact is hollow or solid, referred to as a vibrosensoric experience [23] . Another study of blind peoples' art making experiences found that the participants explained their aesthetic elements using musical concepts, such as scale, shape, line and rhythm [24] (pp. 87-89), indicating that there seem to be a natural link between the elements of space, shapes and musical concepts. This is something that has also been noticed in research within the area of blindness [25] (p. 231), because as sound comes from different directions it simultaneously indicates the space between the source of the sound and the receiving person.
doi:10.3390/mti2020024 fatcat:glknchhrdjhrnpcod57wk3mcgm