Artificial Synthesisia
Artificial synesthesia is a deliberately evoked or induced sensory joining in which the real information of one sense is accompanied by a perception in another sense through the use of a cross-modal mapping device. It can also be known as virtual synthesia or synthetic synthesia.
The additional perception is regarded by the trained synesthete as real, often outside the body, instead of imagined in the mind’s eye. Its reality and vividness are what makes artificial synesthesia so interesting in its violation of conventional perception. Synesthesia in general is also fascinating because logically it should have been a product of the human brain, where the evolutionary trend has been for increasing coordination, mutual consistency and perceptual robustness in the processing of different sensory inputs.
However, a main drive for investigating artificial synesthesia (and cross-modal neuromodulation) is formed by the options it may provide for people with sensory disabilities like deafness and blindness, where a neural joining of senses can help in replacing one sense by the other: e.g., in seeing with your ears when using a device that maps images into sounds, or in hearing with your eyes when using a device that maps sounds into images. The former synesthetic vision is the main focus of this site. The use of a device to map one sensory information stream into an information stream for another sensory modality clearly distinguishes artificial synesthesia from spontaneous synesthesia or developmental synesthesia as well as from cross-modal associations in non-synesthetes for normal sensory inputs. In other words, we are interested in forms of learned synesthesia (acquired synesthesia) that might result from machine-generated crossmodal mappings, particularly in studying the feasibility of functionally relevant auditory-to-visual synesthesia for blind people - to allow for a non-invasive visual prosthesis via sound-induced visually meaningful percepts (detailed mental imagery, in contrast to natural synesthetes who usually report relatively simple or non-generic visual percepts for color, textures or shapes in association with certain sounds, with different synesthetes reporting different percepts). This can be viewed as a form of synthetic vision, as well as functional synesthesia, where mental images are synthesized by the brain in close correspondence to the visual information as conveyed through sound. The soundscapes thus serve to scaffold mental imagery. The subject can also be related to sound symbolism and artificial or induced phonesthesia (phonaesthesia), a cross-modal mapping where certain sounds become associated with certain meanings.
Hildur Gudnadottir - Composing Scores
Professional musician and composer who uses found sound and site specific sampling to create scores for films and television programmes. Most notably, Gudnadottir composed the score for the HBO/Sky series Chernobyl for which she visited a decommissioned power plant in Lithuania where the series was filmed. The field recorder Chris Watson took recordings of reactive materials such as a large door which Gudnadottir refers to as having the only solo in the score. The reactive properties allowed the microphones to capture an almost melodic set of noises from the material.
Chris Watson - Field Recording
'Beginning in the magical ambience of the gloaming, through arcane animal activity of the short summer night.'
Chris Watson is a field recorder who has worked for the likes of Hildur Gudnadottir and David Attenborough. Watson, through varied techniques and implementations of equipment, has the ability to record minute sound of ants as easily as the noises of radioactive doors.
Alongside his work for TV and film, Watson also streams 9-hour dusk-til-dawn field recordings amongst other things, such as his piece The Sylvan Space, for which he travelled to the Holystone Oak Woodland in Northumberland National Park, recorded for 9 hours from dusk until dawn, and subsequently capturing the sounds of roe deer, fluttering bat wings, bird songs and more.
'In the golden hour of evening light I fixed my favourite microphone array under a stand of ancient oaks and recorded in a surround format throughout the night and across the dawn. I gathered a continuous long form piece within the woodland of stillness, drama and song as the night flowed into sunrise.'
Comments