Herein, we demonstrate a method for creating a sound installation that uses sensors and computers to present computer-generated sound and novel pictogram imagery using information obtained from cameras and thermometers. The computer-generated pictogram sound and image data are presented in a different location from the collection point, and the audience can recognize the changing states by listening to the sound and/or watching the imagery. The system uses thermometers and cameras that are installed in both interior and exterior spaces to detect the state of those spaces based on the obtained image and temperature data. When the state of the captured images or the temperature changes, such as when a moving object is detected, a signal tone and a pictogram are created and assigned to describe the state based on a temperature and image data-mapping table. The designed sounds change according to the detected states, and the proposed installation simultaneously produces sounds and pictograms in different spaces in order to convey information related to motion or temperature states. For example, if the temperatures of the interior and exterior spaces are different, different sound pitches are assigned to each space. Thus, individuals walking from outside to inside, or vice versa, will experience a change in sound. If the installation were demonstrated in both interior and exterior spaces, the temperature in the spaces would also change with the season. For example, if the installation were demonstrated in winter, the inside space would be hot, and outside would be cool, or vice versa in summer. If the installation is considered from a long-term point of view, individuals can experience sounds of varying pitch, which change with temperature, as the seasons change. Consequently, the assigned sound provides information on both motion concerning a moment and temperature regarding a long period of time.
Ref "Designing Sound Representations for Responsive Environments" Takuya Yamauchi, THE 22nd ANNUAL INTERNATIONAL CONFERENCE ON AUDITORY DISPLAY (ICAD2016),Australian National University, Canberra 3-7 July 2016
My name is Takuya Yamauchi. I am freelance programmer, Ux designer and media artist.