Contributions of Gibson's Ecological Psychology to the Creation of Interactive Sound Installations

Abstract

This article explores the potential contributions of Gibson's Ecological Psychology to artistic creation within the scope of Interactive Sound Installations. It provides a historical overview of the evolution of this form of art, highlighting the growing emphasis on the role of space and body (audience action) in shaping interactive art concepts. The article also delves into how ecological acoustics research has integrated interdisciplinary perspectives from sociology, anthropology, and cognitive science to promote a more equitable and fair acoustic environment. Considering that the concept of the body embedded in the world is both central to interactive arts and Cognitive Science, particularly to Gibson's Ecological Psychology, this text presents its main aspects. It then discusses how these aspects can be applied in Interactive Sound Installations. In conclusion, it suggests that applying Ecological Psychology concepts to Interactive Sound Installations can contribute to the goals of recent acoustic ecology in creating a fairer acoustic environment.

Keywords:
Interactive Sound Installations; Acoustic Ecology; Ecological Psychology; J. J. Gibson; Musical Cognition

Resumo

Este artigo explora possíveis contribuições da Psicologia Ecológica de Gibson para a criação artística no âmbito das Instalações sonoras interativas. Fornece uma revisão histórica da evolução dessa forma de arte, destacando a crescente ênfase no papel do espaço e do corpo (ação do público) na formação do conceito de interatividade. O artigo também investiga como a pesquisa em acústica ecológica integrou perspectivas interdisciplinares da sociologia, antropologia e ciência cognitiva para promover um ambiente acústico mais equitativo e justo. Considerando que o conceito de corpo situado no mundo é fundamental tanto para a Arte Interativa quanto para a Ciência Cognitiva, em especial para a Psicologia Ecológica de Gibson, este texto apresenta seus principais aspectos. Em seguida, discute como esses aspectos podem ser aplicados em Instalações sonoras interativas. Para concluir, sugere que a aplicação dos conceitos da Psicologia Ecológica em Instalações sonoras interativas pode contribuir para os objetivos da recente ecologia acústica em criar um ambiente acústico mais justo.

Palavras-chave:
Instalações sonoras interativas; ecologia acústica; psicologia ecológica; J. J. Gibson; cognição musical

Interactive Sound Installation is an artistic mode of expression with a long development history. There are several panoramic reviews considering artistic works and concepts related to interactive arts offered by artists from different artistic languages, such as Graham (1997, 2014), Cornock & Edmonds (1973), and Mamedes (2015, 2018), demonstrating the richness and plurality of possible approaches to the field. We must remember that a new systematic review is always susceptible to flaws or incompleteness. Therefore, the path presented here is supported by the authors who have most directly influenced our academic and artistic journey. Complementary to the aspects discussed by these authors, we will outline the key points of J. J. Gibson's Ecological Psychology (Gibson, 1966, 1979). Then, we present practical examples of applying Ecological Psychology concepts in one of our Interactive Sound Installations. These examples will be accompanied by discussions that address the issues raised by the authors of the brief review.

1. Interactive musical performance in perspective

Interactive musical performance and installation may be considered a modality of Interactive art, or even Interactive Electronic Art. As Beryl Graham demonstrates, they have “explicit relationships with certain types of contemporary art, including installation, site-specific (…)” (Graham, 2014, p. 65), among others. Glusberg (2011), in his text on the history of Performance Art, remembers how significant changes in the context of contemporary art (body art, site specifics, environments, happenings) were fundamental to the development of Performance Art. These changes include transformations in the presence of the body through its de-fetishization, contrasting it with centuries of the “cult of beauty” imposed by literature, painting, and sculpture , and of space, through the reconfiguration of the boundaries between the work and its surroundings and, as Alan Kaprow intended, through the consideration of the audience as actively belonging to the space of the artwork.

Dinkla (1994) also considers the technological development of interactive arts as a practice that emerges from Performance Art. He describes the birth of two basic tendencies established at the end of the 1960s: a) the development of responsive environments within the US-American Art & Technology movement; b) the development of the head-mounted displays (HMD) to enhance the immersion feeling of the audience in the artwork. For Dinkla, these pioneering tendencies have had a significant influence on the current state of Interactive art, leading, at the end of the 1970s and during the 1980s, to a broad theoretical and practical discussion about the differences between Participation and Interaction. His text summarizes how artists such as Myron Krueger, Jeffrey Shaw, David Rokeby, Lynn Hershman, Grahame Weinbren, Ken Feingold, among others have created interactive works exploring concepts like proximity to the work, the ability to manipulate its content, intimacy, subjectivity, sensuality, non-linear narratives, remembering and forgetting. Such concepts were fundamental to both the theoretical discussion and the solidification of the field. Dinkla also highlights how a new generation of artists emerged from the advances driven by the pioneers of the 1960s. They explored more varied forms of interaction supported by more natural and invisible interfaces, overcoming the limits imposed by devices like joysticks or touch screens. This provided opportunities for works that emphasized the environment and the interaction itself.

Over time, it has become widely accepted that interactive works must involve public participation. As early as 1967, Roy Ascott (Ascott, 1967), within the context of cybernetic studies, defined what he termed Cybernetic and Behavioral Art. He proposed that works should have adaptive structures capable of accommodating spectators' responses, laying important theoretical foundations for the practice of interactivity art. According to Ascott, Cybernetic and Behavioral Art, which relies on audience participation for its realization, stands in contrast to deterministic works, where the artist conceives and executes the work independently. In Brazil, Spanish artist Julio Plaza argues that interactivity in such works should not be viewed merely as a “technical or functional convenience” but as something that “physically, psychologically, and sensitively involves the spectators in a practice of transformation”¹ (Plaza, 2003, p. 20). He characterizes public participation at three levels: passive participation (contemplation, interpretation), active participation (exploration, intervention), and interactivity (reciprocal relationship between the spectator and the artificial system of the work). He refers to these levels of participation, from the perspective of the work's openness, as 1st, 2nd, and 3rd-degree openings. Plaza considers the form of opening discussed by Eco in “The Open Work” (Eco, 1990) one of the texts often regarded in the interactivity field as a 1st-degree opening because Eco only considers the multiplicity of interpretations allowed for the public. Cláudia Giannetti (2006), on the other hand, is more emphatic in considering that interactive works are only those that use technical interfaces to foster relationships between the public and the work. The author categorizes the level of interaction provided by the interfaces as follows: a) mediating system, in which there is simple and punctual interference; b) relative system: which allows the user to access the content based on possibilities delimited by the system; c) interactive system: in which the user has the role of sender of information that can intervene, manipulate, and generate new content in the context of the work. Likewise, David Saltz (1997) proposed a classification of performance works that use technological devices based on the levels of interaction they provide. He categorizes them into two types: stage interaction, where the artist interacts on stage with a system while the audience observes, and participatory interaction, where the audience directly interacts with the artwork.

Analyzing several of his works, Simon Penny (2015) suggests that interactive artists should pay attention to the spatial and material details, moving away from the traditional object/spectator relationship and placing the interactor in a continuous sequence of "I'm doing this now" (Penny, 2015, p. 272). Penny criticizes the hierarchical dualism between matter and information prevalent in computer technology of the 1990s, which was reflected in interactive works, particularly those created using virtual reality or virtualized spaces.

Although it was aimed at the stage arts, Rancière’s The Emancipated Spectator, Five Lessons in Intellectual Emancipation is consistently referenced in theoretical discussions about interactive art. His concept of the emancipated spectator, which involves transforming the traditionally passive spectator into an active participant, is considered essential by many artists working with interactive art. In defining the emancipated spectator, he states:

First, viewing is the opposite of knowing: the spectator is held before an appearance in a state of ignorance about the process of production of this appearance and about the reality it conceals. Second, it is the opposite of acting: the spectator remains immobile in her seat, passive. To be a spectator is to be separated from both the capacity to know and the power to act. (Rancière, 1990, p. 2)

In music, the discussion of interactivity has been more common in the scope of electroacoustic music, especially in live electronics. Garnett (2001), in the context of computer music, considers musical interaction as the reciprocal influences that occur between the interpreter/performer and the computer/interactive system. Rowe (1993, 1999) and Miskalo (2009) corroborate this idea by considering interaction in music as the relationship established between the performer and the computer/interface/system. For other authors, interactivity occurs in the collaboration between the composer and performer. Bonfim (2009), supported by Roy Ascott's concept of telematic art, emphasizes how creation in telematic environments can contribute to interaction between participants, transcending the physical limits of distance and broadening the search for global awareness. Hayden & Windsor (2007) classify collaboration between composers and performers into three categories: a) directive: the composer is responsible and usually codifies all their choices into a music score. Interactions between the composer and performer are limited to more pragmatic issues commonly related to performance; b) interactive: the composer is still the author, but the participants are more involved in the process of artwork development so that various aspects are more open and not rigidly encoded in music scores; c) collaborative: the pieces are created as a group and are generally not fixed in musical writing. Cardassi & Bertissolo (2019), based on their own collaborative compositional experiences, emphasize the need to develop specific methodologies to promote more collaborative interaction between the various members of the process, helping to reduce the distance between composers and performers.

From another perspective, Di Scipio (2003) defines interactive musical systems as tools that can react in real-time to changes in external conditions. However, he notes that interactive systems have typically been developed using linear communication models, where a computer algorithm processes information, provided by an agent to determine the system's output. In this setup, agents act and computers react. Di Scipio suggests composing musical interactions instead of interactive music. To achieve this, the work should be projected as a community of self-organizing agents, encompassing both computer systems and the environment, including the audience. This concept, inspired by Maturana, results in what he calls Audible Eco-Systemic Interface: a “structurally closed” system in which the removal or alteration of any component can lead to its collapse, and an “organizationally open” system meaning that the configuration of variables and the sequence of internal states are not predetermined, depending on constant contact with the environment.

Solomos (2020) explores the connection between musical creation and space, examining Xenakis’ visual-architectural sound experiences and the sound spatiality developments in electroacoustic music. He also delves into Christina Kubish's sound installations and Di Scipio's Audible Ecosystems. In this context, Solomos revisits concepts from acoustic ecology, referencing the pioneering work of Murray Schafer (Schafer, 1997) and emphasizing the need to encompass ethics, economics, and sociology, among other aspects, in sound ecology studies. Waters (2007) similarly addresses musical interactivity from an ecological perspective. He highlights the negative tendency present in works created in technologically driven contexts that prioritize the technology over the music itself. The author is concerned about the tendency of some musicologists and composers to prioritize acoustic facts over social and cultural contexts. In a study of interactive works by artists such as Nic Collins, Agostino di Scipio, Stef Edwards, Jonathan Impett, Adam Green, David Casal, John Bowers, Phil Archer, Shigeto Wada, and herself, Waters analyzes the interconnected relationships between composers and performers, performers and instruments, and instruments and environments. He concludes by stating that interactive works, from an ecological perspective, should enable human participation in creating their own musical meanings, in reference to the fourth musical stage that French economist Jacques Attali outlined in Bruits. Essai sur l'économie politique de la musique (Attali, 1986)².

On the other hand, Jean-Luc Hervé suggests that interactive musical works should be considered concerning space, not just situated in it. Hervé (Hervé, 2015) presents several of his works inspired by the concept of the biotope³. He emphasizes that when creating music, one must consider the specific location where the music will be played. The sounds of that place should be incorporated into the music, and the compositions should not be confined to the neutrality of concert halls. Instead, they should be conceived as works that seamlessly blend with gardens and other environments, accenting that electroacoustic tools are valuable assets in this creative process.

We can observe that discussions about interactive music in ecological contexts are beginning to encompass increasingly broad issues that undoubtedly transcend the artwork’s scope, particularly as the artwork becomes more intertwined with the human, social, biological, and aesthetic elements that constitute it. Brandon LaBelle (2021) explores multiple types and modes of sound action that can foster acoustic justice. In a comprehensive and integrative perspective, LaBelle traces how we audibly inhabit our environments and how our sound production is connected to political and economic powers, shaping our attention, what he calls “economy of attention”. He illustrates how the practice of a poetic ecology of resonance has transformative potential to cultivate alternative ways of living:

Poetic ecologies of resonance are suggestive for capturing this unique relationality-this acoustic potentiality that generates and gives way to the formation of other architectural experiences, other bodily orientations, as well as other pathways toward cohabitation, a being alongside and with the unfamiliar, even the unknowable. Resonance is posed, as Lucier and Wollny highlight, as the animation of static form, a performative transformation of space leading toward conditions of “musicality”-space as something to be heard. (Labelle, 2021, p. 82)

LaBelle draws, among others, on the philosopher Jean-Luc Nancy, who posits that the Self, as a body, arises from the act of listening a process that involves being penetrated by resonances, which even shapes the space itself. Interestingly, this notion is closely connected to the concept of a Self emerging from perception, as Gibson suggests.

2. Towards Gibson's Ecological Psychology

Interactive art has followed a path that has developed in parallel, sometimes influencing each other, with what has occurred in the scope of cognitive science. It is not surprising that various artists in interactive art reference thinkers from cognitive science, an interdisciplinary field that intersects with philosophy, computing, neuroscience, psychology, linguistics, anthropology, communication, and obviously, the arts. Concepts developed by Roy Ascott originating in Cybernetics, the initial paradigm of cognitive science, are evident. Simon Penny is influenced by Ross Ashby and Grey Walter’ cybernetic machine concepts, while also proposing to transcend the hierarchical dualism between matter and information - stemming from the cognitivist dualism underlying the initial stages of computing - present in interactive works like virtual reality or virtualized spaces. Penny draws on Humberto Maturana and Francisco Varela's Enactive and Embodied Cognition theory for this purpose (Varela, Maturana, Uribe, 1974; Varela, Thompson, Rosch, 2017). Similarly, Di Scipio relies on the concepts of self-organization and community of self-organized agents from cybernetics pioneer Heinz von Foerster (1961), as well as Maturana and Varela's concept of autopoiesis (Varela; Maturana; Uribe, 1974). Waters, in turn, mentions diverse canonical authors for cognitive science such as Peter Cariani (1992), George Lakoff and Mark Johnson (2003), Eric Clarke (2005), Humberto Maturana, and others.

The parallel paths of the arts and cognitive science are closely linked by the increasing emphasis on the central role of the body integrated into its environment both in theoretical and practical conceptions. The arts have been exploring interactive forms that allow the audience to actively participate in the creative process, engaging practically, politically, and socially with the artwork. Meanwhile, cognitive sciences have moved away from a Cartesian/cognitivist paradigm that conceives the mind as a purely mathematical-rational function detached from the body. Progressively, it has transitioned to a connectionist approach, drawing inspiration from studies of the brain's cortex types, which reestablish the connection between the mind and biology, emphasizing the importance of the body, and entailing the emergence of artificial neural networks. Furthermore, influenced by Ponty’s (Merleau-Ponty, 1945) and post-Ponty’s phenomenology (Jean Petitot et al., 1999), cognitive science has “delved” the mind into a body to paraphrase Merleau-Ponty that exists in a world.

This third paradigm has been given different names, depending on its origin, which has been grouped in what has come to be called 4E Cognition: Embodied, because cognition is considered to be in a body, not only in a brain; Embedded, because this body is in a world that is inseparable from it; Enactive, because cognition occurs within a body with a brain that acts in the world; Extended, because cognition does not reside only in the brain or body, but extends to and from the world (Newen; Bruin; Gallagher, 2018). One of the most important precursors to this paradigm is the American psychologist James Jerome Gibson, who shares the title of the father of Ecological Psychology with fellow psychologist Eleanor Gibson. Ecological Psychology is a field of study that seeks a non-metaphysical or dualistic explanation for the phenomenon of perception. Gibson defines perception as a dynamic process that arises from the mutualistic relationship between organism and environment, rather than a mental process in which a rational mind analyzes isolated objects of perception. This concept was central to understanding that organism and environment are not independent units, but rather entities that mutually constitute themselves and come to exist in this relationship. Considering that interactive art has increasingly sought to investigate the relationship between the body and its environment and cognitive science has scrutinized the concept of Self as something that exists in an inseparable connection with the world, which emerges from action made possible by that world, we believe that Gibson's Ecological Psychology could offer valuable contributions to discussions regarding interactive musical performance.

2.1. Theory of Direct Perception

Gibson’s research in psychology has led to a significant transformation in the understanding of perception and its mechanisms. According to the author, perception is a process that occurs within a mutually informative system that arises from the interaction between an organism and its environment. Gibson argues that even the most intricate activities, such as language and thoughts, are rooted in a foundational level he refers to as first-hand experience. He emphasizes that:

Words, spoken or written, and images, solid or flat, are components of the environment and sources of stimulation, I repeat. But they are the most difficult of all components or stimuli to define in scientific terms. They are facts of such high order that it is easy to be confused about them. They are so familiar to us, we fail to understand their complexity. We tend to think of direct stimuli from the terrestrial environment as being like words and pictures instead of realizing that words and pictures are at best man-made substitutes for these direct stimuli. Language and art yield perceptions at second hand. This second-hand perception no doubt works backward on direct perception, but knowledge about the world rests on acquaintance with the world, in the last analysis […] (Gibson, 1966, p. 28)

Traditional perceptual theories propose that perception is a cognitive process mediated by mental processing within the central nervous system. This processing serves to improve the raw data received from the sense organs, as these organs furnish incomplete and imperfect stimuli and require the enhancement of their inputs through perceptual filters. Both structuralist currents in the 19th century and Gestalt theory in the 20th century suggest that the central nervous system is responsible for enhancing the incoming stimuli received through the sense organs, which are conceived as neutral anatomical units transmitting captured impulses to central processing. According to these theories, perceptual meaning only emerges at the central processing level after all stages of correction and enhancement have been completed. For example, for traditional (Indirect) perception theorists an image captured by vision and formed on the retina is two-dimensional, while the real world is three-dimensional. The two-dimensional stimuli that reach the brain through the channels of perception are limited in comparison to the real world and require processing and interpretation by the central nervous system for meaningful three-dimensional perception to occur. Moreover, the visual perception of movements occurs through “snapshots” that capture each moment and store these sequences in short-term memory. Once a certain number of samples are stored, central processing can infer the movement of an object within our field of vision⁴. Gibson challenges this perspective presenting several important questions: How can impoverished stimuli be organized into meaningful experiences and later enhance the input stimuli? In the context of the disparity between the two-dimensional retinal world and the three-dimensional physical world, how the brain knows the world as three-dimensional when the perception is limited to two dimensions? Gibson argues that conventional theories of perception often inadequately capture the comprehensive nature of perception. They either concentrate on the characteristics of the object being perceived, retaining objectivist remnants, or they emphasize the mental attributes of the perceiver, leaning towards a highly cartesian perspective. As a result, no traditional theory of perception has delved into the essence of perception itself, which lies in the intricate relationship between the individual and the external world. Gibson argues that perceptual meaning is rooted in the afforded action that arises from the perception processes themselves, implying that perception serves the purpose of guiding action and vice versa. In this context, Gibson presents how to differentiate sensation from perception.

2.1.1. Medium, Substances and Surfaces

Theories of Indirect Perception define the sensation of time and space physically based on physical parameters processed by the central nervous system. In contrast, Gibson approaches the perception of space and time from an ecological perspective. In the context of space, physicist definitions supporting Indirect Perception theories divide the environment into various levels, from sub-atomic structures to galactic dimensions⁵. On the other hand, an ecological perspective focuses on what is perceptually available to animals in their relationship with the environment. In this view, divisions and dimensions do not explain or produce perceptual phenomena. Instead, what matters ecologically is how the medium can be: smelled, seemed, touched, heard, or tasted. Our perceptual systems⁶ are limited to a specific scale or range of objects and events, rather than operating in sub-atomic or galactic dimensions. Considering the time, what is usually regarded is on a scale that lies between sub-atomic events and light years. The crucial aspect of ecological time perception is not the abstract times derived from physical phenomena, but rather the ability to perceive time as a change, a process, or a transformation in the environment. Therefore, according to Gibson, space and time are primarily perceived through their qualities of persistence and mutation.

The animal can recognize and interact with its environment through what remains constant and what changes within it. While traditional physics divides matter into solid, liquid, or gaseous states, ecological psychology conceives it as Medium, Substances, and Surfaces. The interface between these three ecological properties allows organisms to have a diverse range of interactions with their environments. The environment can facilitate an animal's movement as a gaseous or aquatic environment provides minimal resistance to living beings' motion. As a result, these thin environments allow light to be reflected, enabling vision. On the other hand, a minimally resistant elastic medium allows mechanical vibrations to be propagated, enabling hearing. In gaseous and, to a lesser extent, liquid mediums, chemical diffusion can occur, allowing organisms to smell. Mediums with constant and relatively homogeneous components, such as the amount of oxygen, are extremely important for supporting life allowing organisms to breathe. Another important aspect of the environment is gravity, which provides most living beings with an absolute axis of reference above and below. In gaseous mediums, such as the atmosphere, and liquid mediums, such as seas and rivers, pressure always increases the further down one goes and decreases the further up along this absolute axis the organism is located.

For Gibson, substances are those that afford different types of relationships that distinguish them from the environment. The portions of the medium identified as substances exhibit much more variability and heterogeneity than those constituting the medium. They vary in aspects such as rigidity, density, viscosity, elasticity, and plasticity. Each of these variations enables living beings to perform various actions, such as eating, manipulating, and creating things. Water, for instance, serves as an interesting example of a substance or medium depending on the type of organism interacting with it. For aquatic organisms, water functions as a medium, while for terrestrial animals, it is a substance that can even be consumed.

Surfaces complete the triad in Gibson's description of the ecological environment, particularly in terms of persistence and change. Surfaces represent perceptual relationships that primarily involve the resistance or persistence to change of environmental components. If portions of the environment possess a certain level of homogeneous persistence and stability they can support the movement or stability of organisms, and thus be perceived as Surface. Surfaces exhibit a structure and layout that facilitate movement across them, reflect light in a distinct manner for recognition, and possess resistance to deformation, disintegration, and specific textural characteristics, among other attributes.

For Gibson, the meanings of the world are discovered through the possible actions that arise from the connection between a being and its environment. As he states:

The world of physical reality does not consist of meaningful things. The world of ecological reality, as I have been trying to describe it, does. If what we perceived were entities of physics and mathematics, meanings would have to be imposed on them. But if what we perceive are the entities of environmental science, their meaning can be discovered. (Gibson, 1979, p. 28)

Therefore, symbolic meaning is not created by a mind scrutinizing the world. As Gibson states: “No symbol exists except as it is realized in sound, projected light, mechanical contact, or the like. All knowledge rests on sensitivity” (Gibson, 1966, p. 26). Meanings emerge from an organism’s connection with the ecological world, representing the possibilities of action available to that organism. These possibilities are logically limited by the characteristic thresholds of their perceptual systems, which encompass the individual's life history and the evolutionary development of their species. We perceive what is possible for our species to perceive, and by extension, what is possible for the individual. This is made possible by the historical interaction of the individual with their environment and vice versa.

2.1.2. Theory of Information Pickup: Affordances and Invariants

One of the central points of Gibson’s Ecological Psychology lies in his definition of Information. Information is shaped by what individuals' perceptual systems are able to acquire in their environment. Certain transformations or persistencies in the environment are not available to some living beings, but are to others, so some phenomena are information for some organisms and not for others. Gibson conceives the concept of information by dividing it into two, or as he calls it: two inseparable bidirectional arrows, one pointing to the organism and the other pointing to the environment. One is what he calls affordances and the other invariants.

Affordances⁷ are the possibilities for action that a cognitive agent can acquire from its immediate environment. As he states: “The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill” (Gibson, 1979, p. 119). If a surface is reasonably horizontal, long enough, and rigid enough, it will allow some organisms to support themselves and move around, especially for terrestrial animals. The “support” for insects, for example, will be different because non-horizontal and non-extensive surfaces allow support and locomotion for this type of organism and are “afforded” as a surface. Therefore, affordances depend on the animal. The characteristics of this surface can be measured in terms of their physical attributes, but these features are not crucial for the emergence of affordances. Instead, their ecological characteristics are essential, as we mentioned earlier. Affordances are the objects of perception, constituting the perceptual “values” and “meanings”, which is why perception is conceived as an action-perception in which actions and perceptions are dependent on the characteristics of the individual's (and species') perceptual systems, incorporating ecological aspects. Therefore, perceptions are direct and do not require any mediating process, strongly opposing indirect perception theories that postulate that the agent becomes aware of the world and its properties through elaborate mental inferences that replace objective reality. After these processes of inference, the agent can conclude about the nature of the perceived object. The paradigm shift proposed by Gibson centers on the idea that there is no cognition without an individual in its world, and therefore there is no separate object of perception in this process.

In addition to affordances, invariants are characteristics of the environment that are maintained in some way, or which change following some kind of pattern directly perceived by individuals, depending on the capabilities and characteristics of their perceptual systems. Instead of the traditional psychological view in which movement is the mental processing of a succession of “nows”, organisms actually perceive a dynamic pattern of environmental transformation. The perception of a surface or texture is influenced by both the pattern of light reflection on the surface and variations in this pattern when the individual moves their gaze or due to changes in light in the environment. Understanding invariants can help elucidate two key aspects of Gibson's Ecological Psychology. The first concept is variation, which explains how the same perceived object can be seen in different contexts or from different angles and still be considered the same object. The second concept refers to the lack of need for inferences to explain perceptual phenomena. If there are specific characteristics (invariants) in the environment that are available to the animal's perceptual systems, there is no need for inferences for affordances to occur. All that is needed is for the animal to have its perceptual systems tuned up to perceive those invariants.

2.2. Partial Equivalence of Perceptual Systems

Gibson's concept of Partial Equivalence of Perceptual Systems can provide fruitful creative strategies when applied to Interactive Sound Installations. Traditional theories of perception usually isolate the sense organs by focusing on their functionalities. For Gibson, the sense organs (perceptual systems) are active and interrelated. The author points out that: “These five perceptual systems overlap one another; they are not mutually exclusive. They often focus on the same information  that is, the same information can be picked up by a combination of perceptual systems working together as well as by one perceptual system working alone” (Gibson, 1966, p.4).

Complementarily, as well as overlapping, they cooperate by establishing hierarchies, as exemplified:

Consider the main perceptual system, the visual one, and note its various levels of function. The single eye is a system of low order, although it is already an organ with an adjustable lens for sharpening the retinal image and a pupil for normalizing its intensity. The eye with its attached muscles is a system of higher order; it is stabilized in the head relative to the environment with the help of the inner ear, and it can scan the environment. The two eyes together make a dual system of still higher order; the eyes converge for near objects and diverge for far ones. And the two-eyes-with-head-and-body system, in cooperation with postural equilibrium and locomotion, can get around in the world and look at everything. (Gibson, 1966, p. 42)

Cooperation between perceptual systems is essential for the emergence of meaningful experiences in organisms. Gibson illustrates how multiple perceptual systems organized in: Basic Orienting System, Auditory System, Haptic System, Taste-Smell System, and Visual System  work together. The Basic Orienting System, utilizing the inner ear, detects acceleration forces that specify gravity and the start and end of bodily movements, while the Haptic System senses variations in gravitational forces. These systems collaborate to provide a reference frame for other systems. The Auditory System detects environmental vibrations in conjunction with the Visual System to determine the direction and source of the vibrations. The Haptic System, comprising numerous subsystems collaborates with the Visual System to enhance the understanding of tactile sensations. The Taste-Smell System inherently operates as a cooperative system, as activities like eating involve both tasting and smelling, along with tactile feedback from the haptic system for feeling food textures. Smelling also interacts with other systems, such as the visual system, when identifying the source of a scent. Finally, the visual system integrates with the other sensory systems, as we have already discussed. As an individual moves around or interacts with the environment, there is a continuous alteration in the environmental invariants due to changes in light within the visual field.

For the author, there is a partial equivalence of perceptual systems when they all receive the same informational stimulus, resulting in redundant information. This is exemplified by:

Consider a fire - that is, a terrestrial event with flames and fuel. It is a source of four kinds of stimulation, since it gives off sound, odor, heat, and light. It crackles, smokes, radiates in the infrared band and radiates or reflects in the visible band. Accordingly, it provides information for the ears the nose, the skin, and the eyes. The crackling sound, the smoky odor, the projected heat, and projected dance of the colored flames all specify the same event, and each alone specifies the event. [...] For this event, the four kind of stimulus information and the four perceptual systems are equivalent. (Gibson, 1966, p. 54)

Stimuli captured by only one of the perceptual systems - for example, only seeing the fire and not smelling it, hearing its sound, or feeling its heat - will trigger memories in the other perceptual systems (self-adjustments). The recalled memories will be based on a previous experience in which all perceptual systems, or at least more than one, experienced the phenomenon simultaneously. Gibson emphasized the importance of this contrast in distinguishing between what was experienced and what was remembered. For the author, these memories are self-adjustments⁸ actions occurring in perceptual systems throughout the organism's life and the species' evolutionary process.

3. The Interactive Sound Installation as an environment

Um outro lugar (Another Place) is a work in progress in a continuous process of re-composition that comprises a sound installation capable of capturing sounds and movements from both the audience and the environment⁹. The installation has been presented in various versions over the past three and a half years. It consists of a loudspeaker set that can vary from two to more depending on the space, a webcam for capturing the movement of the interactors passing through the installation site, one or two microphones, and a computer with an audio interface programmed in SuperCollider¹⁰, Processing¹¹ using OpenCV¹² library tools. The work is installed using elements of the environment, such as foliage, and obstacles, among others, to conceal the electronic equipment (loudspeakers, computer, audio interface, microphones, and cameras) and to avoid revealing interference in the space. The choice of location typically living areas like social spaces, public squares, and walkways, and diligence with the layout of equipment are essential to maintain the natural flow of people and prevent them from altering their behavior when interacting with the installation, which often occurs when the public is aware that they are participating in an artistic work.

FIGURE 1
The use of vegetation to conceal equipment in the site of one of the versions of the installation. The microphone is partially visible in the image. There are loudspeakers hidden within the same vegetation.

The installation programming consists of the following processes:

• Motion detection module: responsible for extracting motion information from a webcam, obtaining intensity, angle, and density information, scaling it, and sending it via OSC¹³ to the control modules.

• Controller modules: responsible for analyzing the motion capture information received via OSC protocol and rescheduling it for use in the audio synthesis and processing modules.

• Audio synthesis and processing modules: Including FFT analysis, processing, and resynthesis (iFFT) that vary according to the controller modules; sound synthesis of various types, and processing such as filters, and reverbs, among others.

• Memorization modules: consist of an audio buffer lasting approximately 8 minutes, which is responsible for recording audio from either the microphones or the synthesis and processing modules, which can overwrite or mix with what has been previously recorded, depending on the controller modules.

• Sampling modules: consist of pre-prepared audio buffers chosen and sent to the processing modules according to variations established in the control modules.

• Spatialization module: responsible for controlling all synthesis and processing flows for the installed loudspeakers which varies according to controller modules.

The organization of the modules ensures the core features of the work: self-organized and unpredictable behavior even in the absence of people or environmental movements detected by the sensors. The way sounds are created and processed changes based on how the interactors move, how intense their movements are, and how quickly they move, at the same time, there is still a high level of unpredictability in the way sounds are created and processed.

Delving into some technical aspects of the installation relevant to the discussion, the motion detector utilized the Gunnar Farneback¹⁴ algorithm from the OpenCV library (see Figure 2). This algorithm employs a dense optical flow object tracking method (dense means that algorithms compute optical flow for all the points in the image), which segments the image captured by the webcam into square sectors of x by x points (defined in the initial configuration) and detects movements within these regions, indicating both the direction and intensity of the movement. The algorithm gives an angle value in radians for the direction and a normalized vector for the intensity of movement for each sector of the image. These values are generated for each image frame based on the system's fps settings.

FIGURE 2
In the first image, we see an example of the motion detector (set to 20 by 20 points) capturing an arm moving mainly downwards¹⁵. The red figure marks the center of the area with the most prominent movement but serves only for debugging purposes. Below, the five images represent different moments showing how the up-to-down position of the arm oscillates one of SuperCollider's Control Channels.

The challenge we faced in our implementation was to interpret this data in an ecologically inspired manner, drawing from Gibson's theories. We aimed to transform this data into a flow that simulates the transformational invariants of the environment as closely as possible. Rather than focusing on mapping coordinates on a screen, we were interested in identifying movement patterns. Therefore, the data was processed to extract general directions of movement (upwards, downwards, right, left), the overall intensity of movement (ranging from aggressive to subtle), and whether the movements were concentrated in specific areas or dispersed throughout the space.

The transformation we implemented (inspired by the shift from the physical to the ecological level proposed by Gibson) utilized a feature offered by the SuperCollider language. In SuperCollider, data flows can be discrete or continuous¹⁶. Discrete streams are typically associated with code execution¹⁷, while continuous streams are responsible for audio synthesis. However, in SuperCollider, it is also possible to transform discrete numerical data into continuous streamings using Control Channels and Control Oscillators. Control Channels compute data similarly to audio channels but at slower sample rates to reduce computational costs. Instead of using logic operators to process information from the motion sensor, we directly link the values to control oscillators with their own limits and operating speeds. This way, we avoid using conditional statements like "if this value is equal to, greater than, or less than this value, do this or that". Each module exchanges information with the others, sending information based on its characteristics and receiving information based on its limitations, according to the configurations set in the Control Channels. This setup mirrors the concept of Partial Equivalence of Perceptual Systems, where a bundle of information arising from the organism's interaction with its surroundings simultaneously reaches various perceptual systems, enabling redundant affordances.

To elucidate the general behavior of the installation and help understand the subsequent discussions, we will provide a brief overview of how the installation operates and its interactive aspects. Positioned in a passageway to blend seamlessly with the environment in such a way that people don't notice any kind of interference in the environment, the installation initially replicates, slightly transformed, the ambient sounds it captures. Minor fluctuations occur based on changes in lighting or movements of environmental elements like vegetation, altering the processing module parameters. The sound reproduced by sampling modules may occur sporadically, with the frequency of these interferences correlating with the intensity of captured movements (motion detection module) organized by control modules. When an interactor passes through the installation, their faster movements compared to those within the space are detected and influence the synthesis modules, generating sounds closely mirroring the captured gestural dynamics, i.e., shorter sounds, with more prominent dynamics, standing out from the behaviors in the background sounds, with a more abrupt variation in pitch etc., inspired by Denis Smalley's Gesture Indicative Fields (Smalley, 1996). These sounds may or may not instigate the individual to explore the installation space. The individual becomes an interactor who starts trying to figure out the logic of what is happening, moving around, sometimes walking from side to side, trying to understand the correlations between the movements they make and the sounds that start to stand out from what should be the environment sonically. The way the modules exchange information with each other ensures a degree of unpredictability in the correlation between gestures and sounds, instead of a simple action-reaction behavior, maintaining the interactor’s curiosity. The gradual increase in activity captured by the motion sensors starts to transform the behavior of the modules more aggressively, generating more variability and unpredictability. The memorization module that records what happens in the environment is programmed to vary between recording or not and overwriting or not the data already in the buffer, depending on the levels of movement. This mode of operation allows the interactor to listen and recognize themselves¹⁸ as the producer of the sound event that the “installation's memory” remembers. This event never appears exactly in the same way because both the recording and its reproduction go through processing modules, which vary as they are influenced by the control modules. Over time, as more individuals interact with the work, the installation sonic behavior tend to become denser. On several occasions when the work was installed (always lasting more than 6 hours), an interesting dynamic sometimes occurred. Collectively, without any control proposed by any individual, the interactors began to slow down their movements and tried to remain silent. This gradually led to a decrease in the overall sound density, directing the installation back toward its initial state, blending again with ambient sounds. Subsequently, the interactors resumed “playing” with the installation. Several times, participants remained engaged in the activity for extended periods, sometimes exceeding an hour or two. In some cases, participants experimented with the equipment (when someone discovered them), particularly the microphones and webcam, and adjusted their approach. For example, some played music from their cell phones directly into the microphone, anticipating that the installation systems would transform these samples. Others started dancing for the webcam, exploring how different movements affected the sound output.

With this description in mind, we can now connect them to ecological psychology and the other concepts introduced at the beginning of this text.

4. Sound Installations and Gibson in interaction: some contributions from Ecological Psychology

Gibson’s Ecological Psychology, particularly the concept of affordance, guided the development of the installation, both the version reported here and the previous ones. The work explores the idea that meanings are the actions of organisms afforded by the environment. The modules were programmed as simple units that interrelate, generating behaviors that are not entirely predictable, in accordance with Penny and Di Scipio’s alerts of the cognitivist computing tendencies that have influenced interactive installations. We planned a certain level of correlation between actions and sounds, ensured by some of the modules, to encourage the engagement of the interactors. However, inspired by Rancière’s concept of the emancipated spectator, we aim to surpass the idea that the interactor must do something for the work to occur. In this sense, the work was planned to behave like the environment instead of manifesting aleatoric behaviors generated by randomization algorithms.

Jean-Luc Hervè suggests that “Thinking about a work of art in relation to its environment means situating it in its place”¹⁹ (Hervé, 2015, p. 1). He is looking for works that are intertwined with the environment itself. However, we believe that ecological psychology allows us to take it a step further. In Hervè's “garden works”, the interaction occurs in the environment, and the environment inspires the sounds that will be created to be reinstalled in that environment, engaging in a dialogue with it. In our case, the installation, inspired by Gibson's concepts, was conceived without extracting sounds or inspiration from an environment. The work was designed to become the environment itself and behave as such, encompassing everything that can (in this installation) be heard and seen, allowing behaviors to emerge that will, in turn, shape the installation itself. In the environment where the work is installed, certain actions and behaviors are permitted before its transformation through the installation, e.g., in a public square, individuals behave in ways that the environment allows them to. However, after the work is installed, new behaviors become affordable to the interactors. The work was designed to enable affordances that were not previously present in that space to emerge, rather than having sound events sequenced or triggered by predefined algorithms. In agreement with Di Scipio, we should aim to compose interactions rather than interactive music. Drawing from Ecological Psychology, we can further assert that in our installations, we strive to compose affordances. By creating affordances in sound installations, particularly those not tied to a specific environment, we offer participants a unique opportunity to reconsider their role within the environment. They can reflect on how they and the environment are intertwined in actions and relationships that emerge within the space, including their interactions with others.

As we have seen, interactive art has played a significant role in reshaping the interaction between the public and space. However, there is a certain emphasis on people’s connections with the space, as highlighted by the various authors we have discussed. They usually point out the importance of considering the public's actions integrated with the environment, but there don't seem to be many tools to delve into the actions themselves. LaBelle's text is one of the few to focus on sound performativity, on an acoustic agency as a tool for transforming acoustic reality. As he states: “Acoustic Justice addresses questions of sonic agency by making a number of key moves in order to extend the work of listening as the basis for civility (...)” (LaBelle, 2021, p. 10). Most strategies mentioned in the literature focus on action from a positive perspective of integration with the environment. However, the actions themselves do not appear to directly impact the poetics of creating the works; they are generally treated as consequences of the installations. On one hand, when actions are included in installations, they are usually done through action-reaction strategies as pointed out by Penny and Di Scipio. On the other hand, Graham (1997) demonstrates that on the scope of art, there is a greater emphasis on considering the artist and their production rather than the audience's experience. In this sense, the strategy we selected for the installation aimed to create a space suitable for the audience’s body-sound action integrated with the environment. To reshape the way individuals engage with the medium, as LaBelle suggests, our installations aim to manipulate affordances as a means of transforming people's habitual everyday behaviors and routines, which are often influenced by unjust political, social, and economic structures. We believe that the root of such injustices lies in resistance to overcome Cartesian assumptions as discussed by various authors from Arts, Philosophy, Cognitive Science, Social Sciences, etc., which have been ingrained in Classical Cognitivism and continue to persist in political, economics, certain artistic and even some ecological perspectives.

5. Conclusions

Understanding, as Gibson suggests, that perception is action and the environment emerges from the mutualistic relationship between its components, including organisms, instead of being a conceptual abstraction that separates the organism on one side and the environment on the other (to be further studied together), allows us to indicate that perhaps this is an efficient way to foster the reduction of acoustic violence. This understanding reinforces the idea that beings and the world are co-dependent. Although this concept is widely accepted in ecology, it is often overlooked in other areas, such as music, politics, and especially in common sense. It is important to recognize the inseparable connection between beings and their environment to promote harmony and reduce conflicts. The concept of affordance is central to this understanding, as it emphasizes that the meanings of the world are shaped by the actions enabled by the world itself in our inseparable connection with it. This completely transforms our perspective from acting “in” the world - which presupposes an objectivist and even extractivist approach towards the world - to acting “with” the world. This is the central motivation of our installation. At the same time, composing affordances involve a fundamental shift in musical composition strategies, focusing on the interactor's actions rather than organizing musical parameters as discretized in music theory for someone to interpret. There is still a lot of artistic research to be done, but we believe this is a promising first step towards emphasizing ecologically oriented action in the context of Interactive Sound Installations.

Art, especially in the context of power and technological fetishization, as Santaella (2003) points out in "Cultures and Arts of the Post-human," has the power to explore the fractures and problems of our time. It is the responsibility of those creating in technological contexts to be aware of the euphoria and dysphoria, as the author terms it, of these contexts and their positive and negative aspects. If contemporary societies have been so strongly detrimentally influenced by technology, art has the potential to occupy this space and redefine these relationships, as interactive sound installations artists and researchers have done. In this sense, we believe that Gibson's Ecological Psychology, which is a fundamental part of acoustic ecology, can contribute to the creation of interactive sound installations that resonate more fairly and positively and enhance our relationships, to paraphrase LaBelle.

ACKNOWLEDGMENT

We would like to thank all the members of the Interdisciplinary Group of Interactive Art (GRITARIA: GRupo de estudos InTerdisciplinar de ARte InterativA) who have been researching and creative partners for years and who actively allowed the work described in this research to exist, especially the composer and partner of one of the versions, Camila Fernanda Silva de Souza.

REFERENCES

• ASCOTT, Roy. Behaviourist art and cybernetic vision II. Cybernetica, v. 10, n. 1, p. 25-56, 1967.
• ATTALI, Jacques. Bruits. Essai sur l’économie politique de la musique. Paris: Presses universitaires de France, 1986.
• BONFIM, Cássia Carrascoza. Telematic Music: Six Perspectives. Leonardo Music Journal, v. 19, p. 95-96, 1 Dec. 2009.
• CARDASSI, Luciane.; BERTISSOLO, Guilherme. Colaboração compositor-performer: uma proposta de metodologia. Anais do XXIX Congresso da Associação Nacional de Pesquisa e Pós-Graduação (ANPPOM). Proceedings. In: XXIX CONGRESSO DA ASSOCIAÇÃO NACIONAL DE PESQUISA E PÓS-GRADUAÇÃO. Pelotas: Associação Nacional de Pesquisa e Pós-Graduação em Música, 2019.
• CARIANI, Peter. Emergence and Artificial Life. Artificial Life II - Santa Fe Institute Studies in the Science of Complexity, v. 10, p. 775-797, 1992.
• CLARKE, Eric Fillenz. Ways of Listening: An Ecological Approach to the Perception of Musical Meaning. Oxford; New York: Oxford University Press, 2005.
• CORNOCK, Stroud.; EDMONDS, Ernest. The Creative Process Where the Artist Is Amplified or Superseded by the Computer. Leonardo, v. 6, n. 1, p. 11, 1973.
• DI SCIPIO, Agostino. ‘Sound is the interface’: from interactive to ecosystemic signal processing. Organised Sound, v. 8, n. 3, p. 269-277, 2003.
• DINKLA, Söke. The history of interfaces in interactive art. Proc. of the 1994 Int. Symp. on the Electronic Arts (ISEA). Proceedings. In: ISEA94: FIFTH INTERNATIONAL SYMPOSIUM ON ELECTRONIC ART. Helsinki, Finland: ISEA, 1994. Available at: https://www.kenfeingold.com/dinkla_history.html Accessed on: 01 Mar. 2023.
  » https://www.kenfeingold.com/dinkla_history.html
• ECO, Umberto. Obra Aberta: forma e indeterminação nas poéticas contemporâneas. São Paulo: Perspectiva, 1990.
• GARNETT, Guy. The Aesthetics of Interactive Computer Music. Computer Music Journal, v. 25, n. 1, p. 21-44, 2001.
• LAKOFF, George.; JOHNSON, Mark. Metaphors We Live By. London: University Of Chicago Press, 2003.
• GIANNETTI, Cláudia. Estética digital: sintopia da arte, a ciência e a tecnologia. Belo Horizonte: C/Arte, 2006.
• GIBSON, Eleanor Jack. Principles of perceptual learning and development. New York: Appleton-Century-Crofts, 1969.
• GIBSON, James Jerome. The Senses Considered as Perceptual Systems. London: Gorge Allen & Unwin LTD, 1966.
• GIBSON, James Jerome. Ecological Approach to Visual Perception. Hillsdate: Lawrence Erlbaum Associates Publishers, 1979.
• GLUSBERG, Jorge. A arte da Performance. São Paulo: Ed. Perspectiva, 2011.
• GRAHAM, Berly. A Study of Audience Relationships with Interactive Computer-Based Visual Artworks in Gallery Settings, though Observations, Art Practice and Curation. Doctor of Philosophy-London: University of Suderland, 1997.
• GRAHAM, Berly. Histories of Interaction and Participation: Critical Systems from New Media Art. In: BULLEN, J. B. (Ed.) Performativity in the Gallery: Staging Interactive Encounters. Cultural Interactions Studies: in the Relationship between the Arts. Oxford: Peter Lang, 2014. v. 31, p. 65-84.
• HAYDEN, Sam.; WINDSOR, Luke. Collaboration and the composer: case studies from the end of 20th century. Tempo, v. 61, n. 240, p. 28-39, 2007.
• HERVÉ, Jean-Luc. La musique et son biotope. Analyse musicale, v. 76, n. June, 2015.
• PETITOT, Jean; VARELA, Francisco J.; PACHOUD, Bernard; ROY, Jean-Michel (EDS.). Naturalizing Phenomenology: issues in contemporary Phenomenology and Cognitive Sciences. Stanford: Stanford University Press, 1999.
• LABELLE, Brandon. Acoustic justice: listening, performativity, and the work of reorientation. New York City: Bloomsbury Academic, 2021.
• MERLEAU-PONTY, Maurice. Phénoménologie de la perception. Paris: Editions Gallimard, 1945.
• MAMEDES, Clayton Rosa. Design sonoro e interação em instalações audiovisuais. Thesis (PhD in Music)-Campinas: Universidade Estadual de Campinas, 2015.
• MAMEDES, Clayton Rosa. Participação em Instalações Interativas. Art&Sensorium, v. 5, n. 1, p. 39-54, 10 June 2018.
• MICHAELS, Claire F.; CARELLO, Claudia. Direct Perception. Englewood Cliffs: Prentice-Hall Inc, 1981.
• NEWEN, Albert.; BRUIN, Leon de; GALLAGHER, Shaun. (EDS.). The Oxford Handbook of 4E Cognition. Oxford: Oxford University Press, 2018.
• PENNY, Simon. Emergence, Agency, and Interaction-Notes from the Field. Artificial Life, v. 21, n. 3, p. 271-284, Aug. 2015.
• PLAZA, Julio. Arte e interatividade: autor-obra-recepção. ARS (São Paulo), v. 1, n. 2, p. 09-29, Dec. 2003.
• RANCIÈRE, Jacques. The Emancipated Spectator, Five Lessons in Intellectual Emancipation. New York: Stanford University Press, 1990.
• ROWE, Robert. Interactive Music Systems. Massachusetts: The MIT Press, 1993.
• ROWE, Robert. The Aesthetics of Interactive Music Systems. Contemporary Music Review, v. 18, n. 1, p. 83-87, 1999.
• SALTZ, David Zucker. The Art of Interaction: Interactivity, Performativity, and Computers. The Journal of Aesthetics and Art Criticism, v. 55, n. 2, p. 117, 1997.
• SANTAELLA, Lucia. Culturas e artes do pós-humano: da cultura das mídias à cibercultura. São Paulo: Editora Paulus, 2003.
• SCHAFER, Raymond Murray. A afinação do mundo. São Paulo: Ed. Unesp, 1997.
• SMALLEY, Denis. The listening imagination: Listening in the electroacoustic era. Contemporary Music Review, v. 13, n. 2, p. 77-107, 1996.
• SOLOMOS, Makis. From music to sound: the emergence of sound in 20th- and 21st-century music. Abingdon, Oxon; New York, NY: Routledge, 2020.
• VARELA, Francisco. J.; MATURANA, Humberto; URIBE, Ricardo. Autopoiesis: The organization of living systems, its characterization and a model. Biosystems, v. 5, n. 4, p. 187-196, 1974.
• VARELA, Francisco. J.; THOMPSON, Evan; ROSCH, Eleanor. The Embodied Mind: Cognitive Science and Human Experience - Revised Edition. Massachusetts: MIT Press, 2017.
• VON FOERSTER, Heinz.; PASK, Gordon. A Predictive Model for Self-Organizing Systems, Part I. Cybernetica, v. 3, n. 1, p. 258-300, 1961.
• WATERS, Simon. Performance Ecosystems: Ecological approaches to musical interaction. EMS07 - The ‘languages’ of electroacoustic music - Leicester. Proceedings. In: EMS07. Leicester: EMS, 2007. Available at: http://www.ems-network.org/IMG/pdf_WatersEMS07.pdfb Accessed on: 01 Mar. 2023.
  » http://www.ems-network.org/IMG/pdf_WatersEMS07.pdfb

• 1
  “ela implica física, psicológica e sensivelmente o espectador em uma prática de transformação.”
• 2
  In his analysis of music as a cultural form closely linked to modes of production, Attali divides musical practice into four cultural stages: a) sacrificial, which encompasses prehistory. During this stage, music was part of the oral tradition, and its production aimed to counter the noises of nature, death, chaos, and destruction; b) representative, which includes forms of musical production from the period of the development of writing, where the musician becomes a “producer and seller of signs”; c) repetition, which began with the advent of recording, where the musician's performance became a reproduction of the recording; and d) composition, which is not deeply defined by Attali. This stage corresponds to the post-repetition period, where re-use and remixing are available to everyone. It can represent both the search for novelty in the modes of production and their tragic continuity.
• 3
  In ecology studies, a biotope is a living place defined by its physical and chemical characteristics under relatively uniform conditions, including all flora, fauna, fungi and microorganisms.
• 4
  For Michaels and Carello (1981), traditional theories of perception (indirect) are impregnated with remnants of the Euclidean view of movement.
• 5
  In the case of music, much of musical theory is supported by indirect theories of perception that discretize pitch, intensity, and duration into physical measurements such as Hz, dB, and seconds.
• 6
  The replacement of sense organs by perceptual systems is central to J. J. Gibson's theory. To the author, traditional theories of perception view sense organs as passive morphological units that only and neutrally transmit stimuli to the brain's processing. In contrast, Gibson posits that perceptual systems are active entities with unique characteristics. These systems dynamically adapt and self-adjust to better capture information from the environment, are situated within a mobile body, and evolve according to the individual's life experiences and the species' developmental history.
• 7
  A neologism created by the author from the verb To afford.
• 8
  The self-adjustment processes of perceptual systems to generate affordances are linked to the learning theories developed by Eleanor Gibson and extend beyond the scope of this text. For an introduction to perceptual learning, see: (Gibson, 1969)
• 9
  Videos of excerpts captured during one of the performances can be accessed at the composers page: https://raelgimenes.com
• 10
  SuperCollider is both a programming language and a development environment for digital audio synthesis, processing and algorithmic composition. For more information visit: https://supercollider.github.io/
• 11
  Processing is both a programming language and a development environment Java-based for audio-visual programming. For more information visit: https://processing.org/
• 12
  OpenCV is an open source library for image processing and Computer Vision, available in versions compatible with the most popular programming languages such as C++, Java, Python, among others. For more information visit: https://opencv.org/
• 13
  According to Open Sound Control Specification 1.0 (see: https://opensoundcontrol.stanford.edu/spec-1_0.html#introduction), “Open Sound Control (OSC) is an open, transport-independent, message-based protocol developed for communication among computers, sound synthesizers, and other multimedia devices.”
• 14
  For more information about the Gunnar Farneback algorithm in the OpenCV library, see: https://docs.opencv.org/3.4/dc/d6b/group__video__track.html
• 15
  The example provided here was generated in the laboratory and was not captured during installation. Since the equipment is concealed, it is not feasible to capture screenshots in the installation site.
• 16
  In the context of discrete computing systems, continuity is obviously virtualized.
• 17
  Unlike visual languages like MaxMSP or PureData, SuperCollider is a text-based interpreted programming language.
• 18
  The author informally collected feedback from the interactors as they left the work, inviting them to share their thoughts on the experience if they wished.
• 19
  “penser une œuvre d’art en relation avec l’environnement c’est donc la situer dans le lieu”

• Guest Editors:
  Dr. William Teixeira; Dr. Jacek Smolicki.

Publication Dates

• Publication in this collection
  04 Apr 2025
• Date of issue
  2024

History

• Received
  29 July 2024
• Accepted
  10 Oct 2024
• Published
  03 Dec 2024