Calendar: 2006-07 | 2005-06 | 2004-05 | 2003-04 | 2002-03 | 2001-02 | 2000-01
OP_ERA: sonic dimension / Daniela Kutschat and Rejane Cantoni
| January 14 - March 19, 2005 Opening reception: January 13, 6 - 9 p.m. |
 |
 |
 |
OP_ERA: sonic dimension is an immersive and interactive installation designed as a music box. The box is a black, open cube (three walls of 3x3m each) filled with hundreds of visually identical violin-like strings. Tuned with the specific tension, each virtual string vibrates with a visual-sound frequency (light and sound waves) that varies according to its relative position and mode of interaction. The object of this project is to research the geometry of sonic spaces and to develop sonic interfaces.
In OP_ERA: sonic dimension there are two modes of interaction. In the first case, the input interface is a 360º microphone that collects the user’s sounds. When the microphone picks up a sound, the software analyses, filters and converts it into equivalent visual outputs. If the user sounds like an F, he will see the relative F frequencies vibrating as strings according to Pythagoreans principles. In the second case, the input interface is an array of sensors. Used to detect the user’s position this interface allows the system to interpret any action as a gravitational force. For example, when the user points to a string set, the action not only vibrates the selected strings, but it also changes the shape of space time.
OP_ERA is an immersive and interactive interface designed for multisensorial experimentation of space concepts. Conceived as a virtual reality environment, this project focuses on research and development of: (1) scientific and artistic models of space (the first version encompasses: 1D, 2D, 3D, 4D); (2) human-machine interfaces (hardware and software) specially designed for environments where a human agent and artificial engine are symbiotically interconnected; and (3) alternative ways of spatial perception and cognition.
Immersive-interactive environments, a mix of physical and data spaces, are more than powerful engines to promote spatial illusion; rather they are promises of interfaces through which human and machine communicate symbiotically. In the optimal case, in these environments the 'natural' behaviour of a human agent is associated with the computer’s 'artificial' behaviour in an inseparable way. Any action taken or contact made under those circumstances equally generates understandable information to any of the parts.
It brings up an old problem of human-machine interfaces. How and through what kind of interfaces one system may best interact with another? Or, from the perspective we know best, through what kind of interfaces may we enter a data world without being disturbed by unnatural (weird) devices?
Inspired by such questions we have being developing (since 1999) an immersive interactive environment named OP_ERA . It stands for a world, shaped as a set of interconnected logical dimensions. The dimensions –which derive from scientific and artistic concepts of space— were conceived to generate spatial cognition through multisensorial experimentation of space models that evolve in relation to the human body.
OP_ERA consists of interconnected dimensions structured by logical linkages. Each dimension leads exclusively to the next one and simultaneously to all previous ones. In a sense, OP_ERA has a beginning, a narrative hierarchy when going from the first dimension to the fourth, but it has no end or any kind of narrative path when going from a higher dimension to a lower one. Such a structure intends to create feedback loops, which allow events occurring in lower dimensions to affect the outcome of events in higher ones.
The integration of the human agent and the environment follows the same geometry of the dimensions. The whole scheme is structured in increased levels of space/interface complexity. The higher the dimension, the more the agents are integrated —expanded or interfaced— in a way that not only space becomes more complex but also the interagents' semiotic relations become ever greater. For example, if the interactor proceeds through dimensions 1D to 2D, the prior dimension is incremented to the next (2= 2+1), a rule that also correspond to the integration of the body: Dimension one requires auditory interfaces; Dimension two auditory plus visual interfaces; Dimension three auditory plus visual plus haptic interfaces; and so on.
Dimensions
The four dimensions in OP_ERA (2001 and 2003), namely X, XY, XYZ and XYZT relate to the history of concepts of space. As in “Lineland” in Abbott’s romance [1], X is a straight-line realm. Space is a finite segment composed by a multitude of points that are sound-based elements. These sound elements represent pre-programmed computational objects that compose the world —sounds. Their nature is to transmit —attack, sustain or release from reverberation to echo— sonorous information. In this dimension, the user distinguishes the shape of space and his or her relative position in it, emitting and receiving sound information. Interaction, or space cognition, are limited to ear perception. The overall concept of space is placed in reverberation.
The second dimension, XY, is a flat realm. The shape of space extends in two dimensions: the ‘imported’ one, X, or length, plus width. Like a huge flat screen it contains all the objects of the world, which ‘exist’ only within the limits of length and width. Within this dimension of XY are four orientation points —N, S, E, W. Therefore, objects and the human interactor are free to move in four directions —up or down, right or left. All objects independently of their nature (sounds, shapes and avatars) are rendered as light waves but perceived as vibrating lines with all occupants of this dimension, including the user, having a common boundary –a space confined to a finite and limited plane. Touching allows the human interactor to know the nature of an object, if it is a shape or a sound element. Since in XY, space is confined to a finite and limited plane, the logic follows that if we try to exceed its limitations we will step out of it.
The third dimension, XYZ, is a cubic realm. Space is essentially an empty box. Three linear dimensions —X, XY and depth— extending vastly as a limitless void in which all things are contained and through which they move. Within this imaginary box three forms —a green triangle, a red square and a blue circle— perform a kind of Oskar Schlemmer’s mathematical ballet. All forms have ‘behaviors’ attributed to them. They translate randomly according to the intrinsic qualities of their shapes. The triangle moves through the diagonals, the square through the orthogonal axes and the circle by rotating like a satellite. Without users’ interaction, that order would go on forever. As the human body is incorporated into the spatial scheme, the choreographic algorithm tracks its presence and triggers responsive events by changing and flipping the plane and direction.
In the fourth dimension XYZ and T are a time evolving landscape. Space evolves as a condensation of all three realms —X, XY, XYZ— composed by a manifold of emerging Lorentz attractors evolving in time according to the interactor's position. In this dimension, space visualization and cognition is only possible through simulation.
Implementations
OP_ERA is an ongoing research project. We have been developing it since 1999. The actual implementation won two national art and technology awards in Brazil: in 2002, the Itaú Cultural Transmídia; in 2003, the 4º Prêmio Cultural Sergio Motta and in 2005 a grant for exhibition at the Beall Center for Arts and Technology.
The implementation of the project started in 2000. At that time there was no digital CAVE in Latin America; so the solution was to develop a computer with three graphic outlets, a three-wall projection system and a custom made tracking device that simulated a portable CAVE.
The first public demonstration of this prototype took place on 24 and 25 May, 2001, at Centro Cultural Banco do Brasil (CCBB) in Rio de Janeiro [2].
In 2003, with the Transmídia-Itaú Cultural grant for production, we developed a second prototype [3] for an automatic virtual environment – the CAVERNA Digital of the LSI, at the University of São Paulo
This 2003 implementation also focused the development of a custom software and hardware architecture to model the visual (stereoscopic) and sound dimensions in the digital CAVE system.
OP_ERA is an ongoing research project and we are now developing new approaches for the 4th dimension. As in the previous implementations, the logic and esthetics of this work is also guided by the research and experimentation with space concepts, language code and human-computer interfaces (software and hardware development). For example, to promote alternative ways of spatial perception and cognition, this time, we are developing interfaces, machines and sign systems that translate space-time concepts into visual-sonorous-tactile information to the user.
The development of human-computer interfaces that enable a symbiotic interaction between human and artificial agents is a very important subject of our research. Up to now, it has led us to design and develop alternate tracking systems and experiment with immersive and interactive environments (virtual reality systems).
Examples are: OP_ERA: hyperviews [2004 > OP_ERA: hyperviews] , OP_ERA: haptic wall [2004 > OP_ERA: haptic wall] , OP_ERA: haptic interface [2004 > OP_ERA: haptic interface] and OP_ERA: sonic dimension [2005 > OP_ERA: sonic dimension].
The ultimate objective of our work and continuing research is to improve the way we perceive and acquire knowledge of space-time phenomena.
Who can benefit from this research?
Besides the aesthetic experimentation, other possible applications of our work include:
1. educational features (e.g., immersion into space-time concepts, interaction with scientific and artistic models, simulations of mathematical and physical space dimensions (1-11D);
2. research material for cognitive sciences (e.g., tests and training through multisensorial experience);
3. further developments in design and technology (e.g., alternate ways for human-computer interaction).
Biographies:
Daniela Kutschat, of São Paulo, Brasil, is an artist, lecturer and researcher. PhD in Visual Arts, she coordinates the Post-Graduate Studies in Interactive Media of SENAC in São Paulo. Her research focuses computer systems, digital media, cognition and human-computer interfaces. She also develops body-space integrating systems for interactive and VR environments. More recently, the project OP_ERA was sponsored by Transmídia, Itaúcultural (2002) and awarded with the 4th Prize Sergio Motta (2003). More about her work and exhibitions can be found at www.danielakutschat.com .
Rejane Cantoni is an artist and new technologies researcher who develops human-machine interfaces within intelligent environments. She has a Ph.D. in Communication and Semiotics from the Catholic University of São Paulo, where she works as an Assistant Professor in Technology and Digital Media at the Computer Science Department. www.rejanecantoni.com