Seeing Sounds and hearing movements

Researchers from the Móbile project go on tour to show the results of blending art with technology

Lílian Campesato in the work Conexões dispersas/dispersões conexas (Scattered connections/Connected scatterings), 2011

Imagem: Public domain image Lílian Campesato in the work Conexões dispersas/dispersões conexas (Scattered connections/Connected scatterings), 2011Imagem: Public domain image

Published in May 2013

When the Móbile project was started in 2009, Fernando Iazzetta of the University of São Paulo’s School of Communications and Arts (ECA-USP) described his intention to “combine theoretical and artistic production, making it possible to develop creative work in our proposal.” Near project completion, Móbile researchers demonstrate their dedication to that promise by showcasing the Móbile project on an international tour from which they have just returned.

Five groups conducted presentations during the tour: the Sonic Arts Research Centre at Queen’s University in Belfast, Ireland; the Seia Conservatory of Music and the University of Aveiro in Portugal; La Haceria in Bilbao, Spain; and Hangar in Barcelona, Spain. “For the first time, FAPESP funded a tour of musicians abroad. We saw that in addition to the published works, the artistic component generated by our research was as important as the written results,” explains Iazzetta, author of Música e mediação tecnológica (music and technological mediation) from the publishing company Perspectiva.

In addition to the project coordinator, Lílian Campesato, Michelle Agnes, Julian Jaramillo, Rogério Costa and Vitor Kisil comprise the members of Móbile. The professors and students who were not able to attend the tour were represented by the artistic production created as a result of their thematic research. The show, which was named Transparência (Transparency), consisted of six scenes in which “traditional” works were combined with instruments and sheet music. Three works employed improvisation, and the three works were based on an attempt to create interaction among music, technology and other types of art.

Móbile Ensemble at the show Por trás das coisas, (Behind things) October 2010

Imagem: Public domain image Móbile Ensemble at the show Por trás das coisas, (Behind things) October 2010Imagem: Public domain image

In the interactive scenes, the group used a black table on which a mini camera was installed and operated by Lílian Campesato. “She manipulates objects so there are variations in the projection of sound and images, for example. In another part, we show a film of a subway platform. On stage, Lílian appears to be tearing pieces of paper, which above her come to slowly reveal themselves as the image that goes with the film,” says Iazzetta. “We attempted to pass on this feeling of revelation by making the images transparent,” says the musician.

Móbile’s objective is to assemble researchers from the fields of music, visual arts, performing arts, and computer and engineering science to develop new musical processes that focus on the integration of various unconnected sectors. In particular, the project questions the obsession with technology, whereas the initial experimental models focused on studios and high-tech equipment. “There was a time in 20th-century music when exaggerated enthusiasm for novelty emerged. Technological advances were revered as if they were the stars of the show on their own. For us, what matters is the result,” says Iazzetta. “Often, a lot of technology can even cause interference. We took a low-tech approach and used the most efficient technology we could find in everyday life. The most complex link has to be artistic thought rather than engineering.”

The researcher cautions that although this concept may not be original, the mystification with advanced technology is prevalent even in major music research centers. He believes that the reassignment of the role of virtuosity, which has divided creators and audiences into two separate groups, and a certain distancing in terms of tradition results in the combination of the music of other artistic models with various arts. “It is in this context that we are able to develop interactive music because the only way to access this new form is through the full sensory experience,” says Iazzetta.

Móbile Ensemble presenting a work at the ¿Música? 5 at the University of São Paulo, in 2012

Imagem: Public domain image Móbile Ensemble presenting a work at the ¿Música? 5 at the University of São Paulo, in 2012Imagem: Public domain image

The basis of the project is a reaction against technology as a solution to every problem without disregarding history. This is the case with music. The term “electroacoustical music” emphasizes the technology used by this genre. This notion was initially criticized in the 1980s. The idea is to remove art from its pedestal, which isolated it as something for the few and by the few, present it as an accessible and ironic game and make music and its creation compatible with daily life. Technological mediation facilitated the connection between sound and visual elements.

“The audience at a concert, for example, looks on passively at something they are unable to do or fully understand, in total awe and from a distance. Even with a piano, people do not really understand the complex mechanics,” states Iazzetta. Technology can compensate for this lack of virtuosity. It can positively forge a “new amateurism”: the ability for anyone to create music. The asymmetry between persons who create and persons who appreciate the work of art would disappear. Machines would replace technique and experimentation would replace tradition. “Playing with machines is an experience in which art tends to stop being art. These are artistic proposals that leave technique behind and focus on the experience of playing with materials. In turn, art is increasingly immersed in the use of technological junk, and, paradoxically, the technique itself gets diluted,” notes Iazzetta.

Michelle Agnes playing prepared piano music at the ¿Música? 3 event at the Cultural Center in São Paulo in 2011

Imagem: Public domain image Michelle Agnes playing prepared piano music at the ¿Música? 3 event at the Cultural Center in São Paulo in 2011Imagem: Public domain image

However, these ideas derive from technology for the paradox to work. “Our work complements the musicians. We have many questions about the sound and they are objective, carrying no aesthetic bias. These are signals that are treated scientifically,” observes Marcelo Queiroz, professor of computer science at USP and a member of Móbile. In addition to technological knowledge, Queiroz has a degree in composition from ECA-USP. “But when I’m on the ‘other side’ I just work with the scientific variables in search of a horizontal partnership with the group’s musical research,” says Queiroz.

Queiroz indicates that this interaction occurs whenever an artistic question generates a technical challenge, such as the analysis of voice signals. “I prefer to see music and art as places where problems of technical interest that require technical solutions arise naturally. After all, since computers are more flexible, they offer more opportunities to expand upon and experiment with sounds and signals than by creating new instruments,” says Queiroz. For Queiroz, music and the visual arts of the past are following a similar trend. “From the standpoint of sound data, we are able to modulate human hearing and transpose it into a computational tool. We will soon have the same sound synthesis that has been achieved in the past by using graphics programs,” states Queiroz. He alludes to computer-generated simulations of acoustic spaces, in which a group can be heard in a room that does not even exist.

Cesar Villavicencio playing the hyperflute

Imagem: Public domain image Cesar Villavicencio playing the hyperfluteImagem: Public domain image

“Working on the Móbile project has been a unique opportunity for us. For the first time, we have succeeded in assembling an interdisciplinary team of scientists, artists, and technology, and art students working together on research in cutting-edge artistic production,” says Fabio Kon, also from the Department of Computer Science at USP and a member of the Móbile project. “Since the project began, we have sought to approximate the artistic creation of cutting-edge production technology and exploit the synergies between these two forms of knowledge. This is no small feat because artists and technicians tend naturally to work in isolation, but this project has taken a first step towards changing this. We are pleased with the results, but there is still much to be done and the road ahead is long,” says Kon. He also notes that “this is a unique way to produce and disseminate knowledge that does not fit into the traditional patterns of research funding.” According to Kon, everything transcends musical studies.

“There is much talk about technology and machines as the opposite of human beings, as though the soul of music was sold for a penny. But to imagine that a machine takes what is human in music is to forget that nothing is more representative of human beings than the machines we make,” notes Iazzetta. Musicians treat the relationship between music and technology in terms of dependency rather than in terms of symbiosis.

According to Iazzetta, today’s use of technology is no longer essential as in the early days of electroacoustical music. Instead, it has become incidental. Technology is no longer the focus and can be used as a tool to create interest and demystify the act of musical creation.

Móbile: Interactive Musical Processes (Nº. 2008/08632-8); Type Thematic Project; Coord. Fernando Iazzetta (USP); Investment R$515,936.56 (FAPESP).