Creation 2015

Artistic research context
METABODY addresses the importance of non verbal communication and embodied expressions for cultural diversity as a fundamental form of cultural heritage that is not adecuately taken into consideration, and which is being undermined by current information technologies, which induce unprecedented forms of homogenisation of non verbal expressions while subjecting people to an increasing control, thus undermining fundamental freedoms.

Piece concept
An intimate and cosy small space. 2 persons sitting on chairs, facing each other.
A Meta-Interviewer sits in the room and invites the visitor into a nonverbal conversation.

The Meta-interviewer mediates the experience by guiding the interaction with discrete facial gestures. Micro movements and breathing echoes on the walls of the room. Everything is visible and audible. This is not a place to hide. What is happening is interaction in real time. Right here, right now.
Using motion tracking and sensors, each gesture triggers light or sound and change the atmosphere of the space. As it becomes obvious to the visitor that his/her movements also control light and sound, the performance becomes an exchange, a non verbal conversation. Along the way, the interaction becomes tactile and enter the private space of the body to challenge its intimacy.
The nonverbal and touch will give the interaction a particular rhythm and feeling, thus a unique experience to each visitor.

Duration : 2-3 min.
No phones are allowed in the Meta-interview installation.

This piece is a collaboration between Instituto Stocos and Palindrome Dance Company e.V.
Artistic Concepts: Delphine Lavau and Robert Wechsler
Musical Composition: Pablo Palacio (Instituto Stocos)
Technical developments: Palindrome, Stocos, Steim, Marcello Lussana

Premiered at Medialab Prado, IMF Madrid, 22-24 July, 2015


with the support of the culture programme of the European Union








We have noticed in increasing detail and sophistication, the aspects of human movement which, when sonified, are most meaningful to movers in their movement-music expression. While there is important diversity in range of expression, ability and body type, we found the disalignment context important in designing systems that accommodate aberrant behavior. Specifically, this means:

1. systems with the broadest possible range of mappings
2. systems that are equivocal, employing for example fuzzy logic, rather than strictly 1- to-1 mappings
3. using activity-based parameters (as opposed to position- and shape-based parameters). Many people cannot (or do not want to) use fixed-measurement controllers.
4. systems for which there is no "wrong" way to play them

On a deeper level, a profound re-thinking of system design may be needed. As one of the designers, Andreas Bergsland put it: "The concept of affordance can be useful when designing interactive environments, because it invites thinking about users, technology and audience as an ecosystem where reciprocal interchange of information and sensation take place. It highlights the fact that both thinking and sensing are distributed and embodied processes, where environment, technology and users constantly feed back on each other."(19) This dynamic looping process enriches the experience and contributes to the creation of scenarios helpful in integrating the experience on a collective level.

Performing, that is, the showing of what one can do, also plays a role in the process of raising awareness of diversity. Indeed, the interactive motion tracking with its "play area" offers a unique stage for this process to unfold. The workshop leader assumes the role of director/conductor, orchestrating the set through storytelling, theater and dance.

Rather than leading to exclusion, awareness of our differences in a creative setting can have the opposite effect. Listening and observing the other, imitating the workshop leader and following the same rules together allows a freedom of expression in a co-footing environment. Everyone is differently the same. The disability doesn't exist anymore or is perceived as a poetic difference. "Listen to my body talking" promotes diversity through original movement and sound.

In designing music-movement tools for persons with disabilities, we face large, but also very interesting challenges. This user group is not only incredibly diverse, but also incredibly open. One of our main challenges has been to ensure inclusion for users with all abilities, so that all types of movements can in fact render musically interesting and pleasing results for the user. The overruling strategy we have taken in that respect has been to strive for variation and richness in mapping strategies, interaction metaphors and in sound and music. Simultaneously, we have maintained activity, being something truly universal across abilities, as the central parameter for all our environments. For both Frederick, Anna and Daniel, feeling the music follow their activity level seemed to be sufficient to generate a rewarding experience. We have realized that its counterpart, stillness, is also very important, and as for Daniel, can be a crucial component in perceiving the causality between movement and sound.

We have made many surprising revelations in our workshops as users would play the MotionComposer “incorrectly”, and in doing so discover brilliant creativity, inventiveness and musicality. To wit Frederick (described above) played the tonality chair environment, in which arm height along the vertical axis is tracked. But Frederick was almost horizontal in his special wheelchair and thus his arm movements did not follow the intended trajectories. This lead to unintentional, yet interesting consequences. Other examples include persons who reach both arms to one side of their body (fairly common), twist around in their wheelchairs, or who reached towards the floor or towards the audience instead of upwards. From a choreographic standpoint, these ways of playing are expressive and completely justified even though the logic of the system as a musical instrument is not what was intended.

The question for us, then, is how to design dance-music systems which offer "rules" for their control, and yet for those cannot or choose not to follow those rules, allow alternative mappings and modes of playing. This dichotomy -- rules and freedom -- cannot be resolved through compromise. A system that sometimes does what you want, will always be frustrating. The question is how to accommodate multiple modes of playing. They can be concurrent or alternating and if the latter, where there is a switching back-and-forth, how is the choice made of when to switch?

    by the therapist (or other person pressing the buttons)
    by the user (for example, through a particular gesture)
    via an intelligent system, which analyses the style, range of movement, etc. of the user

Finding good answers to these questions depends on amassing experiences with users who appropriately reflect an extremely broad range of abilities. Disabilities Studies professor Devva Kasnitz spoke up during a seminar we gave at University of California at Berkeley to say, "we are not interested in having you develop tools for us, but we do want you to develop tools with us" (Block 2015). Different users can contribute to the development in different ways, some more actively and directly with own suggestions and ideas, and others more indirectly, through showing what they like and what they don’t like so much.

Future Work

With MetaBody partners InfoMus and Steim, we are looking into extending the range of human movement properties which we believe could improve the technology and make a richer user-experience. For example, higher order movement qualities, such as softness, lightness, tension and so on are important to how we feel when we move and yet are largely out-of-reach to the technologies used in this study. Shape-based aspects -- twists of the torso, twist of limbs, bending of torso and limbs, extension and contraction -- represent a similarly out-of-reach area.(20) The dancer Muriel Romero pointed this out at the 2015 MetaBody Conference in Madrid when she said, "As soon as I do something interesting with my body, the technology gets confused".

Finally, the way we look at interactive technologies could lead to a new perspective of the body and by extension, society. Soft skills, as artistic and creative expressions, promote the vision of a sustainable and inclusive culture. Beyond therapeutic and pedagogical interests, we observed that the joy and pleasure felt by participants has a universal echo concerning the perception of the body. What is imperfect, what we call dis-able falls away in this universal perspective.


We obviously could not have undertaken a project of this size without help from many directions.  We would like to acknowlege first of all our co-organizers at the MetaBody. These include STEIM, InfoMus, Stocos, and Reverso. They assisted not only in ..............  LINKS


Principle authors: Robert Wechsler, Andreas Bergsland, Delphine Lavau, Marcello Lussana, Ekmel Ertan

Additional researchers/contributors: Josepha Dietz, Annika Dörr, Pablo Palacio, Alicia Penalba, Marije Baalman, and particular thanks to Jaime Del Val, who had the idea for MetaBody and was instrumental in designing this subproject. ..................................................










1.  "Using motion tracking technology (including video-based and controller-based systems), dance and music, the questions the research poses include: How can we remove barriers to expression? Technology tends to reduce gestural expression, how can we expand it -- expand range of movement, range of expression? How can we promote a more positive awareness of difference, through disalignment from normative conception of ability or intelligible expression? How do we generate affordances that invites deviant (alternative) behaviors which foster plurality and not homogenization? What are the cultural differences in the perception of difference? How do these differences relate to acceptance and integration (inclusion)? How do different societies (including governmental and non-governmental organizations) approach inclusion?"  Jaime Del Val