I am on top of a little mountain now, overlooking the sea between escalante and laredo on the Northern coast of Spain. i see a blue ocean sparkling below me. I am alone.

I was thinking on my way up about sound bubbles, or the chewy kind of particles that Takehisa Kosugi seems favour (aka soundance). They have a biological sound (as pablo's filmmaker friend would say), and yet are not necessarily something one could identfy as "nature".  Indeed, they are not a sound you would or could ever hear in nature.

I for one would like to wade through such sounds, disturbing them as I go -- waking them up, exciting them, confusing them...

Multi-dimensional transformations of particle arrays mapped in some intuitive way to tactile (haptic) and body (kinesthetic) experiences.

In simpler words, ways to make things that could make it really fun to move, to move sound.  To be sound.

Moving beyond "triggering sounds with movement" is a great challenge. It presents us with the chance to reach a more sophisticated, intuitive and ultimately more powerful model for movement-sound expression.

In my opinion, it depends on two things. 

One is the right kind of sound and movement.  Some sounds, sound like body, or at least natural in some way.  That is, they sound either like sounds that a body could make, or

like sounds that a body could cause.  A body could walk through gravel, stir up leaves, cause bubbles to be released.  A body could creak, stretch, vibrate, shake, elongate, rumble, break crystals of glass, ...

The other thing it depends on is a strong mapping.

Most mappings are pretty useless (are far too subtle to be of implicit or explicit expressive value). There are three that I call "Strong Mappings":

1 Particle-based sound correlation
2 Layer-addition and subtraction
3 Bending


1 Particle-based correlation

Discrete mini or micro sounds. They can occur in groups or alone, and are triggered by dynamic changes in movement.

Most complex movement has within it numerous stops and starts, things like direction changes, glances, jerks, ducks, sharp drops, torso position shifts, foot-position adjustments, sharp gestures, and so on.

By connecting, linking enough of these to particles to such movements, you get the a strong feeling the body is making the sounds.

This approach concerns density and other complex sound processing, including:

grain density
grain size
envelope
freq.range (tendency mask)

We might also include degree of randomness / degree of order or chaos (particlarly applied to patterns) to this category.


2 Layer-addition and subtraction

It is possible to control longer sounds with shorter gestures and movements through the addition and subtraction of layers. The layers maybe be textural in nature, or, concerning addition but not subtraction, rhythmic.

Both sound interesting. Right now, as I write this on a hillside above the sea, I can hear the distant sea, some small insects and something like the sound of air-in-motion. Imagine how arresting it would be if one of these were to suddenly stop. One forgets such sounds, until they stop or start. Like a quick inhalation followed by holding the breath. A blink in time.

Intuitive mapping suggests lower movements and gestures going to lower-sounding sounds. Heavier movements to heavier sounds, whispy to whispy, open to open, closed to closed, intimate to intimate, etc. Muffled (dampened) to muffled (dampened), inverted to inverted. These latter two are more modulations than additions / subtractions, but they will function similarly in this context.


3 Bending

Bending refers to continuous variables. While by far the subtlest of the Three (correlations are much more difficult to attain), bending is important. I think that just as with pulses above, we have to think of it in terms of multiple coupled parameters. These things seem to need a certain complexity in order to live.

Bending overlaps with particles, since particles can form clouds, rivers, swarms.

slow to fast (bursts of movement), momentum
slow to stop to slow again (creeping, exploratory types of movement)
tentative initiations of strong, busy sounds (like a motor getting starting, or the first few pebbles of a landslide)

Activity level, height, position left-right and width work well as continuously variable data sources.



The point is that by blending together all three -- Particles, Layers and Bending -- I believe highly intuitive and powerful interactions will be possible.

more essays