<nettime> corporeal performances

[Scott deLahunta <sdela@ahk.nl>]

[An essay posted to the dance-tech list a few days ago in response to a 
posting about interactive sensor systems]

++ invisibility/ corporeality ++

The process of computation is invisible in the simplest sense that the 
labor of the software programmer or engineer is largely taken up in the 
'writing' of an instruction that tells the computer hardware and connected 
peripherals how to execute (perfectly) an operation. (1) This writing and 
subsequent rewriting/ editing is part of the creative process whereby 
something gets 'made' in terms of digital technologies. While some might 
decide to write code (or simply 'coding') for code's sake -- generally this 
activity is done in order to enable something else to happen or get made. 
Where the primary creative activity is in relationship to the 'second 
order' activity can only be determined on a case/ field by case/ field 
basis... but it will vary (i.e. there is variation in this regards between 
the music field and dance field)

The other day I was having a discussion with someone about dance making and 
interactive systems and the 'transparency' issue as regards the receptivity 
of an audience to the aspects of the work that might be invisible. What is 
being considered *invisible* in this context is the mapping from input to 
various forms of output -- and this mapping is essentially the consequence 
of someone providing the instructions for the computer, telling it what to 
do. Input in the case of BigEye occurs through the analysis of performer 
movement/ action in a video image that then generates a stream of 'movement 
tracking' data. Thus performer movement/ action is used to trigger some 
sort of event (sonic, visual, robotic, etc.) in the space around or in some 
proximity to the performer. The connection between the performer action 
that activates the stream of data and the output event is determined by 
'mapping' the input to the output in the computer in some way. Christopher 
Dobrian (U. of CA, Irvine) expresses it simply: "The interpreted data 
provides information about the speed, direction, and location of moving 
objects in the video image, and that information can be used to provide 
input control data to music-generating software". This is essentially what 
is referred to as an interactive system. (2)

Mapping is at the heart of the creative process as regards these systems -- 
which Marcelo Wanderley (researcher based at IRCAM) in a detailed 
presentation on interactive systems at ISEA in December 2000 pointed out. 
In a paper entitled *Towards a model for interactive mapping in expert 
musical interaction*  (2000), Wanderley and Ross Kirk review the ways 
"performer instrumental action can be linked to sound synthesis 
parameters". (3) Their precise definition of 'mapping' uses the word to 
refer to the "liaison or correspondence between control parameters (derived 
from performer actions) and sound synthesis parameters. Within this they do 
not include in the concept of mapping the "actions related to data 
preparation, such as segmentation, scaling, limiting, etc." In the paper, 
they point out that generally two main 'mapping' directions can be derived 
from an analysis of past work: a) the use of generative mechanisms (e.g. 
neural networks) to perform mapping; and b) the use of explicit mapping 
strategies. For Wanderley and his fellow researchers in the field of 
electronic music, mapping is clearly a topic of immense creative interest 
and focus of artistic practice. However, it is the manifestation of mapping 
that enters the field of perception of the viewer/ listener, not the 
mapping itself. Once completed, the instructions that comprise the mapping 
itself are relegated to the invisibility of computation. How this invisible 
mapping works or how it might work is of interest primarily to those who 
are engaged in its construction. [These last few sentences form the core of 
a polemic -- which is completely refuted by the work of medical researcher 
Greg Kramer -- see note 4 below]

There might be two directions for artistic work with interactive systems -- 
One: towards allowing everyone audience/ user access to all facets of the 
systems -- input, mapping and output. Today, dance performances using 
interactive systems tend to allow an audience access only to the output, 
installations allow access to the input and the output -- so why not 
include exposure to the mapping itself?

Another direction is towards the user/ performer who might work with 
(practice) these systems in order to enter into a realm of higher level 
skill and 'virtuosic' activities within them. Combining input measurement 
that responds to a higher level of detail in performer action with more 
complex mappings. Wanderley/ Hunt conclude their article by stating that 
"complex mappings cannot be learned instantaneously, but then again, we 
have never expected this from acoustic instruments". Assuming that the 
reference to learning can be seen as a reference to training -- it begs the 
questions, where in the dance field do we discuss and debate notions of 
dance 'learning' (training/ technique) overlapping with the development of 
interactive systems? There are a handful of practioners/ artists (i.e. 
troika ranch/ palindrome) whose efforts over time are accumulating richness 
and depth through personal determination and diversification // but their 
activities are focussed on artistic output, not training.

To return to this concept of the invisibility of computation -- in 
relationship to interactive systems. I am curious about the long term 
outcome of creative activity that is proportionately shifting its centre of 
labor from the physical spaces to the virtual spaces. Any dance artist 
working with interactive systems will tell you that so far the amount of 
work involved in 'getting the technology' to work is immense and seems 
disproportionate to the amount of work done in the studio, perspiring and 
flexing. A shift away from the physical is by consequence in aesthetic 
terms a shift away from the formal and towards the conceptual... could we 
see audiences who develop a taste for mapping coming better prepared to 
watch/ contemplate dancing in interactive systems?

Perhaps there is a future that holds both possibilities -- where the 
invisibility of computation is displaced by its contemplation in the 
context of corporeality. Where 'interactive systems' are infiltrated 
increasingly by sweating/ flexing bodies who spend more time in them 
sweating and flexing... while an audience/ user's understanding of 
performance may increasingly see this body as one that is intimately 
involved with the invisible electro/ digital spaces within which it performs.

***********************

1) Writing is arguably not the best descriptive metaphor for software 
programming -- building is preferred as often coding more often requires 
the reuse or reassemblage of previously written code.

2) For a very useful and straightforward breakdown, I recommend Dobrian's 
website 'video motion tracking for musical input' 
http://www.arts.uci.edu/dobrian/motiontracking/default.htm

3) You can download the Wanderley/ Ross PDF here: 
http://www.ircam.fr/equipes/analyse-synthese/wanderle/Gestes/Externe/Hunt_Towards.pdf

4) For a completely radical and fascinating look at 'mapping' -- take a 
look at this article on *Mapping a single data stream to multiple auditory 
variables* by Greg Kramer on the sonification of data from Radionuclide 
Ventriculography (RVG)—a non-invasive means for obtaining the blood volume 
change of the left ventricle. The article details how by working with 
complex mappings rather than a 'one to one' mapping auditory 
representations of a 'diseased heart' for example can become much more 
compelling to listen to -- don't miss downloading the .wav files 
http://www.santafe.edu/~icad/ICAD96/proc96/kramer.htm

#  distributed via <nettime>: no commercial use without permission
#  <nettime> is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: majordomo@bbs.thing.net and "info nettime-l" in the msg body
#  archive: http://www.nettime.org contact: nettime@bbs.thing.net