Introduction
This semester’s focus study was based on real-time computer graphics and how a CPU/GPU controls and processes graphics-based instructions in real-time. In compliance, with this theoretical study the major submission for the course was a demo composed using Virtools. A demo is a multimedia presentation that can incorporate interactive elements. The demo scene arose as a means for coders and multimedia specialists to show-off their skills amongst their friends and other underground organisations. A demo looks similar to an animation sequence although the difference it a demo is rendered in real-time with code determining the functionality as opposed to keyframes. For those who may wish to delve further into the demo scene, please visit the following sites:
Concept
The demo's content is based on the selected soundtrack, Dance 4 Life – Breathe (Black Sun Empire Remix One). I spent a great portion of time simply re-playing the track annotating keywords in the lyrics as well as imagining the aesthetic of the track. Essentially, the song talks of two souls embracing each other in a world of freedom: “You and I, spinning round.” Intentionally, my demo visualises the relationship between two datasets, webcam and microphone input. I attempted to recreate the message of the song by creating a dynamic relationship between the two datasets, which are represented as two spheres (red and blue). The red sphere depicts microphone input while the blue sphere embodies the webcam input. It was quite challenging to maintain a balance between data representation and integrity with the soundtrack, so I tried to represent data via properties that did not effect position or camera angle. Hence, the two datasets affect properties such as colour, scale and speed. The soundtrack was quite interesting and diverse in its nature with moments of both calmness and climax, allowing for a wide spectrum of actions.
Inspiration
Andrea Lau's Multiplicity played an integral role in providing me with ideas for the aesthetic that I was attempting to create. Her piece
exploits the simplicity of spheres to create interesting patterns and behaviours. Her piece was similar to mine in respect to the use of and
synchronisation with the soundtrack. Both our soundtracks have calm and climactic ranges, but most importantly is our soundtracks lead the
demo's behaviour.
Mapping
As previously mentioned the data primarily controls properties such as colour, scale and speed. The reason for this was to have some control over the objects being displayed. I limited the number of dynamic properties in order to assist the audience with the process of mapping their motion and sound to behaviour of the demo. Ideally, this demo would suit a large-scale setup, where the sound and motion of users could provide feedback in a more immersive manor. The motion and sound data is collected in real-time and often displayed in real-time, which makes the process of mapping simple. The following screenshots depict the affect of real-time data input on the demo:
 |
 |
|
Scene 01 This screenshot depicts the dramatic influence of heavy webcam activity. Blue spheres dominate the screen due to intense webcam activity. |
Scene 02 This screenshot depicts the influence of both webcam and microphone activity simultaneously. Unlike other scenes, scene 01 captures data and displays that instance of data rather than display the data in real-time. |
 |
 |
|
Scene 03 Shown here is the effect of microphone activity dominating the dataset in scene 01. |
Scene 04 Scene 02 is interesting because it displays a collection of sensor instances. White spheres depict data instances over a 10 second period. This shot depicts heavy webcam input. |
 |
 |
|
Scene 05 Shown here is the influence of significant microphone activity. The sphere always emerge into an interesting molecular object. The size and behaviour of the molecules is determined by the sensor input. |
Scene 06 Scene 03 shows a direct, real-time display of captured sensor data. This screenshot depicts heavy webcam activity. |
 |
 |
|
Scene 07 This is a combination of both microphone and webcam influencing the colour of emitted particles. |
Scene 08 Heavy microphone input led to a highly sasturated red emission. |
Approach
In order to send real-time captured sound and motion data to Virtools proved to be quite complex. I used Adobe Flash 8 to interface
with the microphone and webcam. Flash captures the motion and sound data in the form of integers ranging from 0 (no activity) to
100 (maximum activity). The dataset did not just use real-time captured data, but averages of the last ten capture values. Flash
then sends the dataset to a text file where it is stored. Although, the flash does not connect to the text file directly, a PHP
script takes the values from flash and writes it to the text file. The same PHP script is used to interface with Virtools by
accessing the text file and sending it to a Virtools array. I think the most interesting part of the way I implemented this is the
fact that the data capture application and the demo can run at completely different locations as it is all server-based. This could
make for some interesting implementations. Guy Watson's tutorial proved to be invaluable for the Flash implementation:
Gallery
Most of the following images show off the influence of sensor data on the demo:
 |
 |
|
Scene 01 Introduction/title screen |
Scene 02 Both webcam and microphone data is subdued. |
 |
 |
|
Scene 03 Calm scene depicts the coming together in both the soundtrack and the two datasets. |
Scene 04 Purple emission depicts heavy webcam activity. |
 |
 |
|
Scene 05 Heavily saturated red emmision illustrates intense microphone activity. |
Scene 06 This is a capture from the climax scene where the tempo booms. |
 |
 |
|
Scene 07 The climax scene use multiple camera angles to create chaos and stimulation. |
Scene 08 In order to follow suit with the soundtrack lyrics, the camera zooms out to depict "spinning round". |
 |
 |
|
Scene 09 The background colour of scene 05 changes dynamically with sensor input. |
Scene 10 This shot depicts the two sphere entities being free in a concentric universe. |
 |
|
|
Scene 11 The finale uses a global blur to create a scattered effect, which provides a nice conclusion. |
Downloads
In order to correctly run dnb demo, you must have a microphone and webcam. Also, you must run the flash applet whilst viewing the demo in order to capture the webcam and microphone input.
» Run dnb demo
» dnb demo source
» Flash source
» Get Virtools Player
» Get Flash Player
About Me
Adam Rusell Stone
SID 200417265
email stone.adam@gmail.com
web stoneykun
Andrew Vande Moere
course Real-Time 3D Multimedia
web infodemo