VJing using Quartz Composer
The project took a new turn when I came across Synapse for Kinnect.
Synapse is an app for Mac and Windows that allows you to easily use your Kinect to control Ableton Live, Quartz Composer, Max/MSP/Jitter, and any other application that can receive OSC events. It sends joint positions and hit events via OSC, and also sends the depth image into Quartz Composer. In a way, this allows you to use your whole body as an instrument.
Synapse for Kinect. Site Accessed Dec 10th 2011
After downloading the source files I quickly had the depth image running on the Mac and from this point I decided to develop the entire project using Quartz Composer.
Quartz Composer is included with Xcode and is a visual programming tool similar in many ways to Max MSP in that you drag patches into a composition and then connect these to create an API (Application Programming Interface).
With my new-found interest in using Quartz Composer I began researching for methods of creating the Video Wall from the video clips we recorded. Having already created a number of Quartz Composer compositions from Tutorials I had a good idea that the patches I would need would include the ‘Billboard’, ‘Keyboard’ and ‘Midi’.
To start the new composition I added the video loops to a folder and then in turn dragged each of these into the composition. Each video file creating its own patch (Fig 1.0) from the image output of these patches I connected these to the image inputs of the Billboard patch. This I repeated for the 8 video loops which would make up my video wall.
You then need to set the X and Y coordinates for the video loops using the Patch Inspector on the Billboard patches. These I arranged around the centre of the screen so that I could leave the centre space free in order to display the image from the Kinect device, in effect the conductor of the video loops.
Input Patches for Keyboard and Midi
To test the video wall functions as expected I dragged a ‘Keyboard’ patch onto the composition and linked the outputs of this to the enable inputs of each of the Billboard patches. The Keyboard keys are configurable so I changed these from the arrow key defaults to the number keys 1 to 8. (See Fig 2.0)
Now when I run the composition and pressed the number keys each of the video loops played in turn corresponding to the keys set to enable them.
The next step was to test that Midi would also be able to control the video loops and so I dragged a Midi patch onto the composition, which I set for middle C and 5th Octave. The outputs are labeled as for a keyboard and so it was a simple matter to link the outputs of the keys to the enable inputs of the Billboard patches.
However when I’d attached a Midi Keyboard to the USB of my Laptop nothing appeared to work. Going into the Audio Midi setup I made sure it was setup for the M-Audio Oxygen keyboard I was using and I also ran the test program which confirmed the keyboard was talking to the Laptop correctly. Unfortunately there was still no response from the Quartz composer composition so I had to go into the Midi patch and use the patch inspector to re-configure the settings. In the end the correct settings were:- Midi Channel 1, Middle C as 5 and Midi Source as ALL and then on the Keyboard itself setting to Octave 1. With these changes it was now possible to control the video loops using the keys on the piano keyboard.
To complete the design I added an Image patch to act as a background for the video wall it was at this point that I noticed that the layer order needed to be changed so that the background did not overlay the video loops. The completed patch looks complicated but it’s quite easy to understand. (See Fig 3.0 click on images to see bigger versions)
Finally the completed Quartz Composer composition with Video display can be seen in Fig 4.0 along with a short video showing the Midi Keyboard controlling the video loops. Please note these are not the final videos to be used in the project but are just temporary videos to prove the operation.[youtube.com/watch?v=05ra1f9BsjU]
Project Blog Entry Links
- Performance Video – Conclusion
- Performance Video – Wiimote MAC
- Performance Video – VJ’ing using Quartz Composer
- Performance Video – Kinnect on MAC and PC
- Performance Video – MAX MSP Jitter
- Performance Video – Modul8
- Performance Video – The Human Orchestra
Word Count 697