Senior Design Project: Implementation of a Public Exhibit for the Kemper Museum of Contemporary Art
Here is the inscription the museum wrote for us to put next to it:
About his sketches of nightclub scenes, Denzil Forrester says, "I just wanted to draw movement, action, and expression." Consider the Digital Organum your sketchpad.
The Digital Organum is a visual and musical instrument that translates the body's movement with light and sound. It was developed in partnership with University of Missouri-Kansas City (UMKC).
Stand on the carpet and move your body or interact with others. The Digital Organum uses a body-tracking camera to display individuals' changing positions in dynamic figure outlines and musical sound.
IDEAS
The local museum of contemporary art provided the initial impedus for this project by reaching out to our university to explore the possibility of student collaboration. They were looking for more "contemporary" ways to get people to interact with art other than by providing a corner with crayons. UMKC faculty saw the opportunity to promote our science and engineering department. Our senior design coordinator suggested to our group that we explore the possibility of creating engagement material for the museum, when we expressed our interest in making some kind of synthesizer.
Credit for the inital 'big theremin' idea goes to our teammate Jordan, things sort of took off from there.
ARCHITECTURE
The system uses the Kinect V2 Python library to get the position data for each person interacting with the exhibit. I assisted with planning the general architecture and interaction between subsystems, but our teammate, Martin, did the heavy lifting with regards to implementing and troubleshooting our kinect-to-midi quantizer and adding features.
The visual aspect was fun to program! Firstly, the PykinectV2 library provides convenient keyframes for each person in the frame, meant to ask as a mask for anyone in the frame. I simply performed an erosion on the mask and subtracted it from itself, to make a fun outline for each person that looks rotoscoped. The rendering system is a simple OpenCV fullscreen image window (good enough for a hillbilly!).
For our audio stack, one of the faculty members put us in contact with a master's student in composition at our conservatory.