Audiovisual Playground is a 3D, interactive music-making application developed in Unity for the HTC VIVE.
We wanted to design a music-making interface that took advantage of what consumer virtual reality (VR) has to offer. Today’s VR technologies are more accessible to developers and more consumers own VR products more than ever before. Software developers can now utilize game engines like Unity and quickly design 3D worlds that surpass what is possible or feasible to build in the real world. Hardware like the HTC VIVE, which allows for head and motion tracking, allows users to be further immersed through using physical gestures to interact with virtual worlds.
The overall design of music production software hasn’t changed much in the past several years. For the most part, music production software is always a 2D interface where users design audio left to right separated by tracks. However, now with VR we have the ability to design 3D interfaces that can be used almost anywhere.
The way Audiovisual Playground users compose music is through the use of pedestals, similar to the way tracks are used in most music production software. Each pedestal represents an instrument: snare drum, kick drum, crash, hi-hat, and keyboard. Each pedestal has a sequence of eight programmable beats represented by floating icospheres, and the current sequence will loop at 120 bpm and synchronize each of the pedestals. Users can walk around the virtual environment and using a controller they can point and click at the different icospheres around each pedestal to compose different songs.
Source code and Unity executable can be found here: https://github.com/shadowfax6894/audiovisual_playground
This project was developed along with Kristof Klipfel, Christopher Koski, and Alexander Goodwin.
The source code for the sequencer system we used came from Nidre’s Github: https://github.com/Nidre/Unity-Audio-Sequencer
The audio backing track was created by Drumless Backing Tracks and retrieved from their youtube channel here: https://www.youtube.com/watch?v=qTn42JnH6Os