From the outer reaches of space to the depths of the ocean floor, Aquaero offers an immersive experience to interact with hand-like creatures and environment through gesture and audio recognition.
Players interact with creatures and environment through audio events triggered by the audience, such as bubbles or lightning strikes. Flocking creatures such as jellyfish, fish and birds can be found in levels and add to the interaction. Audio and Environments transition seamlessly as the characters move around.
The creatures in Aquaero are designed to be whimsical and dynamic, where no two creatures look the same. They are a combination of eyes, mouths, horns, and tails, and where the location of each is determined by the hand or object that appears on the table.
Aquaero uses a tabletop covered with dark fabric, combined with a PlayStation 3 Eye camera, to detect players hands, and objects in the field of view.
The initial concept was to use a light table, which is illuminated from below. This allows for Aquaero to be played in dark lighting environments. Light tables also have a few drawbacks however, such as the complexity required to build, and transport them. In addition, it is very important to have even lighting from below, which we foresaw challenges with.
We decided that using a standard black background would be sufficient for this project. This also mean the setup could be easily portable, and adjusted based on table size. If we were to revisit it, using a proper light table would be a great improvement.
Interactions in Aquaero are driven by movement and shape. When you place your hands or objects above the table, they are captured by the PlayStation Eye camera. Keeping your hands still will create creatures in the shape of your hands.
When an object is placed over the table, it’s captured by the PlayStation Eye. This camera data (1), is then converted to greyscale (2), compared to an original image (3), which has no objects in view, and then performs a contrast increase and substraction of the background (4). Once this background subtraction has been performed, the image is then converted to a shape (5), and then smoothed (6). These smoothed shapes are then used as the basis, to attach the eyes, mouths, horns and tails.
In addition to creating creatures, you can also create wind, by moving your hands in the environment. Creatures are affected by this wind, and are thrown around the world. This wind is powered by the velocity and direction of the hand movement which then affects vector fields (the arrows you see below), which keep track of all the directional information, and the velocity of the objects moving through the frame.
Each level of the environment is designed to be distinct, but seamlessly transition into the next. Each level is composed of multiple layers giving depth, and interest as the creatures are created, moved around, and destroyed by environmental effects such as lightning.
The key challenge for audio design, was finding thematic music that â similar to the environment â was distinct, but smoothly transitioned, so that players felt that they were in a new place, but still felt somewhat familiar. See the introduction video for the finalized soundtrack. The looping tracks were built in Ableton Live, and faded in and out dynamically.
In addition to the soundtrack, we built in audio detection, where if the players make enough noise, e.g. by clapping their hands, it will trigger thunder and lightning, occasionally striking the creatures. The video above captures it in action.
Branding consisted of two goals:
Brand the team: Projector Face
Create a unique logo that could be reused for future projects. Inspired by the Iron Giant (a long time favourite of mine), and the excessive need for projectors our projects relied on, married with a typeface that is reminiscent of movie poster credits.
Brand the experience: Aquaero
Create a logo that represents that the environmental diversity in which the experience exists. In hindsight, I do wish I found a more elegant solution to combining water and air into a unified concept. The water and cloud feels adequate, but overall a bit clumsy graphically.
Aquaero was an amazing experience for me â it was the first project where so much technical, artistic, and creative energy came through into an experience that anyone could partake in.
It was a real gift to get to work with such a talented bunch of folks.
Audio Recording + Programming, Environmental Design, Physical Setup
Creature Asset Design, Flocking Programming, Documentation
Backend Programming, Shader design