On the Boil
Auckland museum’s tale from a virtual topographic ocean.
Text:/ Derek Powell
Moana – My Ocean, a recent exhibition at the Auckland War Memorial Museum provides an excellent example of how audiovisual techniques can be used to not just see below the surface, but provide entirely new perspectives into the behaviours of the natural world. Although museums may no longer be characterised by rows of dusty glass display cases and dioramas of stuffed birds foraging amongst the painted trees of a faux forest, this exhibition’s use of Artificial Intelligence to drive a sophisticated audiovisual installation, created an immersive experience to instruct and entertain a wide spectrum of visitors.
“We [New Zealanders] like to think of ourselves as a marine nation, and Aucklanders love the sea and boating,” reveals Victoria Travers, senior exhibition developer at the museum. “But all we really see is the surface. There’s a lot going on that we wanted to be able to bring into an exhibition. Moana – My Ocean actually resulted from the strong public response to a blog, posted by one of the museum’s curators during an expedition to the remote Kermadec Islands, far to the North-East of New Zealand.”
Sparked by the social media response to marine science, planning for this exhibition started as far back as 2011. “First, we started looking at the concept, the scope and what stories the Museum had to tell,” Travers explained. This first stage is very content focused as the development team and the curators decide on the content of the exhibition. Things start to get interesting as the team of 3D designers get to work mapping out exactly how the content is to be displayed and what techniques will work best in bringing the objects in the collection to life.
Quite early on, decisions were held about whether the exhibit should be object focused or technology-based, with more audiovisual content to explain and reveal the stories behind the artefacts. Given the enormous scope offered by this subject matter, which ranges from the smallest of microscopic plankton to whales and sharks, it was decided that the exhibition should be a mix of objects and a multimedia interpretive display.
Moana – My Ocean takes visitors on a marine journey through the Hauraki Gulf, New Zealand’s most-used stretch of water, famous for recreational fishing and boating. The shallow, warm waters produce an abundance of phytoplankton and zooplankton that makes the gulf an incredibly productive area, brimming with life. It is famous for regular ‘boil-ups’, where a large school of fish is literally herded into a tightly-packed formation close to the surface by predators, causing the surface to boil as fish desperately try to escape the predators below.
Four Sony VPL-SW535C projectors display the real time animation as it is created directly by the server. Image courtesy of Auckland War Memorial Museum. Photographer: Tāmaki Paenga Hira.
each on-screen experience is unique, generated afresh for that viewing, unlike a film which creates every element according to a fixed script
ON THE BOIL
All agreed that a boil-up would make a wonderful visitor experience, but creating and interpreting this natural phenomenon would be a challenge. The museum design team considered a sculptured exhibit but Travers recounted that the team felt a static exhibit wouldn’t adequately capture the drama of the event. “Anyone who’s been stuck in the centre of a boil-up before, and I have been,” Travers recalled, “knows that it’s really disorientating. It’s teeming with abstract colours and shapes, really noisy and all around there’s just frenzy and drama. Yet at the time, you don’t know what kind of fish are around you and what predators are below. We wanted to be able to capture that experience, and show what was really happening.”
A brief was put out to specialist contractors, and along with a number of conventional proposals involving underwater and aerial filming of a real boil-up event, one pitch that came back really stood out. Rawstorne Studio, an Auckland-based creative design agency, proposed the idea of using Artificial Intelligence (AI) to reconstruct the fish behaviour in the boil-up system and display it using computer-generated animation. Essentially, the animation would combine elements of AI with mathematical models derived from research on flocking behaviour, to create a virtual representation of how large groups of marine animals behave when faced with given stimuli like a predator threat.
BUILDING A VIRTUAL OCEAN
Rawstorne Studio then brought on creative company Pleasure Kraft to help create the audience environment – a 270° immersive animated projection that recreated the experience of a boil-up from within the bait ball itself. To generate the on-screen action they engaged American animation artist Robert Hodgin. “Robert was key to the project,” said creative director Robin Rawstorne. “He writes code just like a composer writes a score”.
Hodgin, who has an interest in simulating group dynamics, jumped at the chance to team up with the museum’s marine scientists for this unique project. The original concept was to represent thousands of fish as simple geometric shapes that would school together to represent the bait ball. But as the project developed Hodgin was able to recreate lifelike representations of each animal. Tom Trnski, head of natural science at the museum, worked with the Hodgin and his team to ensure the correct species were represented and that they interacted in a realistic way. With Trnski’s guidance the swimming style, schooling patterns (and, for the predators, hunting techniques) of each species were tweaked so the science was spot on, while retaining the drama and intensity of a life or death natural struggle.
Rawstorne explained that each on-screen experience is unique, generated afresh for that viewing, unlike a film which creates every element according to a fixed script. “It is actually AI driven,” he emphasised. “Around 7000 individual virtual fish are created in the computer and animated on screen. They swim about, school together and react to each other in this virtual ocean just as they would in real life, according to some quite simple algorithms that mimic natural behaviour patterns”.
“Then we introduce animated predators into the scene, who operate under their own set of behaviours.” The predators, including dolphins, sharks and finally a massive Bryde’s Whale, attack the bait fish who instinctively attempt to escape by schooling into ever-tighter formations – ultimately resulting in the boil-up. All this is created by the individual animated fish objects, each reacting to the predators in real time, so while the on-screen action is predictable, a viewer will never see exactly the same phenomenon twice – just like a real ocean boil-up. The entire virtual ocean scenario is created within a single computer, that maps the animations onto multi-head video output cards.
THE EVOLUTION OF A VIRTUAL FISH BOIL-UP
PUTTING ON A SHOW
With the content correct, the next step was to decide how it would appear on the enveloping circular screen. As creative director Robin Rawstorne puts it: “We were able to position virtual cameras within the virtual world, which observed the unfolding maelstrom. Our only ‘influence’ was to time the release of each predator species to build the intensity and excitement of a boil-up.” As the three-minute sequence unfolds, the camera viewpoint changes from outside the school to a predator’s-eye-view as sharks feast on the ever-tightening ball.
Meanwhile, another hunt was underway: to find the right projectors to fit within the tight confines of a 6m display space. To meet the requirement for a 270° immersion, four projectors would be required, set at approximately 65° spacing and edge blended. Short throw was a must, so that visitors did not interrupt the beams as they moved in and out of the space, but that raised potential issues of hot spots and light scattering in the overlap regions.
After a careful shoot-out between candidates, the team eventually settled on four Sony VPL-SW535C projectors which at 3000 lumens and WXGA (1280 x 800) resolution had the right grunt to match the specialised optical requirements. Lamp life was also a factor, and with a rated 6000 hours (in low mode) the Sony’s stacked up well in the value for money stakes. The projectors were mounted on a custom circular support above a 2.5m-high screen constructed with a specially-welded seamless PVC projection surface to preserve every pixel of resolution.
So how did the ocean-savvy New Zealand public take to this unique view of life in their beloved gulf? The Auckland War Memorial Museum takes a rigorous approach to analysing the visitor response to each of their exhibits, so Victoria Travers had an array of facts and figures to determine just how effectively each aspect of the exhibit captured the public attention. “Normally, we work to a rule of thumb that says visitors will spend just three minutes in front of any filmic or animated experience before moving on, so the Boil-up experience was timed to a three minute cycle,” she explained. “But when we studied the patterns of movement throughout the exhibition, we found most visitors stayed for six minutes in the boil-up theatre, which meant on average, they saw two complete shows.”
And that, my friends, in the museum world, gives the Boil-up a blockbuster rating.
MUCH MORE INFORMATION:
You can find fascinating insights into the design and lots of technical illustration at:
For those more interested in the coding of the animation sequence for the boil-up, Robert Hodgin gives an in-depth, behind-the-scenes exposition of the programming process at:
Creative design: Robin Rawstorne – Rawstorne Studios
Creative and technical producer: Olivier Jean – Pleasure Kraft
Creative software coder: Robert Hodgin
Sound design: Peter Hobbs – Harmonic
Theatre construction: Cutting Innovations
Workstation: Dell Precision T5600 (8-core Xeon CPU)
Video processor: Nvidia Quadro 5000
Router: Kramer VS-84HDCPx1 DVI matrix switcher
Projectors: 4 x Sony VPL-SW535C ultra short throw LCD
Projection surface: Harkness Stagelite Matt White 100 screen