As part of our evolving Digital Culture Strategy, we are sharing a series of case studies profiling how leading practitioners are embracing digital platforms and mindsets to make innovative new work and build connections with audiences.
It’s not often a designer gets a gig for a major organisation that wants to leave the brief open just to see where the designer will take it. But that’s what happened to Melbourne-based creative coder, designer and developer Mel Huang, when she began working with the Australia Council to present its new Digital Culture Strategy.
“It’s super rare that someone says, ‘What do you want to build?’,” Mel said.
Rather than simply uploading the strategy as a document on its website, the Australia Council decided to release it by commissioning an interactive artwork or experience that would exemplify the principles of the strategy itself – a platform for the strategy that would practice what it preaches.
Co-developed through 2020 with industry leaders, experts and creatives in response to the COVID-19 pandemic, the Digital Culture Strategy outlines six key priorities that set a path to increased digital engagement with Australian arts and creativity leading to greater community connection, wellbeing and more dynamic and resilient cultural industries.
As well as bringing in Mel Huang, the Australia Council reached out to Tasmanian-based developer Daniel Reid, whose practice focuses on building digital products for arts and cultural festivals. Daniel and Mel had worked together before, first collaborating on the website for Hobart’s Dark Mofo Festival in 2018.
“Our first reaction was, ‘these guys probably just want a slick web interface to read the strategy on with some fancy digital transitions and a nice reactive design’,” Daniel said. “When we presented a range of options, to our surprise they picked one of the most ambitious ones.”
“They said, ‘Go for it – this is the kind of stuff we should and could be doing as an organisation that represents the cultural industry in Australia. We should be at the forefront of doing interesting digital things and communicating with digital artists in a way that connects to them’,” Daniel recalled.
Recognising how accessibility was at the centre of the Digital Culture Strategy, Mel and Daniel decided to start there.
“So often [accessibility] is something that falls by the wayside,” Daniel said. “In prioritising aesthetic experience people often sacrifice accessibility.”
Rather than simply taking a kind of “checklist approach”, looking at text size and colour contrast, Mel said they instead wanted to explore how captioning and audio could be at the very centre of their design. An accessibility audit, as well as a series of one-on-one sessions with different kinds of users, enabled Mel and Daniel to overcome a series of assumptions they had made.
“It gave us completely fresh eyes,” Daniel said. “Opening up the design process to others was just so helpful.”
Building a voice-double
The result is an immersive, AI-driven, 3D world. Each priority area of the strategy is represented by a coloured sphere that rises and falls as the strategy is read to you, an experience a bit like taking an audio tour through a sculpture garden.
Built utilising three.js, a JavaScript library that creates 3D graphics in a web browser, it is then housed in a React interface.
“But it’s actually more of a sound project than a visual immersive project,” Mel clarified. “It’s about being about to wander around and hear the strategy. That was the focus.”
So what you hear in the experience is not simply a recorded voice being played to you. The audio excerpts that narrate the strategy are actually an AI-driven robotic voice trained on human voice samples.
“Initially we wanted many different voices to represent the diversity of the strategy, but achieving that in a COVID context was quite challenging,” Mel said.
So they used podcasting software Descript, that allows you to train an audio model using your voice. Just 30 minutes of Mel’s voice was enough to train a model to say pretty much anything they asked it to.
“It was equal parts incredible and equal parts terrifying,” she said. “It is quite creepy. But there are nuances of human language that the AI can’t really replicate which is great because then you know that it’s a robotic voice.”
After some massaging to improve a few strange clauses and intonation, the audio clips were then embedded into the interactive experience, with an acknowledgement of Country spoken by Gumbaynggirr technologist Liam Ridgeway and a soundscape from Corin Ileto.
Mel and Daniel hope to release the platform as an open project so that other users can upload their own 3D models and audio.
”It’s all part of that idea of sharing process, sharing code and sharing discoveries,” Mel said. “I think it’s the way of the future – being more open about our information, our resources, our time, and opening that conversation up to anyone who wants to be involved around the work is an incredible opportunity for anyone working at this particular time,” she said.
Daniel agrees, but emphasises that despite the rich experimentation with virtual and hybrid worlds that has occurred during the COVID-19 pandemic, it has also highlighted that nothing quite replaces an ephemeral or live experience.
“A product needs to be made for a virtual space in order to thrive in a virtual space,” he said. “A digital gallery with digital artwork is a very different experience to walking around a physical gallery space. That structure, scale and physical form of real life is still hard to replicate. We’ll only get better and better at it, but they’re not equivalent experiences. Longer-term virtual-specific experiences will take off in a very different way than making the physical experience virtual.”
Explore the interactive Digital Culture Strategy here.
In response to our Digital Culture Strategy, we have launched a four-year Digital Culture Program to develop practice, share knowledge and invest in innovation. Applications are now open for a range of professional development opportunities and we will be announcing new programs and opportunities regularly over the next twelve months.