David Bowie is - AR | Pilar Aranda

DAVID BOWIE IS

On the 72nd anniversary of David Bowie’s birth, the David Bowie is AR exhibition application launched on iOS and Android platforms.

My role

Designer, Developer, and 3D scan coordinator.

How was the

process like?

Multiple iterations on a tight deadline in close collaboration with a small team of developers and designers, who’s paths intersected constantly. Methodic and rapid prototyping, plus fluid communication were a key factors.

Who is our user?

Curious David Bowie Fans and Augmented Reality fans that desire to enjoy the AR exhibition.

Outcome

DavidBowieisReal.com was launched in 2019 for iOS and Android. The app yield outstanding reviews from users, critique, and awards.

I remember, as a child, watching VHS tapes on my family’s old tube TV, mesmerized by a man that looked like he’d just arrived from Mars. Watching him performing in his seemingly endless parade of eye-popping costumes, I never thought that one day I’d be standing in the same room with that eclectic collection of sparkly shoes, floor-length cloaks, and impossible onesies. As the principal coordinator of our effort to scan and document the Bowie Archive’s store of memorabilia, I was thrilled to have such a hands-on role in capturing such an important part of Rock history, but I also felt a heavy sense of responsibility. This was not only the legacy of a man from the stars but a very real one named David Jones, who was leaving behind a wonderful wife, children, and grandchildren.

Why AR?

As the famous world-traveling exhibition “David Bowie Is” was closing its doors permanently, our team was charged with the challenge of translating it into a fully digital immersive experience. We needed to create an intuitive, democratic AR exhibition tool that allowed the audience to enjoy all the assets from their homes.

When you get the call to start a 6-month project that requires more than 500 assets to be imported into Augmented Reality and display them in a way that it hasn’t been done before, you know that the road is about to get delightfully bumpy.

AR Design Process

I was in charge of designing, iterating, refining and finishing most of the AR scenes in Unity, working in synergy with the art director and lead developers. On a regular basis I was working on 3-4 scenes per week, some of them iterating until launch day.

We modeled each scene, or room as we referred to them, by starting with a curated list of items, followed by sketching, and a rapid prototype build that could be demoed the same day. Each scene was iterated at least a dozen times with ongoing user testing.

3D Scanning and Photogrammetry

The 3D scanning of costumes and accompanying assets occurred in parallel with the prototyping and production of the app. I was fortunate enough to be a part of both processes. Due to some prior experience with 3D scanning, I became the 3D scan and photogrammetry coordinator. Scanning of all pieces was completed in 10 days, a timeline only possible with constant coordination with the V&A teams, Brooklyn Museum, 3D scan contractors, and photographers. Further processing of the scans went on for another 4 months due to the unique nature of our technical needs.

Pain points: There was at no point during the process a straight pipeline for generating 3D scans to the specs required for AR (low poly + high detailed textures). The budget for post-processing was also a pressing concern. I developed a strategy to create cost-efficient results, curating costumes that would be processed in more detail, while assigning less time-consuming to different teams.

Findings: we learned that more 50k polygons per costume were our limit, with 4K textures. Photogrammetry textures combined with 3D scans and manually retopology to 50K yielded better results.

First Prototype

I was incredibly honored to be the first one on the team to prototype a full scene for our experience. I created this prototype as I was coordinating the 3D scanning process of all the costumes, as you can imagine it was an extremely agile process. We chose Ashes To Ashes because we already had the costume fully post-processed. We learned a lot about what it means to bring the exhibition to the user's table while I was iterating this fist prototype in Unity.

Pain points: we were undecided about creating a tabletop experience or a full real size version of the exhibit. The prototype helped us to understand the scale. We also realized that the viewer needed more intimacy with the pieces that were displayed. Some times when some unique piece of art is presented in the user's room without special lighting or context, the experience loses power and the item seems mundane.

Findings: we wanted to create an intimate atmosphere with the viewer without losing the benefits of presence in AR, so I opted to prototype a semi-translucent background for this experience that transforms the scene lighting into a mysterious twilight zone.

Sound

Challenges: managing and displaying dynamic 3D audio sources, at multiple angles.

Pain points:

Too many audio sources can take too much computational power.

The 3D audio effects were lost if the app was launched on a mobile phone without headphones.

Findings: While creating immersive aural environments for the app, we determined that using 5 discreet audio sources with customized EQ yielded better results than employing a simple stereo (left and right) audio configuration. We prompted the user to use headphones during the onboarding, but kept this audio setting optional for users.

UI and onboarding

Equally challenging were our directives to create a compiling user interface, navigating system, app menu, and on-boarding.

Pain points: Translating a 3D space into an intuitive 2D user interface took months of discussions, iterations and the help of external UX and UI teams.

Findings: We realized that representing a map would be the most effective way to create intuitive navigation for our 25 virtual “rooms.” Inspired by David Bowie's love of Berlin, we decided to create a navigation menu based on Berlin’s subway map as an homage. This process ultimately added several layers of complexity to the workflow,, so we consulted a map specialist who pointed us in the right direction for our customized map menu.

On-boarding: Due to the fact that this was going to be many user's first mobile AR experience, we needed an extremely self-explanatory onboarding experience. We opted for clear instructions with minimalist images. The iteration process was continual until the launch day.

Voice Over

Finding the right voice for the exhibition was a challenge in itself. Discussions of who we should record the voice over were ongoing until almost the due date. We were extremely lucky to recruit as our narrator Oscar-winning actor, and David Bowie’s friend,Gary Oldman.

Pain points: We had to create a sophisticated Unity event system to ensure that the voice over and the scene audio/music not in conflict.

Findings: giving the user the option to have or not the audio guide was what got better results.

Lighting

Proper lighting in AR is extremely important especially if the asset can match the user's room light. Although many of our rooms transition into a blackout background we had to be very mindful of the amount of light sources, angle, and temperature to enhance the art pieces.

Pain points: for dark rooms more than 3 light sources at a time showed to be quite expensive on mobile processors, leading to glitches and malfunctions. So we opted for baking as many lights as possible.

Findings: without any background in AR we utilized Unity ARKit 2 to match the user's room light worked best, although we realized that in the case of the costumes baking a think layer of shadows into the current texture helped to make it feel more realistic.