After building out a robust platform for experiencing augmented reality and powerful tools for learning and teaching coding with Swift, Apple is tackling AR creation and exploration in its retail stores with new Today at Apple Labs, Walks, and an interactive art exhibit visible only through the lens of an iPhone. Here’s what you can expect the next time you visit an Apple Store.
Nick Cave’s “Amass”
The free Apple Store app has transformed from a portal to Apple’s online store into a portal to another dimension. Download the app on an iPhone before arriving at any brick and mortar Apple Store across the world and you’ll be ready to step into an interactive AR installation by artist Nick Cave. Once you’re on the store’s network, the app will offer an option to launch the [AR]T Viewer.
Nick Cave’s experience is called “Amass,” and is made up of bright metallic objects called Ikon Elements. There are five Elements in total, and you’ll have to move around the store to find them all. It’s a little bit like a game.
Apple’s in-app tool for viewing the exhibit is based on ARKit and works remarkably well. Unlike normal AR apps, there’s no set up and tracking your phone to a flat surface required, the experience just begins. A simple camera interface allows you to capture screenshots or video, and five dots along the edge of the screen show which Elements you’ve collected.
As you move through the store, colorful and spinning 3D objects will appear to float in a swarm from the ceiling above you. The Ikon Elements look a bit like kaleidoscope wind spinners you may have seen decorating gardens. After you navigate the store, you’ll have found all five Elements. Sound effects add to the experience, but can be a little difficult to hear in a noisy space. You might want to wear AirPods.
Once you’ve discovered the full installation, you’re presented a button to release the Elements. This was my favorite part of the experience, so I won’t spoil what happens next — you’ll have to try it yourself. The [AR]T Viewer is a fun way to introduce augmented reality to customers, and Apple’s implementation is slick and non-technical, exposing little of the underlying complexity. The art installation isn’t promoted outside of Apple Stores right now, but I think it’ll be a popular pastime after visitors learn it exists. The only caveat is that you’ll have to own a recent iPhone. The Apple Store app for iPad doesn’t appear to support the feature yet, and store demo iPhones don’t have the app installed.
[AR]T Lab: AR Experiences Co-created with Sarah Rothberg
Coding Labs aren’t new to Today at Apple, but augmented reality is. Digital artist and teacher Sarah Rothberg’s new AR Experiences session is a great introduction to both Swift and AR for beginners as well as a perfect companion to Nick Cave’s installation for customers interested in learning more.
An opening video from Rothberg asks of AR “What can I do with this thing that’s positive?” Apple Store Creative Pros expand on the topic by explaining that you’ll be using Swift Playgrounds to write code that activates an AR scene of your own design. Rothberg stresses the point that code shouldn’t be scary. Think of it as another tool like a pencil or paintbrush.
“Mismatched Time”
We started by viewing Sarah Rothberg’s AR experience “Mismatched Time” on iPads. The project walks you through setting up a scene and triggering animations. Each animation is accompanied by an audio message relating to the passage of time.
After seeing what’s possible with Swift Playgrounds, it’s time to dive in to making your own scene. Teaching the fundamentals of Swift in the confines of a 90 minute session is tough, but the main concept was communicated clearly. Labs like this are usually capped at 10 participants, but the inaugural August 10th session I took at Apple Aventura was a packed house with individuals representing a wide variety of skill levels. We were all able to complete the lesson successfully.
The AR experience we created in Swift Playgrounds during the session.
I’m completely unfamiliar with writing code, but Apple’s sample project is annotated well and provides easy fill in the blank placeholders to help you understand what each line represents. We added Rothberg’s 3D models to our scene, applied animations to them, and added Actions that triggered when you moved near the models in 3D space. At the end of the session we were able to AirDrop our work to personal devices for sharing with friends or to take the project further on our own.
AR For All
In six cities around the world, Apple is also offering an [AR]T Walk co-created by seven artists partnered with the New Museum. If you’re in New York City, San Francisco, London, Paris, Tokyo, or Hong Kong, the experience offers an even more immersive window into augmented reality through the lens of art. Custom creations dynamically interact with the cityscape around you. Early impressions from the [AR]T Walk in New York show how Apple has lowered the barrier to AR by offering an experience you can simply enjoy without having to think about the technical aspects of the platform.
The message of Apple’s new in-store AR experience is clear: augmented reality is for everyone. Just like the technologies we already use every day, AR should enable us to create new things and enhance our lives while receding to the background. With augmented reality, Apple is aiming to evolve the intersection of technology and liberal arts.
