Warpin Media is known for making magic with immersive technology. Brands turn to the Sweden-based company for virtual and augmented reality experiences that change the way consumers interact with products.
Warpin’s early-access partnership with Magic Leap led them to create a high-profile spatial computing experience for a recent H&M-Moschino collaboration. Warpin showcased the Magic Leap One experience at the global launch event for the collaboration in NYC, as well as at several other events in the US and Sweden afterward.
We wanted to learn more about the tech-forward creatives behind Warpin’s flagship experience with H&M-Moschino, so we asked Warpin’s very own CTO Patrik Buckau and 3D Lead Artist Robin Perdén about their design and development process.
How long did it take the team to develop the experience? Can you describe the process from concept to technical implementation?
Patrik: From start to finish, we had about 7 weeks. We were already well-acquainted with Magic Leap’s hardware and software from before, which certainly helped with that aggressive schedule. When we got the brief, there was already a lot of video content created for the H&M-Moschino collaboration, so we realized that we wanted videos to be a core part of the experience. We just had to figure out interesting and intuitive ways of incorporating those videos in a mixed reality context.
Editor’s note: The H&M-Moschino campaign theme happened to be "Moschino TV,” which helped shape the Magic Leap One experience.
Robin: We used Unity and our standard production tools for creating the assets. From an art perspective, it was mostly about taking the 2D videos and transcending them into something new and exciting. We achieved this by creating animated screen surfaces, splitting the videos into 3D layers and having them react to the user’s gaze. We also produced a number of 3D assets that either depicted products from the collection or were props featured in the video material. For some of the 3D assets, we also added the ability to interact with them by “touching” them with your hand.
The experience has a bold, unique vision and surreal aesthetics that transcend any one category. It’s not a “fashion” app any less than it is an artistic experience. Can you talk about working with the creative teams at Moschino and H&M?
Robin: We worked most closely with H&M Labs, Sandberg&Timonen (the creative leads of the “Moschino tv” campaign) and Chimney (producers of the video material). Even with so many moving parts and parties involved, I’d say it was a smooth ride from start to finish. Putting enough time into creating well-thought-out concepts was really helpful. That said, since it was such an artistic experience, it could be hard to explain sometimes. To make sure everyone was able to follow our progress, we did weekly live demos and recorded Capture videos of the experience for those unable to attend.
What inspired the decision to create a spatial computing experience as opposed to a “traditional” 2D or VR app?
Patrik: At Warpin, it’s our mission to help our clients discover and understand how new technology can impact their business in the future. For fashion in particular, we see so much potential for spatial computing. We see the ability to deliver new experiences where the consumer connects with the brand, collections and individual garments on a completely new level. Fortunately, H&M shared our vision and took this brave first step with us. We’re just getting started.
Robin: It’s simply not possible to truthfully recreate pants covered with paillettes in a real-time environment yet, haha. So why not let the user stand in front of the real physical garment — where they can see all the details, move around it, feel it? Using spatial computing, we could let the garments speak for themselves and enhance the experience by introducing these fantastic elements around them.
The experience is built in a flexible, modular way that enables a customizable “fashion room” recreation in any given space. What drove that decision? How do you see the challenge of designing for unknown spatial constraints?
Robin: Since this was a quick turnaround project involving a lot of parties, it was best to make it as modular as possible. This lent us the opportunity to start working on the assets way before we knew the final layout of the physical space. (The final physical space wasn’t even built until a few days before the show!) We designed smaller “stations” that were tied to physical props, those props were podiums, shelves and mannequins. ... On top of those stations, we were able to place single instances of videos and 3D assets in a similar fashion to how Magic Leap’s own “Create” app works — which allowed us to effortlessly add finishing touches to the space just days before the show.
Patrik: We put a lot of effort into developing our scene editor mode where you basically create an experience from within the application using assets from a content library. It allowed the team to test anything from small aspects to how the entire user flow would work by dynamically adding and moving things around the physical environment. ... It also enables the experience to be scaled easily to different venues and events.
In the experience, you cleverly utilize gaze detection as well as hand tracking. What informed the input design choices? Did you experiment with alternative input schemes or did you know what you wanted from the start?
Patrik: We have experimented with a lot of different input methods and found that gaze and hand gestures generally are the ones that are both easy to grasp and feels very natural.
Robin: We found that having videos automatically play as you gaze at them felt futuristic and matched the “zapping between channels” feel that the campaign really emphasized. We also start the experience by having a handbag that melts if you reach out and touch it, as this is many people's first instinct when they see a digital 3D object through Magic Leap for the first time.
The experience is loaded with rich sensory stimuli — beautiful, detailed meshes, surreal GIFs and animations, otherworldly soundscapes and elaborate textures. Did you have to optimize the application and if so, how did you go about it?
Robin: An interesting aspect of doing graphics for the Magic Leap One is that your real-world environment is part of the experience you’re building, which means you get a much higher budget on your individual assets.
Patrik: Most of our optimization efforts were focused on video playback and dynamically switching in and out high-resolution videos as the user shifted context. To be honest though, the Magic Leap One packs quite a punch which had us spend less time than you would think on optimization. Instead, we could put more of our efforts into “creative programming” and tuning user interactions.
Warpin is well-versed in immersive work — your body of work includes many examples of stellar VR and AR experiences. What was it like working with the Magic Leap One in comparison to other platforms? What experiences proved valid, and what did you have to learn anew?
Patrik: Thank you! As a company that has worked a lot with both VR and AR, for us Magic Leap One delivers the best of both worlds. We could utilize a lot of our previous experience with phone-based AR while at the same time not having to care about the restrictions that come with that medium. The biggest learnings come from working with the additive display and the multiple input methods the device offers.
Robin: At first, it feels limiting to not be able to show pure blacks — but once you get used to it and adapt your visual style language to it, you start to realize that you can use the transparency for interesting effects. In this project, we used occlusion geometry to create a portal in the wall, but in some of our R&D projects, we have really utilized it for all kinds of interesting blending and ghosting effects.
Notably, this augmented showroom has no “click to buy!” links. This is not an e-commerce app. The intention here clearly goes way beyond selling products — tapping into the rich history of fashion, art and commerce. What do you hope that people who go through the experience will feel and take away from it?
Patrik: We set out to deliver a very unique experience that people would find exciting and delightful enough to leave a lasting impression. Judging by the reactions we got from our visitors at the NY launch event, we think we managed to achieve that goal.
Robin: This experience was about inviting the users into the world of “Moschino tv” — to truly be enveloped by it. To stand next to the real physical clothes and accessories while at the same time having all of these fantastic virtual elements around them.
Patrik: While this particular experience was all about the fashion and art, the concept can of course be extended to include a direct shopping component going forward.