The augmented reality (AR) market often prioritizes the visual experience, emphasizing the strength of a device’s optics. But audio is another element that is critical to AR immersion. While Magic Leap 2’s proprietary optics breakthroughs offer an expansive field of view and world-class image quality with industry-first dynamic dimming technology, the sensory audio experience is key to greater AR immersion.
True AR immersion requires advanced spatial audio. In the physical world, our psychoacoustics enable us to naturally locate sources of sounds and formulate a basic understanding of the space they occupy. In other words, we are able to determine where a sound is coming from and estimate how far away the sound is from our position in a space.
When we hear sounds, we perceive the sound’s position based on temporal and qualitative differences between that sound’s arrival at the closer ear (ipsilateral) and further ear (contralateral), combined with conditioned subconscious interpretation of head- and ear-related filtering effects. We perceive the space that a sound occupies by interpreting the differences between the direct path and the reflection paths that seem to characterize the space's reverberation.
But what does this really mean? Let’s say you’re in a room full of people and there are several conversations going on around you. Someone across the room yells your name, trying to get your attention. Even though there is a lot of noise around you, you can still distinguish who is calling you because you know where that voice is coming from.
This is spatial audio -- and it can unleash enterprise productivity by making AR solutions more immersive and useful. To blend the physical and the digital worlds, which is at the heart of AR experiences, the AR device must let you perceive the location and direction of sounds to realistically place these sounds into a 3D space -- with greater clarity than you would have typically.
Magic Leap 2 represents a number of innovations that elevate the user experience of augmented reality. Powered by impressive compute performance and rigorously tested in custom-built labs, Magic Leap 2 features best-in-class spatial audio capabilities, including:
These capabilities are supported by robust developer tools that make it easy to integrate spatial audio, as well as access and fine-tune more sophisticated acoustic design.
Spatial sound can enhance the immersive AR experience that blends our digital and physical worlds in a broad range of enterprise scenarios and environments.
Steering your attention to the right place
Attention steering guides your awareness to specific directions. For example, your attention can be redirected to something outside of your field of view using spatial audio sources. And multi-sensory spatial awareness is critical in enterprise settings.
For example, you can predict the kinds of noises you’ve heard before on the manufacturing floor or in the operating room. With all the sounds you’re familiar with, your brain actually builds a model for how the sound sources typically sound. Your attention is then more easily directed to what’s important, which can be signaled by an unfamiliar sound -- a beeping of a machine that wasn’t beeping before, or a signal from the device to focus on a key piece of data.
Spatial audio can steer a technician’s attention to the correct place they need to focus on to fix a machine or an area that needs to be repaired. It can also steer a warehouse worker to the right place in a bustling facility to locate items for customers or prepare a shipment for delivery.
While Magic Leap 2 has industry-leading optics with a 70° field of view that delivers a more expansive digital workspace, spatial audio adds another dimension to the immersive nature of the AR experience beyond the field of view. You can hear three-dimensional audio rendered from specific locations in space outside of what you can actually see, which is a key benefit for enterprises.
Being present and connected with others -- virtually
Compelling spatial audio depends on more than what direction a sound is coming from. The interaction a sound has with its environment to produce indirect sound assists localization and externalization cues, in addition to informing one’s physical surroundings.
In most video calls, you view a flat 2D screen that doesn’t prioritize who is speaking, which conversations are most important, and where to direct your attention. This can make it difficult to understand and engage in conversations. Magic Leap 2’s co-presence capability, powered by the spatial audio engine, lets you focus on, and tune into, specific conversations.
Co-presence is a communication feature that refers to being in the same physical setting as the communication participants -- face to face. When you’re co-present, the people you are communicating with are more “real” or “salient.” In increasingly virtual or remote work settings, this is important, because without it, our awareness of others is reduced, we aren’t as responsive to each other’s ideas, and we have less public awareness. With co-presence in an enterprise AR environment, you get the crucial sense of being present and connected with others in that virtual environment, and you’re more agile, adaptive, and engaged.
Hear what I hear -- with remote assistance
Along with co-presence, remote assistance is an important piece of the spatial audio puzzle of enterprise AR. Using remote assistance on the Magic Leap 2 platform lets you meet virtually in a physical space to collaborate and learn, while driving productivity and situational awareness.
In the telehealth market, for example, AR-enabled remote assist lets surgeons receive real-time intraoperative guidance from other physicians, surgical specialists, or medical device/technology experts. Such guidance can be critical when a surgeon is presented with a complex procedure and may benefit from additional technical skills or expertise or during procedures that involve devices or implants that necessitate input from an industry representative. And if a procedure is unplanned and time-sensitive, it may not be feasible for a remote expert to travel to the site of the operation or transport the patient to another facility. Remote assistance capabilities can alleviate these constraints, particularly in rural settings, where surgical resources might be limited.
Manufacturers are also using remote assistance on the Magic Leap platform to enable technicians to meet virtually in a physical space to collaborate and capture knowledge from more seasoned machinists, many of whom are retiring. Such collaboration has helped them realize improvements in savings, productivity, quality, recruitment, and employee retention. They’re reducing training time from weeks to days, delivering new efficiencies and process improvements. And along with reduced training time, they’re saving money by reducing scrap and minimizing human errors.
Magic Leap 2 empowers developers with easy-to-use spatial audio features and finer controls. Automatic acoustic mapping, sound occlusion, and hardware acceleration are simple to implement and use in a custom Unity plug-in via C APIs. And Magic Leap 2 features finer acoustic scene control to completely customize the sonic environment.
Industry-first automatic acoustic mapping takes the guesswork out of audio development.
Magic Leap 2 can match reverberation to your environment to replicate human perception. It’s the first AR platform to offer automatic acoustic mapping of your environment for realistic and spatially-aware AR experiences.
The device’s microphones can listen and measure the reverberative properties of your current acoustic environment -- a novel technique that allows virtual sounds to be automatically rendered with reverberation that matches your physical environment to more seamlessly blend real and virtual content. It’s a game-changer for AR developers, who previously needed to guess the settings for unknown environments.
Acoustic scene design allows for flexible, custom audio experiences.
Acoustic scene design gives developers increased flexibility and customization to define the acoustic properties of rooms, regions, and particular bodies, with the option of incorporating automatic acoustic mapping. This enables more realistic, useful audio for solutions. For example, developers can customize the amount of sound obstruction when something is placed between you and the sound source.
Hardware acceleration reduces latency and improves performance.
For improved performance and a seamless audio experience, Magic Leap 2 can offload audio processing. Hardware acceleration lets developers offload the audio processing from application resources to dedicated resources, which helps improve performance and free up application resources for other tasks. Developers can simply enable this in our Unity tools with a checkbox or keep the audio processing in the main CPU in order to create more custom AR experiences.
Magic Leap 2 provides best-in-class spatial audio for the most immersive AR experiences. With these capabilities, you get realistic, high-quality audio with industry-first capabilities, a custom combination of hardware and software, and a suite of easy to use tools for developers.