Spatial Computing: An Overview for Our Techie Friends

By: Rony Abovitz, Paul Greco, Yannick Pellet, Hudson Welch, Sam Miller

August 27, 2018

A Personal Spatial Computer

Magic Leap One allows digital content to move beyond the confines of the 2D screens and computers of today. It is a biomimetic system that, through a combination of Digital Lightfields, sensing, and compute - acts as a co-processor to the already present, and incredibly powerful human brain. We focused on making a proprietary Digital Lightfield signal to work with - not against - the human eye-brain system. We use the visual cortex as our display. The Lightwear and Lightpack’s sensing and compute provide the platform of the future for developing environment understanding, context awareness and human-centered AI.

Here are few key properties of Spatial Computing:

  • __Fields:__ The spatial computer merges the analog light- and sound-fields we see and hear from the real world with the digital light- and sound-fields born from the imaginations of creators.
  • __Persistence:__ The sensor suite allows digital content to maintain their position in the real world as users live and move around (“pixels can stick to the world”).
  • __Interactions:__ The digital content presented by the spatial computer can respond to natural human signals (head motions, eye gaze, hand gestures, speech) as well as handheld controllers.
  • __Context:__ The digital content presented by the spatial computer has explicit geometric and semantic knowledge of our real-world surroundings.
  • __Connections:__ Just as humans have face-to-face and remote connectivity (telephony, social networks), data access (web, IoT), and action-from-afar (IoT, robots), the digital content we access through spatial computers are part of our broader network. 
  • __Respect:__ Through context, connection, and knowledge-of-the-past from persistence, digital content that exists with us can respect and enhance the human experience.

Dynamic Digital Lightfield

Light comes from multiple sources in the environment around us and bounces in every direction. This is referred to as an analog lightfield. When light enters our eyes, our binocular vision provides depth, color, size and speed information to our brain allowing us see a full picture of our world.

photo1 Analog lightfield Signal interacting with human eye-brain system.

A Dynamic Digital Lightfield is a binocular display that can project digital objects into the world such that light enters the eye as if it were reflected from a real object. Our lightfield is biomimetic (mimics our biology/physiology). Magic Leap One creates a seamless experience where digital and analog lightfields combine into a single scene. For example, with Magic Leap One, you can render digital flowers in a physical vase so they can appear fresh every day.

photo2 The Dynamic Digital Lightfield photonics chip combing the unobstructed analog light field from the real world with a digital lightfield.

Magic Leap One Volumetric View

Spatial computing is about volumes and spaces, so a traditional 2D field of view measurement does not adequately capture the volume of depth that someone can access through Magic Leap One.

To fully describe the three-dimensional attributes of space directly visible through Magic Leap One, we use the term volumetric view.

The Digital Lightfield can display volumetric content in a frustum that stretches from your hand (37 centimeters / 14 inches) to the horizon (infinity). The volumetric view has a 50 degree diagonal and a 4:3 aspect ratio.

The Digital Soundfield can create digital sounds from your hand to the horizon, 360 degrees around you. In the natural world, sounds draw your attention to where you should look. The Soundfield effectively extends the Lightfield volumetric view by drawing your attention towards digital content.

FOV image 0 To enable true spatial computing, Magic Leap One combines Digital Lightfields, Digital Soundfields, and the ability to operate with no external sensors or compute. Digital content can live 360-degrees around you.

Keep in touch