Spatial mapping and the companies trying to digitize our existence
Sure, it sounds cool, yet this “virtual twinning” effectively extends the possibilities of both the “world of atoms and the world of bits“ as HoloLens inventor Alex Kipman eloquently put it in a TED talk which has been viewed over 3.5 million times.
This is not some distant blue-sky thinking. There is already an entire ecosystem emerging around the challenges of sensing, mapping, and replicating objects from the real world into the digital – and vice-versa. In fact, one of the main applications of the 5G connectivity infrastructure rollout will be to mirror physical objects and environments digitally in real-time. That, in turn, would enable XR (extended reality) applications to be built which bring quantifiable value to users. 5G applications that consumers will “get” “I don’t think the average person really comes close to realizing just how much of a quantum leap we’re talking about,” says Deeyook CEO Gideon Rottem. “They think it will be faster, but literally it will enable things that will make what we have now pale in comparison,” he adds. Rottem’s Israel-based company has developed a self-learning tracking solution that measures the angles of wireless transmissions – tapping into the existing 1.7 billion wireless access points that already exist globally – to determine positioning of employees, items and assets (within 10 cm) without tapping into a company’s IT network and without invasive personal tracking or data capturing. "Although consumers are intrigued by 5G, speed on its own is not enough to sell them on the idea of upgrading." These mapping technologies are going to interact with the cloud differently, as it no longer becomes necessary to upload images in order to understand what the user is standing in front of. According to Rottem, once we achieve accurate positioning, we can also create accurate virtual worlds (largely automatically) so the integration between the physical and the digital can become seamless. A newly released report by research and consulting firm Strategy Analytics revealed that although consumers are intrigued by 5G, speed on its own is not enough to sell them on the idea of upgrading. This is partly because it is difficult to grasp what it will enable them to do. CSPs will need to solve existing pain-points and entice customers with new experiences. In practical terms, spatial mapping technologies are crucial in enabling the type of integrated applications that not only address practical customer needs but also create these “wow” factors that will prompt mass adoption of 5G. The above research found that what got consumers most excited about the prospect were immersive experiences such as holographic conversations and XR field training. It’s not a surprise, therefore, to see many companies in the connectivity space embracing immersive technology and content as they seek to bridge this “imagination gap”. An example of a CSP doing just that is UK mobile giant Three, which recently released a highly produced tongue-in-cheek advert which shows us what a 5G-enabled holographic future could look like. “Hold on to your crumpets,” says Three in their recent 5G ads. Moetsi is a company working to build the spatial mapping infrastructure to make that future happen by offering “sensor stream processing on the edge.” This effectively boils down to processing vast amounts of data in real-time and delivering it in a format that makes sense and is relevant to the context of where it is being applied. To illustrate how an application could leverage this, Moetsi co-founder Adam Polak gives an example of someone about to cross a street getting alerted that the car moving towards them isn’t slowing down for the red light. At that point, all our decisions would potentially become augmented and informed by tapping into these edge-processed datastreams, fundamentally changing our relationship with technology, the world around us, and one another. 3D mapping in arts and entertainment There are use cases already emerging of spatial mapping being applied to create, transform, and augment art. Award-winning immersive content studio HEAVY has partnered with the Marriott hotel group to build a “secret digital garden” at the JW Marriott Anaheim Resort which was due to open in April. The project leveraged mapping technology from 6D.ai, a company that set out to build “a 3D map of the world” and has since been acquired by Niantic, the makers of Pokémon GO. This feature of persistence is key to understanding the value these real-time mapping technologies bring to such experiences. In this case, HEAVY produced a fine art installation blending physical, custom-made sculptures with augmented reality elements to create a serene space where resort guests and visitors can plant their own fantastical trees while enjoying playing among digital caterpillars and butterflies. What makes the zen-like experience more than a bit of fun, however, is the narrative backdrop for the magical plant’s evolution from sapling to tree. By performing the digital meditations, each guest will figuratively transform into a different kind of tree, personalized around their choices, which can then be further nurtured over time like a flora version of Tamagotchi. This means that the way users interact with and nurture their trees will affect how it develops, and consequently what type of experience each person generates and enjoys. For HEAVY’s Head of Studio Chris Nunes, this crossover between data and art, and between the physical and digital elements, comes together through the persistent context that spatial mapping technology enables. In other words, it is about creating a hybrid world which users can navigate naturally and instinctively. “To really explore this landscape well, we need to use best-in-class technologies like computer vision, cloud and edge computing, machine learning, and 6DOF display tools in interesting and approachable ways,” he explains. How will this technology come to life? Improving the speed at which connected devices can stream information to an authoritative server will be a crucial piece of that puzzle according to Polak, as improvements in networking such as 5G will help reduce the latency and increase the amount of sensors that can communicate in a local area, something that the current generation of networking infrastructure simply cannot cope with. “The engineering challenge is enormous: being able to interpret any sensor data correctly in a manner that can be used by application developers to provide value to end-users, basically not only seeing, but understanding anything a device can sense in the world,” Polak adds.” So, are we there yet? Well, yes and no. A lot of this technology already exists but it is hampered by the fact that nearly every XR developer has to roll out their own solutions to build cross-platform applications that can be shared with all users. According to Polak, initiatives like OpenXR are addressing the problem of having to build separate implementations for ARKit, ARCore, and MRTK (Microsoft’s Mixed Reality Toolkit) by enabling developers to access XR device data through a single interface. Once we arrive at that convergence of 5G connectivity and the ability to interact with digitized objects in platform-agnostic experiences that are embedded in our own physical reality, our world as we know it will never be the same. We will see an explosion of XR applications that provide immense value industry and enterprise, as well as mind-blowing experiences for individual consumers. So while wrapping your head around the concept of building a “Mirror World” might be difficult at first, once it clicks into place we’re likely to wonder how we ever settled for such an ordinary reality in the first place.