Stanley Weinbaum in his short story Pygmalion Spectacles introduced the world to the idea of goggles that allow users to experience an artificial reality. Weinbaum pondered over this idea and went as far as creating a reality that also has tastes and touch along with vision. Though works of Weinbaum were considered to push the field of fiction away from the doldrums of the early 20th century, the idea of this reality within reality was still considered a bit of a stretch.

However, today, after decades of innovations, we have developed wearable devices that transfer us to an almost life-like reality where one not only gets to ‘see’ but also ‘be’.

As the efforts were being put into making AR/VR devices accessible, making machines to think also have gained traction at the same time. Today, at this intersection of interdisciplinary associations, artificial intelligence is helping to make virtual reality more real.

Currently, Facebook’s Oculus is one such endeavour that has been at the forefront of bringing AR and VR devices to the fore. Oculus Insight marks the first time that fully untethered six-degree-of-freedom (6DoF) headset and controller tracking has shipped in a consumer AR/VR device. Built from the ground up, the Insight stack leverages state-of-the-art computer vision (CV) systems and visual-inertial simultaneous localisation and mapping, or SLAM.

How AI Improves Oculus

Before we get going into how AI is changing the reality, here’s a quick overview of AR/VR/MR:

• Augmented reality – Digital information overlaid on the actual world of reality
• Virtual reality – Full digital representation of our reality
• Mixed reality – A combination of virtual reality and reality

Facebook Reality Labs (FRL) has in-built virtual spaces that allow AI researchers to train a model to understand digital spaces. For example, one such task can be to find missing keys in a life-like digital living room.

The accuracy of these approaches rely on a combination of a well-engineered and integrated camera rig with a high-accuracy depth capture system, a state-of-the-art simultaneous localisation and mapping (SLAM) system, and a dense reconstruction system. 

Research is being done in these domains for decades and many techniques were experimented with. Of which, SLAM techniques are one. However, the technology has only recently become mature enough for consumer applications, such as driverless cars and mobile AR apps.

AI can help solve the challenges of rendering highly compute-intensive visuals in VR.

DeepFocus, which also is aimed at enhancing VR images, relies heavily on deep learning. It constitutes a novel end-to-end convolutional neural network that produces the image with an accurate retinal blur as soon as the eye looks at different parts of a scene. The network includes new volume-preserving interleaving layers to reduce the spatial dimensions of the input while fully preserving image details. The convolutional layers of the network then operate on the same, reduced spatial resolution, with significantly reduced runtime.

To establish the location of objects in relation to other objects within a given space, these current systems use a custom hardware architecture and advanced computer vision algorithms — including visual-inertial mapping, place recognition, and geometry reconstruction. 

A Future Of Dream Like Reality

The idea of this immersive realism can be dated back as far as to the paintings of the 19th century’s Franz Roubard’s panoramic paintings. So far, the applications of AR and VR have been associated with the entertainment industry like gaming and movies. 

However, today, augmented reality has forayed into more pragmatic ventures like training neurosurgeons to gain expertise, design interactive tutorials for schools and many more.

The rise of smartphones with high-density displays and 3D graphics capabilities has enabled a generation of lightweight and practical virtual reality devices. The video game industry has continued to drive the development of consumer virtual reality unabated. Depth sensing cameras sensor suites, motion controllers and natural human interfaces are already a part of daily human computing tasks.

Recently, companies like Google have released interim virtual reality products such as Google Cardboard, a DIY headset that uses a smartphone to drive it. Companies like Samsung have taken this concept further with products such as the Galaxy Gear,

 Any sufficiently advanced technology is indistinguishable from magic

-Arthur C. Clarke 

Over the last 40 years, personal computers, smartphones, and tablets have given us a constant, near-instantaneous access to the digital world through 2D screens, in the process touching almost every aspect of our lives. The team at Oculus believe that the next 40 years will see AR and VR radically change the way we work, play, and communicate. As more research steers towards mixed reality, the reach of AR/VR is turning speculation to reality. And, what new avenues it will uncover is waited to be seen. 

Watch Hyper-Realistic digital re-creation video:

https://www.facebook.com/TechAtFacebook/videos/459546424812742/