Ivan Sutherland’s Sword of Damocles, a head-mounted virtual and augmented reality system, was ungainly but remarkably forward-thinking. Developed over a half-century ago, the demonstration in the video below includes many of the components that we recognize today as critical to VR and AR displays, including the ability to display graphics via a headset, a positioning system, and an external computational mechanism.
Since then, AR and VR have experienced waves of hype that builds over a few years but reliably fades in disappointment. With the current excitement over consumer-level AR libraries (such as ARKit and ARCore), it is worth asking if anything is different this time.
The Augmented Connected Enterprise (ACE) team at FXPAL is betting that it is. We are currently building an AR-based remote assistance framework that combines several of our augmented reality, knowledge capture, and teleconferencing technologies. A future post will describe the engineering details of our work in more detail. Here we explore some of the problems that AR has faced in the past, and how we plan to address them.
In their paper “Drivers and Bottlenecks in the Adoption of Augmented Reality Applications” , Martinez et al. explored some typical pitfalls for AR technology, including No standard and little flexibility, Limited (mobile device) computational power, (Localization) inaccuracy, Social acceptance, and Amount of information (Distraction). We address each of these in turn below:
- No standard and little flexibility
- Limited (mobile device) computational power
Advances in contemporary technologies have largely addressed these two issues. As mentioned above, the market appears to be coalescing into two or three widely adopted libraries (specifically ARKit, ARCore, and Unity). Furthermore, limited computational power on mobile devices is a rapidly receding concern.
- (Localization) inaccuracy
Caudell and Mizell echoed this issue in their paper introducing the term, “augmented reality” . They wrote that, “position sensing technology is the ultimate limitation of AR, controlling the range and accuracy of possible applications.”
Addressing this concern involves scanning several real world objects in order to detect and track them in an AR scene. Our experiences so far reveal that, even if they aren’t yet ready for wide deployment, detection and tracking technologies have come a long way. The video below shows our procedure for scanning a 3D object with ARKit (adapted from this approach). We have found that ensuring a flat background is paramount to generating an object free of noisy background feature points. Other than that, the process is straightforward.
Scanning an object in this way generates a digital signature that our app can recognize quickly and accurately, allowing us to augment the physical object with interactive guides.
- Social acceptance
The many issues associated with the launch of Google Glass made it clear that HMD devices are not yet acceptable to the consumer market. But our intuition is that focusing on the consumer market is inappropriate, at least initially, and that developers should instead target industrial settings (as Caudell and Mizell did at Boeing). A more appropriate metaphor for AR and VR devices (outside of their use in gaming) is a hard hat—something that you put on when you need to complete a task.
- Amount of information (Distraction)
Martinez et al. are concerned that the “amount of information to be displayed in the augmented view may exceed the needs of the user.” This strikes us less as a bottleneck and more a design guideline—take care to make AR objects as unobtrusive as possible.
In addition to the issues above, we think there are at least two other problems standing in the way of widespread AR adoption:
There are a variety of apps that can help AR content creators author scenes manually, including Amazon Sumerian, Apple Reality Composer, Adobe Aero, and ScopeAR WorkLink. However, with these tools designers still must create, import, place, and orient models, as well as organize scenes temporally. We think there are opportunities to simplify this process with automation.
Finally, as with any technology, users will not adopt AR unless it provides value in return for their investments in time and money. Luckily, AR technologies, specifically those involving remote assistance, enjoy a clear value proposition: reduced costs and time wasted due to travel. This is why we believe the current wave of interest in AR technologies may be different. Previous advances in the quality of HMDs and tracking technologies were not met with similar increases in teleconfercing technologies and infrastructure. Now, however, robust, full media teleconferencing technologies are commonplace, making remote AR sessions more feasible.
Many tools already take advantage of a combination of AR and teleconferencing technologies. However, to truly stand in for an in-person visit, tele-work tools must facilitate a wide range of guided interaction. Experts feel they must travel to sites because they need to diagnose problems rapidly, change their point-of-view with ease to adapt to each particular situation, and experiment or interact with problems dynamically. This type of fluid action is difficult to achieve remotely when relaying commands through a local agent. In a future post, we will discuss methods we are developing to make this interaction as seamless as possible, as well as approaches for automated authoring. Stay tuned!
 T. P. Caudell and D. W. Mizell. “Augmented reality: An application of
heads-up display technology to manual manufacturing processes”. In
Proc. Hawaii Int’l Conf. on Systems Sciences, 659–669, 1992.
 Martínez, H. et al. “Drivers and Bottlenecks in the Adoption of Augmented Reality Applications”. Journal of Multimedia Theory and Application, Volume 1, 27-44, 2014.