Augmented reality goes to work

Last summer, Pokemon Go introduced the concept of augmented reality (AR) to millions of people. But AR is about more than just using your phone to chase virtual creatures in the real world. At warehouses, on factory floors, and in manufacturing facilities around the globe, AR is putting on its hard hat and getting to work.

At Samsung's European Global Parts Center in the Netherlands, for example, warehouse employees have been running a pilot using Ubimax software and Google Glass Enterprise Edition. Workers scan barcodes on incoming orders, and then Google Glass displays the aisle, row, and box where the part can be found. They then scan a QR code on a Samsung Smartwatch to indicate when a part has successfully been picked.

Samsung's Vision Picking program has increased production speed by 12 to 22 percent and slashed errors by 10 percent, according to Robert van der Waal, director of logistics operations for Samsung Electronics. He says Samsung is looking at deploying the technology to other warehouses across Europe.

Samsung isn't the only company seeing big productivity gains from AR: 

• Boeing has used Google Glass to display technical diagrams to workers assembling electric wire harnesses for aircraft, leaving their hands free to work while cutting assembly time by 25 percent.
• GE is using AR to measure gas turbine nozzles at a facility in Florence, Italy. A process that used to take most of a day now takes less than an hour, says Shayam Rajan, Edge Lab leader at GE Global Research. 
• At Mortenson Construction, a $3.7 billion firm based in Minneapolis, contractors can don a $15,000 DAQRI headset, walk through a 3D model of a hospital under construction, and see where the electrical wiring might interfere with the plumbing before either is actually in place.
• Technicians at German elevator manufacturer thyssenkrupp are using Microsoft HoloLens to visualize and diagnose technical problems, reducing the time needed to complete repairs by up to two-thirds.

Besides introducing hands-free computing to blue-collar workers, AR is also bringing new challenges to enterprise IT staff who need to manage and accommodate the new technology.

"Right now, the biggest technical challenge is that AR is the Wild West of everything," says Marc Kinsman, an immersive technologies developer for Mortensen Construction. "There are so many different hardware and software options. It's hard to ask someone else, 'How would you do this?' because we're just making it up on the spot."

Smartphones on steroids

In some ways, AR gear poses many of the same challenges smartphones and tablets PCs did in years past. The problem is that AR today is a bit like smartphones in 2005. There's a wide range of hardware platforms—more than 50 sets of smart glasses are currently in production—along with multiple operating systems. 

"A lot of these products are in beta, which means support resources are scarce," says Ross Rubin, principal analyst at Reticle Research. "Unlike an iPhone, Android handset, or Windows PC, they're not well understood."

With the HoloLens, at least, enterprises are working with a relatively well-known OS, notes Aviad Almagor, director of the mixed reality program at Trimble, which has developed a HoloLens app that allows architects to create 3D holograms of buildings in the design phase. 

Because the HoloLens runs Windows 10, managing and securing it is not that much different than doing so for any other network device. "Once devices are enrolled, the IT team can utilize Azure Active Directory functionality, manage settings, install apps, and control updates based on company security policies," Almagor says.

Still, mixed reality devices will raise a slew of new security, privacy, and compliance issues.

"Security is going to be front and center, and the way IT addresses it will depend to some extent on the device itself," says Tom Mainelli, a vice president at IDC.

Head-mounted displays (HMDs) that run on Windows or Android—or tether to a device that does—can rely on well-established security tools and practices. But standalone devices that use their own OS are another story. "As more of these next-generation operating systems appear, managing and securing them will become critical," Mainelli says.

Then there are issues like bandwidth, battery life, and maintenance.

Delivering complex 3D images to headgear over a wireless network could max out the bandwidth of some companies, says Rubin. Devices that are used remotely, like outdoors or on a factory floor, may need to operate without the network, then sync their data when connected again.

Storing content on the device itself may solve some bandwidth problems, but that usually requires a more expensive device with a shorter battery life, adds Mainelli. It's a difficult balancing act. "Products need to be durable enough to withstand factory floor abuse, but light enough to wear all day," he says. "They need to have all-day battery life or a way to hot swap the battery."

Another consideration is user training and support. Employees who are replacing a clipboard or paper manual with AR gear may be using computers in a work setting for the very first time.

Enterprises may find themselves relying heavily on third-party vendors for support and training. For example, Ubimax provides training on the shop floor for its customers, says Percy Stocker, the company's COO. He adds that even new users usually get up to speed pretty quickly, thanks to the devices' intuitive interfaces.

It's the content, stupid

As with smartphones and tablets, AR is useless without the right content. However, creating industrial-strength augmented or mixed reality solutions is far more complex than developing a mobile app.

Developing high-quality HoloLens solutions is more like developing game software, because a major part of the user experience depends on how well the app takes advantage of mixed reality. "The HoloLens requires a different kind of developer than in the past," says Björn Schulte, head of IT management at thyssenkrupp Access Solutions. "It's less about process efficiency, data structures, or user interfaces." 

"Today's AR standards are very immature," adds Rubin. "Even for something like HoloLens, which has a native Windows underpinning, there are whole new sets of APIs you'll need in order to create 3D objects and overlays. It will require thinking in three dimensions—a game-like mentality in terms of graphical objects and interactions."

Content creation will be key, says GE's Rajan, but it's complicated by the profusion of different processors, operating systems, and ways of accepting and displaying content on the various devices.

Another problem: There just aren't that many tools available for building AR apps, says IDC's Mainelli.

"To date, I’d say most large enterprise and industrial shops have had to not only build their apps, but build their own toolboxes to build the apps," he adds. "There is a growing ecosystem of tools out there, but right now, the tools are one of the most limiting factors. Once that improves, we’ll start to see what developers can really do with this technology."

The augmented office

Ultimately, industrial settings are the proving ground for AR in the workplace. Right now, most AR gear is too bulky and expensive for use outside a factory or warehouse. As the technology matures, AR will find its way off the factory floor and into corporate offices. 

Apple's new ARKit development package will likely spur a lot more interest and experimentation with AR devices, says Mainelli.

"The challenges right now revolve around the limited tools, limited access to expert developers, and limited access to affordable HMD hardware," he says. "As these challenges are met, the biggest issue a forward-thinking company is likely to face is managing the massive amount of change that AR could bring to wide a range of business processes."

Rubin sees AR becoming prevalent as a data visualization tool in a diverse range of environments.

"AR will allow employees to play with different data sets and explore them from different perspectives," he says. "An employee could be walking through a retail environment and see overlays of how foot traffic looked in certain sections of a store. Or they could look at a piece of merchandise and overlay the sales figures for that SKU over the past three months. It will be useful for any application that frees people from having to look at a screen at a particular time, and overlay information in a way that sparks insights that might not otherwise be available in that environment."

As the physical and digital worlds converge, AR will become a fundamental way we interact with machines, just as voice and touchscreens have supplemented—but not replaced—keyboards and mice.

All of which will increase even more the demands on IT to support, maintain, and develop for the technology.

Augmented reality: Lessons for leaders

• Security will be front and center, and the way IT addresses it will depend to some extent on the device itself.
• Enterprise AR development requires a kind of new developer who can think in three dimensions, a game-like mentality in terms of graphical objects and interactions.
• Content creation will be key but complicated by different processors, operating systems, and ways of accepting and displaying content on various AR devices.
• Industrial settings are the proving ground for AR in the workplace.
• AR could become prevalent as a data visualization tool in a diverse range of environments. 

Image credit: Google Glass by Kārlis Dambrāns, used under CC BY / Colors adjusted from original