Microsoft Takes Augmented Reality Deeper with HoloLens 2, Kinect Dev Kit

MOBILE WORLD CONGRESS — Microsoft has taken the wraps off of its new HoloLens 2 commercial 3-D headset, a new developer kit for its Kinect sensors and new Dynamics 365 AI tools, designed to simplify the delivery of “augmented reality” solutions.

Introduced Sunday on the eve of the annual Mobile World Congress (MWC) in Barcelona, the new HoloLens is a significant upgrade over the inaugural version of the software giant’s headset that includes a holographic computer, letting users view their surroundings in 3-D with holograms.

HoloLens 2 will arrive in the second half of this year as a more comfortable and flexible headset with significantly improved visualization and double the field of view, Microsoft officials said during a launch event at MWC. The company also is expanding its ecosystem and tools to bring new solutions to market. The enablement efforts will touch on software developers, ISVs, integrators and vertical solution providers.

Microsoft Dynamics 365 Remote Assist

Microsoft aims to make it easier to integrate HoloLens with its own offerings, notably in the planned April semiannual update to Dynamics 365, which will include new AI-enabled point solutions. For example, Dynamics Remote Assist will enable HoloLens to provide remote troubleshooting by adding visualization of the scenario.

In addition to providing deeper integration with its own software, including Microsoft Teams and Power BI, Microsoft also is opening HoloLens to an expanding ecosystem of partners including Mozilla, which is enabling support via the Firefox browser (currently the Windows 10-based HoloLens requires Edge).

HoloLens 2 headsets will cost $3,500. Though formidable, Microsoft officials said that’s less expensive than the current model, which carries a $5,000 price.

Among the notable improvements to headset and its embedded holographic computer, HoloLens 2 will have a new display engine that will double the 3-D headset’s field of view, while maintaining the 47 pixels per degree of sight, which is critical to view text as small as eight point.

Microsoft technical fellow Alex Kipman introducing HoloLens 2 at Mobile World Congress.

“This is the equivalent of moving from a 720p television to a 2K television for each of your eyes; no such technology exists in the world,” said Microsoft technical fellow Alex Kipman, speaking at the launch event.

Besides the improved field of vision, HoloLens 2 will use new sensors with embedded AI and semantic understanding, including support for Windows Hello authentication. Microsoft also said HoloLens 2 will offer more precision; for example, when viewing something, it will better understand the user’s intent and render the holograms in real time.

More Comfortable

In addition to the improved capability, Microsoft said HoloLens 2 is lighter and more comfortable than its predecessor. It’s manufactured with a lighter carbon fiber material. Along with various other improved ergonomics, including a flip visor, Microsoft said the new headset will better lend itself to …

… switching tasks and more suitable to wear for several hours at a time.

HoloLens 2 sports a new visor flip.

“HoloLens 2 sets the new standard for comfort in mixed reality headsets,” Kipman said.

The launch signals Microsoft’s plans to expand its investment in HoloLens, which the company first demonstrated in early 2015, at the same time Google suspended sales of Google Glass, which was in beta for two years at the time.

Google Glass, which was marketed for consumers, from the outset was criticized because of privacy concerns. In contrast, Microsoft launched HoloLens as an industrial-grade, mixed-reality headset that enables 3-D experiences.

HoloLens and Kinect have become key components of Microsoft’s strategy to enable AI using 3-D augmented reality, making it easier for simulation in functions such as manufacturing, health care and architecture, among others.

Microsoft’s HoloLens came to market as a developer edition in mid-2016. Various customers, including BMW, Boeing and Virgin Atlantic. have tested or implemented it for specialized use cases  Late last year, the U.S. Food and Drug Administration (FDA) approved a solution developed by partner Novarad, for use in presurgical planning.

“Our success lies in the success of our partners and customers and our ecosystem,” said Microsoft CEO Satya Nadella, at Sunday’s HoloLens 2 launch event. “This requires that we collectively move from being just passive consumers of technology to active producers of technology; we create technology so on customers can build their own technology.”

Partner Enablement and Ecosystem

During the hour-long presentation and demo, Kipman demonstrated the new HoloLens 2 with various partners, including PTC and Trimble, and emphasized the opportunity for partners. Trimble, which last year announced its XR10 hard hat, says it will use HoloLens 2 to provide access to holographic 3-D information at construction sites.

At MWC, Trimble vice president Roz Buick joined Kipman on stage to explain how the XR10 can both improve construction activities and safety.

“The XR10 lets architects, engineers and contractors bring the 3-D constructible model detail and process all the way from office to field and onto the tool, so now you can visualize and build exactly what you want,” Buick said.

New Dynamics 365 solutions for HoloLens 2, announced in September and slated for the April update, include:

Microsoft’s new Azure Kinect DK, available now for $399, brings together the Kinect AI sensors for building elaborate …

… visualization and speech models. The 7-inch device has a sensor and spatial microphone with a video and orientation sensor and comes with SDKs and related documentation.

Julia White, Microsoft’s corporate VP for Azure, introduces Azure Kinect DK at a MWC launch event.

“Azure Kinect is an intelligent edge device that doesn’t just see and hear, it understands the people environment, the objects, the actions — and the level of accuracy you can achieve is unprecedented,” said Microsoft Azure corporate VP Julia White.

The tool is the latest iteration of Kinect, the motion-detection sensor device first introduced as an add-on to its Xbox gaming system and later for vertical solutions in retail, manufacturing and health care. Microsoft and ISV partner Ocuvera is using the kit to help aids and nurses at Cleveland Clinic predict when a patient is about to fall, White said.

Nadella noted in his opening remarks that 1 million hospital patients in the U.S. fall each year, 11,000 of which result in fatalities, which White emphasized.

“It’s in fact Ocuvera that’s building this solution with Azure Kinect to proactively alert nurses to a likely patent fall, so the nurse has the ability to reach that patient before the fall occurs,” she said. “We’re very excited to be working with Ocuvera to get those fatal patient falls from 11,000 down to zero.”