On Tuesday, Ars had the opportunity to sit down with Microsoft's Director of Communications, Greg Sullivan, to walk through a quick demonstration of the company's new mixed reality headset, the HoloLens 2. HoloLens 2 isn't a full-immersion virtual reality device like Oculus Quest; it's an augmented reality device—think Pokémon Go on steroids. Sullivan never used the word "augmented" at all, preferring instead to talk about "mixed reality."

When I asked the difference, he explained that the majority of the content and techniques a user experiences with HoloLens2 would translate directly to a hypothetical full-immersion VR headset with the same sensors. "Mixed reality" is a blanket term that encompasses both augmented reality—which specifically mixes elements of virtual and real world—and virtual reality, which at least in theory immerses you entirely in the virtual. Sullivan went on to point out that even what we usually think of as fully virtual reality can't avoid the real world entirely—the six degrees of freedom (6DOF) a VR user experiences end abruptly if the user bonks into a real-world wall.

Ergonomics and eye tracking

If you're accustomed to VR headsets, one of the first striking things about HoloLens 2 is its weight—or lack thereof. Most VR headsets feel something like a diving helmet—they're heavy and oddly balanced, turning every VR session into a neck workout. HoloLens 2 is considerably lighter, easier to adjust, and pretty much perfectly balanced. It has a single adjustment knob in the back, like a hard-hat or welder's mask, and it slides on and off easily. Adjusted properly, it "floats" across the bridge of the nose, resting a little but not all of its weight there.

With the device on and adjusted, Sullivan led me through its features. I paid particular attention to tracking—most mixed reality devices know where you're pointing your head but have no idea what you're actually looking at. HoloLens 2 actually tracks your eyeballs themselves, so it knows what you're really looking at, and even how far away it is. The effect is uncanny—in one demonstration I was asked to read a block of text, which automatically scrolled down as my eyes tracked across it.

The result of this eyeball-tracking reading experiment felt oddly like telepathy—slightly frustrating telepathy, in this case. The text scrolls automatically as you look directly at the bottom row of text in a block of six or so rows; I normally read two or three rows in a glance, so it felt like needing to look slightly past the bottom to make it scroll.

Obviously, this is something that can be easily tuned—but it's intriguing how unfair it feels when something seems like a psychic power but still doesn't quite understand you.

Hand tracking

The lack of handheld controllers with HoloLens 2 also grabbed my attention—I'm used to pinching and grabbing, but with most VR rigs, this is done with the aid of handheld controllers that track your hands' position and give you triggers to squeeze. That effect eventually feels pretty natural, but it's unnecessary with HoloLens 2, which instead tracks your bare hands' position and actions visually. You can reach out and pinch hologram elements between thumb and forefinger, tap buttons with an extended finger, or "visually tap" far away items by looking at them, then pinch your thumb and forefinger outside your own field of vision to "click."

HoloLens also gives you the chance to feel like a cyborg; several parts of the demonstration utilize a virtual control panel mounted on your left wrist. If you raise your hand with palm facing you and look at your left wrist, you get a series of buttons apparently embedded in your arm which can be tapped; some applications also offer a larger, hand "held" control panel which appears inside your opened left hand.

These virtual control panels came in handy in a simple drawing app, which allows a user to free-form sketch three dimensional shapes in the air. Depending on the tool selected from a palette in your left hand, you can draw, rotate, clone, or move these free-drawn shapes; I very quickly littered the whole room with bizarre arcs and patterns only I could see.

Visual effect

The one negative thing I have to say about HoloLens 2 is the visual effect. Microsoft describes it as injecting "holograms" into the real world, and this description isn't far off the mark—the effect is a lot like primitive holographic displays you might remember from Epcot in the late 1980s. Microsoft's own PR pictures show mockups in which hyper-realistic image overlays are blended seamlessly into a worker's vision; if HoloLens 2 really is capable of that kind of thing, I didn't see it.

My visual experience with HoloLens 2 was very retro-futuristic, reminiscent of movie graphics from the late 1980s. Pixels were individually distinguishable in most cases, and the images in general were shimmery and tended to fade in and out at the periphery of vision in unhelpful ways.

The virtual holograms are also not entirely opaque; while easily solid enough to focus on, they can be seen through by a user willing to focus at a distance. This isn't necessarily a bug; the kit costs $3,500 as an upfront purchase or can be leased for $125 per month, and is intended for industrial use. If you're on a factory floor or climbing a utility pole, you probably don't want "helpful" overlays you can't see through if you need to. (Industrial users can also flip up the display screen entirely if holograms get in the way of work at a crucial moment.)

But if you were hoping to play games with overlays as solid-looking as those on a smartphone's display while playing Pokémon Go, you'll be disappointed.

Conclusions

HoloLens 2 represents some big and impressive moves forward in the mixed-reality industry. The sheer ergonomics of the headset make extended use a pleasure, and the advanced eye and hand tracking open up new possibilities in both control and immersion.

While its relatively unimpressive visuals would disappoint a gamer, it's important to note that gaming isn't the target for HoloLens 2—and that Sullivan made a point of saying that everything in HoloLens 2 would translate well to a hypothetical, similarly equipped full VR rig as well. We suspect Microsoft finds the current $3,500 price-point too steep to bother making a gamer version, but once the cost drops, it seems certain that the company's gaming division will develop similar consumer technologies.

Listing image by Jim Salter