Matthew Renze

Hello HoloWorld

First impressions developing holographic applications for the Microsoft HoloLens

Author: Matthew Renze
Posted: 2016-05-01

Microsoft HoloLens

I received my Microsoft HoloLens Dev Kit a couple weeks ago. Since then I've been exploring the device, the apps, and development tools. Since the HoloLens is such a new and potentially revolutionary technology, I thought it might be valuable to compile my initial impressions from my first couple weeks of developing software for the HoloLens.

The HoloLens Device

At first glance, the device itself doesn't look and feel like a developer prototype at all. It feels more like a finished product in terms of its construction and aesthetics. In fact, even the box and packaging seem more like a finished product than a dev kit (at least compared to my Oculus Rift DK2).

The visual display projects images in front of your eyes in a way that makes things appear as if they are really in front of you. It's hard to explain unless you've actually used one, but the effect is very convincing. In fact, after a few minutes of use, your brain seems to forget that this is all just an illusion.

The down side is that the augmented-reality view port is limited to a medium-sized rectangle in the center of your total field of vision. This, in my opinion, is the biggest downside to the first iteration of the HoloLens hardware. However, if this limitation can be overcome before the commercial version, Microsoft will have a truly immersive and game-changing technology on its hands.

The HoloLens is a bit heavy, so I don't expect you could wear it for too long without a bit of neck tension. However, I assume that the weight will be reduced with future iterations of the design as well. Adjusting the head band so that it puts more of the weight towards the top of your head, rather than on your forehead, seems to help reduce the stress on my neck though.

The battery life appears to be about 2-3 hours of continuous use. While this might not seem like much, I still haven't run out of power while using it in my first two weeks. I've typically interacted with the device for 30 minutes or so at a time. Usually just enough time to try a new application or deploy and test an application that I've created. In addition, the device can be used while plugged in, so if you are developing at your desk, you can just keep it plugged in while developing.

Operating System

The HoloLens runs Windows 10. The operating system and the user interface work very well with the device. You use the bloom gesture to launch the Start Menu. You gaze at icons and air-tap or say the word "select" to navigate. In addition, there are other gestures like pinch-and-drag for scrolling, zooming, etc. and voice commands as well.

Unfortunately, text entry is a bit slow and awkward. There's an on-screen keyboard that pops up that allows you to air-tap to press each key. However, it's far less efficient than using a physical keyboard, especially for entering passwords. Ultimately, it's no worse than entering text on a video-game console or interactive television though.

All in all, interacting with the HoloLens device via Windows 10 is pretty effortless. What's even more impressive is that it works this well already, given that it's only in the developer-prototype stage at this time.

Holographic Apps

The HoloLens comes with a few pre-installed apps and there are a few more available on the Windows Store right now. I started by creating simple holograms using the pre-installed Holograms application. You can drag and place holograms around your room. For example, there's a tiny ballerina hologram that plays music and dances when you air-tap on her. She seems surprisingly real when she starts dancing to the music.

Next, I downloaded, installed, and played RoboRaid. This game has been the "aha" moment for everyone that I've showed the HoloLens to so far. It's the app that really sells the idea of the mixed-reality computing paradigm. It's hard to explain until you've seen it, but it really seems like there are robots busting holes in your walls and trying to attack you. I've never experienced a video game like it before. It is truly remarkable.

Then, I tried Fragments, a sci-fi detective game that challenges you to solve crimes by walking through the holographic memories of the victims of the crime. This game transformed my living room into an abandoned warehouse filled with rats, a missing child, and a gun-wielding abductor. There are moments where you honestly forget that you're in your own home and you feel like you're actually re-living someone else's memories. The effect is very convincing.

Finally, I tried Young Conker, a video game involving a toy-sized squirrel, who runs around your house trying to solve puzzles. It's a very weird experience being an adult and walking around your living room, guiding a tiny imaginary squirrel to solve a puzzle on your couch. Even more unusual is seeing this little squirrel jump on to and off of your couch as if they are really co-present.

I tried several other apps including HoloStudio, Edge, and Skype; however, the four applications I mentioned above were the ones that really sold the vision of the technology for first-time users.

HoloLens Development

To begin developing for the HoloLens, I started by watching the online videos and reading the online documentation to get a general feel for everything. Next, I installed the development tools (i.e. VS 2015 Update 2, HoloLens Emulator, and Unity HoloLens Tech Preview). Then, I attempted to get the open-source code for Galaxy Explorer to compile and run on my HoloLens.

The process wasn't effortless, but within an hour, I was able to take the Galaxy Explorer code from GitHub, load it into Unity 3D, build a Windows 10 Store deployment project from Unity, and then deploy the solution to both the HoloLens emulator and my HoloLens device. All-in-all, once you know what you're doing, the process is actually pretty simple.

After I finished exploring the Milky Way galaxy in my living room, I decided to watch the tutorial videos at the Microsoft Holographic Academy to learn how to build a holographic app myself. The tutorials walk you through all of the basics of building a holographic app, including gaze, gestures, voice commands, spatial sound, and spatial mapping.

I've used the Unity 3D engine to develop software in the past, so everything was very familiar to me. In fact, I had almost no issues what-so-ever walking through each of the tutorials. I did not copy-and-paste the sample code, as the tutorial suggested. Instead, I typed everything out by hand, like I usually do with tutorials, so that I would understand what each line of code was doing. Within a couple hours, I went from starting with a new project, to interacting with a small holographic world on my kitchen countertop.

Opportunities for Improvement

There are definitely a few things that could be improved for the development experience. The most significant is the length of the feedback loop for debugging. Essentially, you have to build the Unity project as a Window 10 Store project, then open the build output project in Visual Studio, and finally deploy to the HoloLens (or emulator) to test your code changes.

This process can take several minutes to test a single one-line code change. To be optimal, this cycle needs to be significantly reduced. In the future, I assume there will be a way to simulate the HoloLens gaze, gestures, and voice input directly within Unity so that you can get immediate feedback without having to deploy to the device or emulator (if this feature does not already exist).

In addition, Unity 3D isn't the land of best practices when it comes to writing C# code. The code that Unity framework encourages you to write isn't easily unit testable without significant mocking. It will also encourage you to violate the Law of Demeter, use the service-locator antipattern, tightly couple via inheritance hierarchies, and make extensive use of out-parameters, etc. So, you have to work quite a bit harder to write good clean code in Unity.

Final Thoughts

In the end, I was very surprised how easy it was to build my first application for the Microsoft HoloLens. Having previous experience with Unity 3D was probably my best asset during the learning process. In fact, for anyone interested in learning how to develop apps for the HoloLens, I would highly recommend becoming familiar with developing regular 3D apps for Unity first before developing for the HoloLens.

Finally, I need to thank Microsoft for inviting me to be part of wave 1 of the HoloLens Developer Program. I'm very excited to be one of the first people on the planet to be creating holographic applications for the Microsoft HoloLens. In addition, I'm really looking forward to creating my open-source immersive data visualization tool using this technology. I'll be sure to keep everyone posted on my experiences over the course of its development.

Share this article:  Share on Facebook Share on Twitter Share on LinkedIn Share on Google+