Since 2018 I have been working at Microsoft. First contracting, then full time.

This page documents surface details related to HoloLens 2 Tips & HoloLens 2 Setup. I was one of the core team members for Tips and a support team member for Setup helping with it’s final waves of polish and bringing over work I had done for Tips.

HoloLens Tips

HoloLens 2 Tips is a teaching experience for the purpose of introducing new-to-Augmented-Reality users to common controls and interaction modalities.

I was one of the core team members that built and refined Tips leading up to the launch of HoloLens 2.

There was a high bar for the visual effects, they had to breathtaking, unique and interactive. An example of this was the iconic holographic and iridescent hummingbird. The hummingbird was a highly complex model and was the flagship avatar for both Tips & HoloLens Setup experiences.

I kept track of the general interaction code, event synchronization and visual quality of assets throughout the reviews. I worked closely with the design leadership, as well as with various 3D artists and animators to hit a stunning fidelity while keeping everything performant.

Tips was built using additive scene loading and a persistent controls scene (lessons I learned from working on Attunement). This allowed for individual modules to exist in isolation of each other, and to allow for future modules to be added. Additionally, I implemented the introduction, near select, far select, rotation, resize, movement & start menu modules.

I aggressively hunted bugs and reviewed user trials to find more unintended behavior.

During development, my duties also put me on a collision course with Mixed Reality Toolkit issues that weren’t yet up to our target standards. My contributions to MRTK included building Pressable Buttons, as well as highlighting issues with the Localization & Speech functionalities with the toolkit. Those issues would all go on to be fixed prior to release of MRTK 2.0.

HolOLens Setup

HoloLens 2 features an Out-of-the-Box setup experience, a custom set of guided instructions to make the device intuitive and approachable.

The experience begins with the user viewing a colorful spiraling digital entity moving about. A button is presented (along with a ghostly instructing hand) to begin the experience. The colorful nebula digitizes and shapes into a Hummingbird, which will hover and follow your hand. Again, there is ghostly hand prompting for users who do not reach up on instinct to hold or touch the hummingbird (many users do just that).

This covers topics such as initial wifi setup, account binding, setting up iris login, introducing speech, menu functionalities and more.

Due to many of the hardware specifics, this application had a wide breadth and a high technical reliability bar. No errors or bugs could be allowed across the finish line in any of the features.

Furthermore, the usability had to be ironclad. The device as a whole relied upon a user’s first experiences being positive. This means getting through this application without getting caught or confused. The Setup experience went through numerous refinements to sand away rough spots and unclear language.

It was a unique career experience to work along so many domain experts tracking metrics and potential improvements across the many different areas of functionality, from eye tracking, to 3D artwork, to hardware performance. I was glad I could contribute to the pressable buttons, porting over several of the HoloLens 2 Tips and to stamp out bugs for the release.

Closing

I am proud to have contributed to HoloLens Tips and OOBE, they were fantastic projects. The lessons I learned about adjacent fields, the nitty gritty workings of Unity, and design expertise about Augmented Reality was tremendously valuable. I couldn’t have asked for a better experience in shipping a product that would affect numerous enterprise workplaces.