Manual

User Manual:

Open the PDF directly: View PDF .
Page Count: 17

Download
Open PDF In Browser	View PDF

HoloLens AR Interaction Tasks Manual

Contents
•
•
•
•

Setup
Usage Instructions
Make Changes
Resources

2

Set Up
This guide assumes starting from scratch (no toolkits or editors installed) and walks through the steps to
get the tasks running in Unity and deployed on the HoloLens.
Set up is separated into three stages.
1. Download and install necessary editors and toolkits
o Windows 10 update and settings (you need to be running Windows 10)
o Unity
o Microsoft Visual Studio 2017
o Mixed Reality Toolkit (Unity asset package)
2. Create Unity projects for each task
o Get scene preview running
o Build project solution
3. Deploy project solutions to HoloLens
o Set up HoloLens
o Deploy app using Visual Studio
o Windows Device Portal

Download and Install Necessary Editors and Toolkits
This section is summarized by the following links.
•
•

https://docs.microsoft.com/en-us/windows/mixed-reality/install-the-tools
https://github.com/Microsoft/MixedRealityToolkit-Unity/blob/master/GettingStarted.md

Windows 10 update and settings (PC)
•
•

Install the Windows 10 April 2018 Update by running Windows Update
o If Windows is up-to-date the Update installed
Enable Developer mode
o Go to Settings > Update & Security > For developers
o You may need to reboot

Unity
•

Download latest Long Term Support (LTS) Release of Unity Editor Download Assistant (Windows)
o Projects tested on LTS Release 2017.4.10f1 (installer is provided)
o Note a newer release may be out. A list can be found here:
https://unity3d.com/unity/qa/lts-releases?_ga=2.10765437.818138280.1527115303289721018.1521153098
o When installing, make sure to select Windows Store .NET Scripting Backend (you may
install the docs as well)

3

o

Specify install folder

o

Create an account if you are a new Unity user

4

Microsoft Visual Studio
•

Download and install Microsoft Visual Studio Community 2017 (installer included)
o Select Universal Windows Platform development workload
o Select Game Development with Unity workload
o Deselect the Unity Editor optional component (we’ve already downloaded the latest
release)

5

Mixed Reality Toolkit – Unity
•
•

No action needed
HoloToolkit 2017.4.10 packages are included in each project

Create Unity Projects for Each Task
This section walks through setting up the tasks in Unity.
•
•
•

Cube task
Cube random plane task
Pointing task

The goal is to run scene preview for each task. When scene preview runs, the task is ready to be loaded
onto the HoloLens.

Cube task
•

Open Unity and create a new project

o

Title the project according to task and choose a location for the project folder then
select Create Project

6

•

o

An empty project will open in the editor (you may keep the editor open till the end)

o

Navigate to the project folder at your specified location

Open a new file explorer and navigate to the provided corresponding task folder
o Copy the ProjectSettings and Assets folders into the newly created project folder

7
o

•
•

Opt to overwrite all files when copying
▪ These are the ProjectSettings files

Return to the open Unity project and wait for the project to load
Open the main scene
o File > Open Scene > scenes > main.unity

8
•

Apply mixed reality project settings
o Mixed Reality Toolkit > Configure > Apply Mixed Reality Project Settings

•

Click the play button to run the scene preview

9
•

The play button will appear pressed signaling scene preview is running

Cube random plane task
•

Follow the cube task steps above to run scene preview
o The initial scene preview view is the same

Pointing task
•

Follow the cube task steps above to run scene preview

10

Deploy Project Solutions to HoloLens
This section outlines how to export a project from Unity to the HoloLens.
•
•

•

•

Set up HoloLens
o https://docs.microsoft.com/en-us/hololens/hololens-setup
Build and export the solution from Unity
o https://docs.microsoft.com/en-us/windows/mixed-reality/exporting-and-building-aunity-visual-studio-solution
Use Visual Studio to deploy the task to the HoloLens
o https://docs.microsoft.com/en-us/windows/mixed-reality/using-visual-studio
o I have only deployed over USB
o Wi Fi is an option
To configure and manage the HoloLens (e.g. record or take screenshots) use the Windows
Device Portal
o https://docs.microsoft.com/en-us/windows/mixed-reality/using-the-windows-deviceportal

Usage Instructions
These sections describe each task and their features. Screenshots are included at the end.
Each task can be played within the Unity Editor using scene preview (see Setup). Scene preview controls
are as follows. These controls translate into gesture controls on the HoloLens.
•
•
•

•

Hold right-click to look around
o Locate the cursor in the center of the view
WASD to move around
Hold shift+right-click to ready the tap gesture
o Cursor will slightly enlarge
o One of the hands will turn blue
In the ready tap gesture state orient cursor on an interactive object
o Tap or hold left-click to interact with the object

Cube Task
Objective
Tap and hold select white cubes to drag them into randomly spawned trigger points (white spheres). A
cube is rendered transparent and colored red when it collides with a trigger point. A trial consists of 27
cubes populating a 3x3x3 array of trigger points. After a trial is completed a new one automatically
begins, and the trial counter is updated. The task is complete after three trials.

Usage
•
•
•

Orient cursor on a white cube near its center
Hold shift+right-click to ready the tap gesture
With the cursor on or near the center of the cube hold left-click to drag
o You should be holding shift+left-click+right-click
o Both look around and use WASD to move the cube while dragging

11
•

Drag the cube into a trigger point

Cube Random Plane
Objective
Identical to cube task. This task randomly spawns trigger points by frontal plane. A frontal plane is one
of the 3x3 front facing trigger spawn arrays. There are three frontal planes front, middle, and back. For
each trial, the plane spawn order is randomized, and trigger points are randomly spawned within the
planes. An entire plane’s trigger points must be triggered for the next plane (if any) to begin spawning
points.

Usage
•
•
•

•

Orient cursor on a white cube near its center
Hold shift+right-click to ready the tap gesture
With the cursor on or near the center of the cube hold left-click to drag
o You should be holding shift+left-click+right-click
o Both look around and use WASD to move the cube while dragging
Drag the cube into a trigger point

Pointing Task
Objective
Tap select trigger points (red spheres) from a circular array of static points (white spheres). Trigger
points will spawn indefinitely. Interactive sliders change the height, distance, and size of the circular
static point array and the point sizes.

Usage
•
•
•
•

Orient the cursor on a trigger point
Hold shift+right-click to ready the tap gesture
With the cursor on the trigger point left-click to tap select
Adjust sliders as you would drag a cube (see Cube Task)

Web Browsing Task
Objective
Simulate web browsing. All keyboard input has been disabled.

Usage
•
•

•

Navigate to https://web-browsing-task.appspot.com/ using the HoloLen’s web browser
To move the app (browser window)
o Gaze at the app bar and tap and hold to select it. Move your hand to position the app,
then raise your finger to place it.
To resize the app (browser window)
o Gaze at a corner or edge of the app window, and tap and hold. Move your hand to
change the app window’s size, and raise your finger when you're done.

12

Screenshots

Cube task before third trial completion.

Cube task after third trial completion.

13

Cube random plane task with front plane filled.

Pointing task initially set.

14

Pointing task after slider adjustments.

One of many web browsing task pages.

15

Make Changes
This section describes how to make changes to the tasks. Save the scene and project for changes to
persist.

Camera distance
The camera parent position can be changed to adjust the initial view of a task. The process is the same
for each Unity task.
•
•
•

Open the task in Unity and select the MixedRealityCameraParent game object in the scene
hierarchy
Switch to the Inspector tab
Adjust the Transform component’s Position property then scene preview to see your changes

Sliders (pointing task)
The slider effects can be adjusted.
•

Open the pointing task in Unity and select a slider from the scene hierarchy

16
•
•
•

Switch to the Inspector tab
Expand the Slider Gesture Control (Script) component
Adjust the Min and Max Slider Values then scene preview to test your changes

Resources
Majority of the links in previous sections stem from these.
•
•
•
•

https://docs.microsoft.com/en-us/windows/mixed-reality/
https://docs.microsoft.com/en-us/hololens/
https://support.microsoft.com/en-us/help/12644/hololens-use-gestures
https://github.com/Microsoft/MixedRealityToolkit-Unity

To aid with troubleshooting, specifically script functionality, development commit history can be viewed
on GitHub. However, latest developments may not be pushed yet.

17
•
•
•
•

https://github.com/wilsjame/OEBL-Cube
https://github.com/wilsjame/OEBL-Cube-random-frontal-plane
https://github.com/wilsjame/OEBL-Pointing-Task
https://github.com/wilsjame/OEBL-Web-Browsing

Please email James jgwilson1214@gmail.com if something is not working as expected or with questions.
Manual’s last major changes on 14 September 2018.



---

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.7
Linearized                      : No
Page Count                      : 17
Language                        : en-US
Tagged PDF                      : Yes
XMP Toolkit                     : 3.1-701
Producer                        : Microsoft® Word for Office 365
Creator                         : Wilson, James G
Creator Tool                    : Microsoft® Word for Office 365
Create Date                     : 2018:09:14 15:04:56-07:00
Modify Date                     : 2018:09:14 15:04:56-07:00
Document ID                     : uuid:3EF5F335-9A40-41BB-9A66-A880FD9EF983
Instance ID                     : uuid:3EF5F335-9A40-41BB-9A66-A880FD9EF983
Author                          : Wilson, James G

EXIF Metadata provided by EXIF.tools