Path Planning For Hololens

As part of our movement to develop learning tools using new technologies, I’ve create a demo app for Microsoft Hololens. It consists of different demos on its capabilities.

The Hololens has a particularly powerful feature, it is able to scan around you and create a 3d map of your environment. And hence you can make a virtual mouse move on your sofa and jump onto your table since the virtual mouse knows the 3d environment around you. But this poses a problem, the mouse need to know how to move around the 3d environment. This is called path planning.



Unity has a native path planning algorithm. It involves defining which part of your 3d objects is “walkable”. Consequently, it will calculate possible paths for those 3d objects. This is their Navmesh system. However, the calculations must be done prior to building the game whereas for the Hololens the 3d map is created during runtime. So we can’t use the Navmesh system.

I therefore used an existing path planning program called Simply A* from the Unity Asset store. It is a grid based A* algorithm. It requires a lot of modification in order to be used with the Hololens.

And once a path is created, correct animation has to be run with the path. So a straight path will use a walking animation, while a vertical path will use a jumping animation.

SLAM - Placeholder RTABMAP

Research on Simultaneous Localization and Mapping (SLAM)

Doing a Part-time PhD from Newcastle University on the Simultaneous Localization and Mapping. Autonomous robotics and self-driving cars are hot topics today. In order to achieve autonomy, the robot would need to know:

  1. Its own location
  2. A map of its surrounding

This turns out to be a chicken and egg problem. In order to know its position (and orientation), the robot will need a map to reference to, in order to create a map, it needs to know its position. The solution is to do it at the same time in a probabilistic framework, and this is called SLAM.

I will post updates on my work on SLAM.

Living Textbook - Making the textbook come alive with Augmented Reality(AR)

Living Textbook: Augmented Reality App

This project is created to enable on demand video tutorials for students embedded straight from their textbook. Often while revising, students need explanation to refresh their understanding of the topic. Using augmented reality, this is made possible. The mobile phone will detect a marker which will then give the position and orientation of the marker’s plane. A 3d render can then be super imposed and position on the plane, giving the illusion that the 3d render is living in the real world.



This would also enable the 3d render to point onto the text book itself in its explanation creating a very intimate and lifelike guided learning environment.