Path Planning For Hololens

As part of our movement to develop learning tools using new technologies, I’ve create a demo app for Microsoft Hololens. It consists of different demos on its capabilities.

The Hololens has a particularly powerful feature, it is able to scan around you and create a 3d map of your environment. And hence you can make a virtual mouse move on your sofa and jump onto your table since the virtual mouse knows the 3d environment around you. But this poses a problem, the mouse need to know how to move around the 3d environment. This is called path planning.



Unity has a native path planning algorithm. It involves defining which part of your 3d objects is “walkable”. Consequently, it will calculate possible paths for those 3d objects. This is their Navmesh system. However, the calculations must be done prior to building the game whereas for the Hololens the 3d map is created during runtime. So we can’t use the Navmesh system.

I therefore used an existing path planning program called Simply A* from the Unity Asset store. It is a grid based A* algorithm. It requires a lot of modification in order to be used with the Hololens.

And once a path is created, correct animation has to be run with the path. So a straight path will use a walking animation, while a vertical path will use a jumping animation.

Living Textbook - Making the textbook come alive with Augmented Reality(AR)

Living Textbook: Augmented Reality App

This project is created to enable on demand video tutorials for students embedded straight from their textbook. Often while revising, students need explanation to refresh their understanding of the topic. Using augmented reality, this is made possible. The mobile phone will detect a marker which will then give the position and orientation of the marker’s plane. A 3d render can then be super imposed and position on the plane, giving the illusion that the 3d render is living in the real world.



This would also enable the 3d render to point onto the text book itself in its explanation creating a very intimate and lifelike guided learning environment.

Gamification of Syllabus

Technologist: Experiment on Gamification of Syllabus

An on-going project. This involves the gamification of learning. A scenario is produced such that the protagonist is a technologist in a village that require the technology to survive. This is in a medieval setting where modern technologies are absent. This creates appreciation among students on the importance of engineering and technologies. Specifically, they will learn about motors and how it can be used in the village (for farming etc).

The flow of this demo video seems slow, I am tweaking it to incorporate technical problems very step of the way so that students do not have to wait in length for the story line to progress in order to start learning on the topic. At the end of the video, a browser is opened up that access a Drupal (a cotent management system) mutiple choice question page.

The student proceed to solve and the results is saved for viewing by the teachers. Since every motion of the learner can be recorded in the 3d game, many analytics can be extracted from this concept, to estimate the competence of particular students on specific topics.