Augmented Reality for Visually Impaired People

About 36 million people worldwide are blind. For many of these people, navigating new spaces can be a cumbersome or frustrating experience as they listen and feel their way around their environment.

Our project seeks to utilize the spatial mapping power of the Microsoft HoloLens augmented reality device in order to help users navigate in unfamiliar environments.  This could improve independence for blind people by allowing them to find and read objects such as bus stops without sighted assistance.

By making audible the 3D mesh of the user’s surroundings that these devices generate, we will create a soundscape that will let users hear the objects around them. Thus, visually impaired users will be able to hear the presence of obstacles like tables and chairs, or the lack of obstacles that represents a doorway, from up to five meters away.

In combining this capability with machine vision-powered text recognition in one hands-free headset, we hope to give blind users a new level of independence in unfamiliar spaces.

Source code available via Github: https://github.com/arvips/Mesh-Manipulation