The goal of the present Bachelor thesis is to enable comparing different approaches of integrating Neural Networks in HoloLens 2 applications in a quantitative and qualitative manner by defining highly diagnostic criteria. Moreover, multiple different approaches to accomplish the integration are proposed, implemented and evaluated using the aforementioned criteria. Finally, the work gives an expressive overview of all working approaches. The basic requirements are that Neural Networks trained

more »

... TensorFlow/Keras can be used and executed directly on the HoloLens 2 without requiring an internet connection. Furthermore, the Neural Networks have to be integrable in Mixed/Augmented Reality applications. In total four approaches are proposed: TensorFlow.js, Unity Barracuda, TensorFlow.NET, and Windows Machine Learning which is an already existing approach. For each working approach a benchmarking application is developed which runs a common reference model on a test datatset to measure inference time and accuracy. Moreover, a small proof of concept application is developed in order to show that the approach also works with real Augmented Reality applications. The application uses a MobileNetV2 model to classify image frames coming from the webcam and displays the results to the user. All the feasible approaches are evaluated using the aforementioned evaluation criteria which include ease of implementation, performance, accuracy, compatibility with Machine Learning frameworks and pre-trained models, and integrability with 3D frameworks. The Barracuda, TensorFlow.js and WinML approaches turned out to be feasible. Barracuda, which only can be integrated in Unity applications, is the most performant framework since it can make use of GPU inference. After that follows TensorFlow.js which can be integrated in JavaScript Augmented Reality frameworks such as A-Frame. Windows ML can currently only use CPU inference on the HoloLens 2 and is therefore the slowest one. It can be integrated in Unity projects with some difficulties [...]