Microsoft Hololens Data Visualization
Note: The Big Picture
As part of our overall concepts, we are putting together a "big picture" overview of the OSS spectrum of available projects for a Kubernetes based end-to-end Machine Learning distributed processing continuum from Ingestion to Analytics to Visualization.
- https://github.com/cognonic/awesome-AI-kubernetes covers the distributed platform for the machine learning processing elements
- https://github.com/cognonic/on-premise-kubernetes covers the often neglected on-premise Kubernetes configurations in the range between development laptop and full cloud build-out. This will also include some basic Kubernetes and Docker container bits.
- https://github.com/cognonic/The-Cognitive-and-ML-List covers the wide variety of specific machine learning type libraries including a detailed ONNX focus
- https://github.com/cognonic/The-React-List covers the mysterious common usage in the Data Science field of JavaScript for the presentation and visualization of the hard earned results of all that distributed processing power
- (this effort) https://github.com/cognonic/Augmented-Reality-Hololens covers how that hard earned visualization really deserves to move fully into this millenium as real-time Augmented Reality and Virtual Reality (i.e. Mixed Reality) walk-throughs of your data to fully embrace the potential of our modern tools.
- https://github.com/cognonic/The-Big-Tech-Players-GitHub-List covers many of the major companies on GitHub which can indicate the "backing" power of certain OSS projects, a concept more of interest to enterprise consumers of OSS.
Comments, any general feedback, and additional content are always welcome and easily acheived by just opening an issue.
The Holographic Academy - The Holographic Academy is a space for learning. We help the best and brightest teams transform the world through best-in-class HoloLens experiences.
HoloToolkit - The HoloToolkit is a collection of scripts and components intended to accelerate the development of holographic applications targeting Windows Holographic.
HoloToolkit-Unity This is effectively part of the existing HoloToolkit, but this is the repository that will contain all Unity specific components. The HoloToolkit is a collection of scripts and components intended to accelerate development of holographic applications targeting Windows Holographic.
HoloLens Companion Kit - HoloLens Companion Kit -- This is a toolkit style repo for code bits and components that do not run on HoloLens but pair with HoloLens to build experiences. HoloLensCompanionKit contains the following feature areas:
RoomAlive Toolkit - The RoomAlive Toolkit calibrates multiple projectors and cameras to enable immersive, dynamic projection mapping experiences such as RoomAlive. It also includes a simple projection mapping sample.
Joint Orientation Basics - Joint Orientation Basics for Unity - This sample, written in C# for Unity 5, demonstrates how to use data provided by Body.JointOrientations collection. Shows a skeleton that represents a hierarchical model of all joints. Each joint is visualized to show all quaternion values.
Kinect for Windows Samples - This repository contains samples submitted by the Kinect for Windows Developer Community. Submitters intend to help fellow Developers hit the ground running while creating Kinect apps for Windows.
Kinect Common Bridge - Kinect Common Bridge is a complement to the Kinect for Windows SDK that makes it easy to integrate Kinect scenarios in creative development libraries and toolkits.
Hololens Listner - Hololens Voice controller
LeapMotion_Hololens_Asset - leap Motion support for Hololens Asset - LeapMotion_Hololens_UnityProject-RootProject for developing the Leap Motion Hololens Asset
Spatial Mapping Shaders - Some different shaders for visualizing Hololens spatial mapping data.
HoloLib Unity3D C# Hololens Framework - This framework provides:
- A demo application.
- Easy setup.
- Fast development cycle.
- Support for object interactivity.
- Support for spatial mapping.
- Support for gaze interactions.
- Support for voice controls.
HoloLensDX3M - HoloLens Direct X Models Mesh and Materials Toolkit is a Windows Runtime Component project which makes it easy to import Models inside DirectX and SharpDX projects for use inside of HoloLens, Holographic, and 3D based DirectX UWP applications.
HololensDXTutorials - Hololens DirectX Tutorials in C# and C++
Hololens_Share - Hololens Network Share
Holo Tag Recognition - An example on how you can identify specific tags in your environment with HoloLens. The code is based on the blog post: http://www.andrei-ermilov.net/virtual-reality/identifying-tags-in-3d-space-with-hololens/. Please note it does not work in the emulator, you will need a HoloLens device to deploy this solution.
Holo Assisted Training - A holographic assembly line training prototype based on Microsoft HoloLens. In this demo, you are instructed on how to assemble a few parts into an engine. Make sure you are fast and don’t make mistakes! You will receive a final score based on your performance.
San Francisco-Oakland Bay Bridge
Logo Lens - Microsoft HoloLens application for HackMIT. Logo Lens presents relevant stock quote data when a user sees a company logo while wearing the HoloLens.
Neuron Lab - HoloLens App simulating neuron activity and structure
Virtual Foliage - A holographic application for the hololens in which flora begins to bloom on nearby surfaces
MyApartments a dump of my apartments from HoloLens + UrhoSharp - demo of Spatial Data
HoloWorld - This is a Unity Project for HoloLens used in the #HoloTouring conference sessions Real World Holographic App Development by Rene Schulte presented at DevSum 2016, Unite Europe 2016 and NDC Oslo 2016. The demo project is a simple rigid body physics-based game with a plane and some cubes on top which the user can shoot spheres/balls at from the camera (the user's head) which then bounce off in the real-world environment.
HoloFame - This is a Unity Project for HoloLens used in the #HoloTouringFall conference sessions Real World HoloLens Development with Unity by Rene Schulte presented at Unite LA 2016 and VRDC 2016. The demo project is a simple HoloLens demonstrating how to interact with thereal-world environment. The project implements the basic HoloLens input and output paradigms like GGV:
Planetarium - A sample HoloLens app that uses spatial mapping to automatically place holograms within a room. This sample uses the HoloToolkit for Unity. - A sample HoloLens app that uses spatial mapping to automatically place holograms within a room. This sample uses the HoloToolkit for Unity. This is the completed version (with modifications in profress) of the Holograms 230 tutorial from the Holographic Academy, which walks developers through a complete sample project, built in Unity, that demonstrates:
- Scanning a room to create a spatial mesh
- Importing the spatial mesh within the Unity editor
- Visualizing the spatial mesh
- Processing the spatial mesh to find planes (e.g. walls, floor, ceiling, etc)
- Testing the placement of holograms based on the processed planes
- Occluding holograms based on the real world spatial mesh
HoloTank Demo - Simple holographic viewer demo to show a 3D model in a mixed reality setting using Microsoft HoloLens. The demo shows a holographic tank, which was imported from the Tanks! Tutorial from Unity Essentials, introduced at Unity Training Day 2015.
- Simple model viewer app that displays a tank floating 2 meters in front of the user at launch
- Gaze around the space to move the tank and tap to lock the tank in place
- Say "Rotate model" out loud to make the tank spin in place, or "Stop rotation" to stop it
- Show or hide the spatial mesh of your environment with the "display mesh" and "hide mesh" voice commands
- Tank idling sound now uses spatial sound on HoloLens
HoloBot - A starter mixed reality Unity project for Microsoft HoloLens that lets you interact with a floating robot head using speech recognition, and getting answers spoken back by the "bot" using Text-to-Speech. The commands sent to the bot are powered by the Microsoft Bot Framework and LUIS from Microsoft Cognitive Services.
- Hovering bot head (aka HoloBot) with looping ambient sound
- Tap the HoloBot to move it to a different location, tap again to place
- Gaze at the HoloBot to trigger the speech recognizer, you will hear a "Ping" sound
- Speak "commands" to HoloBot using natural language
- The HoloBot speaks back to you using Speech Synthesis (aka Text-to-Speech, or TTS)
- All sounds and speech use spatial sound that originate from the HoloBot
- The "brain" of the HoloBot can be any public bot built with the Microsoft Bot Framework. See bot integration instructions below
HelloHolograms - A sample HoloLens technical demo initially based on the Holograms 100 tutorial from the Holographic Academy. This one uses the HoloToolkit for Unity, and it also includes additional features like shooting balls around the space, and spatial mapping to use the real world in the virtual balls physics. The spatial mesh can be visually turned on and off, though it always remains active for collision detection.
- The latest version was built in Unity 5.5.0b6 (Preview).
- Displays a cube 2 meters in front of the user.
- Use voice commands to switch between "Pickup mode" and "Shooting mode" (default at start).
- Pickup Mode: Air tap the cube to make the cube follow your gaze for placement. Tap again to place the cube.
- Shooting Mode: Shoot white balls into the environment by air tapping in the space; watch them bouce around the real-world space thanks to spatial mapping on the HoloLens.
- Spatial sound effects when shooting balls and when balls collide with the floor, environment and each other.
- Show or hide the spatial mesh of your environment with the "display mesh" and "hide mesh" voice commands.
HoloExplorer Mars - Mars exploration simulation for Windows Holographic. Work in progress, stay tuned.
DesertShooterVR - Sample first-person-style shooter game for virtual and mixed reality. The goal is to demonstrate how to create a Virtual Reality-style experience using the Microsoft HoloLens. This is inspired by the kind of semi-immersive reality found in other projects like the HoloTour. HoloLens-based "VR" also has the advantage of allowing room-scale movement without any separate tracking equipment. The military base in this sample is actually quite huge and it would otherwise be very hard (i.e. impossible) to track the user accurately as they walk around the base if you had to install external tracks (e.g. HTC Vive) and if the user was tethered to a PC. You'll need a "convention center"-size space to truly walk all over the map, but most of the action is meant to be played from the center of the map. This is a work in a progress that I am sharing publicly as I add features. The latest version was built in Unity 5.5.0b10 (Preview).
Holo-Roll-a-Ball - Holo-Roll-a-Ball is a HoloLens version of the starter tutorial for Unity with adapted features. This is a work in progress. The latest version was built in Unity 5.5.0b6 (Preview).
- Playing board floats in front of the user when the game starts
- Move the board with your gaze and tap to lock in place
- Move the player ball with an Xbox One S Bluetooth controller, or the WASD / arrow keys on a Bluetooth keyboard
- Pick-up yellow cubes around the game board to score points, without falling off the board. If the ball falls off, it'll drop to the physical floor where the board is placed
- Show a basic UI for scoring & end of game
- Say the voice command "Reset Game" out loud to reset the player ball to the center of the board
- Show or hide the spatial mesh of your environment with the "display mesh" and "hide mesh" voice commands
- Ambient spatial sound during the game
- NOTE: I deliberately did NOT add walls to make the game more challenging.
- The current UI implementation follows the basic Unity UI created in the Roll-a-Ball tutorial from Unity. It's basic for standard games, but doesn't work by default in VR and Mixed Reality projects. I set the UI Canvas Render Mode to Main Camera (Camera) which works in VR & HoloLens, but again isn't ideal. My next goal is to add a better Spatial UI in a future version.
- I was initially forced to play the initial version of Holo-Roll-a-Ball with a Bluetooth keyboard due to Unity-HTP issues, but I added support for the Xbox One S controller via a plugin. I will eventually remove the plugin and upgrade to the full Input Manager in Unity to support all Bluetooth controllers for a better experience.
SamplyGame-Hololens - A re-implementation of SamplyGame (https://github.com/xamarin/urho-samples/tree/master/SamplyGame) for Microsoft Hololens
