In the previous posts I wrote about the journey of the development of a Mixed Reality based application exploring several topics. If you missed any of them, here is a quick recap.
- Week 1: “Flying Turtles: Nice to Meet You”
- Week 2: “Let's Get Started! Hardware Development Kit and Software Tools”
- Week 3: “Power is Nothing Without "Controllers"”
- Week 4: “VR UX: We Are Not Gorillas!”
- Week 5: “My superman vision: the power of a Shader”
- Week 6: “Lost in Cloud: Cloud Architecture to stream a Mixed Reality Training Experience”
- Week 7: “The incredible power of voice”
In the several articles I suggested how to approach the design as well as provided a few ready-to-use code snippets. You will find parts of the source code both on my GitHub as well as in the new version of the MixedReality Toolkit.
We came to the end of this long challenge, and it is time to reveal what we developed in the past 2 months: TraIND40, the first training platform designed for industry 4.0
Industry 4.0 is a name given to the current trend of automation and data exchange in manufacturing technologies. It includes new technologies such as cyber-physical systems, Internet of things, cloud and cognitive computing. Industry 4.0 is commonly referred to as the fourth industrial revolution because of the gradual combination of traditional manufacturing and industrial practices with the increasingly technological world around us.
In FifthIngenium we often deal with the industrial world and Industry 4.0. One of the main problems of the industrial processes is the new workers' training on machineries, which becomes more complex and technologically advanced every day. Today the modern production lines are built by specialized companies and the products are sold in many parts of the world. For this reason, staff training is very expensive and is also made more and more complex because of the size and complexity of the machinery.
One of the key concepts of the new Industry 4.0 is the SMART PRODUCTION: new intelligent technologies add a "Brain" and "Voice" to the traditional machineries and empower a new generation of workmen. VR/MR can be used to facilitate advanced training of workers with a new inclusive and simplified approach.
The name TraIND40 come from the fusion of “Training” and “industry 4.0” and it is a solution designed for modern factories.
TraIND40 has been designed for the both individual and shared training sessions, with the ability to interact remotely and also using different devices.
During the entire experience, RB2 will be at your side: a friendly robot which will be at your disposal to help you and clear your doubts.
TraIND40 contemplates several usage scenarios:
Virtual individual training: The trainee lives an individual VR experience, with the support of a Virtual Teacher, who guides him through exploration of the machinery, the procedures of ordinary maintenance and the checklist of the operations to do for discovering the problems and resolving them.
Virtual individual assessment: In this phase the trainee is in the VR experience without the support of the Virtual Teacher and he is asked to perform the procedures learned, in different scenarios, according to typical uses cases such as execute a cleaning procedure or identify the machine problem in a fault simulation scenario. The trainee will receive a score, depending on the correctness of each procedure and the time spent on it.
Live shared class: The VR Experience can be shared between more trainees who can connect to the VR Shared class, each one with his VR device. In the class one person is the teacher (with a VR device) who can interact during the training on the virtual model, while the class can see what he's doing.
Remote training with HoloLens: The teacher can use alternatively an AR device (as HoloLens) and be physically present with the real machinery and perform operations on it. The trainees in the virtual class can both observing what the teacher is doing and see on the virtual model of the machinery what's happening.
Traditional Learning Contents: Inside TraIND40 users can utilize also traditional 2D contents such as videos and schematics.
All these features have been developed in the prototype that you can see in the video.
We weren't able to fully develop all the features that we had in mind for our prototype, but we'll be integrating them in the near future:
Remote training: In this phase the trainee is in AR with HoloLens on the physical machinery and remotely the Teacher, both in VR or AR, can support the training and interact in the holograms space, adding elements, giving instructions and reporting problems.
IoT data: In this phase the data in support of uses cases during the training in AR with HoloLens or VR will come directly from physical machinery through IoT solution. During each phase, all the data are collected and stored to perform analysis and set up test specifically design ad hoc for each trainee.
Learning Management System: integration with a backend learning system which can manage the content dynamically, as well as a point-based system and the customization of the single training plan.
My Personal Final Considerations
After this long challenge and much time at work with the DevKit made available by Intel I can draw some considerations.
From the Consumer side, Mixed Reality is super cool and easy with zero-setup time. From developer side, still some fragmentation, but the environment is getting better and better, and tools such as MRTK can make your life easier. But keep an eye on performance.
HP VR1000 is a great device, sufficiently comfortable, with a reactive controller tracking. It only lacks a lenses focus management and balancing system.
How cool is the INTEL SKULL NUC! it never slowed down even when rendering the APP with Unity and Mixed Reality Portal open with 2 different cameras, 2 additional remotely connected devices and active video recording. This is the perfect machine and will be my choice for all the future installation and demo.
Is this the end? Definitely not!