This application abstract introduces the first functional prototype of XR system for additive manufacturing (AM) system’s maintenance support. It has been developed in European Commission funded H2020 project called “QU4LITY – Autonomous Quality Platform for Cognitive Zero-defect ManUfacturing Processes through DigitaL ContInuity in the ConnecTed FactorY of the Future ”. The development is following main principles of Human Centred Design (HCD) (ISO 9241-210:2010).

The results of this application abstract are from the first development cycle of HCD and user evaluations at Prima Industries premises in Turin with the real AM system on February 2020.

The XR-system is based on IEEE Draft Standard for an Augmented Reality Learning Experience Mode (P1589:2020) and it could be used with the Microsoft HoloLens 1 & 2 (Microsoft, 2020) and latest mobile devices (iOS and Android). The whole system is configured around the Activity and Workplace JSON files. The Workplace JSON describes workplace-related information such as points of interest, sensors, etc.  It is parsed with the Workplace   manager and information is transferred to the data layer.  The Activity JSON describes all action steps and the content that should be active for each step.  It is parsed with the Activity manager and information is transferred to the XR layer via local storage. The user can interact with the XR player via a multi-modal user interface (see Figure 1). The following modalities can be used simultaneously (1) Gesture, e.g. doing a “Click” gesture to go to the next work step, (2) Voice commands, e.g. saying “Next” to go to the next work step, and (3) Physical button, e.g. “Click” to go to the next work step.

The evaluation case was Prima’s Print Sharp 250 AM system’s maintenance work. AM system could be seen in left hand side in Figure 1. The evaluation was done Prima Industries premises in Turin on February 2020. The evaluation/use case contains 10 steps procedure of powder bin removal. Three (3) Prima Industries experts participated in the user test. After the test, each participant was interviewed. They were also asked to fill in SUS (System Usability Scale) questionnaire. Users were testing HoloLens and iPad Pro version of XR system.

SUS is a tool for measuring usability and learnability. SUS scores have a range of 0 to 100. SUS score starting from 68-70 represents the level of acceptable system usability. Acceptability ranges: 0-50 not acceptable; 50-70 marginal; 70- acceptable (Bangor et al., 2009; Brooke, 2013).

Average SUS results:

–              Tablet version: 84

–              HoloLens 1 version: 70

The SUS scores indicate clearly acceptable usability for the tablet solution, whereas two experts evaluated HoloLens solution with not acceptable scores. The results are still approximate and preliminary in nature due to low number of participants. They are however in line with the expectations of the developers, since the XR content was less optimized for the HoloLens solution.

The interview results on the overall experience with the both solutions were quite positive. The users’ expressions on the experience with the tablet solution emphasised good usability and usefulness, such as “easy to use, well developed, appropriate for demoing, user friendly, useful”. The expression related to HoloLens focused more on the novelty of the solution and its smart interaction: “Interesting, very interactive, smart, surprisingly easy to interpret”. HoloLens was also experienced as strange to use and not easily adjustable. The strength of HoloLens is that it enables hands-free working. It was suggested that tablet solution is more suitable for on-site training purposes, whereas HoloLens would be good for demonstrating XR for customer. Additionally, a number of minor improvements for content and usability were mentioned in the interviews.

As the XR-system usability has reached a reasonable level (average SUS scores: 70 HoloLens 1 & 84 iPad Pro version), even the test population was quite small. Both the pragmatic and emotional aspects of the user experience were considered fulfilling. It can be suggested that the current XR-system is good bases for the second development cycle of HCD. Also, system will be updated to support HoloLens 2 and base on the first experience of the HoloLens 2 the new version should solve most of the usability issues such as the narrow field-of-view. Future work also includes project monitoring features such as real time process info via IoT interface and video stream from AM machine chamber.