Nexco-East Leverages Microsoft’s Metaverse – XR Today

Microsoft is already working with an esteemed client list to distribute its immersive solutions portfolio to suit a range of enterprise use cases. A range of partnerships come as the leading technology firm ramps up for the debut of its industrial Metaverse next year. 

One of Microsoft’s XR clients is Nexco-East Engineering, which operates roughly 4,000km of all expressways in Japan. The firm recently deployed an immersive training application for ETC equipment and tunnel emergency facilities that leverage MR and digital twin technologies.  

Currently, the region is facing an emerging social issue whereby Japan’s 10,000km of expressway is experiencing damages following more than 30 years of public use. This leads to Japanese expressways requiring extensive maintenance for road surfaces, bridges, and tunnels – according to Nexco-East.

At its Technical Training Center (TTC) located in Takasaki, Gunma, Nexco-East trainees can access the Microsoft-powered training service. The firm states that the training service equips every engineer with the necessary knowledge and skills to complete repair and maintenance tasks.  

Tetsuo Hideshima, the Managing Director and General Planning Manager of Nexco-East Engineering, added:  

We at Nexco-East Engineering ensure that drivers can safely use expressways by providing inspections and troubleshooting all day, all year. For continuous maintenance and management of expressways, it is critical to conduct thorough checks and diagnoses without overlooking any tiny changes in facilities and structures. To that end, streamlining and labour saving based on ICTs are imperative.  

Moreover, Masayuki Namiki, the General Manager of TTC, also noted:  

At TTC, over 1,000 of our engineers participate in training programs every year. They go through training to enhance skills based on the internal skill check system to become experts in expressway inspection and maintenance, called ‘highway doctors. 

Solving Pain Points with XR Solutions 

Due to expressways becoming more complex in design, Nexco-East trainees require increasingly sophisticated and relevant immersive training scenarios that leverage 3D visualizations – to leverage new training opportunities, Nexco-East turned to MR to help visualize mission-critical training data. 

According to Nexco-East, the firm faced hurdles during traditional hands-on learning experiences when covering complex building materials, therefore requiring deep digital standardization of training via Microsoft immersive services. 

Microsoft and Nexco-East’s immersive training solution focuses on translating practical, real-world usage of ETC equipment within a safe and repeatable virtual training space.

Yoshitaka Nakamura, Nexco-East Engineering’s Manager of Facilities Construction Work Division, explained how ETC equipment comprises of multiple devices, leading to extensive training required for workers. 

Nakamura also noted that training sessions must also show “how ETC equipment operates was explained with extensive materials such as paper-based radio communication and control signal flow charts, PowerPoint slides, and chalk talks of instructors and then tested with actual ETC devices.” 

Nakaruma added: 

However, it requires considerable time to gain understanding of associations among the training materials. Also, radio communication and control signal flows are impossible to see, so we had to rely on trainees’ imaginations. This resulted in gaps in understanding.

Working with Microsoft’s Immersive Portfolio

 Nexco-East leveraged Microsoft brand MR and digital twin solutions to solve knowledge gaps and optimise learning outcomes via its internal “MR for ETC equipment training” project. 

The firm launched the project, leveraging digital twin imagery to simulate ETC equipment alongside integrated systems such as infrared sensors, radio communication, and data communication routes. 

Moreover, the firm also works with DataMesh to leverage MR experiences, digital twins, and XR services. 

 Kensho Kashimada, the Director, Marketing and Sales of DataMesh Japan Co, added: 

At that time, there had been no similar attempts at least in Japan. The point was how we could build up MR materials that are easy-to-understand for every trainee while there was no correct answer about what the outcome should be like. 

Kashimada also noted how his firm “carefully proceeded” to develop the XR training service through “repeated hearing sessions with the project leading department and TTC instructors who would actually use the tool in practice.” 

Moreover, Kashimada explained that when trainees receive lectures with just “oral explanations and 2D materials,” Nexco-East witnessed how students found it difficult when trying to understand ETC mechanisms “firsthand.” 

Kashimada also added: 

We could also appreciate the potential value of the tool we were trying to create. So I think it was a very valuable process. 

Therefore, to provide suitable training procedures, Nexco-East underwent a development period of six months to enhance its training procedures by combining its MR solutions with traditional teaching methods to improve learning outcomes across all trainees. Nexco-East also leverages Microsoft’s Hololens 2 headset along with tablet devices to secure the training.  

Kashimada again explained: 

Normally, accuracy of SLAM (Simultaneous Localization and Mapping) in positioning of MR superimposition depends on hardware performance. However, for this MR for ETC equipment training, SLAM features of DataMesh’s proprietary digital twin platform (FactVerse) were used to make SLAM accuracy levels of multiple devices uniform.  

Following a successful project launch, the company plans to debut another MR learning intuitive called “MR for tunnel emergency facilities training.” 

The Power of Digital Twins in Immersive Training

The XR industry is full of trending technology terminology and buzzwords – akin to other emerging sectors. However, with the rise of the industrial Metaverse and its real-world application comes a legitimization of certain integrated technologies.

A key immersive enterprise technology leading alongside the industrial Metaverse future of 2024 is digital twins.

Digital twins are already a commonly leveraged solution for many enterprise use cases, notably AEC, product design, and immersive training. The technology holds similarities to CAD files, providing a level of familiarity to users.

However, digital twins stand out from previous 3D file types thanks to the technology’s ability to recreate objects with immense detail – including replicating multiple small components and faults. Moreover, the technology can create 3D representations of objects in motion or in use to assist with training use cases.

For Nexco-East, the firm worked with its project partners to reproduce equipment movements. DataMesh, for example, created visualizations of internal structures to develop situationally relevant  MR-based ETC equipment training. 

Moreover, by leveraging immersive technology, Nexco-East can create unique training opportunities unavailable via traditional methods.

Namiki explained:

In the past, we presented pipe samples from real equipment and illustrations to teach their structures and operations, but we couldn’t actually let water through them to demonstrate the mechanism. Also, explanations somewhat varied among instructors. As with ETC equipment, the gap in levels of understanding among trainees was recognized as an issue.

The firm is leveraging Microsoft’s and DataMesh’s immersive solutions to provide fresh training abilities, such as a feature that allows mentors to adjust water transparency via an XR digital twin animation showing water flowing inside a pipeline. Therefore providing a precise simulation of how water flows and stops inside a pipe.

Moreover, the firm is integrating other unique simulation methods, such as the implementation of how fire affects equipment, letting trainees realistically experience situations which are challenging to teach outside of digital simulations – “from water flow and the valve opening mechanism to things going on behind pipes are all comprehensible at a glance,” Hideshima remarked.

The latest developments on Microsoft’s Industrial Metaverse

This time last year, many assumed the HoloLens was facing a demise. Following internal controversies, layoffs, and reorganizations, Microsoft’s immersive division looked almost stagnant as the firm moved towards genAI solutions.

However, quietly in the background, Microsoft was working on optimizing its XR product portfolio – targeted at enterprise end-users. The move came as other firms like Apple and Meta unveiled MR headsets that could – still – overtake Microsoft’s HoloLens range. However, Microsoft is solely focusing its XR portfolio on enterprise end-users with no clear ambition to reach a consumer market – perhaps allowing it to stand alone in the marketplace.

Following various breadcrumbs from company representatives, Microsoft finally gave audiences an insight into its immersive roadmap, which aligns with the firm’s new-found genAI goals.

Microsoft quietly teased information regarding an industrial Metaverse roadmap at its Inspire 2023 showcase event covering the firm’s latest innovations. Following comments from Lili Cheng, Corporate Vice President of Business Applications and Platforms at Microsoft, it is clear that Microsoft is launching its industrial Metaverse and MR product roadmap with plans to leverage the firm’s recent AI innovations.

Cheng recently explained that AI and XR should exist side-by-side to improve outcomes for various enterprise use cases. According to the Corporate Vice President, Microsoft’s investments in the emerging generative AI-powered service CoPilot will also improve MR dynamics guide service for XR headsets.

Cheng also noted the importance and roles of integrated technologies such as digital twins, explaining that RT3D visualization and data allow frontline workers to “receive a better understanding of the machines and processes at hand.” The VP said that AI enhances integrated XR technologies.

Cheng added:

Mixed reality is the eyes and ears of AI. Delving into operation nuances for personalized, in-depth learning becomes easier. When integrated, mixed reality and AI accelerate worker training, shortening steps and supplying users with the working knowledge they need for the task at hand.

Microsoft’s industrial Metaverse roadmap will also offer a “curated learning path” to provide end-users with resources to introduce workplace XR technology, with considerations for integrated first-party technologies, such as Microsoft Co-Pilot, Azure OpenAI, AI, ML, Azure Databricks, Synapse Analytics (2024), Microsoft Fabric (2024), Microsoft Purview, Power Platform, D365 Field Service, D365 Guides, D365 Remote Assist, HoloLens 2, Azure Maps, Azure HPC product, and Azure Digital Twins.

Microsoft launched its industrial Metaverse roadmap as part of its AI Cloud Partner Program, which brings significant partners together to fuel growth and innovation towards enterprise-grade AI to boost enterprise use cases of AI, Cloud, and Metaverse solutions to improve productivity, resiliency, and sustainability outcomes.

The AI investments lend a hand in optimizing Microsoft’s XR solutions for frontline operations. Microsoft services such as Dynamics 365 Guides provide a spatial environment that allows frontline workers to access Microsoft brand collaboration tools, such as Teams.

Much like its Nexco-East customer success story, integrated AI features allow XR-ready frontline workers to complete repair procedures – which can become a significant blocker – with increased efficiency, connecting workers “beyond their physical limits,” remarked Cheng.