With the digitization accompanying Industry 4.0, IoT and Cyber-Physical Systems or Digital Twins, data and its processing are becoming a key element of value creation. Despite the efforts of automation, autonomization and the use of artificial intelligences to control the growing data volume, humans continue to be the main decision-making body. As a result of this situation and the paradigm of human or customer centricity, digital human-machine-interfaces (HMI) have gained central significance. It is important to create even more effective assistance systems which continue to give humans the ability to control, monitor and make safe decisions in the work processes.
This is where augmented (AR), mixed (MR) and virtual reality (VR) come into effect as human-centered front-end technologies.
The possible applications of AR, MR and VR extend across the entire product lifecycle, since visualization is increasingly important in every phase of the product lifecycle and the strength lies in spatial and dynamic visualization. With AR, this can be performed in the application context or completely independently of time and place, as with VR.
In the “Assisted Reality” trail, T-Systems Multimedia Solutions GmbH uses the example of previous customer projects to show how assistance systems can be created with AR, VR and MR for industrial use in production, maintenance and service as well as in the fields of marketing, sales and corporate communications. For the user, this creates new possibilities to complete tasks more efficiently and rapidly, make faster decisions and increase the convenience and experience, which boosts user motivation.
The use cases are generally autonomous but can also be linked to smart maintenance and smart products.
Lars Vogel, T-Systems Multimedia Solutions GmbH
- Managing directors and C-level decision makers (CEO, CDO, CFO, CMO, CIO)
- Managers and planners in development, production, operation and maintenance
- Managers of technical field services
- Marketing managers and sales managers
- Machine and system construction, finishers and system operators, airplane and engine construction, automotive production, telecommunications, consumer goods, German Armed Forces, police
The Assisted Reality trail includes five use cases from different customer projects and industries.
IoT gives international system manufacturers and operators the opportunity to monitor the status of their products and control them remotely. Despite many digital advantages, there is an additional need to adjust settings or analyze the system on site. In most cases, however, the staff at the site lacks the required expertise to correctly adjust the system and analyze and remedy errors. The trips required to keep the systems up-to-date tie up many experts and generate enormous costs. The idle time created by the maintenance intervals results in additional high costs and stress; losses from production failures quickly go into the six-figure range. In such cases, VR, AR and MR can help to cover maintenance and work on-site as well.
Troubleshooting by telephone or email is made much more difficult by language barriers, for example due to different native languages or specialized terms as well as cultural customs (e.g. handling faults). Globally operating manufacturers thus look for solutions which enable experts to perform a comprehensive analysis of systems and the system and operating context even remotely.
It also must be possible to quickly transfer the knowledge gained through the troubleshooting (e.g. the correct setting parameters) to the product development unit and into the documentation so that it can be made available for training more efficiently and be updated quickly. This calls for solutions that make it possible to create easily understandable documentation on the fly and continue using it in other contexts.
In system design and planning, it is still customary to this day to plan systems in 2D and then use them in paper format for visualization. As a result, the ideas behind the systems or complex products and a correspondingly precise definition of requirements strongly deviate from those of the real needs and real designs. This leads not only to frustration but also a loss of several months along with increasing costs for modifications that were not previously calculated.
A challenge for system control is that a decreasing number of employees must monitor entire process chains whose speed also requires the ability to quickly respond to faults. Operators must be able to handle more complexity since they need to keep an eye on more things all the time. This requires solutions that make it possible to monitor and control systems at any time from anywhere and quickly get an overview.
Assisted Reality – Remote Maintenance Support
Managers of maintenance and servicing
The maintenance must be completed faster and more efficiently so that every saved hour makes an important value contribution. One significant factor is being able to avoid travel and waiting times, either due to a separation of documentation and installation sites or by experts having to make the trip there. The maintenance progress, which was previously captured manually and with analogue worksheets, will be automated and determinable in real time in the future so that service technicians are relieved of this task, better responses to potential bottlenecks are possible and scarce resources can be optimally planned for. Additionally, paper-based documentations must be replaced by digital formats to avoid media disruptions that may cause errors and to keep them up-to-date more easily. The knowledge dispersed to multiple maintenance teams must also be increased and updated. Digital operating instructions and instructional videos can simplify and speed up knowledge transfer. In addition to challenges in knowledge management, the maintenance and servicing processes should also digitally capture and process wear and tear along with damages. The full integration in ERP, PDM or CRM systems is simultaneously the goal and the prerequisite.
T-Systems Multimedia Solutions shows how service technicians can use AR technologies during servicing.
- Status information for the robot or machine are queried directly at the components in real time via tablet or data glasses.
- Service technicians then receive system or product information, such as sensor data, 3D animations, videos, a fault analysis or even repair steps to be taken.
- For additional advice, other experts in the field can be consulted via tablet or data glasses in a way similar to a video conference. A chat makes it easy to enter work instructions.
- Additionally, the service technicians can virtually mark areas in the recordings that may indicate damage – such as hairline cracks for example – and require further examination.
- The video function of the data glasses makes it possible to create maintenance videos that can be used further for analyses, documentations or as training videos.
Assisted Reality – Training and support in the service process
Managers of Service and Technology
AR-based support of line circuits for technical service by Deutsche Telekom.
Service companies usually have to master a whole series of challenges. The technical development, which results in a greater product and performance variety as well as time and cost pressure make it increasingly difficult to maintain the service quality. In addition to further reductions of error rates, the response times must be sped up and additional subcontractors must be involved to provide affordable personnel. This means that the technical knowledge in the minds of older and experienced employees in particular must be made digitally accessible so it can be scaled. With the help of AR and MR, the company should also become more attractive in its recruitment of younger employees and in customer acquisition while also creating a bridge to older staff members.
T-Systems Multimedia Solutions shows a service app based on Microsoft Hololens for the circuitry of network lines. This is an ongoing project and shown is the result of the first development level, how a switching operation is carried out at the so-called MFH (multifunctional housing).
- The Hololens shows the service technician the order with the customer data, the site of the MFH and the required tools.
- Having arrived at the MFH, it must be opened with a transponder to deactivate the alarm system.
- When the MFH is open, the service technician starts the work process: On panels floating in space, the technician is shown step-by-step work instructions and the correct switching path across the switching blocks.
The clear work instructions ensure that even untrained employees can easily learn the task. The visualized switching path enables a quick capture and high-quality circuitry. As shown by user tests, the Hololens itself creates high motivation among the staff.
The design of this assistance system received a Red Dot Design Award in 2017 for excellent application design.
Assisted Reality – Support of robotics in production
AR-based robot control via IoT Gateway
System operators increasingly have to monitor entire production lines consisting of multiple single system modules. To quickly monitor the status of each module without having to stand directly in front of the respective system panel, the AR or MR application will help to see the system status from anywhere purely with visual contact. Additionally, robots should be more intuitive and directly operable, either to define processes or take over control, generally on a temporary basis.
T-Systems Multimedia Solutions shows an MR app based on Hololens that makes it possible to monitor a robot status, where a Lego Mindstorm is used as a substitute for systems for demonstration purposes. In addition, the robot can be controlled by the Hololens with simple commands.
Assisted Reality in marketing and sales
Marketeers, sales, product developer
Holo-Marketing app and Digital Reality app
It is difficult to market complex products such as systems, airplanes or yachts as well as furniture or other large objects, since they can only be presented to the potential customer on site as an exemplary sample, if at all. It becomes even more difficult when the products are still in development or will only be created once the buying decision is made.
This also applies to smaller products with many variants which are customizable and can be produced in a larger variety – here too, the customer’s imagination is required for a confident buying decision.
Based on a smartphone app and a Hololens app, T-Systems Multimedia Solutions shows how the sales and presentations of products are optimized.
With the Holo-Marketing app, it is shown how three-dimensional, holographic product projections that integrate real space enhance the imagination and enable assembly planning in the space. Furthermore, this application shows how several Hololenses can be combined into a “shared experience”.
- Here two users each get one Hololens that enables an interactive projection of products.
- The viewers can select a product from a catalogue, such as an endoscope which they can experience in stereoscopic and 360-degree views.
- They can either move around the product or rotate it via a tap, scale it or open a panel with additional information.
- The product itself can also be moved to other places, which even enables the complete interior furnishings of a lab.
The scenario shows how entire product worlds can be experiential.
With the Digital Reality App from T-Systems Multimedia Solutions, it becomes possible to use a smartphone like a magic lens to digitally expand an analog print Magazine.
- The static magazine illustration of a smart factory, for example, is brought to life via AR and can even be experienced as VR after a waiting period.
- The scenario is accentuated by text spoken by an expert.
- The Digital Reality app also makes it possible to contact the expert instantly via email or phone with a single click.
The marker-based approach enables a number of additional use cases that combine marketing, sales and training: through digitally stored product information that replaces package inserts or by showing the availability and history or origin.
Assisted Reality in product development
Development and design Managers
Design Review App
Globally operating product manufacturers face the challenge of having to develop products faster and cheaper while also ensuring that they meet the local market demands and requirements. To detect these, many iterations with modifications and adjustments at the digital CAD model are required, often with a real assembly. But this results in enormous costs and delays.
T-Systems Multimedia Solutions shows how AR based on Hololens enables the early visualization of digital drafts for dispersed teams even before the first real assembly.
- The technology makes it possible for companies to integrate their customers into the product’s development and planning process at an early stage, to make the product experiential for the customers, discuss it with customers and determine improvements. This increases the chances for success in the market.
- A presentation of non-transportable products at the customer’s site in the form of a holographic, true-to-scale depiction is also possible.
teaser trail (1 day), intensive trail (2-3 days)
The teaser trail presents the state of the technical development in the area of mobile AR/VR/MR to the participants and shows possibilities based on concrete customer problems. With workshop performances, the teaser trail can be expanded into an intensive trail.
Along with other group units at Deutsche Telekom, T-Systems Multimedia Solutions GmbH is developing solutions for customers in Germany and abroad.
Opportunities for Projects and Funding Instruments:
- Technology partnerships
- Potential participation in joint projects
- Order for T-Systems Multimedia Solutions GmbH
You are interested in this Trail? Contact us!