Mobile Technology with Augmented Reality Bridges Social Distancing

Training methods address loss of human resources and personal contact during pandemic, and improve operations.

 

Over the last decade and more, multiple difficult realities for process manufacturers have emerged, including:

• Internal engineering and technical staff reductions through attrition, voluntary and forced retirements, and outright layoffs.
• Many highly qualified people and their accumulated knowledge and experience have been lost in the process, and the people that remain often lack the same depth of experience.
• Whatever expertise is still available must be spread as widely as possible within the company.
• Any tribal knowledge remaining must be captured in whatever way is practical.

These problems are not new, but they are made more complex because of COVID-19 and the new safety protocols process plant facilities must implement to keep operating without outbreaks. So today, all those challenges remain and must be solved while keeping individuals at safe distances.

IIoT-based technologies provide mechanisms to solve all these, including COVID-19 problems, simultaneously. This is not due to any sudden discovery, but rather a result of mechanisms and methods that have evolved with new technology developments to vastly improve communications. While this new capability is being used many ways, one of the most immediate benefits is improving training under COVID-19 restrictions to make up for the problems just discussed.

 

Technologies replace proximity

The traditional way to train a technician involves some basic classroom time followed by partnering with a more experienced mentor doing actual maintenance tasks in the plant. This is effective, provided there are enough experienced mentors. Unfortunately, for many facilities, new technicians are left to figure out problems by themselves. As a result, troubleshooting and maintenance tasks take longer to complete and the potential for errors is higher. What is the answer?

The traditional approach is for the unassisted field technician to call back to the control room or maintenance shop with a walkie-talkie and try to discuss the problem. It doesn’t take much imagination to picture the difficulties of trying to describe a situation and solution using this approach, but it’s better than nothing. The experienced person in the shop can, theoretically, talk to more than one technician working on more than one problem, so this is a step in the right direction, but inadequate to deal with any degree of complexity.

Improving on this situation requires sophisticated technologies supported by the IIoT. For example, Yokogawa’s SensPlus Buddy platform uses augmented reality (AR) combined with web-based communication to extend beyond the control room or maintenance shop, and perhaps well beyond the larger facility.

 

Using AR in plant environments

Smartphones and the degree of communication they enable are part of everyday life. Texting photos and visually chatting via a video-calling app, such as FaceTime, are part of how things get done today for most people. These capabilities can be used and enhanced with AR for industrial applications.

Just to avoid any confusion, we should clarify the meaning of AR as contrasted with virtual reality (VR). Sometimes these terms are used interchangeably, but this is incorrect. VR tries to create a fully immersive 3D experience and typically involves wearing a headset to provide elaborate visual effects. This is often used as an element of sophisticated industrial training platforms and can require the creation of a simulated plant environment, but no sensible person will want to walk around a working process plant wearing such a headset.

AR works with a conventional smartphone or tablet to overlay a digital image onto an image produced by the device’s camera. For example, a technician trying to pick a specific instrument listed on a work order out of a cluster of similar devices can point the smartphone camera at the group (Figure 1).

Figure 1: Platform uses AR with a smartphone or tablet camera to add information to basic imaging capabilities.
 

If the relevant device is in the picture, the AR function recognizes it and might superimpose an arrow or circle onto the image to point it out. The platform can then take the next step and link the device to information regarding its function, calibration requirements, maintenance procedure, and other information.

The user does not have to wear anything that interferes with normal vision, so there is less likelihood for an accident. This may sound simple, but the underlying technology is very sophisticated.

 

Typical use case

Here’s a typical example of a situation that could happen anywhere. A technician is dispatched to determine why a specific flowmeter is sending erratic readings to a host system. FM-27 is blinking in and out, causing plant operators to lose view of the reading. The way it is behaving suggests it is a power supply issue rather than an internal malfunction. The technician carries a smartphone and has to locate the transmitter on the pipe in the process unit.

It is possible to communicate with remote experts, maybe located in the control room or the maintenance shop, via the video call function (Figure 2) to show what is on the local display at the transmitter. Its blinking is characteristic of a power problem, likely a loose termination. With some advice from the expert via video on how to get the cover removed, the technician can now see the terminal block. The original manufacturer’s installation manual is also loaded onto the phone, so the wiring diagram can be called up as needed.

Figure 2: Communication from a technician can extend to anyone in the company via the Web.
 

After clipping on the leads from a communicator, the technician can discuss the results with the expert, and if necessary, show what’s on the screen via video (Figure 3). All it takes is drawing with a fingertip on the touch screen to mark up the photo and send it back to the control room.

The expert can send back a photo of the terminal block with the specific screws that should be checked circled, while also advising the technician to check the conduit connection at the housing.

The technician tightens the problem terminal, and then checks the others for good measure. When everything is back in place, control room operators can see the local display is correct via video, with that device view restored in the control room. This can be done by a single technician, possibly with insufficient experience to handle the job without coaching, but still able to complete it working alone thanks to the digital lifeline back to an expert.

This scenario illustrates several critical points:

• The technician can share the field situation using the video capability in far more detail than would be possible using just a walkie-talkie. Humans are visually oriented creatures, so the ability to see the picture is critical.
• Both parties in the exchange can share instructions and results graphically, marking up photos and diagrams as needed to emphasize critical points.

Figure 3: Information from a test instrument or communicator can be sent via video.
 

The exchange and resulting images can be retained for review and future reference. This is a very practical way of capturing tribal knowledge so that technician and others can learn from the experience.

Early tests of this approach with novice technicians yielded a variety of insights as to its effectiveness:

• Increased a sense of confidence at work while performing tasks
• Improved safety management during task execution
• Eliminated the need for expert engineers at the work site, making it easier to solve multiple problems, even with a shortage of engineering resources
• Allowed the plant to maintain production quality and avoid interruptions by preventing operational errors.

 

Supporting both sides

So far, we have looked at AR as a teaching tool, but consider it also from the expert’s point of view. In our example the expert did not have to leave his or her location. No additional trip into the plant environment was necessary, avoiding lost time, safety risks, and an opportunity for potential COVID-19 transmission. It was possible to look over the technician’s shoulder virtually and see a high level of detail without close personal contact.

When this is the case, the expert’s location becomes irrelevant and could be anywhere in the world. This is especially important where engineering and technical resources are scarce and not evenly distributed. This is the way to get the most from those resources with the lowest cost. Travel can be curtailed, also minimizing possible COVID-19 exposure.

It is also possible to combine forces and draw more people into the discussion, engaging more than one expert, or using a particular task as a teaching moment so other students can watch the process, regardless of their location.

 

Operational mechanics

Yokogawa’s SensPlus Buddy is a web browser-based application (Figure 4), so there is no software to maintain. As long as there is internet access, it can run on any device capable of hosting a web browser—such as a laptop, smartphone, or tablet. The operation server is managed by Yokogawa, and it supports communication via a Wi-Fi network or mobile via 3G, 4G, and LTE. There is no need for an end user to host server-based software or create additional networks.

Figure 4: Platform is browser-based, so it can be accessed anywhere there is a web connection.
 

All communication is encrypted using HTTPS (SSL/TLS) to prevent interception attempts, or block an impersonator trying to break into an online conversation. Information is not stored on the cloud, with shared images instead staged in the web browser cache, allowing users to retrieve and download these image files after a service conversation.

The AR market continues to grow rapidly as new applications are created, particularly on the mobile side. The spread of mobile devices is making AR ubiquitous in plant environments for the reasons just discussed. Yokogawa is working with customers to identify issues and deliver services for manufacturers working to cope with the growing list of daily challenges.
 
Dr. Kazuya Suzuki is the product manager for plant field support systems at Yokogawa Electric Corp. In 18 years with the company, he has served as an architect of process control, IT, and manufacturing execution systems, with additional specialties in network security. He has a PhD in informatics from Kyoto University.