Nobuo Okabe1
Efficient and safe operation of automation systems requires the knowledge and experience of experts, and the transfer of knowledge and experience from generation to generation is indispensable for its accumulation. That's why the aging of operators and engineers is an important issue in industry today. The Cyber Perception Concept aims to resolve the transfer issue, enhance human adaptability and resiliency, and improve plant efficiency and safety, using cutting-edge technologies. The goals of the concept are those of making knowledge and experience reusable, improving the efficiency and safety of human work, and avoiding risks in human work. This paper describes the research activities undertaken to achieve this concept, including those of anomaly detection using corrosion sensing images, and industrial augmented reality (iAR).
- Research & Development Division, Innovation Headquarters
INTRODUCTION
 |
| Figure 1 An inclusive relationship between goals and their functions |
Automation is defined as "the use of computers and machines instead of people to do a job"(1). However, automation systems cannot actually operate without the support of human beings. Sound operation of automation systems is supported by many experts. For example, as the number of product types produced in a plant is increasing regardless of plant scale, plant operation needs to be backed by a higher level of experience. In addition, during system operations, daily fluctuation of raw materials and changes in external environment such as weather and plant equipment need to be considered. Furthermore, many experts are also needed for prevention and early detection of equipment failure, aspects that form the basis for plant operation.
Usually, because the transfer and accumulation of technical knowledge and experience depend on intangible transfers between human beings, the costs needed for this are indistinct. In other words, they appear simply as labor costs. Meanwhile, the issues of decreases in manpower and the lack of successors at production sites, which are caused by economic reasons such as personnel reduction for improving management efficiency, or social factors such as a decline in the working population due to population aging, have generated concern in various domains. The decreases in the workforce and successors make the costs of accumulation and transfer of knowledge and experience owing to human ability priceless.
The Cyber Perception Concept aims at improving production efficiency and safety, by resolving the issue of transferring the knowledge and experience mentioned above and strengthening human intrinsic adaptability and resiliency through the utilization of the latest technologies and their research results. The objective and goal of this concept, research activities related to it as well as future issues related to it are described in the following sections, in this same order.
OBJECTIVES OF THE CYBER PERCEPTION CONCEPT
The objectives of the Cyber Perception are to resolve the issue of transferring knowledge and experience mentioned above by making use of the latest technologies and their research results. The two following specific objectives are to be achieved. The first is that of changing the costs required for the accumulation and transfer of knowledge and experience from priceless to affordable, by replacing the work of accumulation and transfer, which currently depends on human capability, with machine work as much as possible. The second is that of improving productivity and safety and offering a more pleasant work environment, by providing the persons involved with new roles and relationships.
GOALS OF THE CYBER PERCEPTION CONCEPT
 |
| Figure 2 A conceptual diagram of Cyber perception |
To achieve the two above objectives, the following three goals must be reached. The goals and functions required for them are in an inclusive relationship together as shown in Figure 1.
Goal 1: Making human knowledge and experience reusable.
Goal 2: Improving efficiency and safety of human work.
Goal 3: Avoiding risks in human work.
Making Human Knowledge and Experience Reusable (Goal 1)
For achieving this concept, human experience and knowledge must be converted into tangible data so that they can be reusable. This process is the basis for all services. The main functions required for this goal are data collection through various means, interpretation and analysis of collected data, and provision of data in a reusable format shown in Figure 2-A.
Data to be collected include unconscious information relating to human work and decisions, such as the time and contents of operations, conversations and expressions, and surrounding sounds and visions. Such information can be collected by sensing via human beings making use of wireless sensor network systems, mobile terminals and others.
 |
| Figure 3 An example of wide area corrosion progress monitoring in a plant (monitoring corrosion areas using a threshold) |
The latest research suggests the potential of modern statistics for interpretation and analysis of collected data. One of the most difficult issues during this interpretation and analysis is how to give meaning to collected data. Because most of the data collected in plants are just multi-dimensional and time series data with no particular meaning, giving meaning to the data is needed for their analysis. In addition, human intervention is necessary to do so. Thus, the key is how effectively meaning can be given to the data. Details of this issue are explained in the section "Technologies for Giving Meaning to Data."
To provide accumulated data in a reusable format, a data platform enabling the horizontal integration of applications is essential. Details are explained in the section "Data Platform."
Improving Efficiency and Safety of Human Work (Goal 2)
The next goal is improving operational efficiency and safety by widening the range of human perception and awareness.
 |
| Figure 4 An experiment for detecting localized corrosion locations using four ultrasonic sensor signals |
In order to make use of the information accumulated through achieving goal 1, the surrounding situation must be also grasped as shown in Figure 2-B. For this purpose, a technology called context awareness can be applied. For achieving goal 2, first, related objects and their locations are identified, and then information considered useful is searched for and selected. When searching and selecting, technologies such as a recommendation system or fuzzy search are required. In these processes, it is necessary to adequately visualize attributes, states and the like which cannot be perceived with the ordinary five senses of human beings or are difficult to understand. For example, temperature, existence of poisonous gases and the status of neighboring operational teams are those which need to be visualized.
In addition, as shown in Figure 2-C, it is necessary to support multimodal communications with remote operators in real-time, such as sound, vision, location information and operational status.
These measures for achieving goal 2 will not only improve the efficiency of on-site operations, but also reduce unexpected events, contributing to improving safety.
Avoiding Risks in Human Work (Goal 3)
Even if operation efficiency and safety are improved, hazardous incidents cannot be completely eliminated. Accordingly, preventative measures are necessary. For example, human errors are inevitable, and double-checking and cross-checking by a third person are employed to prevent such human errors from becoming critical incidents. Making use of the functions of goal 2, a person who is not at the site can assist checking as a third person. Furthermore, measures for the heuristics and biases(2) can be computerized, and research results of mental models(3) can be applied to achieve easy operability for prevention of misoperations.
RESEARCH ACTIVITIES FOR ACHIEVING THE CYBER PERCEPTION CONCEPT
 |
| Figure 5 An example of anomaly detection by using image information |
This section outlines some of the research activities which can be applied for achieving the cyber perception concept.
Corrosion Sensing
Plants consist of metal components, and corrosion is a common problem across industry. Corrosion is categorized into two types; general corrosion and localized corrosion. The progress of the former has a certain tendency and can be predicted easily, and so there are many solutions available. In contrast, in cases of the latter, it is very difficult to predict its location and state of progress, and no economically viable solutions are yet provided. Thus, detecting localized corrosion still relies heavily on human inspection, although lapses are occasionally seen. Human inspection of localized corrosion is a problem for productivity and safety.
The example introduced here is a study of technologies and systems for detecting localized corrosion by sparsely installed sensors, monitoring the corrosion's progress, and managing information about corrosion locations. Its conceptual diagram is shown in Figure 3. The major issues for this system are as follows.
- Low costs are required to cover a wider range of a plant.
- To cover a wider range, for example, a size of a measurable area is given higher priority than measurement precision.
- Multiple detection methods need to be combined to monitor structural objects of varieties of shapes and under various environments.
- The system needs to be linked with the location information management of each piece of equipment and device, and others in a plant.
 |
| Figure 6 Seamless collaboration based on iAR |
In this study, a measurement method using ultrasonic waves to measure corrosion is being examined as shown in Figure 4.
Anomaly Detection by Using Image Information
Even though various monitoring technologies exist, in many cases human beings monitor the existence of anomalies except for in inaccessible locations. However, it has been proved that human perception can deteriorate due to various reasons. For example, most people asked to focus on counting basketball passes could not spot a woman wearing a gorilla costume crossing the court(4). The research shown in Figure 5 as an example aims to compensate human overlooking of anomalies by detecting something different in image information.
Industrial Augmented Reality (iAR)
Augmented reality (AR) is a technology which greatly expands the information obtained by the visual sense, which is the most relied upon of the five human senses, for providing a new intuitive user interface
Industrial augmented reality (iAR) aims to improve on- site operation efficiency and safety using AR technology through effectively enhancing the five human senses. Figure 6 shows an iAR application example. The details are described in "Industrial Augmented Reality - Innovative operator assistance in collaboration with Augmented Reality" in this special issue.
FUTURE ISSUES
 |
| Figure 7 An example of data analysis using domain knowledge |
This section explains some of the important issues needing to be investigated in the future.
Technologies for Giving Meaning to Data
Data analysis in the area of electronic commerce (EC) is popular. Because data subject to analysis are those derived from behaviors of individuals such as purchase activities, natural language in text, and personal profiles, a methodology for giving meaning to data is established. When a person purchases something through EC, personal information must be registered, and time and object of purchase are recorded as well. His comments on a social networking service (SNS) can be analyzed using a natural language analysis technology. His activities on the Internet can be traced and logged by using cookie and other technologies used in the browser. Even identifying a person is possible by extracting his activity profile.
Meanwhile, the data collected at a production site such as a plant are multi-dimensional and can be time series data with no particular inherent meaning. However, analyzing the data making use of what is called domain knowledge, expertise relating to a system from which the data is collected, can give meaning to the data as shown in Figure 2-A.
 |
| Figure 8 A dual-layered structure model of data and applications |
Figure 7 shows an example of data analysis using domain knowledge. It is clarified that certain multi-dimensional data consists of two groups of data, and their distributions partially overlap.
As explained above, some kinds of human intervention are necessary to give meaning to data. Giving meaning to data requires the combination of two approaches; finding out data corresponding to events already known to people, and finding out events not noticed by people. Selecting methods and a workflow is the key to its efficiency.
Data Platform
As shown in Figure 1, the three-stage goals are set for achieving this concept, and required functions for a stage include those for a previous stage. This implies that a dual- layered structure as shown in Figure 8 is needed. There must be a strong requirement to integrate stand-alone applications, e.g. those described above, horizontally to increase the flexibility and resilience of human work. The integration layer must be end-user-programmable to adopt a variety of use cases.
CONCLUSION
Yokogawa believes that seeking out ideal automation systems as a whole, including human beings, is required for the future of automation, and it is the responsibility of Yokogawa as an automation equipment vender. Meanwhile, when considering human nature, wide-ranging problems explained in the preceding sections must be resolved. Thus, the way of proceeding with studies on the concept must be elaborated. We will shorten the time required for a cycle of trial and error by giving higher priority to verification of the value of the concept than specific technology development, and conduct investigation, research and development which can quickly arrive at the proper destination.
REFERENCE
- Pearson Education, Longman Dictionary of American English, Pearson Education ESL, 4th Edition, 2008
- D Kahneman, Thinking, Fast and Slow, Farrar, Straus and Giroux, 2011
- D A. Norman, The Psychology Of Everyday Things, Basic Books, 1988
- C Chabris, D Simons, The Invisible Gorilla: How Our Intuitions Deceive Us, MJF Books, 2010