CENTUM CS Wide-Area Communications System

OGAWA Toshiki¹ TATEOKA Minako¹ TANAKA Toshiaki¹

We have developed a wide-area communications system that uses the circuits of a common carrier, such as Japan's NTT, to seamlessly integrate CENTUM CS systems located far from each other. Multiple relay circuits, connected to the CENTUM CS systems by bridges, are used in a load-sharing, wide-area communications system to guarantee high throughput and high reliability. This paper describes the bridges — a key component of this wide-area communications system.

1. Industrial Automation Systems Business Division Ethernet is a registered trademark of XEROX Corporation.

INTRODUCTION

The development of communications technologies has formed a system structure known as the communication network. This network has not only established itself in the industrial world but also in the general household. The recent spread of technologies such as the Internet has been quite spectacular.

No one can predict how such a communication network will affect the general society. The key success factor of the network lies in it's ability to remove the barrier of physical distance.

The CENTUM CS wide-area communications system has been developed with this feature as its focus. The aim then is to make it possible for geographically distant plants, both in Japan and abroad, to be operated and monitored as if they were located in the same place. Another important technical challenge required to implement this approach is how to make the system operate in "real-time." The solution to these challenges involves developing systems based on "load sharing." This method has enabled a wide-area communications system that operates with high stability to be realized.

FEATURES

The wide-area bridges that form the core of the wide-area communications system have the following features:

• The bridges are linked to a common-carrier network using Ethernet. "Socket" communication using the TCP/IP protocol is employed to communicate over Ethernet. Thus, a link to relay circuits can be achieved using the regular routers and remote bridges that are commercially available from network equipment vendors. These routers and remote bridges are designed to suit different types of common-carrier networks; they can be connected to the networks regardless of the type of relay circuits used.
• Wide-area bridges relay communication frames transmitted through the Vnet, the control bus for CENTUM CS, –in other words, transparently. This makes the wide-area bridges and common-carrier networks, which exist between two systems, virtually transparent to users during operation and monitoring. For this reason, two plants can be integrated in a seamless manner.
• As it stands, common-carrier networks do not have sufficient bandwidth, while the Vnet has a transmission rate as high as 10 Mbps. If the traffic volume between systems is too large, the relaying capability between the wide-area bridges can produce a bottleneck. This may lead to transmission delays.

To solve this problem, two or more lines of wide-area bridges and relay circuits are installed between two systems. This network configuration distributes relayed communications into these two lines, increasing the throughput.

SYSTEM CONFIGURATION

Figure 1 illustrates the system configuration. The CENTUM CS contains a Vnet as the control bus.

 

Figure 1 System Configuration

System A is independent and located a long way away from system B. For each of systems A and B, an information command station (ICS) which undertakes operation and monitoring, and a field control station (FCS) which undertakes control, are located along the Vnet. These units connected to the Vnet are referred to as the nodes.

A wide-area communications system is built by connecting a wide-area bridge to the Vnet of each of these two systems and allowing a common-carrier network to run between the systems.

This system configuration interconnects the Vnet's of the two systems. From the Vnet frames that run through their respective Vnet's, each wide-area bridge selects those to be relayed to the remote system. The bridge thus relays the selected frames through such network relay units as an IP router and a digital service unit (DSU) to the common-carrier network. The widearea bridge of the remote system, in turn, relays the Vnet frames received from the other system through the common-carrier network to the local Vnet.

Thus, the Vnet frames, which are directed at either remote system and run through the two Vnet's, are relayed back and forth between the two systems. This system configuration enables the two geographically-separate systems to be operated and monitored from either side.

Figure 2 shows the common-carrier network service provided by Nippon Telegraph and Telephone Corporation (NTT). The use of digital lines as fast as 1 Mbps is suitable for relay circuits serving as control buses to meet such requirements as real-time capability and communication quality. Relay circuits slower than that baud rate may also be accepted depending on the amount of traffic between two systems. Analog lines are not usable as relay circuits since the fastest transmission rate guaranteed by NTT for these lines is as low as 9,600 bps and the network quality is low, compared with digital lines.

Figure 2 NTT Common-carrier Network Services

FUNCTIONS

1. FRAME RELAY FUNCTION
Vnet communication frames are available in two types: unicast frames and broadcast frames.

Communication frames of the uni-cast type are used as the references or settings of function-block data in an FCS, while those of the broadcast type are used to annunciate process-alarm messages.

The wide-area bridges relay each type of frame using the following procedures.

1.1 Uni-cast communication frames
The node address of the Vnet consist of domain and station addresses. The domain address is an address uniquely assigned to segments of the Vnet. The station address is an address that is also uniquely assigned within a single domain.

In the Figure 1 example, domain addresses 1 and 2 are assigned to the Vnet's of systems A and B, respectively. In each domain, station addresses 1 and 10 are assigned to the ICS and FCS, respectively. The wide-area bridges relay frames between the two Vnet's based on the domain addresses. The bridge refers to the destination domain address of each frame received in order to relay only the frames whose domain address matches the preassigned Vnet domain address of a remote system. The widearea communications system is configured so that the domain address of the remote system is defined during system generation.

1.2 Broadcast communication frames
Broadcast communication frames are relayed according to an AP command in a given Vnet frame. The Vnet frame contains a code for identifying the higher-order application that handles the frame. This code is referred to as the AP command.

The AP commands for relaying frames to a remote system are pre-registered with the wide-area bridge during system generation. When the bridge receives a broadcast communication frame, it searches for the AP command in that frame in order to relay only those frames with the pre-registered AP commands.

2. V-NET STATUS ANNUNCIATION FUNCTION
The CENTUM CS can monitor both the operating status of a Vnet bus and the nodes connected to the Vnet. The wide-area bridges announce the status of their local Vnet bus and nodes to the respective remote systems at fixed intervals or during a state transition. In addition, the wide-area bridges mutually monitor the operating status of their remote system and the commoncarrier network as well. If either bridge detects a failure, it informs the local Vnet of the current status, that is, that the Vnet in the remote system has failed.

Conversely, when the status recovers, the bridge informs the local Vnet that the Vnet in the remote system has recovered to normal.

A broadcast communication frame is used for this annunciation. The wide-area communications system is configured so that all nodes connected to the Vnet can simultaneously obtain the same status information.

3. TIME SYNCHRONIZATION FUNCTION
The CENTUM CS has the ability to equalize the nodes connected to a Vnet to have them share the same time information. The wide-area bridges equalize the nodes across the wide-area communications system. That is, when the bridge detects a change in time or received a time-setting message from a Vnet, it relays this latest time information as a communication frame to a remote Vnet. A broadcast communication frame is used for this annunciation. The wide-area communications system is configured so that all nodes in the system, which receive the relayed message, obtain the same time information simultaneously.

4. LOAD-SHARING FUNCTION
The wide-area bridge provides a load-sharing function in order to increase the throughput between two systems. This function is applied to a communications system where the traffic load across two systems exceeds the information-carrying capacity of the bridge and the common-carrier network.

In a load-sharing system configuration, two or more pairs of wide-area bridges and relay circuits are installed between the two systems. Figure 3 shows one such method. The load is shared by duplicating the communication circuit comprising wide-area bridges and relay circuits.

Figure 3 Load-sharing System Configuration

A wide-area bridge in a normal system configuration bases its relaying of frames on the domain address. The bridge in a loadsharing system configuration, however, bases its relaying of frames on both the domain address and the station address. (Note that any particular node in a Vnet is identified by a combination of domain and station addresses.)

In order to share the load, nodes are allocated to wide-area bridges in groups on one side of each communication circuit. The wide-area communications system offers two methods for this grouping:

1. Grouping based on the evenand odd-numbered station addresses of nodes
2. Grouping based on arbitrary station addresses of nodes

The second method makes it possible to configure a loadsharing communications system with three or more relay circuits. The information on load sharing in this system configuration is equalized between the two wide-area bridges that are paired when a communication connection is established between them.

The following explains how the system shown in the Figure 3 example works.

In Figure 3, two pairs of wide-area bridges are installed to duplicate the relay path routed via the common-carrier network. In this configuration, bridges A-1 and B-1 form one pair while bridges A-2 and B-2 form another pair. In this example, the nodes in domain 2 are grouped as noted below, so they are allocated to each communication circuit.

• The communication circuit comprising bridges A-1 and B-1 supports communication between all nodes in domain 1 and the FCS-41 and ICS-31 stations in domain 2.
• The communication circuit comprising bridges A-2 and B-2 supports communication between all nodes in domain 1 and the FCS-42 station in domain 2.

When frames are transmitted from system A to system B, only those frames that ought to be relayed according to the load sharing information noted earlier are relayed by the bridges.

This system configuration has made it possible to limit transmission delays resulting from the restricted informationcarrying capacity of a relay circuit. Consequently, the wide-area communications system can achieve real-time capability, a feature of the Vnet, even when the system-to-system traffic load is significantly quite large.

5. SECURITY FUNCTION
In wide-area networks, especially switching networks, systems are directly linked to public communication networks; thus security becomes a serious issue when using these networks. For this reason, the wide-area bridges have a feature that rejects link requests from bridges other than those that are pre-registered during system generation. This feature is realized by simply checking the IP address of a node that has requested the connection.

It is also possible to build an even more secure network environment by:

• introducing conventional security-oriented routers such as network relay equipment
• encrypting the process data transmitted through commoncarrier networks

CONCLUDING REMARKS

The CENTUM CS wide-area communications system is operating at many user sites, both in Japan and abroad.

The system has realized plant operation that is capable of wide-area communication and features a higher degree of realtime capability. This is especially important when the loadsharing function is applied to system operation across plants with heavy traffic loads.

There is no doubt that this type of communication technology will advance further as a system to include hardware and software. This wide-area communications system is only our first step. We are determined to pursue increased network speed, enhanced security, and reduced costs, while at the same time maintaining a balance between these improvements and the capabilities of distributed control systems.