Companies using or providing industrial automation systems never want to receive news that a vulnerability has been identified that could allow a cyber criminal to break into a system to do damage or steal information. Anyone who pays attention to the cyber security world knows about these sorts of problems.
Microsoft has to correct such vulnerabilities in Windows and other platforms on a monthly basis, or even more often in some cases. The hand wringing about Microsoft discontinuing security updates for Windows XP shows that hackers are still finding new vulnerabilities; others certainly still are waiting to be discovered, even after more than 12 years of security updates and improvements.
Figure 1. Multiple Firewalls: A company should install an Internet-facing firewall as well as ones between its corporate and control networks. Source: ICS-CERT
Vulnerabilities of one sort or another exist in virtually every software program of any size or complexity. While programmers do their best to create reliable and secure software, some weaknesses will arise when programs have many thousands or even millions of lines of code. These vulnerabilities may not interfere with normal operation but one might provide an entry that can be pried open with the right tools and strategy if a cyber criminal wants to break in.
Meet ICS-CERT
The U.S. Department of Homeland Security supports an Industrial Control Systems Cyber Emergency Response Team (ICS-CERT) that provides many helpful resources and services for industrial users. If this is something you never have explored, make a point of checking outhttps://ics-cert.us-cert.gov.
One ICS-CERT service is an index of vulnerability advisories listed by particular products deployed in the field. Most of these advisories relate to a specific piece of hardware or a software platform. The weaknesses might have been discovered by users or as part of laboratory analysis. The response team documents the problem, explains what it means, indicates if it has been used in a documented cyber attack, details what the vendor is doing about it, and provides suggestions for mitigation.
The list covers a variety of issues. Some examples include:
• runtime toolkit null pointer deference;
• data server denial of service;
• improper authorization;
• incorrect DNP3 driver input validation;
• use of hard-coded password; and
• insecure password storage.
The website names the specific supplier affected by each listed vulnerability.
Many varieties of root causes exist but one of the most common is some form of buffer overflow. In simple terms, a buffer overflow occurs when a hacker forces too much data into a given buffer space, which causes it to overflow into adjacent buffers and create interference. This can provide a means to access other parts of a system for potentially nefarious activity.
Cyber criminals typically use these vulnerabilities as tools in one of two ways.
A hacker who has learned how to exploit a vulnerability in a particular piece of equipment can try to use this skill wherever that equipment is deployed. Knowing how to break into a Brand-X Model 200 programmable logic controller (PLC) because it has a hard-coded password allows the person to search for situations where that device is found and attack these locations as targets of opportunity. The motivation may be mainly to capitalize on the weakness of the equipment rather than to target a specific location.
A hacker trying to break into a particular target will take the opposite approach. The person might scan the systems of the company to determine what equipment it uses. When those items are cataloged, it’s a matter of finding out which have the most useful vulnerabilities that can be exploited to achieve the desired result. Some companies have discovered to their dismay that a dedicated hacker has a better picture of their networks than they do.
A Sobering List
After spending a few minutes sifting through the list of advisories, you’ll find that virtually every major automation supplier shows up one way or another, and most more than once. No company is immune to some sort of problem, particularly with its larger platforms. Suppliers don’t like to appear on the list and do everything they can to correct these issues but the realities of cyber security guarantee problems always will exist.
Such problems occur for a number of reasons. Suppliers tend to reuse chunks of programming that work and some of these can go back to a time when security was less of a concern. Such legacy code can remain in a user’s system for many years and function perfectly, all the while hosting a vulnerability waiting to be discovered.
Some users looking at the list of vulnerabilities may find it somewhat overwhelming given the sheer number of items. However, for a given asset owner or industrial network administrator, the overall number isn’t as important as the specific items germane to particular installations.
The more relevant question is how many of these vulnerabilities exist in your plant, and how severe the individual ones are given your particular applications. Of course the information available from the ICS-CERT only is valuable if you know when it applies to you, which requires some sleuthing on your part.
The extent to which this ICS-CERT information will have meaning for a specific company depends upon one critical question: Do you know what is in your plant? This isn’t a general question with a short answer. Instead, you must consider in detail the particulars of your operations because the ICS-CERT advisories are very specific down to the individual device or system and its software revision.
Yes, it is important to know which distributed control system or supervisory control and data acquisition platform is running, but this knowledge must include every item of automation and networking hardware, including but not limited to:
• PLCs, programmable automation controllers and remote terminal units;
• Industrial PCs;
• engineering workstations;
• human/machine and other operator interfaces;
• ethernet switches;
• servers;
• firewalls and security appliances;
• wireless gateways;
• transient, portable and mobile devices;
• modems; and
• anything and everything else that communicates on your automation networks.
The list isn’t complete without the relevant software and firmware — down to the revision level — connected to every one of these items. It’s also important to know how all these devices are connected and what kinds of communication are typical and necessary in day-to-day operation.
If your list lacks the required level of detail, you’re not alone. Few, if any, companies keep their list as up-to-date as they should. Nonetheless, the ability to know compromised devices and systems exist on your networks is critical to defending them against attack.
Understanding traffic on your networks also is an essential step to identifying something that shouldn’t be there, but that’s a discussion for another time.
Using the List
You likely will find at least a few entries on the list that apply to your particular operations.
Each advisory cites the issuing company, the specific products impacted down to their revision level, and the date of the advisory. It provides enough detail so a user can know immediately if a given installation is affected, hence the necessity of your knowing in detail what software is in use.
Advisories will offer warnings such as: “Successful exploitation of this vulnerability may allow remote attackers to execute arbitrary code. Impact to individual organizations depends on many factors that are unique to each organization. ICS-CERT recommends that organizations evaluate the impact of this vulnerability based on their operational environment, architecture, and product implementation.”
The advisory continues with a discussion of where these platforms typically are used and more technical detail on the vulnerability. It also offers analysis as to how the weakness might be exploited, whether any known cases of it being used in an attack exist, and even some mention of the skill level required to carry out an attack. This information is designed to assist an affected user considering what defensive strategy might be best.
Normally, the ICS-CERT allows the relevant supplier to study the vulnerability and to prepare a mitigation strategy before publishing an advisory. So, typically, the advisory will suggest something like: “[The vendor] provides patch software for the latest revisions of the affected products. This vulnerability can be corrected by installing the patch software. The computer must be rebooted to activate the patch software. If the system uses earlier versions of the software than the ones for which the software patches are provided, [the vendor] recommends that users upgrade to the latest revisions/versions and then apply the software patches.”
ICS-CERT also offers a series of general defensive measures that can be used to protect against this specific risk and other related issues; for instance, it suggests the use of multiple firewalls (Figure 1). Those practices suit virtually any situation and are well worth reviewing. In addition, more extensive guidelines appear in other publications, which can be found with a few minutes of exploration on the ICS-CERT site.
Working with Patches
A typical industrial automation system environment includes three main types of software:
1. The main control system platform. Most environments usually consist of one main system supported by ancillary functions or utilities added by the integrator or customer. In any case, these programs are all known and function under their own name.
2. The operating system environment, and the general networking and communication platforms. These serve as infrastructure; while they may support the automation system, they don’t perform any control functions. In years past, these would have been included as part of a larger proprietary system but more recently products like Microsoft Windows have taken over.
3. All other items of the automation system. Suppliers of complex systems or control system integrators may embed third-party software to carry out specific functions. Depending upon the situation, the identity and nature of these might not be obvious — or even known at all — to the user. When a vulnerability is found and the user doesn’t have sufficient information to recognize it, the risk remains that a criminal might find the weakness and exploit the opportunity.
Most solutions require software patches. When dealing with the main control system platform, you can presume that patches issued by the original company will work without disrupting other parts of the system, at least under normal circumstances. They typically do require rebooting the system, which might be difficult depending upon the criticality of the system to the plant’s operation.
Patches issued for operating systems, including Windows and other platforms that perform similar functions, must be checked to verify they don’t disrupt any control system functions. Often testing done by the control system vendor will suffice. However, particularly if an end user’s system contains significant modifications, the user company may have to carry out its own tests before deploying the patch to the operating plant system.
Dealing with third-party software vulnerabilities is the most difficult because such software frequently relies heavily on custom code. In these cases, it’s often necessary to go back to the original creators of the code. Another option is to modify the system such that the custom code no is longer needed.
Five Key Steps
Table 1. Properly addressing cyber-security issues starts with detailed knowledge of your specific systems.
You must understand the particular vulnerabilities your operation faces and deal specifically with each weakness. This requires:
- Preparing a detailed list of your plant's automation hardware and software.
- Examining the ICS-CERT advisories to see which apply to your system.
- Making a list of these issues.
- Contacting your suppliers for patches to resolve your listed issues.
- Developing a plan to install and test patches.
Use Available Resources
Tackling your particular cyber-security vulnerabilities demands a conscientious approach, as summarized in Table 1.
Take advantage of the ICS-CERT. It offers a variety of helpful information and resources for asset owners. This information applies to a wide range of applications and can form a basis for initial cyber-security planning.
The specific vulnerability advisories are more complex; using them effectively assumes a certain level of technical competence and the ability to recognize when and how they apply in a given situation.
Cyber security is a complex issue but help is available from ICS-CERT, suppliers and consultants.