Safeguarding your assets
Published: 08 January, 2015
The increased use of plant floor automation to achieve production goals has created a dependency on PLCs, PC-based control systems, SCADA systems, robotic controllers and HMIs. It is therefore vital that companies ensure that proper safeguards are in place to protect and manage any changes made to control logic across the plant, says Andy Thorogood,product manager at MAC Solutions.
Plant floor automation control systems often involve an array of PC workstations, programmable logic controllers (PLCs), HMIs (human machine interfaces) and robotic controllers. Control logic is stored either in the device or on an associated workstation and can involve a large number of associated files and executable programs. This complexity poses a challenge to detecting unauthorised – and potentially hazardous – program changes, especially where systems from multiple vendors and the associated variety of program development and device management tools exist.
Until recently, proprietary protocols and network isolation provided adequate security from external threats. However, many vendors are abandoning proprietary communication mechanisms in order to lower costs and improve reliability. Similarly, more and more device management is moving to PC-based workstations and other ‘open’ systems. This transition to standard protocols and operating systems is making modern devices and systems more vulnerable to attack.
Change management and the plant
Fortunately, software solutions are now available that can help to safeguard plant-wide automation and control assets. An automation Change Management Systems (CMS) such as Autosave from MDT Software, is a centralised system that manages changes to program logic for controls programs and devices such as PLCs, CNCs, HMIs, PC control systems, robots, drives and general automation programs. A typical small plant will have a few hundred programs that should be managed, while large plants will have several thousand. Over the life of a facility the investment in program logic alone represents a significant expenditure that should be preserved and optimised. In order to do this, a CMS should have the following features:
• A backup/archive of prior revisions of programs.
• The ability to detect changes.
• Tools for documenting changes and making these visible to users.
• A historical record of who made the change, when, and from where it was made.
• Secured user and workstation access.
• Features for controlling editor operations mapped to user permissions.
• Disaster recovery/procedures for recovering from hardware failures.
• Change notification.
As automation devices have grown more complex and have incorporated more plant data in their operation, there is an increase in the need to make adjustments to variables and logic to continue smooth operation. These adjustments may be minor individually, but are directly linked to machine throughput and uptime. If the current device program and configuration are lost, and an old version of the device program must be used, the result is decreased machine performance, decreased quality and/or downtime. While this situation is costly enough, consider the ramifications to plant operation if there are no older versions of a lost program available and the program must be completely rewritten. This can and does happen, and the effects can significantly impact safety and plant throughput for months. These impacts added to the cost to re-rewrite, test and commission a single program are often greater than the cost to implement a plant-wide CMS solution.
Types of risks
There are many events that can have a negative affect on plant performance, and some that represent serious safety hazards. Reliable automation control logic can be compromised by the following events:
• Human Error: If someone makes changes to a program that result in undesired performance, or corrupts the program due to inadvertent changes, the prior version of the program is readily available with a CMS.
• Equipment failure: Equipment can and does fail. If the hardware fails and the only good copy of the program logic was in that hardware, the plant has a problem. With a CMS, the hardware is replaced and maintenance staff download the latest version of the program to the processor resulting in only a few minutes of downtime.
• Sabotage: As unfortunate as this threat is, someone can connect directly to many devices (especially those in remote, unsecured locations) and modify the program with harmful results. A CMS is designed to store processor passwords so these are not available without going through the CMS. Also, the CMS will periodically upload the logic from the processor for comparison with a copy on file. Changes can be identified in graphical detail, and immediate notification can be sent to responsible individuals.
• Power surges / interruptions: Power issues can cause equipment to lock up or go off-line. If these situations result in a loss of the program, it can be downloaded from the CMS after the hardware is reset.
• Fire: Any fire will be a major disruption. Whether a single device or an entire facility is lost, having all program logic stored in a central, organised CMS repository accelerates the time and decreases the cost associated with resuming production. Insurance underwriters are beginning to factor in the use of a CMS in assessing the risk profile of facilities. Without proper system safeguards these events can lead to increased downtime and an increase in “mean time to repair” (MTTR). Recovering from these events quickly requires adequate planning on the hardware and maintenance strategy, and a reliable and recent backup of the automation control program logic. Current and complete backup copies of the program logic require the features of a CMS. While a manual backup approach may appear adequate at first glance, experience has shown that plant personnel have too many tasks that compete for the time to manually back up programs on a consistent basis. Also the increased visibility of changes through better reporting and the potential for process improvement brought about by the effective use of a CMS application can quickly pay for the CMS.
Impact of plant activities on versions of program logic
Each plant has a unique set of change types and frequencies that can affect a CMS strategy. A selected set of activities is outlined below to prompt further thought and highlight the need for a proper implementation of a CMS in order to achieve optimum results.
• Nature and frequency of changes: ensure that an adequate number of program copies are available to ensure that changes can be classified and reviewed. Some changes represent true improvements, while others highlight a process problem or training issue that should be addressed by other means.
• Process enhancements: If changes are made in the process that make prior versions of the program obsolete, these enhancements should be clearly identified so that users do not revert to an older version of a program to fix a new issue. Plant operating guidelines should identify when the deletion of prior programs is warranted, and which users will have this permission.
• Unmanaged changes: Without a CMS the controls engineer would use the editor software on a workstation or laptop to make changes in a device. If multiple people make changes from multiple computers, the documentation of changes is often lost. Using a CMS to compare the program running in the device with the last recorded version, a plant can identify changes that were made outside of the CMS. Once the CMS is implemented and sufficient device networking is in place, edits outside the CMS should be discouraged.
• Temporary changes: It is common to make a temporary change to a program to resume operation while a maintenance task is performed on a failed component. It is also common for these temporary bypasses to be forgotten, which can result in serious safety issues. A CMS is used to note these temporary changes and provide a means of easily restoring a prior version of the program once maintenance is complete.
• Multi-process or recipe operations: In facilities that run different processes or recipes it is important to manage which version of a program is being updated. The creation of specialised copies of programs to use as “master versions” for each of these processes can aid in managing these efficiently.
For further information please visit: www.mac-solutions.net
