OVATION USERS GROUP: Users ponder big issues, knuckle down on knotty plant problems

Encompassing 1300 GW of global capacity, 450,000 MW of that in the US, and 120,000 MW of combined-cycle capability, the Ovation control system platform, according to Robert Yeager, president of Emerson Automation Solutions Power & Water, is Number One in global power generation control systems. But Yeager, in the traditional “bragging rights” opening remarks at the Ovation Users Conference, clearly wanted to put his competitors on notice that he is gunning for more.

“We’re going to knock the wind out of our competitors,” he said, referring to the relatively new OCC100 product, designed to compete head-to-head with the traditional programmable logic controller (PLC). The OCC100 also forms the backbone for Ovation-based microgrid control systems.

Features now fully integrated into Ovation, such as embedded simulation (Fig 1), vibration and condition monitoring, and predictive analytics obviate the need for separate packages—and separate vendors. Yeager claimed that the company is working on 35 active embedded simulator projects around the world. “In five years, operators will spend more time in the ‘virtual’ plant residing in the ‘cloud’ than with the real plant controls,” he predicted.

Underscoring the challenge cybersecurity poses to highly connected digital systems, Yeager also noted that the Ovation cybersecurity team has ballooned from three to 50 members in the last few years. “We’re providing cybersecurity services on competitor control systems, too,” he added.

Steve Schilling, VP of technology, and head of the Ovation Technology Team, amplified some of Yeager’s comments. Ovation’s “remote node interface module,” a/k/a Ethernet I/O, is in final testing, he said. It can be used, along with the OCC100, not only for wind and solar facilities, but also for smaller, dedicated control systems for skid-mounted process units—the traditional purview of the PLC.

Two other remarks by Schilling were, well, darn right chilling. First, he noted that “storage is the wild card in the power equation.” This is yet one more indication that grid-scale storage is seriously penetrating the power industry “psyche.” Commercial storage facilities are already lopping off the peak of the peak demand in major markets around the country and thus compete with quick-start gas turbine units.

Second, he said that half of one large utility’s 6000 employees are eligible to retire in five years. That’s more than a brain drain; that’s a potential vortex, at least in the time scales the electricity industry operates under.

Yeager had noted earlier that one-third of the attendees at the confab had never attended an Ovation User group conference before. Hopefully, some of those newcomers represent youthful energy, not veterans who had to take over control system duties because someone retired.

Schilling also put turbine vendors on notice, saying that Ovation now offers a “completely integrated turbine control solution incorporating new synchronizer, machinery health, and excitation modules.” He also addressed cybersecurity, stating that certification under ISA/IEC 62443 is underway.

Modern to mundane

Up next in the general session was Emerson’s Glen Heinl, director of customer services, who addressed mapping your journey with an Ovation system. He said Ovation offers more than 50 formal classes for employee development and advanced skills, and 20 webinars are available without leaving your desk.

One important question he had for the folks on the deck plates: “Are you checking your power supplies?” Because users often add I/O cards and capabilities, the existing power supplies can quickly become overwhelmed. A preventive maintenance guide, addressing power supplies and other critical items for Ovation systems is expected to be released “within six months.”

Speaking of knotty problems, the modern simulator has come to the aid of the age-old mundane problem of inconsistent operator performance at one plant. Personnel there, suffering from the aging workforce challenge, committed to a formal operator training program anchored by the Ovation embedded simulator, replacing a simulator onsite for eight years but not used.

Plant representatives reported the program led to a 44% reduction in startup and shutdown times. The five operator crews, a mix of veterans and new recruits, had been doing things differently. Following training, the crews are consistently following startup and shutdown procedures, performing tasks simultaneously, getting qualified “expeditiously,” and in general achieving more consistent performance.

Lucky sevens

Replacing seven gas-turbine control systems with Ovation in seven weeks could be like holding your breath at the casino, hoping for lucky sevens. What made this project more white-knuckle is that the seven turbines (at two different locations, one a combined cycle facility, the other a simple cycle peaker facility) were acquired from merchant owner/operators. Lacking experienced personnel in this area, the new owners essentially put their full trust in Emerson as system supplier and engineer—that is, not hiring a separate electrical contractor. All electrical demolition was also left to Emerson.

One of the objectives of the replacement at the combined-cycle facility was to automate the manual power augmentation and water injection NOx control subsystem, dubbed SPRINT™ by the turbine vendor. In fact, this was the core of project justification. At the peaking facility, it was to achieve remote operation.

A few of the candid lessons learned include the following:

  • The due-diligence team for the acquisition did not pay much, or any, attention to field instrumentation. Apparently, this is typical of due-diligence teams, and something others seeking to acquire plants should guard against.

  • While generally pleased with overall outcomes, the site representatives did note the Ovation team’s lack of experience with retrofits of machines from this type—for example, the turbine vendor locates I/O at the GT housing, while Emerson prefers to put the boxes in the control room (Fig 2).

  • Resolving disputes around the HMI (human machine interface) graphics and high-performance screen layouts was also a challenge.

  • The new owners discovered that the air permits for the peaking units were written in a way that prevented them from adequately conducting no-load tests. Only two starts per day were allowed if the units were not going to proceed to full load and compliance operation within 30 minutes.

Actuator acting up

Steam-turbine bypass systems and valves for combined cycles have been giving users fits for years. In one Ovation user’s case, it was the actuators on the HP and IP bypass valves. It would do little good to show the list provided of what was wrong with the original actuator design, because essentially nothing was right with it. However, in fairness, it should be noted that this combined cycle, which came online in 2003, was originally designed for baseload service, but later began to cycle.

According to the facility reps, “the valve would go nuts” on a steam-turbine trip, and for good reason: It has to go from closed to 80% open (11.2 in. of valve stem travel), then throttle as if nothing happened, all in two seconds! Doing both well is difficult for a valve this size, they conceded. They also conceded that the actuator worked fine the first few years, suggesting that cycling may have been the root cause of their issues.

The original valve was fine, they said, but the actuator was poorly designed. For one thing, it was “full of O-rings, and other parts and pieces” and three derivative boosters (used with a positioner to increase stroking speed). Also, the rapid stem movement repeatedly broke the weld between the plug and stem.

The plant decided to go with pneumatic actuation because it would avoid oil leaks and fire hazards and the parts would be more readily available than for a hydraulic actuator. Emerson designed a bolt-on actuator (Fig 3), leaving the original valve in place, with far fewer parts and all the boosters of the same component. The new booster is three to four times the size of the original one.

According to these plant reps, the valve and actuator “haven’t been touched since” and the weld cracking issue has been resolved. Tuning is much easier and more consistent, and the control loop is more stable during startups.

 Sabers for cyber

Protecting against cyber-intrusions and keeping the “bad guys” out consumes more and more of the digital control system community’s energy—and piles on costs. In addition, like environmental restrictions, there are multiple layers of compliance, standards, and jurisdictions.

During a cybersecurity panel, a representative from the US government’s Industrial Control System Cybersecurity Emergency Response Team (ICS-CERT, part of DHS) referred the audience to the C-SET, a cybersecurity evaluation tool.

As described in a document available from ICS-CERT online, “it is a desktop software tool that guides asset owners and operators through a step-by step process to evaluate their industrial control system and information technology network security practices.” Available for free, users answer questions and the software generates a report comparing your practices to recognized government and industry standards and recommendations.

In addition to federal standards and recommendations, states are imposing their own, according to a consultant on the panel. New Jersey, for example, has mandated cybersecurity standards. He noted that costs for complying with mandatory requirements could be challenging for power producers in low-price markets. That includes most everyone in today’s world.

An expert from one of the largest combined cycles in the country, built in the early 1990s, explained how his plant went from “25 years of connecting everything to disconnecting from the sins of the past.” His plant had one supervisory control system for the entire plant, because it was designed to deliver all of its 1640 MW of capacity to one buyer as a baseload facility. The plant comprises 12 gas turbine/HRSG trains and three steam turbine/generators and, to complicate matters, two steam hosts and six packaged boilers.

Now they have to comply as a NERC CIPS 6 medium impact facility. “In the early days of compliance, we were patching one workstation every two to three days.” Plus, they are planning to add an 800-MW 2 × 1 combined cycle at the site.

Plant practices reported to the group included these:

  • Create action plans based on vulnerability assessment results.

  • Patch monthly to keep current.

  • Physical and electronic access control is key.

  • Use E-ISAC for threat intelligence.

The electricity information sharing and analysis center (E-ISAC) is operated by NERC but functionally isolated from its enforcement arm. It’s a central repository for physical threats, vulnerabilities, and incidents. According to information available from the program online, the following benefits are described:

  • Identify adversary campaign tactics, techniques, and procedures and share specific mitigation actions.

  • Reverse-engineer malware to better understand events and develop predictive capabilities.

  • Share tactical information to reduce cyber risk for all participants.

  • Cross-benchmark and evaluate with other critical infrastructure sectors. CCJ