2020 CCJ Best of the Best: St. Charles Energy Center

Cooling-tower fan gearbox mod facilitates oil sampling, recharge

Challenge. Oil in cooling-tower fan gearboxes is sampled/tested monthly. When the tower was built, safe and easy access to facilitate sampling was not a priority (Fig 1). Some personnel considered the sampling process dangerous and time-consuming. It involved (1) LOTO of the necessary breaker and circ-water valves for the affected cell, (2) unbolting of part of the fan shroud, (3) installing a walkway to reach the gearbox, (4) locking the fan to prevent its rotation, (5) putting on a fall protection harness, (6) walking to the gearbox, (7) tying off to an anchor point, and, finally, (8) removing the filler plug and drawing an oil sample.

After the oil sample was retrieved, the entire process was reversed to return the tower cell to service. Weather permitting, this evolution would take at least two hours to complete.

Multiplying that by 10 fans means a minimum of 20 man-hours was required monthly to pull oil samples from the cooling tower.

The same process was followed for gearbox oil changes. The time it took to drain and fill each gearbox made this job difficult to plan and perform.

Solution. Develop/implement a safer, more efficient way of obtaining oil samples, and of performing oil changes. Staff decided the best method would be to modify the equipment so sampling and oil changes could be done without entering the tower cell. Piping was connected to the gearbox and routed outside the fan shroud, and hoses and quick couplings were installed on the piping near the fan motor to allow easy access to the sample/fill line.

Lastly, plant purchased a portable oil pump/filter cart to facilitate sampling and oil changes (Fig 2).

Today, all a technician has to do is hook up the oil cart, turn on the pump to circulate the oil, and retrieve the sample from the sample port. During oil changes, the cart allows for quick draining/filling of gearbox oil and provides the option to valve-in filters to remove particulates and moisture from the circulating oil.

Results. Oil sampling and changes can be done in minutes versus the hours previously. Most importantly, these tasks can be performed without putting employee safety at risk: All work is performed on the cooling-tower deck outside the fan shroud. Added benefit: When the piping was being modified, staff ran an additional line from the gearbox vent port outside the fan shroud. A desiccant filter was added to that to decrease the moisture level in the oil and increase its useful life.

Project participant: Frank Katzenberger

Brush maintenance program improves generator reliability

Challenge. Less than a year after commissioning, St. Charles experienced a ring-to-ring fault causing a collector flashover on the steam turbine/generator. After this event, plant staff established daily and weekly criteria for brush and collector inspections and maintenance going forward.

Staff initially adopted a procedure for weekly generator field brush checks involving use of a clamp-style ammeter, plus a visual check for discoloration of shunts, chipped brushes, excessive wear, contamination, hung up brushes, etc. The action of taking current readings by clamping the ammeter around the shunt pigtails of a running machine is a dangerous evolution even with the required arc-flash PPE—especially given the number of brushes per unit.

Solution. Mindful of the safety concerns related to online amp readings, staff incorporated infrared technology into the weekly brush inspections. An IR imager is now used to perform a thermal scan of each brush while the machine is online (Fig 3). The temperature of each brush is recorded on a spreadsheet and used to track performance over time.

Staff can now determine which brushes are carrying the most load and which are not carrying current as they should be based on the running temperatures. Data are evaluated weekly and compared to prior results to establish a priority level for the replacement of each brush and/or holder.

Results. Use of IR technology has minimized the safety risk of placing hands and equipment inside of a running machine along with eliminating the shock and flash risk associated with those actions. Staff are now able to schedule brush changes based upon the temperature profile of the given brush over time and assign priority to those that need immediate attention.

Using this technology along with normal wear-indicator inspections has greatly increased generator reliability, mitigating the detrimental effects of brush selectivity and the potential for flashover. However, the most important benefit is personnel safety.

Project participants: Bill Bates, Jimi James, Frank Katzenberger, Chris Higgs

Job status board helps keep staff informed, safe

Challenge. During planned maintenance outages it can be difficult for staff to track what and where work is being performed. A plant could have hundreds of contractors on site with many different lock-out/tag-out (LOTO) clearances, confined-space permits, and hot- work permits. In addition to permits and clearances it is particularly important for personnel to be informed of major evolutions—such as lifts involving cranes, foreign material exclusion (FME) zones, excavation areas, and the occasional x-ray zones.

Outages can be ever-changing events so keeping personnel up to date on the status of these items is an extremely important safety concern. Staff needed to find an easy, effective way to communicate the status of these items to all personnel.

Solution. Staff created a status board for mounting in the control room that was simple to read and allowed personnel to make changes easily as the status of jobs evolved (Fig 4). Instead of using a standard LOTO station or outage board, plant staff wanted something visually attractive with pictures, and understandable by the least experienced personnel.

The staff commissioned Powergen Publications Inc to create an artistic rendering of the entire facility using actual plant drawings. This drawing contained all equipment onsite—including piping and proper flow paths. It then was imprinted on a 4 × 8-ft metal whiteboard, allowing technicians to write on the drawing with erasable markers as well as to place magnets onto the surface. Individual magnets were custom designed by the facility to represent LOTO’s, confined space, hot work, crane, excavator, x-ray zones, areas of isolated power, and emergency power locations.

The LOTO boxes used by the facility to hold the keys associated with clearance isolation locks were individually labeled to match the LOTO magnets on the outage board (Fig 5). With the LOTO magnets matching the LOTO cabinets, personnel can easily see where each LOTO is hung and can sign on to the corresponding paperwork. The magnets are easily removed or added as the job status changes.

Results. The outage board has been an huge success and become a focal point during the outage. Before an outage even starts the board is used to help in the planning where LOTOs must be hung for the jobs being performed. During the first stage of the outage, the board is used by the staff to see what LOTOs have been hung, allowing them to plan work and decide where the next LOTOs would be needed.

During the outage, the board is used to identify where all work is being performed in the facility. Near the end of the outage, the board helps staff quickly determine what equipment has been cleared and what is still under a LOTO—thereby allowing for a better and more efficient plant restoration process.

Besides being used for outage work, the board is a primary tool for training technicians— because it is a huge one-line drawing of the entire plant. Staff also has used the board during plant upsets to direct personnel response and identify equipment to check.

Project participants: Nick Bohl, Jonathan Bennett

Vision Screen promotes operational excellence

Challenge. With staff changes following the assignment of a new O&M company to the facility, familiarity with the plant and its nuances raised some challenges for the fresh faces—for example, frequent connection loss with the gas-turbine flame scanners, irregular fuel-gas or plant-air pressure performance, and chemical tanks not being replenished in a timely manner.

Accessing and readily monitoring these factors while learning a new facility and trying to troubleshoot operational challenges led to an unnecessarily convoluted shift. Adding additional screens to the control room seemed to treat the symptom and not the problem.

Solution. The historian PI server captures all the parameters of interest so the complimentary software tool, PI Vision, was used to build the platform that would later be named SCEC Vision (Fig 6). A large screen display was ceiling-mounted and outfitted with a 4.5-in. Intel mini-computer specifically optimized for 4K UHD resolution with a high refresh rate. Internal cyber controls and NERC guidelines were followed when implementing this concept, segregating the device from the controls network.

Screen layout required significant effort. PI Vision inherently has a very limited design toolset, so it was up to the staff to create custom images and data equations. Every icon, animation, and formula has been designed in-house. Once the initial layout was completed, a survey was taken of all data points considered important to plant health so they could be implemented.

The result was a simple but aesthetically pleasing display anchored as the centerpiece of the control room (Fig 7). The historian PI server now feeds data, in real time, to this centralized display, complete with visual cues to keep operators alerted of detrimental change.

Results. Since implementation of SCEC Vision, there has been a dramatic improvement in voltage-schedule adherence. Plus, chemical-tank provisions are ordered without delay, and flame-scanner faults are detected quickly and corrected. The fuel-gas and plant-air pressures now are tweaked to near perfection. A quick look back at the previous eight hours trends the plant’s generation and drum level controls to scan for unusual operation.

There also have been unforeseen benefits. Whether it is during the morning staff meeting, or simply passing through the control room, SCEC Vision had become a conversation piece that helps inspire new ideas and constructive criticism. As the staff overcame many of the above- mentioned obstacles, the mindset moved to improving efficiency and reliability. SCEC Vision became so valuable that screens were placed in the offices of the plant manager and plant engineer for monitoring purposes.

The importance of plant efficiency led to operator training and heat-rate monitoring on SCEC Vision. Condenser vacuum, fuel-gas metering, and vivid palettes featuring the CEMS and HRSG duct burner behavior are all contributions from operator feedback.

There was virtually no cost associated with the development and implementation of this tool— other than a $150 annual licensing fee. Use of SCEC Vision continues to grow to serve the needs of the plant and its personnel.

Project participants: Jonathan Bennett

Keep up with NERC requirements on visitor tracking, control of transient cyber assets

Challenge. With NERC CIP-003-7 introducing additional requirements for cybersecurity and controls, one particularly polarizing topic had become visitor tracking and management. With St. Charles located just outside Washington DC, the plant often hosts guests from government agencies and nearby engineering colleges. The industry-standard visitor sign-in sheet did not meet the latest regulatory standards.

Revision of the NERC standard introduced strict governing of transient cyber assets (TCA) and removable media. Whether it be an OEM sales consultant, electrical relay technician, or GE technical advisor, all too common are devices such as laptops and USB thumb drives allowed to connect freely to what should be controlled digital environments.

Rarely were these devices scanned for malicious code that would contain malware designed to execute immediately upon contact. All of this further highlighted the necessity for more intuitive, and modern access monitoring and control tools.

Solution. Staff decided a digital kiosk would allow the facility to eliminate most of the shortcomings of a pen, paper, and clipboard setup. This would require an easily accessible interface with security against physical theft.

The kiosk design selected was a slim Android tablet and complimentary stand, complete with the security option of a lock and key. Unnecessary applications were stripped upon installation, user access was limited, and an “Always Awake” application was installed with enhanced brightness. LobbyTrack ended up being the visitor management software of choice, configured with a “self-sign-in” lock code to prevent misuse. A placard was affixed above the station to guide visitors to sign in.

Another digital kiosk was implemented to manage the TCA and removable-media concerns. A Windows-based device was chosen this time with security again being the focal point for the choice of stand. The intention of this station is to allow all outside devices and media to be scanned within a sandbox environment where a report can be generated and logged without risk of malicious code—that is, malware or ransomware—compromising the plant network.

The control room houses the HMIs for the plant’s DCS and has a connecting door to the business servers. Previously, guests would sign in at the lobby and have unchecked access to the control room and, arguably, the server room, if not for the CRO monitoring them. Now, biometric smart locks have replaced the original door latches for both control-room and server- room access.

All personnel have their fingerprints coded. Different security levels are assigned to staff based upon specific need—that is, very limited access is granted to the server room to reduce unnecessary foot traffic and threat potential opportunity. Should the fingerprint have trouble for whatever reason the faceplate of the smart lock allows a numerical code to be entered. The digits on the faceplate strategically rotate with every press to eliminate any third party recording the code.

Results. Plant management received immediate positive feedback when the digital sign-in kiosk went live. New visitors simply enter their name and company on the touch screen, select their host, and the rest is done for them—including notifying the host that their guest has arrived.

Visitors can be scheduled, to reduce sign-in time to near-instant, and returning visitors can simply type in a few characters from their name to be presented with a suggestion box for a quick autocompletion. This has dramatically improved visitor accuracy and participation. There is also a web-based control panel where monthly reports can be generated, which ended up providing an unforeseen benefit in our accounting department with monthly accruals.

Part of the initial concern in implementing the security kiosk for scanning TCA and removable media was that the process might be expensive and cumbersome. Rather than use a full workstation plant opted for a two-in-one tablet which reduced the cost dramatically while still being able to manage Windows media quickly and efficiently. The anti-viral/malware software has been configured to scan connected devices immediately, place threats found in a secure vault, and generate a report without any prompts or interaction. The additional security measure has been painless, generally requiring less than 60 seconds for a complete scan.

Upon first installation of the biometric smart locks it took less than an hour in total to add and configure each onsite employee with his/her own unique fingerprint key. The fingerprint scanner has been quick, reliable, requiring zero maintenance, and alleviating the need of metal keys or plastic keycards. These biometric smart locks are also configured with their own independent roster. Universal access is not allowed, everything is on an as-needed basis.

Contractors also can be granted a temporary access PIN, prior to arrival, with an expiration time and/or date. This allows unimpeded workflow with a trusted vendor. The lock also has short-range Bluetooth communication providing a full-access log complete with timestamp for incident recall. There was a period of culture shock with repeat contractors, but the message is loud and clear that the plant is taking security very seriously all while complying with the updated NERC CIP-003-7 standard.

Project participants: Jonathan Bennett, Kelly Swann, Javier Gomez