Generator condition monitor critical to avoiding catastrophic loss

Generator monitoring capability historically has been limited, with many common failure mechanisms monitored imperfectly or not at all. Examples of inadequately monitored thermal failure mechanisms on generator stators include stator core lamination insulation failure (Figs 1, 2), cracks developing in stator conductors (Fig 3), and loss of cooling (Fig 4). On field, unmonitored failures that generate high temperatures include cracking of field conductors (Fig 5), shorts (Fig 6), and grounds developing because of field coil/turn distortion (Fig 7).

Limited monitoring capability has been troubling to both generator manufacturers and powerplant O&M personnel. Failure caused by overheating of the stator core iron is a particular concern. While this type of generator failure is not common, when a core fails it usually causes extensive damage, with associated expensive repairs and outage time.

The Generator Condition Monitor (GCM) discussed below is a thermal monitor developed to address overheating concerns. Today it can detect the very early stages of localized overheating associated with each of the three failure mechanisms illustrated in Figs 5-7 (see below).

However, development of this instrument followed a tortuous path because the technology employed is so sensitive and sophisticated that early versions of the instrument were subject to alarms from causes not related to component overheating. It took several years of intensive effort from several organizations to refine the design to a point where the equipment could be considered reliable.

How the GCM works

Operation of the GCM is based on the fact that thermal decomposition of organic materials—such as epoxy and polyester resins, enamel paint and core laminate enamel—produces large quantities of submicron particulates (pyrolysate products). Their size ranges from 0.001 to 0.01 microns. Under normal operating conditions, there are no particulates of this size in the cooling gas; they are produced only by the decomposition of organic materials.

Key components of the thermal monitor are an ion chamber detector, automatic sampling system, and electronics package. The ion chamber contains a weak alpha source. It produces negative ions which are carried into a collector assembly via piping across the generator ventilation fan.

The negative ions are attracted to a positive electrode in the ion chamber detector, producing a very small current flow. The current is amplified electronically to produce an output that typically is set at 80% of full scale on a monitoring device.

If there is no overheating within the generator, the output remains stable at the 80% base. But if overheating occurs, thermal decomposition will produce a large quantity of submicron particles which will be transported to the GCM via the generator’s cooling gas system.

When the submicron particles enter the ion chamber detector, the negative ions attach themselves to these particles. Because the submicron particles are relatively heavy compared to the negative ions, the attached pair will likely flow past the positive electrode of the collector assembly, thereby decreasing the current and lowering the monitor output below the present 50% alarm level.

Overheating is verified by insertion of a confirmation filter at the inlet to the ion chamber. The confirmation filter will remove the submicron particles, allowing the ion-chamber-detector current to increase and cause the output to return to 80%—thereby confirming the presence of thermally generated submicron particles.

For the monitor to function reliably, the gas flow rate must remain constant and the ion chamber detector and electronics must be operating properly. The generator cooling gas fan essentially assures constant gas flow—unless a restriction occurs in the piping or rotor speed changes.

GCM evolution

In the mid 1960s, GE engineers designed an Ion Chamber Detector that could be used for the detection of particulates produced by the thermal decomposition of organic materials (Refs 1, 2). This device was later incorporated into a prototype instrument, the Core Monitor (CM), designed specifically for the early detection of local overheating in large hydrogen-cooled generators (Refs 3, 4). Several of these instruments were installed on generators at sites throughout the US.

Shortly after installation, a CM installed on a large generator indicated overheating was taking place. Because this was a prototype instrument, yet to be field-proven, a decision was made to disregard the alarm. Several days later the generator failed in service. Inspection revealed that severe core overheating had indeed occurred and that the monitor had provided an early warning.

This incident prompted GE to offer CMs as part of the supervisory equipment made available for turbine/generators. The OEM also signed a licensing agreement with Environment One Corp, Niskayuna, NY, allowing it to manufacture and sell the GCM.

This is the fourth article in a continuing series on generator monitoring, inspection, diagnostics, and root-cause failure analysis developed by Clyde V Maughan, president of Schenectady-based Maughan Generator Consultants, for the [BF, color] CCJ. [LF, black] The fifth article is in the queue and will be published in the 2011 Outage Handbook (May release).

The articles listed below, available at www.ccj-online.com/maughan/, are a valuable resource for owner/operators of turbine-driven generators:

• Maintaining carbon-brush collectors, 1Q/2010.
• Options for monitoring generator condition and their limitations, 2Q/2010.
• Input from monitoring, inspections, tests critical for maintenance planning, 3Q/2010.
• Generator condition monitor critical to avoiding catastrophic loss, 1Q/2010.
• Root-cause diagnostics of generator service failures, 2011 Outage Handbook

Design improvements

The early monitors included a test particle source and a rotometer-type flow gage (Fig 8), in addition to the ion chamber detector, confirmation filter/solenoid assembly, and system electronics. In most installations, a signal alarm contact was connected to an annunciator in the control room.

When a signal was received, operators had to go to the physical location of the monitor—usually under the belly of the generator—and try to assess what caused the output reading to drop.

To eliminate the need for operators having to go to the monitor, a remote panel was developed that could be located in the control room. It provided operators with a recording of the output, operation of the confirmation filter, and operation of the test particle source used to test detector operation.

At the same time that the remote panel was developed, Westinghouse Electric Corp designed an automatic sampling system, which was incorporated into the monitor. It provided a means for automatically collecting a sample of the thermally decomposed products when an alarm condition occurred. The sample then could be analyzed by a laboratory to confirm the validity of the alarm and, in some cases, determine the material that was overheating.

On generators where the hydrogen was not clean and dry, contaminants such as oil, rust, and scale could foul the piping and flow meter. When this occurred, hydrogen flow could be restricted, causing monitor output to drop. The resulting decrease in output signal caused an invalid alarm.

The flow-meter contamination problem was addressed by replacing that instrument with a differential-pressure gage. With this design change, the monitor output became much more stable, although fouling of the piping continued to be a concern.

The remaining issue with the early monitors, and the one of greatest concern, was determining when an alarm was valid. The next series of design improvements addressed this issue, and represented a major advancement in the level of reliability and confidence with this technology.

Specifically, an auto-alarm remote was designed to automatically and accurately verify when alarms were valid. It also indicated when (1) there was a fault with the monitor itself, (2) the supply power was interrupted, and (3) gas flow was too low. These changes, completed by the end of the 1980s, enabled the more reliable GCM (Ref 5, Fig 9).

The post-1989 GCM has had two additional improvements to enhance its reliability and performance. They are:

• The differential-pressure gage has been replaced by a differential-pressure transmitter. This improvement allows tracking of hydrogen flow remotely to determine the influence of flow on the output signal. Today’s monitors also are designed for use in hazardous areas.
• The use of tagging compounds to help identify the location of overheating. Six different chemical compounds are mixed with the generator insulating point. The formulations typically are applied (Fig 10) to the turbine end windings, collector end windings, core ID, rotor OD, bushings and lower leads and transformers and reactors (on GE Generex units).

Alarm conditions

There are four conditions that can cause monitor output to drop. They are:

• Decrease in gas flow.
• Faulty ion chamber detector.
• Faulty system electronics.
• Actual overheating within the generator.

The first three conditions will provide an invalid alarm, but misinterpretation of these erroneous signals essentially has been eliminated by the modifications described above. The fourth condition, actual overheating, is responsive to high temperatures associated with component failure anywhere within the generator. Typical failure modes that generate an alarm are breakdown of core iron lamination insulation, cracking of stator winding electrical connections, and winding shorts/grounds or broken turns in the field.

Another overheating mechanism which may cause an alarm is high local temperatures on the core clamping flanges, or on the copper flux shields used at the ends of the core by some generator manufacturers. As load conditions move toward leading power factor—starting at around 0.98 lag—the components at the ends of the core begin to generate additional eddy-current losses and may become hot (Fig 11).

In general, this condition is not harmful to the generator. But paint or contaminates, such as oil, on these components may overheat and form submicron particles that can alarm the monitor. Since alarms from this operating condition will only occur at higher power-factor loading, it is possible to screen out this type of false-positive alarm.

Operating experience

Early versions of the core monitor were relatively simple, with little signal-verification capability. These devices were viewed as somewhat of a glorified smoke detector, although recognized as much more sensitive and complex. If a valid signal were received, the device was capable of providing the expected alarm, and since the potential benefits were so great, more than 1000 were installed.

However, numerous drops in the output signals occurred as a result of invalid alarms from failures within the monitor, as well as from contaminants—such as normal oil vapor—and changes in gas flow through the monitor (Ref 6).

Oil contamination was plentiful, thus assuring numerous invalid alarms. The high costs associated with diagnosing an alarm quickly caused the monitor to loose credibility. Routine maintenance and operator also were neglected. Result: Early monitors were of little value to the plant operators and they often disregarded alarms.

Although the monitor had developed a dubious reputation, the industry’s need for such a device motivated manufacturers to continue development work aimed at finding a reliable design. During the 25 years of this evolution, reliability gradually improved—as described earlier.

EPRI assisted in the development work by conducting an industry survey directed specifically at monitors produced after 1990. Immediate responses were received from 100 utilities, 29 of which reported having a total of 91 monitors in service. Here’s a summary of the information gleaned from the survey:

• Quality of monitor maintenance—excellent, 6; adequate, 21; poor, 0.
• Quality of operator training—excellent, 2; adequate, 18; poor, 9.
• Experiencing of invalid alarms—yes, 2; no, 19. One utility reporting invalid alarms stated, “Some false alarms on all of our units, new and old.” The other mentioned that the affected generator was 25 years old.

Overall, it appears that maintenance is adequate but training is not. The latter is particularly important because of the infrequency with which operating personnel experience alarms and because of the fundamental relationship between training and correct operator response to an alarm.

Survey respondents reported three additional incidents: two stated difficulties with gas-flow calibration because of high differential pressure across a multistage blower. The third said no alarm was received during a major rotor winding failure because of a clogged sensing line.

Three previously unreported “saves” were included in the comments received, specifically: (1) arcing caused by rotor winding failure, (2) burning from rotor shorted turns, and (3) alarm initiated by a break in an end winding connection.

Several additional comments provided by the respondents that may help guide your plant’s monitoring efforts are these:

• Oil filters and traps are a must in sensor lines.
• Occasionally, non-critical alarms are caused by supply-voltage issues during plant startup.
• Difficulty in obtaining adequate gas flow through the monitor.

Operator training

Recommended steps for responding to an alarm are described in the manual provided with the monitor. The individual checks are not complicated, but alarms are likely to occur so infrequently that a control-room operator may not recall appropriate action to take when an alarm is received.

One solution: Configure the DCS to display a checklist of appropriated actions in either a textual or flow-chart format to help the operator make a timely and logical response to the alarm. The monitor manual probably includes the information needed to implement this solution. If not, contact the instrument manufacturers.

Maintenance

Reliable GCM operation requires relatively simple periodic checks which are detailed in the installation and operations manual. The basics:

Daily. Observe gas flow rate and adjust if necessary. Verify that monitor output is at the 80% set point; recalibrate if necessary.

Weekly. Push the “Verification Filter Button” to confirm proper operation of the filter system.

Monthly. Perform relay, contact, keypad, output, and power tests.

Yearly. Check for hydrogen leaks from tubing, fittings, joints, and valve packing. Activate the “Auto Sampling System” to confirm proper operation.

Alarm successes

There are 20 incidents known to manufacturers of monitors where a valid alarm has resulted from the overheating of a generator component. Thumbnails of several incidents are provided below based on available information, which in some cases is limited. Root causes are many, including spontaneous core lamination insulation failure, field insulation failures, loss of cooling media, baffle rub, etc.

Rough estimates of repairs and repair costs avoided are included. Accuracy of the individual repair estimates is not high, but the estimates are presented to give a sense of the large potential saving associated with a properly functioning monitor. The combined potential saving from these 20 incidents is estimated at about $150 million.

Blockage of liquid hose. An alarm gave indications of slowly developing overheating. The unit was tripped and the inspection team found a stator-winding Teflon cooling-water hose was becoming blocked, causing overheating of two stator bars. Had failure occurred, it is likely that a double ground fault would have resulted in massive arcing and burning.

Worst-case repairs avoided: Full stator rewind, probable field rewind, possible core restacking, and extensive cleaning of the core, frame, and coolers.

Estimated repair cost avoided: $15 million.

Arcing of rotor winding. Several incidents of alarms resulting in shutdowns were attributed to winding faults.

Repairs avoided in each case: Possible arc damage to forgings with major impact on overall repairs.

Estimated repair cost avoided: Uncertain, but possibly significant (refer to Figs 5-7).

Phase connection failure. Alarm at full load automatically verified; reduced load by about 20% and alarm cleared; returned to full load and alarm returned as well. Inspection revealed burning of a failing phase connection. Major stator-winding failure prevented.

Repairs avoided: Full stator rewind, field rewind, extensive machine cleaning.

Estimated repair cost avoided: $10 million.

Intermittent alarm on high loads. Poor contact between several tube-to-copper connection resistors on gas-cooled stator bars caused a burn hole in a cooling duct.

Repairs avoided: Probably minor, but possibly stator winding failure.

Estimated repair cost avoided: $10 million.

Stator-winding water flow blockage. GCM alarmed on load increase. Inspection revealed blocked water flow, thereby preventing winding failure.

Repairs avoided: Probable stator rewind.

Estimated repair cost avoided: $10 million.

Stator core failure. GCM was in constant alarm for 30 minutes prior to stator-core meltdown. Intermittent alarms were occurring during the previous several months.

Potential for avoided repairs: Restack of core and stator rewind.

Estimated repair cost: Uncertain. If core repair had been possible prior to melt down, cost saving would have exceeded $15 million.

Operation without cooling water. GCM alarmed at 150 MW on a 650-MW generator while the unit was being ramped to full load. Alarm was disregarded temporarily, but unit was tripped manually at 250 MW. Inspection revealed that the hydrogen coolers were inoperative.

Repairs avoided: Possibly extensive damage to the generator.

Estimated repair cost avoided: Uncertain, but potentially very high.

Generator field ground. GCM alarmed about 40 minutes before manual trip of the generator. Inspection and test revealed that several rotor coils had elongated and caused multiple grounds to the retaining ring. The associated arcing had caused the alarm.

Repairs avoided: Possible fracture of a retaining ring with total destruction of the generator.

Estimated repair cost avoided: Tens of millions of dollars, plus exposure of plant personnel to severe injury.

OEM test protection. Use of GCM as protection against over-temperature during acceptance test detected a stator cooling-water blockage early and prevented serious damage. Prior tests without a GCM had resulted in over-temperature that destroyed an entire field.

Repairs avoided: Severe overheating damage to a field.

Estimated repair cost avoided: $3 million.

Core burning. GCM alarmed because of a 2-in. core-lamination hot spot which developed during normal operation.

Repairs avoided: Possibly eventual core meltdown.

Estimated repair cost avoided: Uncertain, but potentially extremely high.

Water cooler valved out. GCM detected overheating in a 550-MW generator while it was ramping up to full load following a generator rewind. The GCM was the first indicator that one of the water coolers was valved out.

Repairs avoided: Possibly overheating of the generator.

Estimated repair cost avoided: Uncertain, but potentially very high.

Baffle rub. During restart after stator replacement, the GCM alarmed. Inspection revealed a rub between the field and a gas flow restriction baffle. The GCM correctly identified the particulate matter as a valid alarm condition.

Repairs avoided: Minor.

Estimated repair cost avoided: Minor.

Breaking of stator winding connection. GCM alarmed on fracturing of a connection, allowing unit shut down without experiencing the extensive contamination normally associated with such a failure.

Repairs avoided: Severe contamination of the generator, possible stator rewind.

Estimated repair cost avoided: $5 million.

Field turn shorts. Existence of turn shorts was confirmed by monitor alarm, allowing a shutdown without collateral damage.

Repairs avoided: Probably small impact in overall repairs required.

Estimated repair cost avoided: Probably minor.

Failure to respond

There have been several reported incidents where after failure, review of plant records revealed that the GCM alarmed properly, but immediate corrective action was not taken.

Three of these incidents are presented below:

• Core fault. At a UK plant, a GCM alarm was given 30 minutes before the generator failed in service. Damage resulted from severe overheating caused by the failure to valve back in hydrogen coolers after a generator rewind.
• Field-turn short. A few months after a large generator field had been rewound, the GCM alarmed for 5 to 10 minutes and then cleared. Shortly thereafter high vibration required removal of the generator from service. Two turns in a small coil had shorted, thereby causing the pyrolysate particles which initiated the alarm, followed by bowing of the field, which produced the vibration (Ref 7).
• Core meltdown. GCM alarmed and less than an hour later the unit tripped on ground relay. Massive core meltdown had occurred, requiring a full core restack, full field rewind, and extensive cleaning of the frame and coolers. Cost exceeded $25 million.

Wrap-up

The GCM is an exceptionally sensitive device. Many years of development and design evolution were required before achieving a state of high reliability against incorrect alarms. That point appears to have been reached in about 1989.

The device monitors the generator for several adverse conditions that can initiate a valid alarm. These range from the benign to core meltdown—a spectrum from minor rub to complete generator internal destruction. Unfortunately, there normally will be at least some uncertainty as to the exact source and urgency of the condition initiating the GCM alarm. But since the costs associated with non-response can be exceedingly high, it is prudent to regard an alarm as valid unless known information confirms otherwise.

Also, because of the inherent nature of the failure modes being monitored, expeditious response may be very important. Thus it is advisable to configure the plant DCS to assist the operator in making a timely and correct response to an alarm. A properly maintained and operated GCM can mean the difference between a brief shutdown for minor repairs or a major overhaul involving weeks or months of costly downtime. CCJ

References

1 G F Skala, “The ion chamber detector as a monitor of thermally produced particulates,” Sixth International Conference on Condensation Nuclei, May 1966, Albany, NY

2 L P Grobel and C C Carson, “Overheating detector for gas-cooled electrical machines,” US Patent 342 7880, 1969

3 C C Carson, S C Barton, and L P Grobel, “Immediate detection of overheating in gas-cooled electrical machines,” IEEE Winter Power Meeting, January 1971, New York

4 S C Barton, C C Carson, R S Gill, W V Ligan, and J L Webb, “Implementation of pyrolysate analysis of materials employing tagging compounds to locate an overheated area in a generator,” IEEE Power Engineering Society Meeting, July 1981

5 G F Skala, “The new generation condition monitor and its application to the protection of turbine generators,” Proceedings of the First International Power Conference, 1986, Beijing

6 D J Wallis and S Kilmartin, “Can you believe your GCM?” Eighth EPRI Steam Turbine-Generator Annual Workshop and Vendor Exposition, August 2003, Nashville

7 David J Albright, “Rotor winding shorted turn detection with a generator condition monitor alarm,” ASME Power, July 2008, Orlando