Grid Disruptions & Extreme Weather
Response to factual errors in the the blogosphere about "Global Climate Change Impacts in the US"
Last updated October 1, 2011
Before delving into the details here, it is worth noting for newcomers that the "Global Climate Change Impacts in the U.S." report encapsulates the findings of 21 peer-reviewed Synthesis and Assessment Product reports, produced by 13 USGCRP participating Agencies, together with other vetted sources. This report alone underwent a remarkable degree of public, governmental, and "Blue Ribbon" review. All in all, about 570 pages of comments (three times the length of the report itself) were received and publicly responded to. This, together with dozens of pages of spirited and constructive internal review comments within the writing team, represents a forum for quality assurance that is a more collegial and credible place to do science than the drive-by blogosphere.
Those more interested in what the original report actually says than planting red herrings, cherrypicking, creative reframing, and knocking down strawmen (and the 'waterboarding' of scientists who work in this domain) will certainly benefit from reading the report itself. The following discussion pertains to second- and third-hand misinterpretations of one particular figure that captures so many people's imagination.
Patrick Michaels' Blog in Forbes
This brief, non-peer-reviewed blog post [September 19, 2011] floats a fanciful vision that EPA will back away from regulating greenhouse-gas emissions, and, presumably the Supreme Court will retract their underpinning decision Massachusetts v. EPA deeming carbon-dioxide a pollutant under the Clean Air Act. To bolster this vision, presumably on the basis that climate change does not and will not in the future "endanger human health and welfare", the blog awkwardly picks on a figure showing short-term power outages out of the energy chapter of the 188-page report. The figure has relatively little to do with the Supreme Court's concerns, undercutting Michaels' own thesis. Oddly, no criticism is leveled (at least here) at the report's far-more-relevant chapters on human health, water, agriculture, or society, as documented and discussed widely, even in Forbes.
Let's review Michaels' substantive criticisms on this particular figure:
If the figure were true, there were apparently NO power outages caused by temperature extremes, wildfires, winter weather, thunderstorms and lightning in 1992.; The fact is that both then and now, even a modestly severe thunderstorm knocks out power in Washington, where most of the authors spend a lot of time.
As is evident in the figure, its caption (masked by Michaels), and the source material, these data represent only large events, and only those on the wholesale power grid (neither the distribution network, nor individual power plants are included — see full description at bottom of this page). Thus, one cannot expect each and every event to appear. In any case, the mapping of weather-related events to specific subcategories is clearly difficult in some cases since the primary sources do not use highly standardized language in defining the triggers. That said, while one may map some of the weather-related events to different weather categories, it would not change the total count of events or the meaning of the figure in question. Here are the reported 1992 events:
Severe weather - 875,000 customers impacted
Storms - 100,000 customers impacted
Hurricane Andrew - 1,500,000 customers impacted
Hurricane Andrew - 650,000 customers impacted [same storm, different area]
Storms - 58,000 customers impacted
Where there is more fundamental ambiguity as to the event trigger, the events are mapped into the non-weather-related category. In 1992, for example, two events classified as "fire-tripped lines" were not included because the type and cause of fire was not reported and so it was not clear if they were, e.g., wildland fires or industrial fires. This convention is applied across the entire time series. [The source data for 2000-2008 are here; prior years are no longer posted by NERC but available upon request. NERC's data include a few events from Canada, without indication of which impacted customers were on the US side of the grid inter-ties. Were the Canadian values subtracted from these early years, or added to later years, the curve would slope even more steeply upward, presumably to the chagrin of critics.
Michaels goes on to state:
A "tenfold" increase in weather-caused outages is preposterous. There's been no change in strong tornado frequency, tropical cyclone activity is near historic lows, and any observed changes in mean or extreme winds are nugatory.
Straw-man framing of data can clearly be proposterous, but data themselves are agnostic. Shooting the messenger won't make data go away. In any event, not listed in the rebuttal are temperature and precipitation extremes, both of which have been steadily increasing in frequency. There is no serious dispute of this in the peer-reviewed literature. NOAA's Climate Extremes Index provides more information. That non-weather-related events are not rising like weather-related ones escapes Michaels' attention.
Note that the term "tenfold" is used for context. One could easily have cherry-picked different years to get thirty-fold or seven-fold, or something in between. One could summarily dismiss the values before EIA took over and get ranges of three- to six-fold. But, over this range, the exact factor is a distraction from the point of the figure and accompanying discussion.
As in the flurry of misinterpretation of this chart that took place two years ago, Michaels masked the detailed caption on the figure, suppressing the caveats and cautions therein and in the main body of the report. Among other things, our discussion of increased vulnerability is also ignored as a likely factor in the trend.
Climate Skeptic Blog
Posted June 18, 2009 with subsequent updates: Two non-peer-reviewed blog posts [1-online - June 17, 2009 | PDF] [2-online - June 18, 2009 | PDF] mischaracterize analysis in a new report entitled Global Climate Change Impacts in the United States. The blogger (a self-admitted "amateur") created a straw man argument by asserting that the chart was presented as evidence of global climate change and was not verified with the primary source. The blog's errors have been nonchalantly propagated to other web sites without further fact checking or due diligence. (The use of profanity in the title of the first entry is additionally unprofessional.)
The underlying database contains approximately 930 grid-disruption events occurring between 1992 and 2008, affecting 135 million electric customers.
In the caption to the figure on page 58 of our report (shown above)—which was masked in the blogger's critique—we expressly state a quite different finding than that imputed by the blogger, noting with care that we do not attribute these events to anthropogenic climate change, but do consider the grid vulnerable to extreme weather today and increasingly so as climate change progresses, i.e.:
"Although the figure does not demonstrate a cause-effect relationship between climate change and grid disruption, it does suggest that weather and climate extremes often have important effects on grid disruptions."
The associated text in the report states the following, citing a major peer-reviewed federal study on the energy sector's vulnerability to climate change:
"The electricity grid is also vulnerable to climate change effects, from temperature changes to severe weather events."
There were many reasons to use this figure in the report, including:
To make obscure but important existing government-collected data more accessible to an interested audience
To quantify the weather-sensitivity of the nation's electric grid
To establish a baseline for the discussion of future climate change impacts
To characterize the severity of current weather- versus non-weather-related events
To conduct an initial assessment of differences in the observed trends of weather- versus non-weather-related events, and compell deeper analysis
As with virtually any climate impact analysis imaginable, there are non-climatic influences that can offset or compound the outcomes. We emphasize these factors throughout the report. In the case of grid disturbances, an ageing grid and associated infrastructure, changes in maintenance and management procedures, and rising electrical demand all create stresses that make the grid more susceptible to outages triggered by even unchanging extreme weather (see work by Dr. Amin). Our laboratory leads a major research activity in these areas known as the Consortium for Electric Reliability Technology Solutions.
Contrary to the blogger's inference, the potential for under-sampling or sampling bias was in fact identified early-on within the author team and—contrary to the blogger's accusation—contact was in fact made with the person responsible for the data collection project at the US Energy Information Administration on June 10, 2008 (and with the same individual the blogger claims to have spoken to). At that time the material was discussed for an hour with the EIA official, who affirmed the relative growth was in weather-related events and that it could not be construed as an artifact of data collection changes, etc. That, and other points in this response, were re-affirmed through a follow up discussion in June 2009. Not surprisingly, the individual was displeased with how the Climate Skeptic Blog mis-represented his statements.
In fact, the analysis understates the scale of weather-related events in at least four ways:
EIA noted that there are probably a higher proportion of weather events missing from their time series than non-weather ones (due to minimum threshold impacts required for inclusion, and under-reporting in thunderstorm-prone regions of the heartland).
There was at least one change in EIA's methodology that would have over-stated the growth in non-weather events, i.e., they added cyber attacks and islanding in 2001, which are both "non-weather-related".
Many of the events are described in ways that could be weather-related (e.g. "transmission interruption") but not enough information is provided. We code such events as non-weather-related.
As noted above, a limited number of Canadian events are embedded in the early NERC data, but not in the later EIA data.
Thanks to the efforts of EIA, the data-collection process became more effective after they took over the responsibility from NERC around 1997. Efforts were made in subsequent years to increase the response rate and upgrade the reporting form. It was not until the years following EIA's improvements that the important decoupling of weather- and non-weather-related events (and a corresponding increase in the proportion of all events due to weather extremes) became visible. There was no indication of a trend in the early years when NERC gathered the data using less intensive methods. We thus regard EIA's efforts as very positive, helping to reveal an important signal that merits further analysis.
To adjust for potential response-rate biases, we have separated weather- and non-weather-related trends into indices and found a far stronger upward trend in the weather-related time series.
As confirmed by EIA, if there were a systematic bias one would expect it to be reflected in both data series (especially since any given reporting site would report both types of events).
As an additional precaution, we focused on trends in the number of events (rather than customers affected) to avoid fortuitous differences caused by the population density where events fortuitously occur. With this method, the absence of a given event (however larger or small) introduces much less bias or variance in the results. This, however, also has the effect of understating weather impacts because of EIA definitions (see survey methodology notes below). For those interested in numbers of individuals affected, the figure caption notes a decided upward trend.
The blogger also speculated that many of the "extreme temperature" events were during cold periods, stating:
"if this is proof of global warming, why is the damage from cold and ice increasing as fast as other severe weather causes?"
The statement is spurious. A closer analysis of the data on events triggered by temperature extremes (see figure below) shows that 84% of such events occurred during the period May through September (with none in Feb, Mar, Oct, Nov, or Dec.). If the questioner is referring to snow and ice storms, they need to do their homework on the effects of increased precipitation (during winter as well as during summer).
Lastly, an identical analysis was previously included in a 2008 peer-reviewed publication (see p. 9) about climate change in the state of California.
Notes on EIA's Methodology
As noted in our report, the data series—although rich—does not include all events. The power disruption datasets represent a summary of the pre-selected critical events impacting electrical operations that the Federal Government wants the electric power industry to provide through real-time notices an alert about potential and/or on-going actual events. (After 48-hours, an after action summary report is required.) This is not a 100% census of all the events that occur on the bulk power transmission system; only those that indicate emergency operations (major) practices are being considered or are on-going. There is a reporting requirement of needing 50,000 customers to be without service for events directly linked to major weather events, while events affecting no customers but caused by human error are still logged. So, basic distribution outage information from smaller weather-related events is below the threshold of the DOE reports. The response rates have grown over time and these reporting practices for weather- and non-weather-related events, which improved the precision of our analysis. There are no caveats or warnings on the EIA website about sampling bias or discouraging longitudinal analysis of the data.
Evan Mills
Lawrence Berkeley National Laboratory