The State of our (energy) Storage

Storage-Based Generation

Last week, on October 10th, the EIA released its finalized 2012 utility/generator dataset. In their words, this release covers:

generator-level specific information about existing and planned generators and associated environmental equipment at electric power plants with 1 megawatt or greater of combined nameplate capacity.

making it a rich resource for information on the current state of America's power generation systems. For this post, I am going to focus on storage technologies: their place in our energy system, where they are now, and their future. There are a lot of fun little metrics to work through, although some explanation first is useful.

In reporting generator data, the EIA recognizes 23 "prime movers" and 42 "energy sources." Prime movers are the method of generation, and energy sources are the feedstock or fuel for each generation process.

Prime Movers*

There are a few ways to break down these generation methods into more useful categories. The first is to differentiate between storage and ongoing or direct generation. The storage technologies are extremely useful, because they turn the inefficiencies of traditional generation and demand patterns into usable and highly dispatchable energy. There are 5  somewhat novel energy storage technologies listed (compressed air is listed twice), and 1 traditional technique (reversible hydrokinetic turbines) that can operate in reverse and act as a storage technology. Growth in storage technologies is the key for integrating larger amounts of renewable resources which can vary daily (wind and solar), seasonally (hydro, or even over the course of decades (geothermal, dependent on regeneration of via heat flow rates). With growth in these storage technologies so important, let's take a look at their implementation in the US.**

Out of 19,023 generators in the United States, 172 are storage technologies. In terms of total numbers, storage-based generation accounts for less than 1% of US generators, 0.90%. Despite the vanishingly low number of storage-generators, it's a misleading statistic. There will always be a smaller number of storage generation units than primary generation, a more illuminating factor is to compare the generation capacity totals.  Form EIA-860 reports a nameplate capacity, winter capacity, and summer capacity. While true capacity factors would be the most useful, I took the average of winter and summer capacities for each generator, rather than the larger nameplate capacity.

The total direct generation, and remember, this is an average of winter and summer capacities, which is not taking true capacity factors into account, is 1076 GW. To compare, storage accounts for a measly 22.8 GW. 2.12% of generating capacity could potentially come from storage right now.

Honestly, this sounds a little better than I expected when I started digging into this data, however, there is one major factor that makes the current situation look a bit worse. The cheapest and most well known storage is simply pumped water, or reversible hydro if it's available. Unfortunately, areas with the greatest potential for solar technologies often have less water availability than more temperate regions, and the cost of installing large pumped-hydro storage in the plains dotted with wind turbines could be prohibitive. Removing pumped storage from the storage technologies produces a nameplate capacity of only 0.2916 GW. In fact, with pumped storage excluded, only 16 generating units out of the 19,000+ reporting units were storage technologies. Here's a summary table:

So most of our storage technologies are pumped storage. But how old are these systems?

The first storage generator was constructed in the late 1920's (2 pumped storage generators in 1928 in Connecticut on the Rocky River), most of them were constructed from the 1960's and through the 1970's, then we experienced a large drop going into the 1990's, with a resurgence in the late 2000's. I wouldn't be surprised if this trend was political in nature, but I'll keep that speculation closer to my chest for now.

Again, I'm going to remove pumped storage from the figure. While considerably fewer data points will remain, upcoming technologies should be more apparent.

Because there are so few examples, here's a table describing these installations.

Unsurprisingly, the trend is towards batteries, with a little experimentation in flywheels. Considering the trend towards renewable generation, and the relatively known commodity of pumped storage, it would be reasonable to assume that recent pumped storage installations easily eclipsed those of more esoteric or untested technologies. However, this is not the case.

Surprisingly, despite the maturity of the technology, pumped storage installations have been surpassed by other technologies in installation numbers since the early 2000's. One potential explanation for this phenomenon is that the existing hydro installations that could be retrofitted with this technology are saturated. If that is truly the case, then perhaps investing in new technologies is seen as potentially less capital-intense. Another simple explanation, is that our grid has only begun to feel the strain of variable, renewable generation in the past 10 years, and installations of storage-based generation simply haven't caught up. This also makes sense given the availability of subsidies for investing in green generation, rather than figuring out how to utilize it most effectively.

Finally, let's take a look at why storage generation technologies are so important, through the curtailment of wind generation.

Of the renewable energy technologies, land-based wind is the most variable, and therefore would benefit most from increased utilization of energy technologies. The amount which would be useful is easily to visualize using 2012's curtailment data. Curtailment is the practice of shutting down wind turbines, which can be forced or voluntary. Voluntary curtailment is the process of following the market and determining that running the turbine isn't worth it, and forced curtailment is non-market based, but could be due to grid concerns.  Only ERCOT reports both forced and voluntary curtailment, the other regions only report forced curtailment.

Unfortunately, curtailment information is not as widely disseminated as other generation data. Even going from 2011 to 2012, the DoE experienced different level of participation from utilities in reporting curtailment, shown below***.

This gap in data requires some back-of-the-envelope calculations, something I'm certainly not averse to, although I wish there were more data points to work with. Given the somewhat extreme variability in curtailment losses over the past 5 years, any method I employ will certainly have gaping holes. In the face of these limitations, I'm just going to keep it simple. The average % lost over the 5 reporting groups is 2.36%.

While the EIA's wind data page is several years out of date, an estimation as to installed capacity can be made from the recently released generation report. Summing the nameplate capacity of the operating on-shore wind turbines gives 59.62 GW of capacity. If wind operated 100% of the time, the annual generation would represent the hours in a year (8760) times this capacity: (8760hours) x (59.62GW) = 522,271.2 GWh. This isn't the case, as that figure would need to be reduced by the turbines' capacity figure, and the actual electricity onto the grid would be drop further due to conversion and efficiency losses.

A fairly wide range of capacity factors for wind turbines exists, from 20-40%, but the average reported in the 2012 Wind Technologies Market Report was 32.1%. So,

(522,271.2GWh) x (0.321) x (0.0236) = 3956.51 GWh.

To put that in perspective, the 2011 total electricity sales were 3,749,846 GWh, so the amount of wind potentially curtailed was about 0.105% of our total electricity consumption (reported as sales). That doesn't sound like much, but with an average price of 9.9 cents per kWh it can quickly add up.

($0.099) x (3656.51 GWH) x (1000 MWh/GWh) x (1000 kWh/MWh) = $391,695,252.57

Nearly $400 million dollars of generating capacity left on the table is not an insignificant amount of money, and these calculations are somewhat tip of the iceberg. It might be tempting to compare this "lost" cash to the costs of peaking plants, which have to be maintained, but run relatively rarely. This is a poor comparison because a utility knows the peaking plant will only be run occasionally, and charges accordingly during peak hours. Curtailment of wind generation can be voluntary, but much of what is reported is not, and speaks to the larger issues of reliability and maintenance in our grid.

There are certainly other inefficiencies in electricity generation, and I have seen wind curtailment figures ranking from 25-40 TWh. Although I couldn't find any data or sources to back up those claims, my 2.36% figure is a very conservative estimate.

Hopefully this sheds a little light on the state of our storage deployment, and provides a bit of a backbone for why this is such an important topic. I didn't even discuss the mismatch between wind and solar generation vs. peak demand, but hope to in a future post.


I'm still feeling out this whole posting-independent-thoughts-and-research-online thing, so feel free to call me out on some perceived bullshit, or ask questions about my methodology or sources. I tried to keep this post to a length that would prevent too many eyeballs from glazing over, but I guess I'll just have to wait and see.

*Unless otherwise noted, all information in this post is sourced from the form EIA-860 for 2012, found here.

**Note: I'm only looking at operable plants, the EIA also reports on proposed and retired plants.

*** 2012 Wind Technologies Market Report. US Department of Energy.

WIP - Continental Generation Profile: Africa

It's a work-in-progress (WIP) because it's midterm time this week and last, so my output has slowed dramatically.

The idea here is to take a look at the generation profiles, one continent at a time, in a series of graphs and maps. I'll try to discuss a little about my experience with the region and what notions I've had in the past about its energy resources and generation. My background is there just to be open in case you want to perceive a bias in my representation of the data. Without further ado,


I spent the first half of 2010 mostly in Cape Town, but I also traveled through most of South Africa, some of Zimbabwe and Botswana, and nearly all of Namibia. I was studying abroad at the University of Cape Town, but the semester ended earlier than usual due to the World Cup, and my return flight was a week or two after the World Cup's end in order to avoid the high prices immediately around and during the event.

Before I arrived, my knowledge of energy markets and generation in Africa consisted of a vague idea that Namibia had attempted to build a huge dam, many armchair theorists drew squares on the Sahara showing how little (relative to their countries) space it would take to power Europe via solar, and that there was significant oil production in the middle-western portion of the continent. While there I became at least vaguely aware of some issues in the countries I visited and lived in:

South Africa

  • Significant youth campaigning against South Africa's aging nukes, which was weird to me, coming from the Chicago suburbs, where we have more than people seem to think.
  • Their incredible build-up of CTL (coal-to-liquids) technologies due to their restricted access to liquid fuels during the apartheid.
  • The country had excellent potential for the development of wind and solar, but no actual projects.
  • Had a suspicion that there was huge potential for hydro-power in cooperation with Lethoso, but that the need for water in the relatively-aird South AFrica would supercede that need, at least for now.


  • Never did build the dam with Angola I had heard about previously.
  • The government bought fuel from South Africa, then subsidized it so that prices were actually cheaper there (which pissed off a few vacationing Afrikaners I met).
  • No economic oil (firms have recently been trying to say otherwise, but it hasn't gone well yet).


  • Honestly, I didn't learn anything about energy in Zim (although I would have guessed that very little was happening.


  • All coal, all the time.



If you look at all renewable generation, there appears to be a wide spread across most of Africa, excluding the Saharan nations.

However, a very large portion of this generation is large hydro-electric infrastructure. Removing the hydro paints a markedly different picture. It's interesting here that Egypt and Kenya are the big winners. Egypt seems reasonable as  a state that has existed in its current form for longer than some other African nations, and has always had relatively strong ties to the West. Kenya is more surprising, and speaks to the economic strength and positive growth the country has experienced relative to much of Africa. Oil-exporting nations such as Chad, Nigeria, Cameroon, Equatorial Guinea, have all neglected to pursue development of non-hydro renewables despite the somewhat steady flow of resource profits. While the lack of infrastructure in some states (such as the Congo or Zimbabwe) could be explained by regime instability and corruption, South Africa stands as a shining example of a largely modern economy that has generally failed to pursue renewable resources. This could be a result of significant past investment in traditional fossil fuels and fission plants, but with the high potential for wind and solar and presence of at least one world-class engineering school (University of Cape Town), it seems odd that development has been so slow. If I had to guess, I would place the blame on institutional intransigence within the government, exacerbated by the strength of the department of minerals relative to the government's other interior departments.

Some (bad)Starting Maps: Monthly Electricity Generation and Consumption in 2011

I love graphics. The information density of a good graphic is tremendous. Unfortunately that information density, and frequent use of terms-of-art can easily preclude a meaningful understanding of a graphic, be it a bar chart or a map. I've been there, you look at a map and think to yourself, "well, that's neat", but you don't really gain from it. I know I'll be guilty of this. In particular, I love maps, but I'm horribly rusty at making them look nice.  Essentially, this paragraph is my disclaimer. My maps and figures may look confusing at first, but I'll attempt to thoroughly explain them and improve from one set to the next.  I'll also take any requests for data or more specific maps.

That said, this first map is in awful, awful violation of those principles.

The above map is immediately colorful, has an incredibly weird set of breaks, too many breaks, a typo in the title, a somewhat inscrutable title, and is kind of an all-around train-wreck.

What data went into this map?

  1. Monthly electricity consumption data for each state
  2. Population of each state
  3. Shapefile of the states

What happened to the data to result in that abomination of a title?

  1. The electricity consumption for each month in each state was divided by the state's population in 2011.
  2. Using this data a national average of consumption was created for each month.
  3. Each state's consumption per capita per month was put in terms of the national average as a % difference.

What this map is showing is how much electricity is consumer per capita in a state each month compared to the national average. Except it's every state at once, so it's frustrating to track and distracting to look at.

While the EIA also has monthly reports of cleaned up data, I wanted to practice working with text in Matlab and matching together several values while looping through the data to do some basic calculations and output a decent matrix to join with the database file of the states layer I was using in Arcmap. The generation data reported by utilities to the EIA comes in a monthly format. By linking each utility to a specific state I tracked electricity generation month-to-month for the United States, and mapped the change between each month. If January's generation for Texas was 100GWh, and February's was 75GWH, then Texas would have a -25% change for January-February, and appear as a light green on the map.

The trends evident on the map are largely seasonal, the demand for air conditioning is readily apparent in the eastern and southern halves of the country. New Hampshire and Vermont present an interesting anomaly in October. In 2011, the east coast experienced a massive snowstorm, putting a significant chunk of generation offline. This change is reflected in the map.

Consumption follows a similar pattern.

Hopefully I can come back after midterms and offer a little more analysis for these maps. Honestly, they're not meant to convey a startling new concept, or make you think differently. These maps are simply a graphical reflection of annual electricity use that I hadn't seen presented before.

My basic workflow on these maps goes:


  1. Excel: Glance at the data, check for gaps, make sure multiple datasets will agree in Matlab. Honestly, I'll go back and forth between Excel and Matlab a few times in the beginning to make sure things are lining up. I'm not so confident that I don't check at least part of my Matlab calculations by "hand" in Excel.
  2. Matlab: Any modifications to or combinations of my data happen in Matlab. I also use it to make any figures that are not maps, although you can draw maps from longitude and latitude data if you really want to. I'll also find the min and the max of any dataset I'm using in Arcmap.
  3. Excel: Back to Excel if I'm going to be making something in Arcmap. I use Excel to make sure column headers match if I'm going to join with the .dbf file of a layer, and eyeball the data again for mistakes like unit problems or column/row misalignment in Matlab.
  4. Arcmap: If I'm following the process for the maps above I'll join the numbers crunched through in Matlab to the .dbf file associated with the layer in question. I normally try to join on standardized codes, like FIPS, rather than names, which can be slightly misspelled or just different (particularly for countries).  The min and the max I find above come into play here when choosing a series of breaks in order to display the data with chloropleth symbology. I'll set the symbology in the column (month in this case) with the highest value, then apply it to each other column to remain consistent.  Too many breaks is confusing and boring, too few smooths the curves and presents far less information. Teasing out decent breaks is something of an art, one which I admit I need work at. Normally I use a mix of manual choice and natural breaks (Jenks) depending on the data. In the end, I apply the symbology form the most extreme column to every time-set of data, and export each map accordingly.
  5. Photoshop: Finally, after exporting the maps I need from Arcmap as .PNGs I load them into a stack in Photoshop, create an animation, then export it as a .gif.

“What are you in school for?”

I get asked this question a lot, and generally fail to give a satisfactory answer.  Often it's from family, and I feel bad about my inability to relay what I'm doing or want to do.  I'm going to walk briskly through my undergrad and grad programs to this point to hopefully shed some light.

My undergraduate degree was an interdisciplinary experience magnified through the lens of a program's inception and initial struggle to gain traction.  The School of Earth, Society, and Environment (SESE) was started only a year or two prior to my entering the University of Illinois Urbana-Champaign (UIUC) as an undecided, undermotivated, overenergized freshman. I loved biology and the outdoors, but struggled with math, having paid almost no attention in important foundational classes past algebra. I loved reading and writing (and found it extremely easy to write academically), so I gravitated to history, but dabbled in genetic engineering, geography and geology.

At some point, I realized there was a new school within UIUC seemingly designed around my laissez faire attitude towards the academic advising I had received. In addition, it allowed me to do some basic science (measuring biofuel capacity, designing restoration plans, assessing riparian buffer strips) and get involved in social environmental research, while my math skills continued to wane  It also prevented me from continuing in history, which I knew could be a poor career choice, and wasn't particularly fulfilling, just easy.

SESE allowed me to take courses in any department I wanted, and made it extremely easy to take classes in South Africa during a semester abroad, and have them count towards my degree. The classes were relevant (physical oceanography, sustainability and policy, and ecological change), but the lack of friction compared to other friends in other departments was readily apparent. I took far too much advantage of slippery nature of the requirements during the program's first few years. I literally picked classes based on their description and how interesting I thought they would be, much to the disbelief of my engineering friends who consulted a flowchart each semester to determine their schedule. I also quickly discovered that 300 and 400 level classes were more interesting and engendered better discussion due to the mixture of grad students and smaller class sizes. I also regularly took 18 or more (22 at most) credit hours a semester in order to get he most out of some scholarships. This makes me sound like an over-achiever, but in reality it was the reverse. I skipped over several core courses that seemed hard or boring or seemed to have excess busywork attached. Additionally, the quality of teaching was much improved at higher levels. UIUC's position as a hugely international university often led to language issues between the students and TAs in certain courses, which was never a problem I encountered in higher-level courses.

In the end, I had an incredibly enriching academic and social experience at UIUC, and a life-changing semester in South Africa, but still had no idea what I wanted to do. In the fall of 2010, looking forward to graduation the next spring, the job market looked scarily depressed. I loved school, so I thought applying to grad programs was a decent idea. With no idea of the process, or what I wanted to do I applied to only 3 , totally different, programs. I applied to the University of Oregon's MS in Environmental Studies, the Monterey Institute of International Studies' Peace Corps Masters International Program with the intent to pursue an MA in International Environmental Policy, and to Indiana University Bloomington's MPA degree within SPEA on the recommendation of an email from our program's director at UIUC.

I got in at Monterey and Indiana, rejected at Oregon. Monterey was more expensive, and when I finally started looking at rankings and other criteria, SPEA seemed more and more appealing. People seemed to really like Bloomington, and SPEA was ranked #1 or #2 in the disciplines I cared about, which seemed to ensure a strong alumni network and decent chances at a job following graduation, my whole original impetus for going to grad school. What I was unprepared for was the weird transition into a huge professional degree program, that also had a strong science arm rooted in ecology and chemistry.  Chemistry was one of the reasons I had avoided applying to SPEA's dual MPA/MSES program. Basic chemistry was a prerequisite, but I took genetic engineering as my lab course in undergrad. This has constantly proved to have been a poor choice ( I chose it because it sounded cool and had an easy schedule). A single class in basic genetic engineering doesn't open many doors, but a basic chemistry lab does prepare you for a wide variety of courses and some basic research.

SPEA's students are divided into concentrations, which are declared during your first semester. Concentrations include typical MPA strongholds, such as policy analysis, Non-profit management, and public financial administration, but also drew on the environmental side of the school to offer sustainability and energy. A full list can be found here. While I was initially wavering between energy, information systems, and environmental policy and natural resource management, energy was the most intriguing, and it seemed to me that everything was tied back into energy in one way or another.

Here's a hefty quote from the energy concentration's page:

Students in this concentration focus on energy policies and technologies, exploring the socioeconomic and environmental consequences of both. They learn the life cycle of energy resources and study the economics of energy production and consumption.

In this concentration, you will explore the critical, cutting-edge approaches now being taken in energy conservation and environmental progress, including: tools and techniques for mitigating carbon emissions, new ways to diversify the energy sector, and the development of innovative energy technologies.

By taking an interdisciplinary approach — one that explores the interconnected nature of science, technology, economics and public policy — the Energy concentration positions you to help meet the challenges posed by an increasingly energy-dependent world.

That third paragraph hit on my own personal buzzword, "interdisciplinary." Everything I had done up to that point had been interdisciplinary, so why stop now?

So, for the past two years I have been studying "energy." Energy policy, technology, resources, economics, human behavior, whatever classes were available, while taking tools courses along the way to broaden my employment appeal and open doors for myself. The three major tools courses I took helped me learn some competency with Erdas Imagine (remote sensing, since I had already done a few semesters of GIS with Arcmap), Matlab (for data crunching, graphs, and modeling), and PHP/MySQL (database systems administration). I also learned some SAS and Stata, but need more practice to feel as competent as I do with Matlab.

The one major course correction during my time at IU was a long time coming. I finally took calculus and chemistry, two things I had been avoiding for too many years. I love doing research, reading journals, and fantasizing about my own publications, but a lack of formal knowledge in those topics had hamstrung me for years. I had picked things up, but I wasn't well versed enough to feel confident as often as I should have been. This hit home most strongly in the summer of 2012. I landed a last minute internship at a national lab. An amazing opportunity. Everyone I met was an engineer, computer scientist, or, more rarely, a geologist, geneticist, chemist. As someone with a broad, but not deep undergraduate degree, and pursuing a professional policy and economics-focused graduate degree, I felt woefully out of place despite loving the atmosphere, people, and research I was exposed to.

When I left the lab I had two major goals: learn Matlab and get an MS. I knew that it was possible to add an MSES degree to an MPA, and, outside of the core courses, it would be relatively painless for me. An energy concentration that mirrored the MPA's existed in the MSES program, so I wouldn't have to pick up another concentration, but could drill down more deeply into the topics I cared about. First though, I had to take chemistry and calculus to even be considered for acceptance. I took calculus that fall, and honestly, I struggled. I hadn't taken a math class in 7 years (statistics classes don't count, I had 2 of those), and I had never taken trigonometry. By comparison, nearly all of my classmates had taken AP calculus the year before, in high school. It was a rough semester, but I got through it and unlocked quite a few things in my head that had bothered me for years. Chemistry I took over the next summer and breezed to an A. Again learning to better understand topics I had read about for years.

I had less idea about how to learn Matlab. IUB is not an engineering school, and statistical and GIS packages were the only software I heard discussed with regularity. Fortunately, the geography department has an introductory course in Matlab focused on the manipulation of datasets and creation of figures, perfect for what I wanted. I focused on applying Matlab to all of my classes, from economics to finance, in the hopes of polishing my skills and doubling what I was learning. At the same time I started working for the Indiana Geological Survey on a geothermal heat pump project, and eventually I would build scripts to mimic commercial system sizing software.

Now, I am enrolled in the core MSES courses, and one niggling energy requirement, in what is hopefully my final semester. I'm also one of the co-authors on a talk at the Geological Society of America's annual meeting, and we're polishing up a manuscript to submit to our internal review in the next week or two before sending it out to journals. Between hammering out the manuscript and classes, this semester is probably my roughest ever.

My courses this semester:

  • Wastewater and Drinking Water Treatment Engineering
  • Environmental Chemistry
  • Applied Math for Environmental Science
  • Energy Markets and Analysis

compared to my courses upon arriving into the MPA program 2 years ago:

  • Public Management
  • Statistical Analysis for Effective Decision Making
  • Public Management Economics
  • Law and Public Affairs

speaks to the wide range of topics I've studied, even in the past two years, and why I find it so difficult to distill down exactly what it is I've been doing. Hopefully this post has been enlightening, not just for my family and friends, but for anyone who wonders about an MPA and SPEA in particular, or what to do with their oddball kid who studies all sorts of things and reads way too much, but doesn't slot easily into traditional degrees.