Antifragility

I don't know how we're going to decide how much money to spend in Texas to protect against the next presumably rare extreme cold event, or how to share the cost of increasing the reliability of generating plants and ancillary equipment.  I have, however, learned several things about how we approach the reliability/cost quandary.  One is that Texas's grid operator, ERCOT, is free of Federal Energy Regulatory Commission (FERC) jurisdiction only in the rate-making context, because our intrastate sales aren't considered interstate commerce.  We are still subject to FERC regulatory authority over grid reliability, through the auspices of the North American Energy Reliability Corporation (NERC), a continent-wide non-profit organization that derives its domestic authority from FERC.  There has been a lot of talk about how we in Texas should have learned our lesson from the 2011 winter storm that caused rolling outages, but we suffered polar vortex storms in 2014 and 2015, after which NERC issued a report on all of its dozens of members, showing ERCOT to be in good shape in terms of both energy reserves and winterization progress.  Clearly the 2021 storm blew everyone's weather assumptions out of the water.

The other thing I've gathered about reliability is that we need more attention to linked risks.  It is not anti-fragile to have backup systems that will fail from the same cause that resulted in failure of the primary systems.  Here is a slightly rewritten summary I posted to Facebook of what I learned from the operators of some of our exurban neighborhood infrastructure, which performed much better than infrastructure in the rest of the county or, frankly, the rest of the state.

There are  three different water suppliers on this peninsula, all of which use water wells and RO filtration.  For those of us not on septic tanks, sewage treatment for the whole peninsula is provided by a local municipal utility district (municipal in the statutory sense; we are an unincorporated area of the county).  The MUD also operates one of the three water supply services.  A second water company, ABU, is managed by the local volunteer fire chief as his day job.  A third water company supplies a somewhat detached nearby neighborhood managed by its own HOA, which relies primarily on septic tanks, but may also be connected to the MUD for sewer services to some degree.  So the peninsula has water from ABU, the MUD, and the HOA system, and we all have sewer service from the MUD.

The fire chief's wealthy boss at ABU water company also owns and operates an oil & gas company, which operates maybe half of the local gas wells that supply the county’s utility gas lines.  The fire chief is a talented mechanic who works on his boss's hobby race cars.

The report card:  Although the peninsula had power outages here and there, nevertheless, the MUD, which did not lose power, suffered no interruptions in supply of water or sewer.  The MUD had backup generators that were not used.  ABU lost power but suffered no interruption in supply of water, as its generator worked fine.  The HOA system did have some power outages, and I heard reports of water outages, but I lack details.

Because the MUD didn't lose power, all I really know so far is that it managed to keep operating despite an unusually hard freeze.  I have more information about how well ABU did and why.  For one thing, the fire chief did a lot of last-minute weather-proofing on the his lines and equipment, not all of which was successful, because some lines still broke. What was more important, though, was that although ABU lost grid power, it had a 300kW diesel-powered generator that ran for 4 days straight, using up maybe 2/3 of its 600-gallon diesel supply.

The fire chief chose diesel for this system because he’d been warned years ago that piped-in natural gas can’t always be relied on in a natural disaster. Sure enough, ABU's boss O&G company had to fight to keep its local gas wells operating. They managed it because their workers, and the boss too, got out in the field and weather-proofed equipment, and then when some equipment still froze, used heaters to get it unfrozen. If our county still had at least partial natural gas pressure, much of the credit goes to this boss and his organization. Many other local gas wells just shut in when it got too cold.  Sure enough, many generators in town failed, including the generator at the new ER, because the utility gas pressure fell too low. This reinforces the good sense of the fire chief's decision to run the ABU generator on a diesel tank.

ABU’s water pipe leaks posed a challenge, as did some frozen sensors. The fire chief thought at first that his water tank levels were adequate, until he eyeball-checked some of them and found that the frozen sensors were stuck on a false high reading. That warned him to keep the wells running and the tanks full. The leaks created pressure problems, but again keeping the wells on full blast made up for the struggling pressure. The only trouble was that the increased volume was too great for the RO filters to keep up, so the water supply temporarily bypassed them and converted to pure well water—potable and safe but not as tasty.

The water system in the rest of the county broke down completely, exposing people first to a shut off and then to a water-boil advisory.

What happened in the rest of the state?  It's still very murky, but the upshot is that the same freezing conditions that caused demand to spike also shut off about half of the state's power generating capacity.  Wind and solar primary equipment froze up and stopped operating.  You might think that thermal generating plants wouldn't freeze, but in some cases, not only did critical ancillary equipment freeze and fail, but their fuel suppliers froze and failed.  Gas wells froze and were shut in.  Gas that made it to pipelines couldn't maintain flow or pressure.  There are stories I haven't been able to confirm about power plants losing grid power; apparently they aren't set up to operate independently of the grid no matter how much power they generate internally, which amazes me almost as much as the idea that they don't get first priority from the grid, if that turns out to be true.

Even without spending a mint freeze-proofing equipment that freezes badly enough to cause a disaster only every century or so (because it's a huge problem only if the freeze is not only unusually deep but also state-wide and long-lasting), it seems there are changes we could make to unlink some of these risks.  Plants should be set up to use internal power not only for core functions, but to some extent to heat their ancillary equipment.  It can't be sensible to use unreliable grid power for the very facilities on which the grid is depending.

9 comments:

Grim said...

I only have one piece of advice for you, but I think it's a good one: make it a priority to weld Faraday cages around important electrical infrastructure. One of those 'rare, low chance' events is a major sunstorm, but it's going to happen sooner or later. Faraday cages are super cheap, and when that storm hits your transformers will still work.

It's also good against EMPs, should Iranian terrorists decide to light up a nuke in your upper atmosphere. That's not super likely. The sunstorm is definitely coming, though; we just don't know when. It may be a long time.

Texan99 said...

I've been thinking about Faraday cages, apropos of my comment about always fighting the last war. We could spend a fortune winterproofing only to find we were armoring ourselves against the next hard-to-predict meltdown.

Aggie said...

Tex I have a suggestion to make. There are many facilities around the country that rely on conventional diesel-fired gensets to get them through scrapes, either facilities with critical needs (hospitals and such) or remote power grids that sometimes need a little extra to get over the peak demand loads (peak-shaving). Sometimes the issue is getting the diesel supply to the generator facility is difficult. So many of these setups rely instead on natural gas. They modify the diesels to burn either 100% diesel, or to run on mostly natural gas. It's called a bi-fuel setup.

It's not terribly expensive, and there are no modications to the engine. It sounds like your area has natural gas infrastructure. These things are bolt-on setups. They're computer controlled. What they are used for is extending the diesel supply, so that you are using only about 10% of normal consumption, the rest being made of by natural gas.

I know about these because I've converted the diesels on drilling rigs being installed on offshore gas platforms. Of course there, we had existing gas production. I did it to save money, as essentially we were getting the gas for free, where the diesel had to be purchased and transported by supply boats.

It's a clever, simple, and seamless set up. Houston has the technology support base. If gas should be interrupted for any reason, the engine just goes back to 100% diesel, automatically and seamlessly. Never misses a beat. But this could be a fairly easy way to extend your generator set capabilities for effectively much longer, given your local gas infrastructure.

Texan99 said...

Sounds interesting. I was also surprised to hear in a Khan podcast today, from a guest, that diesel starts to solidify at about 17F, so we came close to having a problem with it, I guess.

Our own generator runs on a propane tank rather than utility-provided gas lines. Of course, we still have to find a propane distributor in business to refill it. After the 2017 hurricane we took out the 175-gallon tank and replaced it with a 500-gallon one. I suspect propane will be hard to find for a while, and expensive.

Aggie said...

Gasoline and diesel mixes are seasonal, and their recipe components are also tailored to the regional weather. A marketer will sell 'summer' and 'winter' diesel as the refiners change their formulations to anticipate the seasons. I would be very surprised to see diesel gel here - it would already be the 'winter' formulation and probably good for considerably colder temperatures. When I worked the severe winter climate areas where it would get to -40°, we used to keep our diesels running when it got really cold but never had to heat the fuel, although we had block heaters for the engines.

500 gallons is a good number! You can probably get about 7-10 days on your generator with wise usage. Once we get to shoulder season the price will start easing down. I usually try to wait to May-June

Texan99 said...

And yet this prepper guy who runs a respectable prepper website had his diesel gel in Austin. It sure surprised him.

Aggie said...

I wonder when he filled his tank.

Texan99 said...

It sounded like he keeps it full all the time; it's supposed to be emergency supplies, not ordinary supplies. He did mention that he thought he had emergency propane supplies, too, but unbeknownst to him, his wife, who uses the Airstream as an office, had been heating it with the propane instead of the electrical grid. Always important to segregate and/or monitor the level of emergencies supplies!

Gringo said...

And yet this prepper guy who runs a respectable prepper website had his diesel gel in Austin. It sure surprised him.

IIRC, I heard from Montana relatives that during the Xmas 1983 cold snap, that reached all the way to Florida, that the propane used for their home heating didn't flow.