U.S. ARMY CORPS OF ENGINEERS INSTITUTE FOR WATER RESOURCES RISK MANAGEMENT CENTER PRELIMINARY LESSONS LEARNED FROM OROVILLE INCIDENT BRIEFING FOR: Eric Halpin, Deputy DSO DATE: 7 April 2017 PURPOSE OF PAPER: Review of Oroville Dam Spillway Incidents, USACE Current Methodology for Evaluating Similar Spillway Potential Failure Modes, and Identification of any Systematic Shortcomings in USACE Practice. BACKGROUND: The recent incident at Oroville Dam was quite serious. At the request of the Deputy DSO, the RMC has taken a rapid look at our processes for evaluating failure modes associated with spillways. This is not a thorough review of the Oroville incident or a comprehensive look at our methodology. That effort should be completed following the review by the Oroville Dam forensic panel. However, the RMC took a quick look at the incident to ensure we haven’t overlooked anything major in our processes. OROVILLE DAM DESCRIPTION: Oroville Dam, completed by the State of California in 1968, is the tallest dam in the United States, an earthfill structure with a height of 770 feet. There are four outlets for releasing water to the Feather River downstream, from most to least preferred: 1. Through the hydro-electric generators, with maximum flow rate of 16,950 cfs. 2. Through a river outlet or bypass valve, with maximum capacity of 5,400 cfs. 3. Through the main service spillway located on the right abutment of the dam, with crest elevation 813 feet. It is controlled by eight 33-foot-high top seal Tainter gates and has a design capacity of 150,000 cfs and a maximum capacity of 296,000 cfs. A concrete lined chute conveys water to the river. 4. Over the top of an ungated emergency spillway consisting of a 1,730-foot-long concrete ogee weir and crest wall with a sill elevation of 901 feet, 21 feet below the crest of the main dam and a height varying from about 70 feet down to 5 feet. Water flowing over the weir discharges onto the unlined rock hillside downstream. Details of the spillway chute design are shown in Figure 1, taken from Drawing IF 262. Note that the concrete slab is only 15 inches (minimum) thick. Although no waterstops were placed in the spillway chute panel joints, steel dowels were placed across the joints and keys were constructed to prevent upward movement of the downstream slab into the flow, as shown in Figure 2 (taken from Drawing A3B9-1). The irregular rock foundation was brought to a more uniform grade using compacted clayey fines. Passive anchor bars to tie the slab down were anchored five feet into rock and “hooked” into the concrete. The slab reinforcement was only placed near the upper surface of the slab. No reinforcing steel was placed in the bottom of the slab. A drainage system was installed at the base of the slab consisting of perforated vitrified clay pipe in a “herringbone” arrangement, with outfalls exiting the spillway walls back into the chute. Significant flows have been observed exiting the underdrain system since original construction. In addition, the herring bone drain pipes extended up into the concrete slab with not much clearance to the top of the slab (minimum 7-inches). This has apparently produced cracking in the concrete. Crack repairs have been ongoing prior to the incidents. 1 Figure 1. Spillway Chute Slab Foundation and Drainage Details Figure 2. Spillway Chute Joint and Drainage Details OROVILLE INCIDENT: In early February 2017, following several years of drought, high inflows to Lake Oroville caused dam operators to open the service spillway gates to control the lake level. Releases were less than the previous maximum releases. On February 8, 2017, a portion of the service spillway chute failed. A 150 foot wide hole formed in the chute around stations 33+00 to 34+00, about half way down the chute, as shown in Figure 3, and a significant amount of the underlying foundation was eroded. Spillway releases were shut off for inspection at a time when the level was rising at unprecedented rates. The upstream remaining spillway slab was cantilevered 2 about 45 feet over a 45-foot-deep hole, as shown in Figure 4. It was decided to release two test flows on February 8–9, following which the upstream cantilevered section of the chute failed and was washed away, and the length of the hole increased from 250 feet to 300 feet. This left two choices for subsequent operations: (1) continue to use the service spillway, knowing it would likely be further damaged, or (2) allow the reservoir to rise until it overtopped the emergency spillway. Initial Hole Figure 3. Initial Hole in Spillway Slab Figure 4. Close-up of Initial Hole in Spillway Chute The decision was made to reopen the service spillway. It was hoped that using the damaged spillway with a limited flow could release enough water to avoid using the emergency spillway, which would potentially damage powerlines servicing the hydroelectric plant. The discharge was reduced from 65,000 cfs to 55,000 cfs, but this flow was not enough to prevent the lake from rising. At some point, the relatively thin spillway walls failed, and debris washed into the river creating a blockage that raised tailwater at the dam, making the powerplant inoperable, and eliminating the ability to release from the units or low level outlets. 3 As the lake level rose, measures were taken to prepare the emergency spillway for use including placing of limited grouted rip-rap in areas thought to be susceptible to erosion, near the toe of the deep section of the Ogee weir, where a concrete apron was not present. On February 10, 2017, power transmission lines were moved, and workers began clear-cutting trees on the hillside below the emergency spillway. Shortly after 8:00 am on February 11, 2017, the emergency spillway began discharging water for the first time since the dam's construction in 1968. Flows peaked at 12,600 cfs causing erosion of soil and rock materials on the hillside below (as shown in Figure 5). If the erosion was significant enough to undermine the right section of the emergency spillway (where the erosion was most severe) and cause it to collapse, a 30-foot-high wall of water could plunge into the Feather River below and potentially flood communities downstream. Fearing such a collapse, officials issued an evacuation order. Figure 5. Damage from Releases over Emergency Spillway (the large hole near the right side of the ogee section is downstream of Monoliths 4, 5, and 6) Figure 6. Damage to Service Spillway and Debris Washed into River 4 Following the evacuation, the decision was made to lower the pool level, using the service spillway, and make repairs. Although additional erosion and loss of the spillway chute and walls occurred (see Figure 6), a somewhat stable condition was eventually reached for releases between 50,000 and 100,000 cfs. Interim repairs to both spillways are in progress. Future major repairs are planned. EVALUATION OF OROVILLE INCIDENT: An official forensic evaluation is under way and this review is not intended for that purpose. However, there are a few noteworthy issues that probably led to the unanticipated damage as follows: 1. The service spillway chute slab is relatively thin for this type of structure 2. Typically two mats of reinforcing steel would be installed on the upper and lower sides of the concrete chute slab 3. PVC waterstops would typically be installed at all slab and wall joints in a concrete spillway chute. Waterstopped or water tight joints/cracks in a spillway chute are key during operation. When the joints/cracks at Oroville were sealed, the underdrain output reduced by about 75%. The "secondary" factors of uplift or erosion beneath the slabs were both highly influenced by flow through unsealed joints/cracks. 4. Drainage pipe would typically not be allowed to project up into the concrete chute slab, especially a slab this thin, which in this case caused large open cracks in the chute 5. The spillway walls were also relatively thin for this type of structure 6. The source of the large drainage flows exiting the service spillway drains has not been definitively determined, nor has the drainage flow been monitored for the presence of fine materials. However, caulking the slab joints and the joint intersection with the chute walls has shown to reduce the amount of water exiting the foundation drains 7. Highly weathered decomposed rock associated with a shear zone passes under the chute in the area of the initial service spillway hole, as shown in Figure 7, and continues off to the left where the bulk of the erosion occurred, as shown in Figure 6 8. Using (erodible) compacted fine soil to fill foundation voids under the service spillway chute is not common practice 9. The passive bars used to anchor the chute are short for this type of structure and were probably not anchored into competent rock in the areas of shear zones, nor was there adequate concrete thickness to anchor them into the slab 10. The erosion downstream of the emergency spillway also tended to follow highly weathered shear zones, and it is not clear how these shears affect the foundation of the ogee weir or crest wall (to the right of the ogee weir), i.e. were they founded on competent rock or was the foundation adequately treated? Based on these observations, the underdrain system, poor geologic conditions with compacted clay leveling fill, and marginal design details likely led to the incident. The drain pipe protruding into the service spillway slab likely caused open cracks to form, and the underdrain flows may have eroded material (possibly altered due to years of drought) from beneath the slab. If the upslope side settled relative to the downslope side, such that the downstream portion projected up into the flow, the situation would be exacerbated. When the spillway gates were opened, high velocity water probably entered the cracks and the stagnation pressures were enough to lift and break the thin slabs, given that the anchor bars were short and anchored into weak materials, and there was no steel mat on the bottom of the slab to resist bending. Once the concrete slab was gone, the underlying highly decomposed rock and compacted clay leveling fill were easily eroded down to lightly weathered rock. Deeper erosion occurred along the deeply weathered shear zones, and the water followed these pathways. 5 When the decision was made to let water go over the emergency spillway, it was expected that the surficial slopewash and weathered rock would be eroded. It may not have been anticipated that there would be highly weathered shear zones where deep erosion channels could form, headcutting back toward the ogee spillway crest. It is not clear if the headcutting would have progressed to the point of undermining and failing the ogee crest (foundation treatment under the ogee is unknown), and even if it did, the water released may have been no greater than what had already been experienced (although the reservoir would have ultimately been lost to the elevation of the erosion), but based on the observed erosion progression, the prudent decision to evacuate was made. The only inundation maps available were for PMF failure of the main dam, so very conservative evacuations were made. Figure 7. Geologic Conditions at Location of Initial Hole (dashed line and fine stippling represent the shear and associated highly decomposed rock) Despite the extensive damage, control of the reservoir was never lost during these incidents. Significant rock erosion occurred, but the underlying slightly weathered to fresh rock was competent enough to resist the flows. From a dam safety perspective, the chance of uncontrolled reservoir release may have been small and the chance of incremental consequences was also probably small. Nevertheless, the incidents created a media frenzy and evacuation of downstream population is never taken lightly, as injury and death can occur from this activity alone (although none was reported for the Oroville evacuations). It is important to understand the potential for this type of incident, possibly leading to potential uncontrolled reservoir releases, to occur in the future. CURRENT USACE METHODOLOGY FOR EVALUATING SPILLWAY FAILURE MODES: USACE uses risk-informed decision making to deal with dam safety issues. To assist with evaluating risks for potential failure modes, the “Best Practices in Dam and Levee Safety Risk Analysis” training course and manual have been developed. There are two chapters in this program that deal with potential spillway erosion failure: (1) Erosion of Rock and Soil, and (2) Stagnation Pressure Failure of Spillway Chutes. A third chapter on spillway erosion is in progress that brings these concepts together. There are other ancillary chapters that are also relevant, but these are the main ones. 6 Potential loss of the reservoir related to stagnation pressure jacking of spillway chute slabs was highlighted with the chute failure of Big Sandy Dam (Wyoming) in 1983. Spalling at a slab joint created a small downstream projection into the flow, as shown in Figure 8. Figure 8. Spalling at Spillway Joint (downstream is toward top of figure) During releases of 400 cfs in 1983, a large “rooster tail” developed (Figure 9). Following releases, it was discovered that a large section of the chute slab had been jacked out of place (Figure 10). Figure 9. Rooster tail during spillway releases 7 Figure 10. Lost spillway slabs The slab was only 15 inches thick, and the anchor bars were thought to be poorly grouted. Fortunately, the releases were small and the underlying rock was able to withstand the flow. This incident prompted research into the phenomenon of stagnation pressure jacking of spillway chutes by the Bureau of Reclamation. Hydraulic models were developed to help estimate pressures that could develop beneath a slab for various joint openings and offsets into the flow. These relationships have been incorporated into “Best Practices” for use in estimating the risks posed by this type of potential failure mode. Recommended details for joint treatment were also developed, as shown in Figure 11. The more of these features that are included, the better the chances that stagnation pressure failure will not initiate. Figure 11. Suggested Joint Details from Best Practices (anchor bars should go well into competent rock, slab thicknesses should be adequate to accommodate reinforcement, and foam insulation would only be used in cold climates where ice jacking could be an issue) 8 It should be noted that the research into stagnation pressure slab jacking was strictly based on hydraulics. There was no consideration for the geological or geotechnical foundation conditions. In several cases, cracks have been found in spillway chutes that related to voids underneath the chute, as shown in Figures 12 and 13 at Hyrum Dam. These voids are related to erosion of the underlying materials either through stagnation pressures pushing material through the drains, drain flows carrying material, or under-seepage erosion. These voids could lead to collapse of the chute during operations, or settlement of upstream sections creating projection of the slab into the flow potentially leading to stagnation pressure failure of the slab. Although the Best Practices chapter on Rock and Soil Erosion contains ways to evaluate this type of erosion, in practice it is difficult to understand the foundation and flow conditions well enough to make reasonable likelihood estimates without some good performance information. The best way to evaluate this type of potential failure mode is to monitor for signs of material movement through the underdrain system outfalls or seepage exit points, cracking in the slab that indicates moment capacity is exceeded due to bridging over a void, or sounding the slab using a hammer listening for “hollow” sounds or using ground penetrating radar in areas suspected of having a void. If this information is collected routinely and rigorously, a reasonable evaluation of the potential for foundation erosion to lead to voids under a chute can be made. Figure 12. Cracking along the Center and Sides of the Chute Suggest Loss of Underlying Foundation Support 9 Figure 13. Erosion of Foundation Material Creating Void under Chute In the end, regardless of whether a chute slab has been lost during releases or an unlined emergency spillway experiences flow over the control sill, the ultimate erosion of the underlying rock or soil must be assessed as well as the potential for headcutting to progress upstream to the point where control of the reservoir is lost, such that uncontrolled releases leading to incremental consequences occur. The Best Practices chapter on Erosion of Rock and Soil provides guidance and tools for evaluating this likelihood. The probability of rock erosion initiating can be estimated using the StreamPower/Erodibility Index method. However, the hydraulic calculations and rock characterization must be done properly. If the conditions can be defined realistically with depth, new tools such as the computer program WINDAM-C can be used to help estimate the depth of erosion and limits of headcutting, and hence the chance of undermining a control structure. Oroville shows the importance of the three-dimensional aspects of the geology. Since WINDAM-C is a two-dimensional program, it is essential to look for the weakest geologic headcutting paths, for example along a highly weathered fault or shear zone. This requires that the geology and foundation treatment be characterized in enough detail to understand where these might occur. However, even then the tools are crude and judgment is needed in interpreting and determining the credibility of WinDAM-C results. With highly varied geology or complicated hydraulics (flow concentrations or tailwater influenced scenarios), the simplified representations result in significant uncertainty. It will be difficult to accurately characterize the conditions under slabs. It will be difficult to accurately characterized the soil and rock and adequately model the layering and structure with our current tools. Thus, competent engineers and scientists must use judgment in using these tools, evaluating the results, and determining the implications for probabilities and risk analysis. Similarly, Best Practices has information and tools to help estimate the likelihood of soil erosion (and removal of any grass cover), should a spillway slab or unlined spillway control structure and/or channel be founded on soil. WINDAM-C also applies to these cases. These evaluations are based on applied hydraulic shear stress vs. soil detachment rate or rate of erosion. Several tests are described to estimate the detachment or erosion rates of soils. USACE has almost every kind of spillway with almost every kind of subsurface condition in its dam portfolio. There has been severe erosion on several unlined spillways under flood discharges 10 (Canyon, Tuttle Creek, etc). Additionally, structures that have high downstream consequences similar to Oroville’s Population at Risk (PAR) are treated consistent with those consequences. The tolerable annual failure probability threshold for a structure similar to Oroville in USACE’s portfolio is less than 1x10-6 and then only if risks are as Low as Reasonably Practicable. On several dams including Lewisville Lake and Center Hill, USACE is modifying spillways to address failure modes due to the high consequences and marginal performance. At Lewisville Lake in particular, the Fort Worth District annually grinds down spillway joints that are protruding into flow. The projects where USACE has spillways and high consequences that are currently being evaluated for failure modes with respect to hydrologic loading are: 1. Bluestone Dam 2. Garrison Dam 3. Pipestem Dam (unlined) 4. Fort Peck Dam USACE METHODOLOGY APPLIED TO OROVILLE: So, what would have been the likely outcome if USACE methodology had been applied to the Oroville Dam spillways prior to the incidents? There were actually two significant spillway incidents at Oroville Dam; (1) failure of the service spillway chute and erosion of the underlying foundation material, and (2) headcut erosion of the hillside material downstream of the emergency spillway. No one knows for sure exactly how the situation would have been evaluated, but if the methodology was followed under the direction of an experienced and qualified facilitator, the following points would likely follow: 1. The two potential failure modes noted above would likely have been identified. 2. The fact that the service spillway had experienced significant flows in the past without incident would probably be weighted heavily by the team in evaluating the probability of failure (although case histories suggest past good performance does not necessarily mean good future performance). 3. Stagnation pressure failure of the service spillway chute (and subsequent heacutting progression to the reservoir) may have risen to a “risk-driver” potential failure mode. The minimal design details, shear zone and compacted clay foundation conditions, cracking of the slab, and large drain flows would have been red flags. However, the estimated probability of reservoir release and associated risks would probably be low due to (1) the long erosion path, (2) competent rock at depth and under the spillway chute and control structure foundation, and likely small incremental consequences due to already high releases and probable evacuations. 4. It is possible but unlikely that it would be recognized that erosion of the service spillway foundation would send enough debris into the river to raise tailwater and render the power outlets and low level river outlets inoperable. Nevertheless, release capacity of the service spillway would probably be judged sufficient to pass large floods even in a damaged condition. 5. It is possible but unlikely that failure of the emergency spillway due to headcutting erosion and undermining would have made it to a “risk driver” potential failure mode. This would be largely driven by the probability of a flood large enough to require discharges over the emergency spillway. It is possible, but unlikely that this potential failure mode would be linked to the service spillway potential failure mode in that discharges would be directed over the emergency spillway at smaller floods due to needed repairs to the service spillway damage. Given the available information, the presence of the shear zones downstream of the emergency spillway and the foundation conditions under the emergency weir would be largely unknown and the assumption that competent rock existed in these locations may have been made. Nevertheless, it would likely be judged that even failure of a few monoliths of the emergency spillway weir would not result in large incremental consequences since the flow would likely be less than that already experienced and evacuations would already have likely taken place (although the reservoir may eventually be lost down to this elevation). 11 So, it is unlikely that all the events that occurred at Oroville would have been envisioned by a USACE risk assessment team. The chance for significant damage would likely have been identified, but the chance of losing control of the reservoir (which never occurred at Oroville) would likely have been estimated to be small as well as the incremental consequences of spillway control structure failure. Therefore, the risks would have also likely been estimated to be small. However, it is also likely that a number of O&M recommendations would have been made to deal with deficiencies in the service spillway chute and underdrain system. While the preceding evaluation is largely conjecture, it is based on participation in a number of risk assessments. PRELIMINARY CONCLUSIONS: 1. The failure of the service spillway chute and the headcutting erosion downstream of the emergency spillway at Oroville Dam (the tallest dam in the U.S.) in 2017 certainly brought renewed attention and concerns related to the potential for this to happen at other dams, perhaps leading to loss of reservoir control. 2. Control of the reservoir at Oroville was never lost largely due to the erosion resistance of deeper unweathered rock at the service spillway and long distance erosion would need to progress or headcut in order to threaten the control structure. However, conservative evacuations were made, based on the only inundation maps available (failure during a PMF event) when undermining of the emergency spillway seemed possible. 3. A previous spillway chute failure at Big Sandy Dam in Wyoming in the 1980’s led to research in stagnation pressure development due to chute slab offsets and joint/crack openings. These findings and methods have been incorporated into the “Best Practices in Dam and Levee Safety Risk Analysis” used by USACE. 4. In addition to stagnation pressure jacking of spillway chute slabs, erosion of the underlying foundation material is an important consideration. Erosion can take place beneath concrete spillway chute slabs leaving voids and the potential for collapse or stagnation pressure failure. Headcutting erosion can progress upstream undermining the control structure for unlined emergency spillways. Best Practices contains information and methodology for evaluating the erosion potential of rock and soil. 5. Every spillway is different, so it is difficult to make general conclusions about how they all will perform. However, the tools are in place to evaluate risks for these types of potential failure modes, provided the design, performance, geologic conditions, and geotechnical characterization are all properly documented. 6. It should be noted that risk assessments typically look at the potential for uncontrolled reservoir release, which means there could be significant damage and erosion requiring extensive repairs, but the chance for pool release and risk could still be small. 7. The one area where this can change how we look at risk is that typically event trees that end without breach of the reservoir are dead-end branches. Damage to Oroville’s spillway cause significant changes to its operation. This could affect other failure modes, and this scenario is rarely considered. FUTURE CONSIDERATIONS: Although the tools are generally in place to evaluate spillway failure risks, the incidents at Oroville and other places indicate that a renewed emphasis on spillway O&M and evaluation is warranted as follows: 1. If concrete spillway chutes show offsets that would project into the flow at joints or cracks, the offsets should be ground down prior to flood season to help reduce the chance of stagnation pressure failure. More guidance is needed on this. Preferred detail(s) of the post grinding shape of joints and/or cracks should be developed and provided to the Districts. Guidance is 12 also needed for caulking or filling the joint/crack. How can this best be done to avoid it getting ripped out, and if it does, what are the alternatives to fill it back in? Its purpose is to keep sand/gravel or ice from jacking the joint open, not save the world. 2. Any spillway drain flows should be monitored for transport of fine grained soil or rock particles, and the origin of the flows should be determined. Additional guidance should be developed and provided to the Districts on this since most spillway under drains systems are complicated as are "monitoring" and access (especially during operations). Since some drains are critical and others are not, guidance for evaluation drains similar to what Reclamation provides for embankment drains would be beneficial. Some need to have access established for monitoring and maintenance, and some just need to be video inspected. 3. Spillway chute designs should be reviewed for areas of weakness that may present problems under large releases. Include consideration of performance and condition assessment. Consider where the slabs are distressed and cracked in comparison to where defenses were built (water stops, drains, anchors, etc.). Is distress just surficial weathering or related to an underlying issue? 4. Concrete chute slabs should be routinely sounded using a hammer to locate hollow sounding areas that might represent a void that needs to be investigated. In cases of severe cracking, Ground Penetrating Radar should be considered as a means to locate such voids. In critical cases, surveys (even LIDAR) can be used to compare existing elevations to theoretical or as built elevations if you have them. At Bull Lake, simple surveys showed the slab had heaved up the exact amount of the depth of the underlying void across the entire chute. Yet the void was blamed on internal erosion. 5. Inundation maps should be prepared for spillway control structure failures so that appropriate evacuations can be made, if necessary (it is believed this is now being done by the Modeling Mapping and Consequences (MMC) center. 6. Attention should be given to these issues and related potential failure modes during Periodic Inspections (PI’s) and Periodic Assessments (PA’s). Case histories, such as Oroville (and others) should form and important part of these evaluations. 7. Specific inspection and maintenance procedures to detect and repair spillway deficiencies should be developed. 8. A screening review of Periodic Assessments (PA) completed to date should be performed to identify dams that may have overlooked these Potential Failure Modes (PFM). 9. A pre-inspection of critical spillway chutes should be made before each operation where possible. The timing could vary from months ahead due to large snow packs to the day before due to a projected thunderstorm. Inspections should commence before spillway operations and continue through operations. The grouted riprap done by the Oroville staff the day before emergency spillway operation was impressive. Depending on when they performed a preinspection, they may have been able to grind an offset or filled in a depressed/cracked slab on the service spillway. 10. Trees growing in spillway wall backfill need to be addressed. These trees might have put root balls into the underdrains, damaged the VCP (these two are common in embankments) or even distressed the walls or slabs. Trees should be removed and the spillway inspected for clogged drains, damage to drains etc. 11. When Potential Failure Modes are evaluated during a risk analysis, events that can be caused by unplanned operation decisions, or due to poor or unexpected performance of another feature on the project, may not be captured. With all of the erodible spillways that USACE has in the portfolio, risk teams should be encouraged to consider whether unanticipated erosion will lead to problems with operating other structures that could threaten the dam or result in uncontrolled release of the reservoir. 13 PREPARED BY: Gregg A. Scott, Lead Civil Engineer, Risk Management Center REVIEWED BY: David A. Margo, Jeffrey A. Schaefer, Charles G. Redlinger, Steven R. Townsley, Dana W. Moses, Nathan J. Snorteland 14