United States Government Accountability Office Report to Congressional Committees April 2019 F-35 JOINT STRIKE FIGHTER Action Needed to Improve Reliability and Prepare for Modernization Efforts GAO-19-341 April 2019 F-35 JOINT STRIKE FIGHTER Action Needed to Improve Reliability and Prepare for Modernization Efforts Highlights of GAO-19-341, a report to congressional committees Why GAO Did This Study What GAO Found In 2018, DOD sent an F-35 aircraft to its first combat mission and started initial operational testing. DOD now plans to spend over $270 billion to buy more than 2,000 F-35 aircraft over the next 26 years. Since 2011, GAO has found the need for more attention to the F-35’s R&M performance to achieve an operationally suitable system. The F-35 program has made slow, sustained progress in improving the aircraft’s reliability and maintainability (R&M). The F-35 aircraft (see figure) are assessed against eight R&M metrics, which indicate how much time the aircraft will be in maintenance rather than operations. Half of these metrics are not meeting targets. While the Department of Defense (DOD) has a plan for improving R&M, its guidance is not in line with GAO’s acquisition best practices or federal internal control standards as it does not include specific, measurable objectives, align improvement projects to meet those objectives, and prioritize funding. If the R&M requirements are not met, the warfighter may have to settle for a less reliable and more costly aircraft than originally envisioned. The National Defense Authorization Act for Fiscal Year 2015 included a provision for GAO to review the F-35 acquisition program until it reaches fullrate production. This is GAO's fourth report under this provision. This report assesses, among other objectives, (1) the program’s progress in meeting R&M requirements (such as mission reliability) and (2) its plans for spending on new capabilities. GAO reviewed and analyzed management reports and historical test data; discussed key aspects of F-35 development with program management and contractor officials; and compared acquisition plans to DOD policies and GAO acquisition best practices. Image of F-35 Aircraft What GAO Recommends GAO is making five recommendations to DOD, including that it identify specific and measurable R&M improvement objectives, align improvement projects, and prioritize resources to meet them. In addition, DOD should complete its business case for modernization before beginning additional development efforts. DOD did not concur with this recommendation, but did concur with the R&M recommendations and plans to take action to address them. View GAO-19-341. For more information, contact Michael J. Sullivan at (202) 512-4851 or sullivanm@gao.gov In 2019, the F-35 program will start modernization efforts—estimated to cost $10.5 billion—for new capabilities to address evolving threats, without a complete business case, or a baseline cost and schedule estimate. Key documents for establishing the business case, such as an independent cost estimate and an independent technology assessment, will not be complete until after the program plans to award development contracts (see figure). Key F-35 Modernization Business Case Documents to Be Completed After Contract Awards Without a business case—consistent with acquisition best practices—program officials will not have a high level of confidence that the risk of committing to development has been reduced adequately prior to contract awards. Moving ahead without a business case puts F-35 modernization at risk of experiencing cost and schedule overruns similar to those experienced by the original F-35 program during its development. United States Government Accountability Office Contents Letter 1 Background Operational Testing Has Started The F-35 Program Is Still Not Meeting All Reliability and Maintainability Targets The F-35 Program Will Start Block 4 Modernization without a Complete Business Case The F-35’s Unit Cost Has Decreased and Its Production Rate Has Increased Conclusions Recommendations for Executive Action Agency Comments and Our Evaluation 16 Appendix I Prior GAO Reports and DOD Actions 31 Appendix II Objectives, Scope, and Methodology 34 Appendix III Status of Selected F-35 Technical Risks 36 Appendix IV Comments from the Department of Defense 40 Appendix V GAO Contact and Staff Acknowledgments 43 Related GAO Products 3 7 11 20 25 27 27 44 Tables Table 1: Planned F-35 Joint Strike Fighter Program Cost and Quantity, 2001–2017 Table 2: The F-35 Reliability & Maintainability Metrics’ Performance as of August 2018 Page i 5 12 GAO-19-341 F-35 Joint Strike Fighter Table 3: Modernization Development Contracting Efforts Planned for 2019 Table 4: F-35 Airframe 2012 and 2016-2018 Deliveries Increased While Other Metrics Varied Table 5: F-35 Engine 2012 and 2016-2018 Deliveries Increased Though Other Metrics Varied Table 6: Select Prior GAO Reports on F-35 Joint Strike Fighter and Department of Defense (DOD) Responses 20 24 25 31 Figures Figure 1: An Image of F-35 Aircraft Figure 2: F-35 Planned and Actual Operational Test Schedule and Delays Since 2012 Figure 3: 2018 Timeline of Major F-35 Events Figure 4: Three Business Case Documents Will Not Be Ready Ahead of the Planned Development Contract Awards Figure 5: The F-35A Negotiated Price Per Aircraft By Production Lot Is Decreasing Figure 6: The Number of F-35 Aircraft in Each Phase of Production Figure 7: The F-35 Helmet Mounted Display Figure 8: F-35B Aircraft Refuel from a KC-130 Aerial Refueling Tanker Using Hose and Drogue Refueling Equipment Figure 9: An F-35B on USS America Page ii 3 7 9 19 21 23 37 38 39 GAO-19-341 F-35 Joint Strike Fighter Abbreviations CAPE C2D2 DOD DOT&E ORD R&M RMIP Cost Assessment and Program Evaluation Continuous Capability Development and Delivery Department of Defense Director, Operational Test and Evaluation Operational Requirements Document Reliability and Maintainability Reliability and Maintainability Improvement Program This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page iii GAO-19-341 F-35 Joint Strike Fighter Letter 441 G St. N.W. Washington, DC 20548 April 29, 2019 Congressional Committees In 2018, the F-35 Lightning II program—also known as the Joint Strike Fighter—saw its first combat mission and began operational testing. The Department of Defense (DOD) is now in its 18th year of developing this family of fifth-generation strike fighter aircraft for the United States Air Force, Marine Corps, and Navy, as well as eight international partners. 1 The F-35’s key capabilities include low-observable, or stealth technology combined with advanced sensors and computer networking capabilities. It is DOD’s largest acquisition program in U.S. military history, with total acquisition costs expected to exceed $406 billion. Currently, the program plans to acquire a total of 2,470 aircraft through fiscal year 2044. Developmental testing for the baseline program is now complete, but the F-35 is still facing affordability and reliability challenges. 2 We have reported on the program’s challenges in the past and made recommendations for improvement. DOD has taken action to address some, but not all, of our recommendations. For a comprehensive list of our recommendations and a summary of DOD’s actions in response, see appendix I. In addition, a list of related GAO products is included at the end of the report. The National Defense Authorization Act for Fiscal Year 2015 included a provision for GAO to review the F-35 program annually until the program 1 The international partners are the United Kingdom, Italy, the Netherlands, Turkey, Canada, Australia, Denmark, and Norway. These nations contributed funds for system development and all but Canada have signed agreements to procure aircraft. In addition, Belgium, Israel, Japan, and South Korea have signed on as foreign military sales customers. 2 The F-35 baseline program, or the baseline acquisition development program, began in 2001 with approval of its first cost and schedule baseline. Developmental testing is intended to provide feedback on the progress of a system’s design process and its combat capability as it advances toward initial production or deployment. Page 1 GAO-19-341 F-35 Joint Strike Fighter reaches full-rate production. This is our fourth report under that provision. 3 In this report, we (1) provide information on the program’s progress toward completing testing of the baseline aircraft; (2) assess the aircraft’s current Reliability and Maintainability (R&M) status; (3) assess the program’s modernization efforts (to add new aircraft capabilities), known as Block 4; and (4) provide information on the program’s production costs and efficiency initiatives. • To provide information on what progress the program has made in testing, we reviewed test event status and schedules, program briefings, and internal DOD briefings. We discussed key aspects of the F-35’s test progress with DOD officials, contractor representatives, and pilots. • To assess the F-35’s R&M, we analyzed monthly contractor R&M reports and compared these to program requirements. We also interviewed DOD officials and contractor representatives. • To assess the program’s modernization efforts, we reviewed documents that should be completed prior to awarding a development contract, according to weapon acquisition best practices identified by GAO. 4 We interviewed DOD officials and contractor representatives regarding the program’s modernization activities and future plans. • To provide information on production progress, we collected and analyzed production performance data from the program office, the prime aircraft contractor, and the prime engine contractor. We determined that all the data we used were sufficiently reliable for the purposes of our reporting objectives. For example, we collected and analyzed the program’s R&M monthly data for calendar year 2018 and corroborated these reports by interviewing contractor representatives and DOD oversight offices such as the Director, Operational Test and Evaluation (DOT&E). In addition, we reviewed official program documentation on the Block 4 efforts and corroborated these with 3 GAO, F-35 Joint Strike Fighter: Development Is Nearly Complete, but Deficiencies Found in Testing Need to Be Resolved, GAO-18-321 (Washington, D.C.: June 5, 2018); F-35 Joint Strike Fighter: DOD Needs to Complete Developmental Testing Before Making Significant New Investments, GAO-17-351 (Washington, D.C.: Apr. 24, 2017); and F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans to Begin Development of New Capabilities, GAO-16-390 (Washington, D.C.: Apr. 14, 2016). 4 GAO, Weapon Systems Annual Assessment: Knowledge Gaps Pose Risks to Sustaining Recent Positive Trends, GAO-18-360SP (Washington, D.C.: Apr. 25, 2018). Page 2 GAO-19-341 F-35 Joint Strike Fighter information officials across DOD involved in the effort, conveyed, such as the Air Force Integration Office and the Naval Air Warfare Division regarding Block 4. Appendix II contains a detailed description of our scope and methodology. We conducted this performance audit from June 2018 to April 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background DOD began the F-35 development program in October 2001 with plans to produce next-generation aircraft to replace aging aircraft in the military services’ inventories. Figure 1 shows the F-35 in flight. Figure 1: An Image of F-35 Aircraft The program has developed and is delivering three variants of the F-35 aircraft; the F-35A conventional takeoff and landing variant for the Air Page 3 GAO-19-341 F-35 Joint Strike Fighter Force, the F-35B short takeoff and vertical landing variant for the Marine Corps, and the F-35C carrier-suitable variant for the Navy. The characteristics of the services’ variants are similar, but each service’s variant also has unique operating requirements. For example, the Marine Corps requires that the F-35B be capable of operating from aircraft carriers, amphibious ships, and main and austere operating bases alike, requiring the ability to conduct short take offs and vertical landings. In March 2005, we found that the F-35 program had started development without adequate knowledge of the aircraft’s critical technologies or a solid design. 5 Further, DOD’s acquisition strategy called for high levels of concurrency between development and production, which runs counter to best practices for major defense acquisition programs. In our prior work, we identified the F-35 program’s lack of adequate knowledge and high levels of concurrency as the major drivers of the program’s significant cost and schedule growth, and other performance shortfalls. 6 Since the development program began in 2001, it has been restructured three times with revised cost and schedule estimates. The most recent restructuring was initiated in 2010 when the program’s cost estimates exceeded certain thresholds established by statute—a condition known as a critical Nunn-McCurdy breach. 7 DOD subsequently certified to Congress in June 2010 that the program was essential to national security and needed to continue. DOD then established a new acquisition program baseline in 2012 that added $162.7 billion to the program’s cost estimate and extended the original delivery schedule by 5-6 years. Since 5 GAO, Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike Fighter Program with Different Acquisition Strategy, GAO-05-271 (Washington, D.C.: Mar. 15, 2005). 6 GAO-05-271; and Joint Strike Fighter: DOD Actions Needed to Further Enhance Restructuring and Address Affordability Risks, GAO-12-437 (Washington, D.C.: June 14, 2012). 7 Section 2433 of title 10 of the United States Code, commonly referred to as NunnMcCurdy, requires DOD to notify Congress whenever a major defense acquisition program’s unit cost experiences cost growth that exceeds certain thresholds. Significant breaches occur when the program acquisition unit cost or procurement unit cost increases by at least 15 percent over the current baseline estimate or at least 30 percent over the original estimate. For critical breaches, when these unit costs increase at least 25 percent over the current baseline estimate or at least 50 percent over the original, DOD is required to take additional steps, including conducting an in-depth review of the program. Programs with critical breaches must be terminated unless the Secretary of Defense certifies to certain facts related to the programs and takes other actions, including restructuring the programs. 10 U.S.C. § 2433a. Page 4 GAO-19-341 F-35 Joint Strike Fighter then, the program’s cost and schedule estimates, as well as the expected number of aircraft to be delivered, have remained relatively stable, as shown in table 1. Table 1: Planned F-35 Joint Strike Fighter Program Cost and Quantity, 2001–2017 October 2001 initial baseline March 2012 baseline Difference from 2001 to 2012 December 2017 estimate Difference from 2012 to 2017 Expected number of aircraft Developmental aircraft 14 14 0 14 0 Procurement aircraft 2,852 2,443 -409 2,456 13 Total aircraft 2,866 2,457 -409 2,470 13 a 2015-2018 2019 7 years 2019 0 years Estimated key dates Initial operational capability Full-rate production 2010-2012 Not determined 2012 Cost estimates (then-year dollars in billions) a Not determined b 5-6 years c Development 34.4 55.2 20.8 55.5 0.3 Procurement 196.6 335.7 139.1 345.4 9.7 2.0 4.8 2.8 5.3 0.5 395.7 162.7 406.1 10.4 Military construction Total program acquisition 233.0 Unit cost estimates (then-year dollars in millions) c, d Program acquisition 81 161 80 164.4 3.4 Average procurement 69 137 68 140.6 3.6 Source: GAO analysis of Department of Defense data. GAO-19-341 a When the baseline was finalized, DOD had not yet identified new initial operational capability dates for the military services. b This is the difference from the October 2001 baseline to the December 2017 estimate. c Annual projected cost estimates expressed in then-year dollars reflect inflation assumptions. We did not assess the reliability of the program office’s cost estimates. Amounts may not sum due to rounding. d The program acquisition and the average procurement unit cost estimates are calculated by dividing the total program acquisition amount by the total quantities and the procurement amount by the procurement aircraft quantities, respectively. Of the F-35’s $406 billion estimated acquisition cost, DOD needs a majority of the funding ($270.3 billion) to purchase aircraft over the next 26 years. Of that future funding, the program plans to spend between $9.6 billion and $14 billion each year through fiscal year 2031. In addition, the program’s sustainment costs to operate and maintain the F-35 fleet over the next 52 years are estimated to be $1.12 trillion. Page 5 GAO-19-341 F-35 Joint Strike Fighter Though the program’s total planned quantities have been relatively stable, the program’s timeframes for procuring these aircraft have changed multiple times. Since the start of development, the program has pushed the procurement of more than half of the total aircraft planned into the future, mostly due to significant concurrency between development and production. Specifically, the program office had originally planned to procure almost 2,000 aircraft by fiscal year 2019. However, according to the current plan, by the end of 2019, the program will have procured just over 500 aircraft. The F-35 baseline aircraft development program was complete in April 2018, when developmental testing concluded. As we reported in June 2018, the program office reported it had met all nine of its capability thresholds—or the minimum acceptable value for each capability—and delivered three of those nine capabilities. 8 However, we also reported that the program has to complete operational testing before DOD can determine if the six remaining capabilities have been delivered. The purpose of operational testing is to assess the effectiveness, suitability, survivability, lethality, and mission capability of the F-35, including the information systems and the air vehicle, in an operationally representative environment. Operational testing includes cyber security assessments, some of which have been conducted. Production of the F-35 began in 2007 while development was in its early stages and before developmental flight testing had started. As a result of this concurrent development, the 357 aircraft delivered through 2018 will need retrofits to fix deficiencies and design issues found during testing. 9 The program’s total estimated cost of concurrency is $1.4 billion. 10 The program office plans for over 500 aircraft to be procured by the time operational testing is completed. Until operational testing is complete, there is a risk that additional problems with the aircraft may be identified. As a result, the concurrency costs of retrofitting delivered aircraft could increase. 8 GAO-18-321. The F-35’s nine capabilities are force protection, net ready, radio frequency signature, combat radius, mission reliability, logistics footprint, sortie generation rates, F35C recovery performance, and F-35B mission performance. 9 This number includes U.S., international partner, and foreign military sales aircraft. 10 This estimate includes deficiencies that may still be identified in operational testing. Page 6 GAO-19-341 F-35 Joint Strike Fighter Operational Testing Has Started The F-35 program started formal operational testing in December 2018 after a 3-month delay. This testing was delayed for two main reasons: (1) to resolve critical deficiencies and (2) to accommodate an unexpected grounding following the crash of an F-35B in September 2018. According to a test official, the program expects to complete testing in December 2019, about three months later than planned due to delays with the simulator that is used for more complex testing. Figure 2 shows the program’s planned end to developmental testing and planned timeframes for operational testing for 2012 and the past four years and the delays the program has realized each year since the program was re-baselined in 2012. Figure 2: F-35 Planned and Actual Operational Test Schedule and Delays Since 2012 The operational testing was delayed for the following two main reasons. Resolution of deficiencies: First, before the program could begin operational testing, it had to resolve critical deficiencies with the aircraft that were identified during development testing. The program categorizes deficiencies according to their potential impact on the aircraft’s performance. Page 7 GAO-19-341 F-35 Joint Strike Fighter • Category 1 deficiencies are considered critical and could jeopardize safety, security, or another requirement. • Category 2 deficiencies are those that could impede or constrain successful mission accomplishment. In January 2018, the F-35 program had 966 open deficiencies—111 category 1 and 855 category 2. At that time, the program planned to move forward before resolving all of them. In June 2018, we recommended that the Secretary of Defense direct the F-35 program to resolve all these deficiencies before the program’s October 2019 full-rate production decision. 11 According to DOD officials, over the past year, the program has made progress in reducing the number of open deficiencies by resolving, re-categorizing, closing, or combining them. For example, in 2018, the program resolved nearly 50 category 1 deficiencies and recategorized over 50 others to category 2. As a result, the program received approval from the Under Secretary of Defense for Acquisition and Sustainment to begin formal operational testing with 13 category 1 deficiencies and almost 900 category 2 deficiencies. According to the Program Executive Officer, none of the open category 1 deficiencies are a safety of flight concern, and all of them have operational workarounds. A current example of an open category 1 deficiency is with lines on the F35’s landing gear, which can rupture when a tire blows, potentially causing loss of a major aircraft system such as the brakes. Such an event requires some repair work to the landing gear, but contractor officials explained that it is not a safety concern. According to the program office, it is not a safety concern because the current workaround for this deficiency is pilot training to avoid braking on the side of the blown tire. Program test officials said that testing with deficiencies is not uncommon and they will continue to work to address them, but some may not be fully resolved for several years. Unexpected grounding: In October 2018, the F-35 fleet was grounded after the program identified a manufacturing fault with an engine fuel tube—a component in the F-35 engine produced by Pratt & Whitney. The fault was found in an inspection that stemmed from an F-35B crash in September 2018. This was the first crash of an F-35. Of the 23 operational test aircraft, the program replaced the fuel tubes on 18 aircraft 11 GAO-18-321. Page 8 GAO-19-341 F-35 Joint Strike Fighter by the start of operational testing in December 2018, which contributed to the 3-month delay. This and other key technical risks are described in more detail in appendix III. In addition to starting operational testing and the unexpected grounding, the program and the airframe contractor Lockheed Martin experienced other major events over the past year, as shown in figure 3. Figure 3: 2018 Timeline of Major F-35 Events For example, the United States completed its first F-35 combat mission in September 2018 when an F-35B successfully hit a target in Afghanistan. The program took steps to mitigate delays to the start of operational testing. For example, the program office, in coordination with DOT&E, received approval to conduct some preoperational testing events starting in January 2018, before the official start date in December. According to DOT&E officials, the outcome of these preoperational test events should count towards the completion of operational testing. This included cold weather testing in Alaska, which took advantage of appropriate weather conditions. Page 9 GAO-19-341 F-35 Joint Strike Fighter Despite the 3-month delay, program officials stated that they consider the F-35 operational test schedule to be adequate for addressing schedule risks, which pertain to unresolved deficiencies and potential problems with the availability of test and support aircraft, ground systems, test ranges, and necessary test models and simulations. According to a test official, as of April 2019, some of these risks have been realized, such as the delay with the simulator, and as a result, the end of operational testing is now planned for December 2019. In addition, there is the possibility of new deficiencies emerging from operational testing. • Unresolved deficiencies: Existing or new deficiencies could negatively affect test results. According to DOT&E officials, since the start of operational testing, four new category 1 deficiencies have been identified, bringing the total to 17. According to DOD officials, it would not be unexpected during the course of operational testing for the program to discover additional deficiencies that may require resolution and re-testing. • Availability of test and support aircraft: According to test officials, F-16s and F-18s are needed to represent adversaries during F-35 operational tests. These assets may not be available because they also support other test programs. According to officials, the F-35 program does not have control over the availability of these aircraft and must work with the Navy and Air Force to negotiate their use. In addition, the limited availability of F-35 test aircraft, in part due to R&M issues and shortages of replacement parts, may also pose a challenge to completing test events, according to officials. • Availability of ground systems: Ground systems required for operational testing, such as the DOT&E developed Radar Signal Emulators, are late in development and may not be available when required. According to DOT&E officials, the emulators imitate modern threat radar capabilities of adversarial nations but their integration with the test range is approximately a year behind schedule. The program is currently using other threat simulators. DOT&E officials stated that they are working to have the radars ready by the spring of 2019, when needed. • Availability of test ranges: Test officials at Edwards Air Force Base expressed concern about the availability of test ranges, which the F35 program shares with other programs. According to test officials, the F-35 was the fifth in line, in terms of priority, to use the range at Edwards Air Force Base, as of October 2018. DOT&E officials, however, stated that they did not observe any range availability issues during the F-35’s first month of operational testing. Page 10 GAO-19-341 F-35 Joint Strike Fighter • Availability of test models and simulations: According to program officials, the program’s testing simulator, which runs the F-35’s mission systems software and provides test scenarios that cannot be replicated in a real-world environment, will not be complete until at least November 2019. Completion of the testing simulator was originally scheduled for the end of 2017. Any additional delays in operational testing could affect another upcoming program decision: DOD’s decision to begin full-rate production in December 2019. 12 This decision is typically made after operational testing is completed. The F-35 Program Is Still Not Meeting All Reliability and Maintainability Targets The F-35 program has made slow, sustained progress in improving the F35’s R&M. R&M determines the likelihood that the aircraft will be in maintenance rather than available for operations. Each F-35 aircraft variant is measured against eight R&M metrics, four of which are in part of the contract. All F-35 variants are generally performing near or above targets for half of the R&M metrics while the other four are still falling short, which is the same as last year. While the program is on track to meet the targets for half of the metrics, the program has not taken adequate steps to ensure the targets for the others will be met. While DOD has an action plan to improve R&M, its guidance does not define specific, measurable objectives for what the desired goals for the F-35’s R&M performance should be. Furthermore, the program office has not prioritized funding for projects that will improve the R&M metrics that are not meeting their targets. The F-35 Program Is Meeting, or Close to Meeting, Half of Its Targets All F-35 variants are measured against eight R&M metrics’ targeted performance levels, and all variants are generally performing near or above targets for four of the eight R&M metrics. 13 This represents little change from their overall performance last year. All eight R&M metrics are described in the program’s Operational Requirements Document (ORD)—the document that outlines the requirements DOD and the 12 Full-rate production is a decision, following the completion of operational testing, to scale up production. 13 According to officials, the current targets are where the program expects the metrics to be based on the number of hours flown. The minimum target is the value each metric should eventually meet by the time each variant has flown enough hours to reach maturity. Page 11 GAO-19-341 F-35 Joint Strike Fighter military services agreed the F-35 should meet. However, in December 2018, DOT&E reported that, although performance for the four underperforming metrics has shown slow growth over the years, none of these metrics were meeting interim goals needed to reach requirements at each variant’s maturity. 14 Each F-35 variants’ R&M performance against these metrics’ targets is shown in table 2. Table 2: The F-35 Reliability & Maintainability Metrics’ Performance as of August 2018 Contractually required F-35A F-35B F-35C Mission Reliability —measures the probability of successfully completing a mission of average duration ✔ ● ● — Mean flight hours between failure (design controlled)—measures time between failures that are directly attributable to the design of the aircraft and are considered fixable with design changes ✔ ● ● ● Mean time to repair—measures the amount of time it takes a maintainer to repair a failed component or device ✔ ◓ ◓ ◓ Maintenance man hours per flight hour—measures the average amount of time spent on scheduled and unscheduled maintenance per flight hour ✔ ● ◓ ● Mean flight hours between maintenance events—also referred to as the logistics reliability metric, measures time between maintenance, c unscheduled inspections, and servicing actions, including consumables — ○ ○ ◓ Mean flight hours between removals—measures the time between part removals from the aircraft for replacement from the supply chain — ○ ○ ○ Mean flight hours between critical failure—measures the time between failures that result in the loss of a capability to perform a mission-critical capability — ○ ○ ○ Mean corrective maintenance time for critical failure—measures the amount of time it takes to correct critical failure events — ○ ○ ○ Metric a b Legend: ●: Metric is at or above current targets ◓: Metric is at or above minimum targets ○: Metric is below minimum targets ✔: Metric is contractually required —: not available Source: GAO analysis of contractor data. GAO-19-341 14 Director, Operational Test & Evaluation, Fiscal Year 2018 Annual Report (December 2019). The F-35 aircraft reach maturity when all variants have flown a combined 200,000 hours, with each variant flying at least 50,000 hours. The F-35A reached its planned maturity in July 2018, but is still not meeting four of its eight metrics. The F-35B and C variants have more time to meet their metrics before they reach their planned maturity in 2021 and 2024 respectively. Page 12 GAO-19-341 F-35 Joint Strike Fighter a Each metric is measured using a 3-month average and reported on a monthly basis; this table summarizes the Joint Reliability and Maintainability Evaluation Team’s review of reliability growth and maintainability improvement data from November 2009 through August 2018. b Mission Reliability is a key performance parameter. Mission reliability, as well as performance against the targets related to all of these metrics, will be evaluated during initial operational test and evaluation. c Consumable parts are nonrepairable items or repair parts that can be discarded more economically than they can be repaired or that are consumed in use (such as oil filters, screws, nuts, and bolts). Since the program began tracking R&M performance in 2009, the program has seen small, annual improvements. Over the past year, all variants showed a slight improvement in targeted performance levels for one metric, the mean flight hours between failures (design controlled), but saw little or no discernable improvement for the four metrics not meeting targets. However, based on current performance, the program does not expect to meet those targets by full aircraft maturity. According to F-35 program officials, the ORD R&M metrics should be re-evaluated to determine more realistic R&M performance metrics, but they have not yet taken actions to do so. Until it does so, the program office remains accountable for ensuring those ORD R&M metrics are achieved. In June 2018, we recommended that the F-35 program identify what steps it needs to take to ensure the F-35 aircraft meet R&M requirements before each variant reaches maturity and update its R&M Improvement Program (RMIP)—DOD’s action plan for prioritizing and funding R&M improvement projects—with these steps. 15 DOD concurred with our recommendation but has yet to take substantive actions to address it. It did, however, complete 16 improvement projects since we last reported on this. Despite completing these projects, there were not significant gains in the R&M metrics not meeting targets. Program officials advised, however, that measurable improvements in R&M can take time to manifest. To speed this process, the program is accelerating planned upgrades to older aircraft where appropriate, which officials stated should translate to an overall improvement in the program’s R&M performance. 15 GAO-18-321. The F-35 program began tracking its R&M performance in 2009 and documented the RMIP’s approach in April 2014. In June 2018, we found that the F-35 program did not have a plan to ensure all R&M targets will be met by full aircraft maturity. Page 13 GAO-19-341 F-35 Joint Strike Fighter The F-35 Program Office’s Improvement Plan Does Not Address UnderPerforming Metrics The F-35 program office has estimated that implementing all of the identified improvement projects currently contained in its RMIP could result in potential life cycle cost savings of over $9.2 billion by improving the F-35’s R&M. As of December 2018, the guidance the F-35 program office has used to implement the RMIP does not define specific, measurable objectives for what the desired goals for the F-35’s R&M performance should be or align improvement projects with R&M goals. Furthermore, the RMIP has not been a funding priority. Federal internal control standards state that programs should define objectives when implementing programs such as the RMIP. 16 Although the F-35 program RMIP’s guidance has a general goal of improving R&M, it does not identify achieving the targets for the eight R&M metrics the program tracks as an objective. Program officials acknowledged that the RMIP’s guidance does not include such an objective. Instead, officials are using the RMIP to prioritize and fund projects that will improve aircraft availability and mission capability—neither of which are included in the eight R&M metrics, but are necessary and important initiatives. 17 Officials stated that by prioritizing these projects, they will eventually improve performance under all R&M metrics, including the four that are not meeting targets. The RMIP’s guidance, however, does not discuss these priorities or align improvement projects with the eight R&M metrics. In our prior work on weapon system acquisitions, we have identified a number of best practices for improving program outcomes, such as clearly establishing well-defined requirements and securing stable funding 16 GAO’s Standards for Internal Control in the Federal Government require agencies to define measurable objectives when implementing programs. Agencies should also consider requirements when defining these objectives. Objectives should be defined in measurable terms so that performance toward achieving those objectives can be assessed. Measurable objectives are generally free of bias and do not require subjective judgments to dominate their measurement. Measurable objectives are also stated in a quantitative or qualitative form that permits reasonably consistent measurement. GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). 17 Aircraft availability (also known as air vehicle availability) and mission capability both measure the percentage of time during which aircraft are safe to fly, available for use, and able to perform at least one tasked mission. The air vehicle availability metric assesses all aircraft in the fleet, including those in the possession of the F-35 units and those at the depots for modifications. The mission capability metric assesses only aircraft that are in the possession of F-35 units. Page 14 GAO-19-341 F-35 Joint Strike Fighter that matches resources to requirements. 18 The F-35 program office has not prioritized or dedicated funding in its budget to improve R&M in part because program officials explained that they have been focused on initiatives intended to lower the cost of the aircraft. Further, any current funding for R&M improvement projects comes from the program’s operation and maintenance funds, which are only available for one fiscal year. Officials further explained that, if such funding runs out or is used by the program for other efforts, then R&M projects will go unfunded or be suspended until new funding is available. In fiscal year 2018, for example, while some R&M improvement projects were completed, several other projects were suspended when that year’s funding ran out. According to officials, these projects may not be started back up until fiscal year 2019. In addition, most of the R&M improvement projects that were approved in fiscal year 2018 were not funded. For example, as of December 2018, according to a contractor representative, all of the identified improvement projects currently unfunded in the program’s RMIP would cost about $30 million to implement, but are on hold and waiting to be funded. Program officials stated that they are in the process of revising the RMIP and have considered including more specific objectives, such as a focus on improving aircraft availability and mission capability and a focus on improving R&M performance where the ORD R&M metrics’ targets are not being met. Additionally, in its 2019 annual lifecycle sustainment plan, the program office noted that a dedicated annual budget for R&M improvement projects would benefit the program. According to the program, any revisions to the RMIP and changes to how it will be funded, however, will not be complete until April 2019 or later. Without defining measurable objectives in its RMIP guidance for meeting all eight R&M metrics and aligning which improvement projects will ensure those metrics are met, the program is at risk of not fully meeting its R&M goals. Further, without prioritizing funding for improving R&M, projects may continue to be either prematurely suspended or never get underway. As a result, the warfighter may accept aircraft that (1) are less reliable than originally described in the program’s ORD, and (2) have operation and sustainment costs that may raise affordability questions. 18 GAO Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.: Mar. 8, 2001). Page 15 GAO-19-341 F-35 Joint Strike Fighter The F-35 Program Will Start Block 4 Modernization without a Complete Business Case With development of the baseline program complete, the program is transitioning to early development and testing for modernization efforts known as Block 4, which are expected to cost about $10.5 billion. The F35 program plans to award Block 4 development contracts starting in May 2019, before completing a business case—a baseline cost and schedule estimate to track the program’s performance going forward. In doing so, the program will commit resources without adequate knowledge of Block 4’s full cost, schedule, and level of technology maturity, putting Block 4 at risk of experiencing cost and schedule overruns similar to those experienced by the baseline program during its development. The F-35 Program Is Transitioning to Early Block 4 Development and Testing The National Defense Authorization Act for Fiscal Year 2017 required DOD to submit a report containing certain elements of an acquisition program baseline—in essence, a full program business case—to include the cost, schedule, and performance information for Block 4. In 2018, we found that DOD’s report to Congress was incomplete but included information on some elements of the Block 4 acquisition program baseline.19 In its report, DOD stated that the acquisition program baseline would continue to be refined over the next year. As a result, we presented a matter for congressional consideration to restrict Block 4 funding until the program established a complete business case. DOD’s report to Congress also outlined the F-35 program office’s new development approach to deliver Block 4 capabilities—new requirements beyond the baseline aircraft capabilities to address evolving threats. As we reported in June 2018, this new approach, meant to deliver capabilities to the warfighter faster, is referred to as Continuous Capability Development and Delivery (C2D2).20 This approach consists of 6-month development cycles in which small groups of capabilities will be developed, tested, and delivered as they are matured. In January 2018, the F-35 program started using this C2D2 approach to develop and test software updates to address deficiencies identified during testing. According to the contractor, the first two software updates also established a foundation for new Block 4 capabilities to be fully developed later. According to program officials, as of December 2018, the program has executed contract actions valued over $1.4 billion to 19 GAO-18-321. 20 GAO-18-321. Page 16 GAO-19-341 F-35 Joint Strike Fighter establish testing facilities and support early Block 4 development of capabilities the program plans to deliver through 2024. According to DOD’s January report, results from this work will help the program inform its Block 4 business case. The F-35 Program Will Start Block 4 Development without a Full Business Case The F-35 program plans to award Block 4 development contracts without knowledge of the effort’s full cost or the maturity of critical technologies. Over the past year, the program has been working to complete its business case for Block 4, including incorporating Block 4 activities into its acquisition strategy—which was approved in October 2018. However, three key Block 4 business case documents will not be ready before the program’s planned May 2019 contract awards for development efforts. • Independent technology readiness assessment: Although the contracts for Block 4 development efforts are planned to be awarded in May 2019, the program will not conduct an independent technology readiness assessment by that time. A technology readiness assessment is a systematic, evidence-based process that evaluates the maturity of hardware and software technologies critical to the performance of a larger system or the fulfillment of the key objectives of an acquisition program. According to a program official, the program will conduct its own assessments on a rolling basis as initial capabilities are developed. The official stated that technologies will not be integrated into the aircraft until they are adequately mature. The program office plans to conduct a partial assessment of initial capabilities sometime between October and December 2019 with additional assessments to follow. However, without an independent technology readiness assessment, the program has not identified potential critical technology elements and as a result, may be at risk of delaying the delivery of new capabilities. • Test and evaluation master plan: Although the F-35 program has begun testing Block 4 capabilities, it does not have an approved test and evaluation master plan. The test and evaluation master plan documents the overall structure, strategy, and objectives of the test program as well as the associated resources needed for execution. It provides a framework for the program office to provide detailed test plans and subsequently determine the resources needed. Test officials have expressed concerns about the lack of an approved test plan, uncertain funding, the number of test aircraft available, and the draft test schedule, among other things. Officials were also concerned as to whether the Block 4 test aircraft would be in the same configuration as fielded aircraft, which are in earlier configurations Page 17 GAO-19-341 F-35 Joint Strike Fighter than the test fleet. Further, DOT&E stated in its annual report that it considers the current Block 4 schedule to be high risk due to the large amount of planned capabilities that will be developed and tested in 6month development cycles. 21 An approved, properly resourced test plan is essential for planning and preparing for adequate testing of the Block 4 capabilities. Without an approved test and evaluation master plan, the F-35 program is providing the test authorities with capabilities to be tested without giving them the necessary direction on how to adequately prepare to conduct the tests. Specifically, test officials stated the F-35 program office has not provided details on which capabilities are planned for each testing development cycle making it difficult to execute testing. While this is still a concern, F-35 program officials explained that over the past 3 months they have been providing the test authorities with the direction needed to conduct testing. • Independent cost estimate: The Block 4 independent cost estimate, which details the program’s total estimated life cycle cost, is not complete. In August 2017, we reported that DOD estimated the development funding needed for the first phase of modernization for Block 4 to be over $3.9 billion through 2022. 22 Since then, the program incorporated more scope and fidelity into the Block 4 cost estimate, which has increased to $10.5 billion for Block 4 capabilities planned through 2024. The program office has provided its Block 4 cost estimate to the Cost Assessment and Program Evaluation office (CAPE) for an independent cost estimate. 23 According to CAPE officials, they will provide the independent cost estimate between October and December 2019 to support the program’s full-rate production decision, but this would occur several months after the program plans to award the Block 4 development contracts. Without an independent cost estimate, Congress does not have insight into the full potential cost of the Block 4 effort. 21 Director, Operational Test & Evaluation, Fiscal Year 2018 Annual Report (December 2018). 22 GAO, F-35 Joint Strike Fighter: DOD’s Proposed Follow-on Modernization Acquisition Strategy Reflects an Incremental Approach Although Plans Are Not Yet Finalized, GAO-17-690R (Washington, D.C.: Aug. 8, 2017). 23 The Director of CAPE shall ensure that the cost estimation and cost analysis processes of the DOD provide accurate information and realistic estimates of cost for the acquisition programs of the Department of Defense including conducting independent cost estimates and cost analyses for all major defense acquisition programs. 10 U.S.C. § 2334. Page 18 GAO-19-341 F-35 Joint Strike Fighter The expected completion dates for these documents are between October and December 2019, at the earliest. Figure 4 shows key Block 4 dates, the planned development contract awards, and planned completion dates for the remaining business case documents. Figure 4: Three Business Case Documents Will Not Be Ready Ahead of the Planned Development Contract Awards Major defense acquisition programs generally follow DOD acquisition policy, which states that prior to the release of a development contract request for proposal, program officials should have confidence that program requirements are firm. Program officials should also clearly state that the risk of committing to development has been reduced or will be adequately reduced prior to contract award. According to best practices identified by GAO, without several of the business case documents completed, program officials cannot have a high level of confidence that the requirements are firm and that the risk to committing an estimated $10.5 billion in funding to Block 4 has been adequately reduced. 24 According to program officials, business case documents have not been completed because they took a step back to re-examine their approach and the cost estimate for Block 4 that DOD established in 2017. Counter 24 GAO-18-360SP. Page 19 GAO-19-341 F-35 Joint Strike Fighter to acquisition best practices, the program plans to initiate additional development work before they acquire the requisite knowledge of the necessary levels of technology maturity and funding. Program officials have reported the planned modernization contracting efforts shown in table 3. Table 3: Modernization Development Contracting Efforts Planned for 2019 Contracting effort Planned award date a Development of the Dual Capable Aircraft for the F-35A May 2019 Development of early Block 4 capabilities for a systems engineering review and activities associated with developmental flight testing May 2019 Source: GAO presentation of F-35 program office information. GAO-19-341 a The Dual Capable Aircraft is funded separately from the F-35 program. DOD plans to integrate this program into the F-35 Block 4. Dual Capable Aircraft refers to the capability to carry conventional and non-conventional weapons. If program officials move ahead with awarding Block 4 contracts without gaining the knowledge that a full business case could would provide, Block 4 modernization efforts will be at risk of experiencing the same kind of cost and schedule growth the baseline development program experienced. The F-35’s Unit Cost Has Decreased and Its Production Rate Has Increased With a few exceptions, the negotiated prices for all F-35 variants have generally been decreasing with each production lot, and more aircraft are being procured in each lot. In particular, the F-35A’s price has decreased in each subsequent production lot, with the most recent price per aircraft at $89 million in lot 11, as shown in figure 5 below. Page 20 GAO-19-341 F-35 Joint Strike Fighter Figure 5: The F-35A Negotiated Price Per Aircraft By Production Lot Is Decreasing In 2018, we reported that while the F-35 program faces affordability challenges, it was investing in several projects to reduce production and sustainment costs. 25 According to DOD, to improve production affordability, the F-35 program office is continuing to make investments to lower the price of an F-35A to below $80 million by lot 13. To realize this goal, the F-35 program office and the prime contractor are increasing the production rate and investing in various initiatives to lower production costs. For example: • According to the program office, it has invested a total of $320.3 million in efforts to improve manufacturing processes that it estimates could result in up to $7.9 billion in savings over the life of the program. In addition, the prime contractor has invested $90 million and plans to invest an additional $25 million to lower its production costs. • DOD issued a contract announcement for economic order quantity purchases for use in production lots 13-14. This approach involves 25 GAO-18-321. Page 21 GAO-19-341 F-35 Joint Strike Fighter making large purchases of components that will be used across multiple procurement lots of aircraft to reduce production costs by buying components in bulk and achieving economies of scale. The program had expected $1.2 billion in cost savings from this effort, but according to estimates from the CAPE, cost savings will more likely be $595 million. In addition, according to program officials, once the program achieves full-rate production, it plans to utilize a multi-year procurement strategy, beginning in fiscal year 2021. This strategy is intended to have similar benefits as the economic order quantity purchases by providing industry with a stable, long-term demand. 26 According to Pratt & Whitney, the cost of the engine is also declining. For example, the price of the F-35A and C engine dropped by $100,000 per engine over the past year. The most recent negotiated price is $11.9 million per engine. The F-35 airframe and engine contractors saw a significant increase in their production rates in 2018, but faced some production challenges as well. The airframe contractor—Lockheed Martin—increased its production rate by 50 percent and delivered a total of 91 aircraft in 2018, with a total of 267 aircraft on its production floor or in contract negotiations as of December 2018, as shown in figure 6. 26 Multi-year contracting is a special contracting method to acquire known requirements, in quantities and total costs not over planned requirements, for up to 5 years. Page 22 GAO-19-341 F-35 Joint Strike Fighter Figure 6: The Number of F-35 Aircraft in Each Phase of Production In addition, Lockheed Martin delivered more aircraft on time. In 2012, none of the planned aircraft deliveries were on time whereas in 2018, 58 percent were on time. To incentivize the contractor to improve on-time deliveries, the program office has added a performance incentive fee to the lot 11 production contract. Table 4 shows some improvements in Lockheed Martin’s production metrics since 2012 and over the past 2 years. Page 23 GAO-19-341 F-35 Joint Strike Fighter Table 4: F-35 Airframe 2012 and 2016-2018 Deliveries Increased While Other Metrics Varied Metric 2012 2016 2017 2018 Aircraft delivered 29 46 66 91 Aircraft delivered late 29 29 28 38 Percentage of aircraft delivered late out of total 100 63 42 42 Average labor hours per F-35A delivered 108,355 47,269 41,541 38,436 Average labor hours per F-35B delivered 107,998 61,928 57,152 55,266 Average labor hours per F-35C delivered NA 65,187 60,121 68,040 Average total hours for scrap, rework, and repair per F-35A 14,332 6,590 5,645 5,156 Average total hours for scrap, rework, and repair per F-35B 17,165 8,420 7,510 7,914 Average total hours for scrap, rework, and repair per F-35C NA 9,552 7,866 11,725 Source: Lockheed Martin data. GAO-19-341 Between 2012 and 2017, Lockheed Martin saw some improvement for all variants’ production metrics, with the F-35A showing improvements through 2018. However, over the past year, several metrics for the F35Bs and F-35Cs saw a decline. According to Lockheed Martin, it faced several challenges with the increased production rate which led to these declines. • For example, since January 2018, the contractor hired around 900 new personnel, nearly 30 percent of its workforce, all of whom needed training. According to officials, this influx of new personnel led to an increase in the average labor hours for the F-35C and the number of hours required for scrap, rework, and repair of the F-35B and F-35C. According to the contractor, as the newly hired personnel gain more experience in the production processes, the average labor hours it takes to build an F-35C should start decreasing again. • The contractor faced several production quality issues and parts delays, which it worked to address over the past year. For instance, we reported last year that due to a fault in the production process, Lockheed Martin halted deliveries after the Air Force identified corrosion between the aircraft’s surface panels and the airframe because Lockheed Martin did not apply primer when the panels were attached. 27 The program office stated that Lockheed Martin and the F35 Program Executive Officer reached a mutual agreement on the 27 GAO-18-321. Page 24 GAO-19-341 F-35 Joint Strike Fighter cost to resolve this issue, the details of which have not been disclosed publicly. • With the production rate increase, the supply chain was strained to deliver parts on time, which led to increases in material shortages for key components, such as the radar. Pratt & Whitney has also increased production over the past year and has shown similar manufacturing performance for the F-35 engine as in past years; however, it had fewer on-time deliveries in 2018 due to the challenges it faced, including an increase in the average number of quality issues per engine. Pratt & Whitney’s production rate increased by 10 percent over the past year, with 81 engines delivered in 2018. Table 5 shows the trends in Pratt & Whitney production metrics’ performance. Table 5: F-35 Engine 2012 and 2016-2018 Deliveries Increased Though Other Metrics Varied Metric 2012 2016 2017 2018 Engines under contract 35 60 73 107 Engines delivered 48 65 73 81 Engines delivered late 41 38 35 70 Percent of engines delivered late out of total 85 58 48 86 1,555 1,239 1,272 1,106 449 183 228 198 811 642 777 941 Average labor hours per engine Average labor hours for scrap, rework, and repair per engine Average number of quality notifications per engine a Source: GAO presentation of Pratt & Whitney data. GAO-19-341 a A small sub-set of parts drive a majority of Pratt & Whitney’s quality notifications per engine. According to Pratt & Whitney, its late engine deliveries increased in 2018 partially due to a subcontractor that did not have all of the needed tooling in place to produce more F-35B engines. To address this and other issues causing the late deliveries, Pratt & Whitney is taking lessons learned from its other production facilities and applying them to the F-35’s engine production. Conclusions The F-35 program has overcome significant hurdles in its 18 years of development of the baseline aircraft, which was completed last year. One recent hurdle that it overcame was resolving many critical deficiencies found during developmental testing, which allowed the program to begin operational testing this past December. Other hurdles remain, including with the F-35’s reliability and maintainability (R&M). Four of the eight Page 25 GAO-19-341 F-35 Joint Strike Fighter R&M metrics continue to fall short of meeting performance targets. Program officials stated that the Operational Requirements Document (ORD) R&M targets need to be re-evaluated to determine more realistic R&M performance metrics but have not yet taken actions to do so. Until the program re-evaluates the targets, it is accountable for achieving those requirements. Furthermore, funding improvement efforts have not been a priority for the program. As a result, over the past year, some projects were started, several were halted while underway, and others are on hold, waiting for funding. As long as targets under all of the R&M metrics continue to fall short, the U.S. military services and the taxpayer will have to settle for aircraft that are less reliable and more costly to maintain than originally planned. Also, with continuing concerns about the program’s long-term affordability, the program is missing a prime opportunity to infuse affordability into the aircraft’s future with better R&M performance. As the program is considering revisions to its R&M Improvement Program (RMIP), it is in a good position to clearly define and communicate its R&M objectives for the aircraft to meet the targets under all of its eight R&M metrics. Until it does so, the program office will not know whether the steps it is taking now are sufficient to ensure each F-35 variant achieves its R&M requirements in the future. As we have reported in the past, the F-35 program started its development before it was ready. It is now at risk of doing the same thing with the Block 4 modernization effort. Since we last reported in June 2018, the program has still not established a solid business case to commit funding and other resources to developing new capabilities for the aircraft. This could result in the program delivering technologies late and over cost estimates. Finally, the program has committed a significant amount of funding to support Block 4, but it has not completed an independent cost estimate of the life-cycle cost. Consequently, Block 4 may follow in the footsteps of the F-35’s baseline program which saw significant cost and schedule growth during its development. This approach leaves the F-35 program, DOD, Congress, and the U.S. military services without key information to make decisions regarding Block 4. Page 26 GAO-19-341 F-35 Joint Strike Fighter Recommendations for Executive Action We are making the following five recommendations to the Department of Defense: The Secretary of Defense should ensure that the F-35 program office assesses whether the ORD R&M targets are still feasible and revise the ORD accordingly. (Recommendation 1) The Secretary of Defense should ensure that the F-35 program office, as it revises its RMIP, identifies specific and measurable R&M objectives in its RMIP guidance. (Recommendation 2) The Secretary of Defense should ensure that the F-35 program office, as it revises its RMIP, identifies and documents which RMIP projects will achieve the identified objectives of the RMIP guidance. (Recommendation 3) The Secretary of Defense should ensure that the F-35 program office prioritizes funding for the RMIP. (Recommendation 4) The Secretary of Defense should ensure that the F-35 program office completes its business case, at least for the initial Block 4 capabilities under development, before initiating additional development work, to include: an independent cost estimate; an approved test and evaluation master plan which addresses resources, aircraft shortfalls, and funding; and an independent technology readiness assessment. (Recommendation 5) Agency Comments and Our Evaluation We provided a draft of this report to DOD for review and comment. Our initial draft report contained only recommendations 2 through 5 above. During the comment period, DOD officials provided additional information about the program’s R&M performance concerning whether the ORD targets continue to be feasible and should be re-examined. As a result, we added our first recommendation above—that the F-35 program office assess whether the ORD R&M targets are still feasible and revise the ORD accordingly. DOD provided written comments on our report, which are reprinted in appendix IV. DOD concurred with our four recommendations on R&M but did not concur with our last recommendation on the Block 4 modernization. DOD also provided technical comments, which were incorporated as appropriate. Page 27 GAO-19-341 F-35 Joint Strike Fighter In concurring with our four R&M recommendations, DOD stated that it would review its R&M requirements and possibly revise them, update its RMIP guidance, and plan for R&M funding going forward. DOD officials did not concur with our recommendation that the F-35 program office complete its business case before initiating additional development work. DOD stated that the F-35 program office has adequate cost, schedule, and technical maturity knowledge to begin the development of initial Block 4 capabilities. DOD also outlined when some of the remaining Block 4 business case documents would be complete. As we stated in our report, these documents will not be complete until after the contracts to initiate additional Block 4 development work will be awarded. We maintain that completing its business case before initiating additional development work would put DOD and the program in a better position to effectively and successfully develop Block 4 capabilities. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Under Secretary of Defense for Acquisition and Sustainment, the Secretary of the Air Force, the Secretary of the Navy, and the Commandant of the Marine Corps. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. Page 28 GAO-19-341 F-35 Joint Strike Fighter If you or your staff have any questions about this report, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Michael J. Sullivan Director, Contracting and National Security Acquisitions Page 29 GAO-19-341 F-35 Joint Strike Fighter List of Committees The Honorable James M. Inhofe Chairman The Honorable Jack Reed Ranking Member Committee on Armed Services United States Senate The Honorable Richard Shelby Chairman The Honorable Richard J. Durbin Ranking Member Subcommittee on Defense Committee on Appropriations United States Senate The Honorable Adam Smith Chairman The Honorable Mac Thornberry Ranking Member Committee on Armed Services House of Representatives The Honorable Peter Visclosky Chairman The Honorable Ken Calvert Ranking Member Subcommittee on Defense Committee on Appropriations House of Representatives Page 30 GAO-19-341 F-35 Joint Strike Fighter Appendix I: Prior GAO Reports and DOD Actions Appendix I: Prior GAO Reports and DOD Actions Table 6: Select Prior GAO Reports on F-35 Joint Strike Fighter and Department of Defense (DOD) Responses Year, GAO report 2001 GAO-02-39 Estimated F-35 development costs, development length, and aircraft unit a Key program event cost • • • 2006 GAO-06-356 • • • 2010 GAO-10-382 • • • 2013 GAO-13-309 • • • Primary GAO conclusions/recommendations DOD response and actions $34.4 billion 10 years $69 million Start of system development and demonstration approved. Critical technologies needed for key aircraft performance elements are not mature. We recommended that the program should delay start of system development until critical technologies are matured to acceptable levels. DOD did not concur with our recommendation. DOD did not delay the start of system development and demonstration stating technologies were at acceptable maturity levels and that it will manage risks in development. $45.7 billion 12 years $86 million Program sets in motion plan to enter production in 2007 shortly after first flight of the non-production representative aircraft. The program was entering production with less than 1 percent of testing complete. We recommended that the program delay investing in production until flight testing shows that the Joint Strike Fighter performs as expected. DOD partially concurred but did not delay start of production because it believed the risk level was appropriate. $49.3 billion 15 years $112 million The program was restructured to reflect findings from a recent independent cost team and independent manufacturing review team. As a result, development funds increased, test aircraft were added, the schedule was extended, and the early production rate decreased. Costs and schedule delays inhibited the program’s ability to meet needs on time. We recommended that the program complete a comprehensive cost estimate and assess warfighter and initial operational capability requirements. We suggested that Congress require DOD to tie annual procurement requests to demonstrated progress. DOD continued restructuring, increasing test resources, and lowering the production rate. Independent review teams evaluated aircraft and engine manufacturing processes. Cost increases later resulted in a NunnMcCurdy breach. Military services completed the review of capability requirements, as we recommended. $55.2 billion 18 years $137 million The program incorporated positive and more realistic restructuring actions taken since 2010, including more time and funding for development and deferred procurement of more than 400 aircraft to future years. The program was moving in the right direction but needed to fully validate design and operational performance and at the same time make the system affordable. We did not make recommendations to DOD in this report. DOD agreed with GAO’s observations. Page 31 GAO-19-341 F-35 Joint Strike Fighter Appendix I: Prior GAO Reports and DOD Actions Year, GAO report 2014 GAO-14-322 2016 GAO-16-390 Estimated F-35 development costs, development length, and aircraft unit a cost • $55.2 billion • 18 years • $135 million • • • 2017 GAO-17-351 • • • Primary GAO conclusions/recommendations DOD response and actions The services established initial operational capabilities dates in 2013. The Marine Corps and Air Force planned to field initial operational capabilities in 2015 and 2016, respectively, and the Navy planned to field its initial capability in 2018. Delays in developmental flight testing of the F-35’s critical software may hinder delivery of the warfighting capabilities to the military services. We recommended that DOD conduct an assessment of the specific capabilities that can be delivered and those that will not likely be delivered to each of the services by their established initial operational capability dates. DOD concurred with our recommendation. On June 22, 2015, the Under Secretary of Defense for Acquisition, Technology, and Logistics issued a Joint Strike Fighter software development report, which met the intent of GAO’s recommendation. $55.1 billion 18 years $130.6 million DOD planned to begin what it refers to as a block buy contracting approach that was anticipated to provide cost savings. In addition, DOD planned to manage the follow-on modernization program under the current F-35 program baseline and not as its own separate major defense acquisition program. The terms and conditions of the planned block buy and managing follow-on modernization under the current baseline could present oversight challenges for Congress. We recommended that the Secretary of Defense hold a milestone B review and manage follow-on modernization as a separate major defense acquisition program. DOD did not concur with our recommendation. DOD viewed modernization as a continuation of the existing program and the existing oversight mechanisms, including regularly scheduled high-level acquisition reviews, would be used to manage the effort. $55.1 billion 18 years $130.6 million The DOD F-35 program office was considering contracts for economic order quantity of 2 years’ worth of aircraft parts followed by a separate annual contract for procurement of lot-12 aircraft with annual options for lot-13 and lot-14 aircraft. However, as of January 2017, contractors stated they were still negotiating the terms of this contract; therefore, the specific costs and benefits remained uncertain. Program officials projected that the program would only need $576.2 million in fiscal year 2018 to complete baseline development. At the same time, program officials expected that more than $1.2 billion could be needed to commit to Block 4 and economic order quantity in fiscal year 2018. GAO recommended DOD use historical data to reassess the cost of completing development of Block 3F, complete Block 3F testing before soliciting contractor proposals for Block 4 development, and identify for Congress the cost and benefits associated with procuring economic order quantities of parts. DOD did not concur with the first two recommendations and partially concurred with the third while stating that it had finalized the details of DOD and contractor investments associated with an economic order quantity purchase and would brief Congress on the details, including costs and benefits of the finalized economic order quantity approach. Key program event Page 32 GAO-19-341 F-35 Joint Strike Fighter Appendix I: Prior GAO Reports and DOD Actions Year, GAO report 2018 GAO-18-321 Estimated F-35 development costs, development length, and aircraft unit a cost • $55.5 billion • 18 years • $140.6 million Key program event Primary GAO conclusions/recommendations DOD response and actions The program office determined that it could not resolve all open deficiencies found in developmental testing within the development program, and they would need to be resolved through post-development contract actions. DOD provided a report to Congress outlining preliminary plans to modernize the F-35. It stated it planned to develop a full acquisition program baseline for the modernization effort in 2018 and provide a report to Congress by March 2019. The program office plans to resolve a number of critical deficiencies after full-rate production. We recommended that the F-35 program office resolve all critical deficiencies before making a fullrate production decision, and identify steps needed to ensure the F-35 meets reliability and maintainability requirements before each variant reaches maturity. We also suggested that Congress consider providing in future appropriations that no funds shall be available for obligation for F-35 Block 4 until DOD provides a report setting forth its complete acquisition program baseline for the Block 4 effort to the congressional defense committees. DOD concurred with both recommendations and identified actions that it would take in response. The National Defense Authorization Act for fiscal year 2019 included a provision limiting DOD from obligating or expending more than 75 percent of the appropriations authorized under the Act for the F-35 continuous capability development and delivery program until 15 days after the Secretary of Defense submits to the congressional defense committees a detailed cost estimate and baseline schedule. Source: GAO GAO-19-341 a The aircraft unit cost is the program’s average procurement unit cost estimate which is calculated by dividing the procurement amount by the procurement aircraft quantities. This is different than the negotiated price for F-35 aircraft, also reported above. Page 33 GAO-19-341 F-35 Joint Strike Fighter Appendix II: Objectives, Scope, and Methodology Appendix II: Objectives, Scope, and Methodology The National Defense Authorization Act for fiscal year 2015 included a provision for GAO to review the F-35 acquisition program annually until the program reaches full-rate production. This is the fourth report under that provision. In this report, we (1) provide information on the program’s progress toward completing testing of the baseline aircraft; (2) assess the aircraft’s current reliability and maintainability (R&M) status; (3) assess the program’s modernization efforts (to add new aircraft capabilities), known as Block 4; and (4) provide information on the program’s production costs and efficiency initiatives. To provide information on progress in the F-35’s development, we reviewed the program’s costs, schedule, and performance plans and compared the actual progress in each area with the goals established in its 2012 baseline to identify any significant trends. We also reviewed the F-35’s selected acquisition report and its fiscal year 2019 budget request. To assess progress in testing, we reviewed test results and associated reports, program briefings, and internal DOD program analyses. We interviewed officials from the program office, military test authorities, and contractors—Lockheed Martin (airframe) and Pratt & Whitney (engine)— on key aspects of F-35 development progress, including flight testing, future test plans, and recent findings from test events. We also interviewed the Director, Operational Test and Evaluation office and F-35 program developmental and operational test pilots. To assess the program’s progress in achieving its R&M targets, we obtained and analyzed its monthly reports on R&M performance from January 2018 through December 2018. We compared these to the program’s R&M targets documented in the F-35 Operational Requirements Document and the Joint Contract Specification. We examined program data for the metrics’ performance across 12 months to identify any trends. We assessed the reliability of this data by reviewing supporting documentation and interviewing program office officials who track reliability metrics and other knowledgeable DOD officials. We also reviewed the program’s Reliability and Maintainability Improvement Program’s guidance to determine if it contained specific and measurable objectives and the projects needed to meet those objectives. We determined that the R&M metric data were sufficiently reliable for our purposes of determining whether the program will meet its targets. To assess the program’s Block 4 modernization plans, we reviewed documents that GAO best practices identify should be completed prior to awarding a development contract. We interviewed DOD and program office officials, and contractor representatives regarding the program’s Page 34 GAO-19-341 F-35 Joint Strike Fighter Appendix II: Objectives, Scope, and Methodology Block 4 activities to date and future plans. We compared the program’s accomplishments over the past year and its future plans to the product development best practices identified by GAO. We reviewed the fiscal year 2019 budget request to identify costs associated with the Block 4 effort. We obtained contract documents for Block 4 activities between March 2014 and December 2018 to determine the total amount of funding that has been obligated to Block 4 and the scope of work that has been contracted. To provide information on ongoing manufacturing performance and the program’s plans to achieve full rate production, we obtained and analyzed the prime contractor’s production metrics and its aircraft delivery rates and from 2012 through 2018. We compared this performance to the program’s procurement plans from its selected acquisition reports since 2003. We reviewed metrics and briefings provided by the program office, Lockheed Martin, Pratt & Whitney, and the Defense Contract Management Agency to identify progress in improving manufacturing processes. We analyzed changes in delivery dates for lot 10 aircraft delivered in 2018. We discussed reasons for any delivery delays and plans for improvement with officials from Lockheed Martin and Pratt & Whitney. We obtained cost investment and savings estimates and discussed cost and manufacturing efficiency initiatives, such as the economic order quantity purchases, with the contractors and program office officials to understand potential cost savings and plans. We also obtained and analyzed metrics on parts and aircraft quality through December 2018 and discussed steps taken to improve quality and deliveries with Lockheed Martin and Pratt & Whitney officials. We determined that the contractor’s production metrics and delivery dates were sufficiently reliable for our purposes of determining production efficiency and deliveries. We conducted this performance audit from June 2018 to April 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Page 35 GAO-19-341 F-35 Joint Strike Fighter Appendix III: Status of Selected F-35 Technical Risks Appendix III: Status of Selected F-35 Technical Risks The F-35 program continues to address technical risks discovered in testing. Since our 2018 report, the program identified new risks with the canopy, fuel tubes, and cockpit, described below. The program has also incorporated design changes that have mitigated technical risks that we previously highlighted. 1 The status of the Department of Defense’s (DOD) efforts to address these issues follows. Newly Identified Technical Risks Canopy Coating De-laminations and Corrosions: The F-35 fleet has experienced approximately 20 incidents of the canopy transparencies delaminating after less than 100 flight hours. The contractor is currently testing numerous solutions for the de-laminations, with intentions of completing testing by January 2019. F-35 aircraft are also experiencing canopy corrosion resulting from moisture intrusion due to the aircraft’s adhesive cracking under pressure and insufficient tape adhesion. The program has identified the need to modify over 173 canopies over 4 years. The contractor has begun to incorporate alternative material and tape into production, and released standardized repair procedures to mitigate this issue. Engine Fuel Tubes: In September 2018, a manufacturing fault in an engine fuel tube caused an in-flight failure, which resulted in an F-35B crash. The investigation identified several other life-limited fuel tubes in each F-35 variant. The fleet was grounded while all aircraft were inspected, and any fuel tubes identified were replaced or will be replaced by June 2019. Cockpit Display: In November 2018, operational test pilots experienced the cockpit display freezing and blanking, and identified the problem as a category 1 deficiency. The display issues occurred after a software update. The start of operational testing was delayed until the contractor could provide a software update to correct the problem, which was accomplished with a work-around in December 2018. 1 GAO-18-321. Page 36 GAO-19-341 F-35 Joint Strike Fighter Appendix III: Status of Selected F-35 Technical Risks Technical Risks Identified In Our Previous Reports Helmet Mounted Display: During low-light flights, the Helmet Mounted Display’s technology cannot display pure black, causing a green glow on the screen which makes it difficult to see the full resolution of the night vision video feed. The contractor is developing a new system to avoid this effect, and the contractor delivered this system to the test fleet in September 2018 with final flight testing planned through January 2019. Figure 7 is a photograph of the Helmet Mounted Display. Figure 7: The F-35 Helmet Mounted Display Aerial refueling probes: The F-35B and F-35C variants use a “hose and drogue” system in which an aerial refueling tanker aircraft extends a long, flexible refueling hose and a parachute-like metal basket that provides stability, the receiving aircraft then connects to the drogue basket with its extendable refueling probe, as shown in figure 8. The refueling probe tips are meant to break in the event there is a stress occurring during refueling. However, the breaking is occurring more often than expected. Since April 2014, more than 20 incidents have occurred where the F-35’s aerial refueling probes broke off while conducting aerial refueling, leading to a restriction of aerial refueling operations. Page 37 GAO-19-341 F-35 Joint Strike Fighter Appendix III: Status of Selected F-35 Technical Risks Figure 8: F-35B Aircraft Refuel from a KC-130 Aerial Refueling Tanker Using Hose and Drogue Refueling Equipment Tire service life: We reported in June 2018, the average service life of tires on the F-35B is below 10 landings. 2 Lockheed Martin is currently working with three tire manufacturers to develop a new design with the goal of 20 landings. Testing of the new tires will occur throughout 2019. Figure 9 shows an F-35B during a landing. 2 GAO-18-321. Page 38 GAO-19-341 F-35 Joint Strike Fighter Appendix III: Status of Selected F-35 Technical Risks Figure 9: An F-35B on USS America Life support system: The program has identified over 35 pilot physiological events, of which nearly 30 occurred in-flight. An action team made of government officials, contractors, and doctors completed their work by May 2018. A root cause investigation did not identify any F-35 system deficiencies, but reported it was difficult to fully determine the problem due to a lack of real-time data. Contracting officials stated that this is partially because the technology has not yet been developed to monitor pilot’s health in flight, in real time. The prime contractor continues to try to develop a means to monitor pilot health. Page 39 GAO-19-341 F-35 Joint Strike Fighter Appendix IV: Comments from the Department of Defense Appendix IV: Comments from the Department of Defense Page 40 GAO-19-341 F-35 Joint Strike Fighter Appendix IV: Comments from the Department of Defense Page 41 GAO-19-341 F-35 Joint Strike Fighter Appendix IV: Comments from the Department of Defense Page 42 GAO-19-341 F-35 Joint Strike Fighter Appendix V: GAO Contact and Staff Acknowledgments Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov Staff Acknowledgments In addition to the contact named above, the following staff members made key contributions to this report: Justin Jaynes (Assistant Director), Jennifer Baker, Emily Bond, Brandon Booth, Erin Butkowski, Matthew T. Crosby, Desirée E. Cunningham, R. Eli DeVan, Laura Jezewski, Jennifer Leotta, Meghan Perez, Hai Tran, Abby Volk, Mary Weiland, Alyssa Weir, and Robin M. Wilson. Page 43 GAO-19-341 F-35 Joint Strike Fighter Related GAO Products Related GAO Products F-35 Joint Strike Fighter: Development is Nearly Complete, but Deficiencies Found in Testing Need to Be Resolved. GAO-18-321. Washington, D.C.: June 5, 2015. Warfighter Support: DOD Needs to Share F-35 Operational Lessons Across the Military Services. GAO-18-464R. Washington, D.C.: April 25, 2018. F-35 Aircraft Sustainment: DOD Needs to Address Challenges Affecting Readiness and Cost Transparency. GAO-18-75. Washington, D.C.: October 26, 2017. F-35 Joint Strike Fighter: DOD’s Proposed Follow-on Modernization Acquisition Strategy Reflects an Incremental Approach Although Plans Are Not Yet Finalized. GAO-17-690R. Washington, D.C.: August 8, 2017. F-35 Joint Strike Fighter: DOD Needs to Complete Developmental Testing Before Making Significant New Investments. GAO-17-351. Washington, D.C.: April 24, 2017. F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans to Begin Development of New Capabilities. GAO-16-390. Washington, D.C.: April 14, 2016. F-35 Sustainment: DOD Needs a Plan to Address Risks Related to Its Central Logistics System. GAO-16-439. Washington, D.C.: April 14, 2016. F-35 Joint Strike Fighter: Preliminary Observations on Program Progress. GAO-16-489T. Washington, D.C.: March 23, 2016. F-35 Joint Strike Fighter: Assessment Needed to Address Affordability Challenges. GAO-15-364. Washington, D.C.: April 14, 2015. F-35 Sustainment: Need for Affordable Strategy, Greater Attention to Risks, and Improved Cost Estimates. GAO-14-778. Washington, D.C.: September 23, 2014. F-35 Joint Strike Fighter: Slower Than Expected Progress in Software Testing May Limit Initial Warfighting Capabilities. GAO-14-468T. Washington, D.C.: March 26, 2014. Page 44 GAO-19-341 F-35 Joint Strike Fighter Related GAO Products F-35 Joint Strike Fighter: Problems Completing Software Testing May Hinder Delivery of Expected Warfighting Capabilities. GAO-14-322. Washington, D.C.: March 24, 2014. F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but Affordability Challenges and Other Risks Remain. GAO-13-690T. Washington, D.C.: June 19, 2013. F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March 11, 2013. Fighter Aircraft: Better Cost Estimates Needed for Extending the Service Life of Selected F-16s and F/A-18s. GAO-13-51. Washington, D.C.: November 15, 2012. Joint Strike Fighter: DOD Actions Needed to Further Enhance Restructuring and Address Affordability Risks. GAO-12-437. Washington, D.C.: June 14, 2012. Joint Strike Fighter: Restructuring Added Resources and Reduced Risk, but Concurrency Is Still a Major Concern. GAO-12-525T. Washington, D.C.: March 20, 2012. (102886) Page 45 GAO-19-341 F-35 Joint Strike Fighter GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to https://www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. To Report Fraud, Waste, and Abuse in Federal Programs Contact FraudNet: Congressional Relations Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Public Affairs Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Strategic Planning and External Liaison James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548 Website: https://www.gao.gov/fraudnet/fraudnet.htm Automated answering system: (800) 424-5454 or (202) 512-7700 Please Print on Recycled Paper.