FOR OFFICIAL USE ONLY Report No. DODIG-2016-107 I nspec tor Ge ne ral U.S. Department of Defense J U LY 5 , 2 0 1 6 Advanced Arresting Gear Program Exceeded Cost and Schedule Baselines I N T E G R I T Y  E F F I C I E N C Y  A C C O U N TA B I L I T Y  E X C E L L E N C E The document contains information that may be exempt from mandatory disclosure under the Freedom of Information Act. FOR OFFICIAL USE ONLY FOR OFFICIAL USE ONLY I N T E G R I T Y  E F F I C I E N C Y  A C C O U N TA B I L I T Y  E X C E L L E N C E Mission Our mission is to provide independent, relevant, and timely oversight of the Department of Defense that supports the warfighter; promotes accountability, integrity, and efficiency; advises the Secretary of Defense and Congress; and informs the public. Vision Our vision is to be a model oversight organization in the Federal Government by leading change, speaking truth, and promoting excellence—a diverse organization, working together as one professional team, recognized as leaders in our field. F r a u d, W a s t e, & A b u s e HOTLINE Department of Defense dodig.mil/hotline 8 0 0 . 4 2 4 . 9 0 9 8 For more information about whistleblower protection, please see the inside back cover. FOR OFFICIAL USE ONLY FOR OFFICIAL USE ONLY Results in Brief Advanced Arresting Gear Program Exceeded Cost and Schedule Baselines July 5, 2016 Finding (cont’d) Objective As a result, major AAG system components required costly redesign, which delayed developmental testing and will further postpone delivery of the full AAG system capability to the CVN-78 aircraft carrier. AAG hardware and software component failures and test site preparation led to the AAG program exceeding the Acquisition Category I threshold for Research, Development, Test, and Evaluation (RDT&E) costs. As of October 2015, RDT&E costs for the AAG program total $743.5 million, which was $571.5 million above the planned costs in the 2005 Acquisition Program Baseline. Developmental testing originally scheduled to end in FY 2009 will continue through FY 2018, and reliability of the system is uncertain. Our objective was to determine whether the Navy was effectively managing the acquisition requirements and testing for the Advanced Arresting Gear (AAG) program. The arresting gear is the system responsible for stopping aircraft while landing on the flight deck of a carrier. Finding The Program Manager, Aircraft Launch and Recovery Equipment (PMA-251), did not effectively manage the acquisition to meet requirements and execute testing for the AAG program. Ten years after the program entered the engineering and manufacturing development phase, the Navy has not been able to prove the capability or safety of the system to a level that would permit actual testing of the system on an aircraft carrier because of hardware failures and software challenges. This occurred because the Navy pursued a technological solution for its Ford-class carriers that was not sufficiently mature for the planned use, resulting in hardware failures to mechanical and electrical components and software modifications to accommodate those failures. In addition, the program manager did not revise the Test and Evaluation Master Plan to address significant changes to the test strategy and schedule. This occurred because redesign changes required significant revisions to AAG key components, and those changes took priority over updating the Test and Evaluation Master Plan. Recommendations We recommend the Assistant Secretary of the Navy for Research, Development, and Acquisition perform cost-benefit analyses to determine whether the AAG is an affordable solution for Navy aircraft carriers before deciding to go forward with the system on future aircraft carriers. We recommend the Program Manager, Aircraft Launch and Recovery Equipment, update the AAG Test and Evaluation Master Plan to revise the planned test strategy, test schedule, developmental and operational funding, and add measures to support the program’s reliability growth plan before the Acquisition Category IC Acquisition Program Baseline is finalized. Management Comments and Our Response Comments from the Assistant Secretary of the Navy for Research, Development, and Acquisition; and the Program Manager, Aircraft Launch and Recovery Equipment, addressed all the specifics of the recommendations and no further action is required. Please see the Recommendations Table on the back of this page. Visit us at www.dodig.mil FOR OFFICIAL USE ONLY DODIG-2016-107 (Project No. D2015-D000AE-0191.000) │ i FOR OFFICIAL USE ONLY Recommendations Table Recommendations Requiring Comment Management No Additional Comments Required Assistant Secretary of the Navy for Research, Development, and Acquisition 1 Program Manager, Aircraft Launch and Recovery Equipment 2 FOR OFFICIAL USE ONLY ii │ DODIG-2016-107 (Project No. D2015-D000AE-0191.000) FOR OFFICIAL USE ONLY INSPECTOR GENERAL DEPARTMENT OF DEFENSE 4800 MARK CENTER DRIVE ALEXANDRIA, VIRGINIA 22350-1500 July 5, 2016 MEMORANDUM FOR ASSISTANT SECRETARY OF THE NAVY FOR RESEARCH, DEVELOPMENT, AND ACQUISITIO N NAVAL INSPECTOR GENERAL SUBJECT: Advanced Arresting Gear Program Exceeded Cost and Schedule Baselines (Report No. DODIG-2016-107) We are providing this report for your information and use. We determined that the program manager did not effectively manage the Advanced Arresting Gear Program acquisition to meet requirements and execute testing. The Advanced Arresting Gear Program exceeded cost and schedule baselines because of hardware and software failures and test site preparation. We conducted this audit in accordance with generally accepted government aud iting sta ndards. We considered management comments on a draft of this report when preparing the final report. DoD Instruction 7650.03 requires that recommendations be resolved promptly. Comments from the Assistant Secretary of the Navy for Research, Development, and Acquisition and the Program Manager, Aircraft Launch and Recovery Equipment conformed to the requirements of DoD Instruction 7650.03; therefore, we do not require additional comments. We appreciate the courtesies extended to the staff. Please direct questions to me at (703) 604-9077 (DSN 664-9077). ~ CJ~arver Assistant Inspector General Acquisition and Sustainment Management FOR OFFICIAL USE ONLY DODIG-2016-lC ~ iii FOR OFFICIAL USE ONLY Contents Introduction Objective _________________________________________________________________________________________1 Background______________________________________________________________________________________1 Review of Internal Controls ____________________________________________________________________6 Finding. Advanced Arresting Gear Exceeded Cost and Schedule Baselines _________________________________________________________7 Mechanical and Electrical Components Required Redesign During Testing ______________8 Software Challenges___________________________________________________________________________ 10 Program Cost Challenges _____________________________________________________________________ 11 Developmental Testing May Delay Shipboard Testing_ ____________________________________ 12 Reliability of the AAG System Is Uncertain__________________________________________________ 15 Conclusion _____________________________________________________________________________________ 16 Management Comments on the Finding _____________________________________________________ 17 Recommendations, Management Comments, and Our Response _________________________ 17 Appendixes Appendix A. Scope and Methodology _______________________________________________________ 19 Use of Computer-Processed Data ________________________________________________________ 20 Prior Coverage ____________________________________________________________________________ 20 Use of Technical Assistance ______________________________________________________________ 20 Appendix B. AAG Test Schedule _____________________________________________________________ 21 Management Comments Assistant Secretary of the Navy for Research, Development, and Acquisition Comments_______________________________________________________________ 22 Glossary___________________________________________________________________________________ 26 Acronyms and Abbreviations______________________________________________ 29 FOR OFFICIAL USE ONLY iv │ DODIG-2016-107 FOR OFFICIAL USE ONLY Introduction Introduction Objective Our objective was to determine whether the Navy was effectively managing the acquisition requirements and testing for the Advanced Arresting Gear (AAG) program. See Appendix A for a discussion of our scope and methodology and prior coverage related to the audit objectives. See the Glossary for the definition of technical terms used in the report. Background The AAG program is a Major Defense Acquisition Program (Category IC1). During the audit, the program was in the engineering and manufacturing development phase of the acquisition process. The Milestone Decision Authority for the AAG is the Assistant Secretary of the Navy for Research, Development, and Acquisition (ASN [RDA]). The Navy developed the AAG system to replace the existing Mark-7 (MK-7) arresting gear. The purpose of the arresting gear is to stop aircraft after it lands on the aircraft carrier. The Milestone Decision Authority approved the AAG program entry into the engineering and manufacturing development phase on February 10, 2005, as an Acquisition Category II2 program. On June 15, 2015, the Navy notified the Under Secretary of Defense (Acquisition, Technology, and Logistics) that the AAG program had exceeded the Acquisition Category I threshold for Research, Development, Test, and Evaluation (RDT&E) cost. The Under Secretary designated the AAG program as an Acquisition Category IC Major Defense Acquisition Program on July 23, 2015. The Aircraft Launch and Recovery Equipment program office (PMA-251) estimates the AAG Milestone C3 decision will occur in the third quarter of FY 2018. 1 Acquisition Category IC is a program for which the Head of the DoD Component estimates eventual total expenditure for RDT&E of more than $480 million in FY 2014 constant dollars, or for procurement of more than $2.8 billion in FY 2014 constant dollars. 2 Acquisition Category II is a program for which the Head of the DoD Component estimates eventual total expenditure for RDT&E of more than $185 million in FY 2014 constant dollars, or for procurement more than $835 million in FY 2014 constant dollars. 3 Milestone C approves entry into the production and deployment phase. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 1 FOR OFFICIAL USE ONLY Introduction Advanced Arresting Gear System Description The Navy established the AAG program in July 2003 to develop a new arresting gear system with increased operational availability, while reducing manning, maintenance, and support costs. The AAG was designed to replace the aging MK-7 arresting gear and provide the Navy with the ability to recover4 all existing and projected carrier-based aircraft such as the C-2A, EA-6B, E-2, E-2C+, EA-18G, various F/A-18 variants, T-45, F-35, and UAV. 5 The Navy planned to replace the MK-7 arresting gear on its Nimitz-class aircraft carriers and the new Ford-class aircraft carriers. The Navy identified several significant shortfalls with the MK-7 arresting gear, including the inability to recover lightweight and heavyweight aircraft, decreased safety and service life, and increased costs for operations and maintenance support. The Navy planned to address the shortfalls of the MK-7 with the AAG system. The Navy projected in its October 27, 2004, Capabilities Development Document (requirements document) for Advanced Arresting Gear that the AAG system would meet the requirements to recover aircraft in 2010. As of March 2016, the Navy had not completed development of an aircraft recovery bulletin (ARB)6 that would allow it to recover an aircraft on the Ford-class aircraft carriers. Additionally, the Navy will not Navy will replace the MK-7 with the current AAG on Nimitz-class not replace aircraft carriers due to cost increases and schedule the MK-7 with delays. PMA‑251 estimated that it would require at the current AAG on least $300 million to perform additional testing of Nimitz‑class aircraft the hybrid AAG system on a Nimitz-class carrier. The carriers due to cost increases and ASN (RDA) told PMA‑251 that funding was not available schedule delays. to support these replacements. Figure 1 illustrates carrier‑installed arresting gear visible from the flight deck (retractable sheave and cross-deck pendent) used to recover a tail-hook‑equipped aircraft. 4 Arresting gear is the system responsible for stopping (to recover) aircraft on the flight deck of a carrier. 5 A UAV (unmanned aerial vehicle) is an aircraft with no pilot on board that can be remote-controlled or fly-based on a preprogrammed flight plan. 6 Aircraft Recovery Bulletins provide standardized operating procedures and technical guidance, and are required to conduct AAG system flight operations. FOR OFFICIAL USE ONLY 2 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Introduction Figure 1. Aircraft Landing on Carrier Source: PMA-251 Program Management and Engineering Support Program Executive Officer Tactical The Program Executive Officer (PEO) Tactical and PMA-251 provide life-cycle acquisition management of the AAG program. PEO Tactical and PMA-251 coordinate with PEO for Carriers to integrate the AAG system on Ford-class aircraft carriers. Program Executive Officer for Carriers The PEO for Carriers focuses on design, construction and delivery, and life-cycle support of all aircraft carriers. The PEO for Carriers reports directly to the ASN (RDA) on acquisition-related matters. Naval Air Warfare Center Aircraft Division The Naval Air Warfare Center Aircraft Division (NAWCAD) Lakehurst, New Jersey, provides engineering support to PMA-251. NAWCAD provides unique test facilities such as the Jet Car Track Site (JCTS) and Runway Arrested Landing Site (RALS), and the expertise necessary to make sure the AAG system meets FOR OFFICIAL USE ONLY DODIG-2016-107 │ 3 FOR OFFICIAL USE ONLY Introduction fleet requirements. Testing at NAWCAD verifies the status of technical progress, minimizes design risks, and substantiates achievement of technical performance. NAWCAD personnel use full-scale testing at JCTS and RALS to measure the system’s ability to meet key performance parameters (primary requirements). The goal of these tests is to: • identify deficiencies early in the developmental process; • demonstrate that design development risks and integration issues are resolved; • confirm that minimum performance for primary requirements is achieved; and • support certification of readiness for initial operational test and evaluation.7 Jet Car Track Site The purpose of JCTS testing is to conduct full-scale system developmental testing and obtain an interim flight clearance. Test personnel8 use the data to substantiate performance and safety for land-based developmental flight tests with aircraft. Test personnel use four jet engines (jet car) pushing a weighted cart (a dead load) down guided rails toward the AAG system to simulate an incoming aircraft. The speed and weight of the dead load is variable depending on the testing requirements PMA-251 needs to test. Once the jet car and dead load get close enough to the AAG system, brakes stop the jet car and the dead load continues moving toward the AAG system. A tail-hook on the dead load (much like a tail‑hook on an aircraft) catches the cross-deck pendent (wire) and the AAG system stops the dead load. During JCTS, testers simulate aircraft landing straight on and at an angle. Runway Arrested Landing Site The purpose of RALS test events is to verify system integrity and aircraft settings (developed at JCTS) and confirm the system is ready for aircraft performance testing on a carrier. NAWCAD personnel conduct RALS testing with actual aircraft using roll-in and fly-in arrestments. Tested aircraft reach the appropriate speed to simulate an arrestment and the AAG system stops the aircraft. The AAG system retracts the wire in preparation for arresting the next aircraft. As each aircraft 7 Initial Operational Test and Evaluation is the dedicated operational test and evaluation conducted on production representative articles, to determine whether systems are operationally effective, and suitable to support a final production decision. 8 The Naval Air Warfare Center Aircraft Division and contractor test personnel. FOR OFFICIAL USE ONLY 4 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Introduction type successfully completes RALS testing, the Naval Air Systems Command, Headquarters issues an ARB. An ARB allows the Navy to begin shipboard testing of the AAG system with the specific aircraft. An ARB provides the crew with instructions using the AAG equipment with that aircraft within specific limits. Funding A July 2015 Acquisition Decision Memorandum9 required the Navy, in conjunction with the Under Secretary of Defense (Comptroller), to use separate, dedicated RDT&E and Procurement funding lines, that are not shared with other non-AAG program activities. The Acquisition Decision Memorandum approved the obligation of funds to allow the AAG program to: • continue in-service support; • maintain the land-based unit in the same configuration as CVN‑78 AAG hardware; • procure spares to support land-based testing; • generate the ARB to support CVN-78 flight operations in FY 2016; • support the construction, certification, and delivery of the CVN-78; and • provide AAG equipment to CVN-79 following the ship’s build schedule. Table 1 shows AAG program funding in the Future Years Defense Plan for RDT&E and Procurement. Table 1. AAG Funding in the Future Years Defense Plan Fund RDT&E Procurement* Total To Complete Total (dollars in millions) Prior Years FY 2016 FY 2017 FY 2018 FY 2019 FY 2020 $596.3 $108.4 $31.8 $6.3 $0.7 $0 $0 $743.5 $71.3 $6.3 $2.4 $2.5 $2.5 $2.6 $85.0 $172.6 $667.6 $114.7 $34.2 $8.8 $3.2 $2.6 $85.0 $916.1 * These procurement funds are only for the AAG land based unit procurement and system modifications. Source: PMA-251 9 Under Secretary of Defense for Acquisition, Technology, and Logistics Memorandum with Subject: Advanced Arresting Gear Program Acquisition Category Reclassification Acquisition Decision Memorandum, July 23, 2015. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 5 FOR OFFICIAL USE ONLY Introduction Review of Internal Controls DoD Instruction 5010.4010 requires DoD organizations to implement a comprehensive system of internal controls that provides reasonable assurance that programs are operating as intended and to evaluate the effectiveness of the controls. We identified internal control weaknesses in AAG program management. Specifically, the Program Manager, Aircraft Launch and Recovery Equipment, did not effectively manage the acquisition to meet requirements, execute testing, and revise the Test and Evaluation Master Plan after significant system changes to the test strategy and schedule. We will provide a copy of this report to the senior official(s) responsible for internal controls in the Office of the Assistant Secretary of the Navy for Research, Development, and Acquisition. 10 DoD Instruction 5010.40, “Managers’ Internal Control Program Procedures,” May 30, 2013. FOR OFFICIAL USE ONLY 6 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Finding Finding Advanced Arresting Gear Exceeded Cost and Schedule Baselines The Program Manager, Aircraft Launch and Recovery Equipment (PMA-251), did not effectively manage the acquisition to meet requirements and execute testing for the AAG program. Ten years after the program entered the engineering and manufacturing development phase, the Navy has not been able to prove the capability or safety of the system to a level that would permit actual testing of the system on an aircraft carrier because of hardware failures and software challenges. This occurred because the Navy pursued a technological solution for its Ford-class carriers that was not sufficiently mature for the planned use, resulting in hardware failures to mechanical and electrical components, and software modifications to accommodate these failures. In addition, the program manager has not revised the Test and Evaluation Master Plan (TEMP) to address significant changes to the test strategy and schedule. This occurred because redesign changes required significant revisions to AAG system key components, and those changes took priority over updating the TEMP. As a result, major AAG system components required costly redesign, which delayed developmental testing and will further postpone delivery of the full AAG system capability to the CVN-78 aircraft carrier. AAG hardware and software failures and test site preparation led to the AAG program exceeding the Acquisition Category I threshold for RDT&E costs. As of October 2015, the RDT&E costs for the AAG program total $743.5 million, which was $571.5 million above the planned costs in the 2005 Acquisition Program Baseline. Developmental testing originally scheduled to end in FY 200911 will continue through FY 2018, and reliability of the system is uncertain. 11 Test and Evaluation Master Plan 1686 for Advanced Arresting Gear, December 6, 2004. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 7 FOR OFFICIAL USE ONLY Finding Mechanical and Electrical Components Required Redesign During Testing The AAG system mechanical and electrical components required significant redesign to meet performance requirements. This occurred because the Navy pursued a technological solution for the Ford-class carrier that was not sufficiently mature for the planned use, resulting in failures to key components. Key Components of AAG System Not Ready for Operations AAG system mechanical and electrical components required significant redesign to meet performance requirements. The Office of Naval Research conducted a Technology Readiness Assessment of the AAG system to assess the maturity of the key technologies in November 2004, before the AAG program Milestone B decision.12 The Assistant Secretary of Defense for Research and Engineering guidance13 states that at Milestone B, the critical program technologies should be demonstrated in a relevant environment.14 The key AAG system technologies Technologies analyzed during analyzed during the assessment were not an accurate the assessment representation of the required maturity needed to be were not an accurate demonstrated in a relevant environment. As a result, the representation of the AAG system required significant redesign of mechanical required maturity needed. and electrical components to meet system requirements during developmental testing. Redesign of Mechanical and Electrical Components The NAWCAD personnel discovered problems during developmental testing. As a result, the contractor redesigned the AAG system water twister, cable shock absorber, and the power conditioning systems. Redesign of Water Twister The water twister is a paddle wheel submerged in fluid designed to absorb the force when the tail-hook of a landing aircraft pulls against an arresting wire to come to a stop. The AAG system was the first time the Navy used variable torque15 technology to stop aircraft. According to PMA-251, in FY 2012 during developmental testing, test personnel identified internal damage from 12 At Milestone B, the program manager seeks approval to enter the engineering and manufacturing development phase. 13 Technology Readiness Assessment (TRA) Guidance, April 2011, Section 1 Summary. 14 A relevant environment is a testing environment that simulates key aspects of the operational environment. 15 A twisting force that tends to cause rotation. FOR OFFICIAL USE ONLY 8 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Finding subcomponents in the water twister. The Navy assembled a team to identify and test a solution to correct the problem. The water twister required significant redesign and in FY 2014, test personnel successfully tested the redesigned water twister. Redesign of Cable Shock Absorber The cable shock absorber is a component used to dissipate the initial force during aircraft arrestment. The AAG system used a fielded shock absorber design that required modification to provide variable damping16 to optimize performance to meet AAG requirements. PMA-251 stated that, in FY 2011, test personnel identified internal damage to the cable shock absorber during developmental testing. The Navy attributed the damage to the complexity of the design and the tight clearances (space) between moving components. The contractor redesigned the cable shock absorber to prevent further damage. In FY 2013, test personnel successfully tested the redesigned cable shock absorber. Figure 2 illustrates the AAG hardware components. Figure 2. AAG Hardware Components Source: PMA-251 Legend (1) Retractable Sheave (2) Cross-Deck Pendent (3) Cable Shock Absorber (4) Electric Motor 16 (5) Mechanical Brake (6) Purchase Cable Drum (7) Water Twister Variable damping is a feature that allows the cable shock absorber to provide different levels of resistance to accommodate weights and speeds of the different aircraft. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 9 FOR OFFICIAL USE ONLY Finding Redesign of Power Conditioning System The power conditioning system is a component that distributes, conditions, and controls the power needed to operate the AAG system. The power conditioning system is composed of three subsystems: the prime power system, the energy storage system, and the inverter system. However, there was no evidence that the power conditioning system design had been implemented and demonstrated in a relevant environment in support of the technology readiness assessment. From FY 2009 through FY 2012, the power conditioning system experienced multiple test failures. The failure of the inverter system during testing led to the redesign of its components. Software Challenges (FOUO) Software modification or rework may be required to accommodate the hardware failures to mechanical and electrical components. As of January 2016, test personnel had only tested the AAG system in a simulated ground environment using dead loads rolling on a track for the F/A-18 E/F. The AAG system relies on software to control mechanical and electrical components, permitting the operator to adjust AAG system settings based on individual aircraft types. . According to PMA-251 officials, the software keeps aircraft from sliding off the carrier deck when it does not land on the deck centerline. The AAG Chief Engineer stated future software releases would address any problems identified during aircraft compatibility testing, including any revisions to the aircraft settings files or other required software rework. (FOUO) NAWCAD officials will verify whether the problem still occurs during testing at the RALS with actual aircraft. If the software does not keep the aircraft in the allowable run-out area, PMA-251 will not receive the ARB it needs to land the F/A‑18 E/F on the CVN-78 during sea trials. Until the NAWCAD completes testing at JCTS, RALS, and CVN-78 for the remaining carrier aircraft, the Navy will not know the magnitude of software problems. 17 At JCTS, the allowable run-out area (includes length and width) is marked with paint on the asphalt to represent the size of the aircraft carrier deck, so testers can identify when the dead load breaches the perimeter. FOR OFFICIAL USE ONLY 10 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Finding Program Cost Challenges The AAG program exceeded the Acquisition Category I threshold for RDT&E costs. As of October 2015, the AAG RDT&E costs total $743.5 million. PMA‑251 planned to submit a revised Acquisition Program Baseline (APB) for approval in January 2016. However, the sponsor18 required additional detail and further analysis before endorsing a full funding memorandum for the AAG program. Without an approved APB, PMA-251 must obtain funding authority from the Under Secretary of Defense (Acquisition, Technology, and Logistics). The initial APB for AAG provided $172.0 million of RDT&E funding in 2005. In June 2009, the ASN (RDA) approved RDT&E costs an increase of RDT&E funding to $364.0 million. We increased determined that the RDT&E costs increased 332.3 percent, 332.3 percent, or or $571.5 million,19 from the 2005 APB. AAG hardware and $571.5 million, from the software failures and test site preparation led to the AAG 2005 APB. program exceeding the Acquisition Category I threshold for RDT&E costs. Figure 3 illustrates the actual RDT&E costs for the AAG program from FY 2003 through FY 2015. The Assistant Secretary of the Navy for Research, Development, and Acquisition should perform cost‑benefit analyses to determine whether the Advanced Arresting Gear is an affordable solution for Navy aircraft carriers before deciding to go forward with the system on future aircraft carriers. 18 Director, Air Warfare (N98), Office of the Chief of Naval Operations. 19 The increase in RDT&E costs was calculated as $743.5 million - $172.0 million = $571.5 million. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 11 FOR OFFICIAL USE ONLY Finding Figure 3. AAG RDT&E Funding (in millions) Source: PMA-251 Developmental Testing May Delay Shipboard Testing In April 2013, a Senior Level Working Group20 reviewed the AAG program to streamline the remaining work. In February 2014, the working group determined that the AAG program would not achieve ARBs for all aircraft types before the CVN-78 returns to correct deficiencies found during sea trials (post‑shakedown availability) scheduled to begin in September 2016. As a result, PMA‑251 focused solely on obtaining an ARB for the F/A-18 E/F aircraft. If PMA‑251 is unable to obtain an ARB for any aircraft before the CVN-78 completes sea trials and enters its post-shakedown availability phase, PMA-251 will not have approval to land aircraft on the carrier. Testing of the AAG arresting aircraft on the carrier cannot resume until completion of the post-shakedown availability phase scheduled to end in March 2017. The AAG program needs to successfully complete testing at JCTS and RALS and obtain an ARB for the F/A-18 E/F before July 2016 to meet the aircraft carrier test schedule. PMA-251 plans to test the AAG at reduced limits on the CVN-78 carrier. However, if the Navy does not achieve an ARB for the F/A-18 E/F before July 2016, the Navy loses the opportunity to test the F/A-18 E/F before the CVN‑78 enters its post-shakedown availability phase. In addition, if shipboard testing 20 The senior-level working group included PMA-251 systems and software engineers, the Program Manager Aircraft Launch and Recovery Equipment, and General Atomics management and engineers. FOR OFFICIAL USE ONLY 12 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Finding with the AAG identifies software deficiencies in the components, any changes to those components or software will require retesting at JCTS, RALS, and on the carrier, leading to further delays. Appendix B illustrates the test schedule for the remaining aircraft. Testing Strategy and Schedule Risks PMA-251 did not update the TEMP to address significant changes in the test strategy and schedule, because redesign changes required significant revisions to AAG key components and those changes took priority over updating the TEMP. The TEMP serves as the overarching document for managing a test and evaluation program. The program manager uses the TEMP as the primary planning and management tool for all program test activities. The TEMP contains an integrated test program summary and master schedule of all major test events or test phases. A program’s test-and-evaluation strategy is also documented in the TEMP. The program manager updates the TEMP as needed to support acquisition milestones and decision points. Throughout the course of developmental testing, the AAG suffered test delays that can be partially attributed to AAG key components. These test delays Test delays caused the program schedule to slip, and developmental caused the testing scheduled to end in FY 2009 will continue program schedule to through FY 2018. In addition, as a direct effect of the slip, and developmental testing scheduled to test delays, costs associated with the program also rose 21 end in FY 2009 will substantially. Navy guidance requires the TEMP to continue through have an integrated test schedule aligned with program FY 2018. objectives and milestone decisions with clear entrance and exit criteria for each testing phase. The Defense Acquisition Guidebook states the test community uses the TEMP to verify that the AAG system meets requirements for effectiveness and suitability. Test Strategy (FOUO) In April 2013, a Senior Level Working Group led an AAG program re‑baseline. In February 2014, the Senior Level Working Group recommended focusing on obtaining an ARB for a single aircraft type before post-shakedown availability and using a phased approach for the remaining aircraft from JCTS to RALS, and to the CVN-78. The original test strategy22 required testers to use the same AAG system to conduct JCTS and RALS testing. After completing testing 21 Secretary of the Navy (SECNAV) Instruction 5000.2E, “Department of the Navy Implementation and Operation of the Defense Acquisition System and the Joint Capabilities Integration and Development System,” September 1, 2011, Section 4.4.7 Test and Evaluation Master Plan. 22 Advanced Arresting Gear Test and Evaluation Master Plan, December 6, 2004, DT-B3 RALS Test Program. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 13 FOR OFFICIAL USE ONLY Finding (FOUO) for all aircraft types at JCTS, NAWCAD would begin testing at RALS using actual aircraft. This test strategy gave PMA-251 the opportunity to resolve aircraft-specific problems before moving to the next testing stage. PMA‑251 modified the test strategy to test each aircraft type sequentially at JCTS, RALS, and the CVN-78 carrier using reduced speed and weight limitations similar to the MK-7 performance limitations. NAWCAD will test the AAG system at the MK-7 equivalent performance requirements. The PMA-251 stated goal is to achieve the AAG requirements specified in the Capability Development Document during follow on Integrated Test and Evaluation. 23 PMA-251 scheduled follow on integrated test and evaluation using full envelope recovery testing at the beginning of FY 2018. Test Schedule and Test Site Preparation (FOUO) In February 2014, the Senior Level Working Group determined the AAG program schedule would not support ARBs for all aircraft types before the CVN‑78 entered its post-shakedown availability phase. PMA-251 modified the AAG test schedule to achieve an ARB at the end of FY 2016 before the post-shakedown availability. To meet the revised schedule, PMA-251 diverted some components of an AAG system intended for CVN-78 to RALS. The change increased test site preparation costs, but allowed concurrent testing at JCTS and RALS. NAWCAD will issue the ARB permitting the CVN-78 to test the AAG with aircraft on the carrier after aircraft successfully complete RALS testing. In June 2015, General Atomics prepared a test readiness-review report and requested approval from PMA-251 to enter performance testing at 23 Integrated testing allows for the sharing of test events, in which a single test point or mission can provide data to satisfy multiple objectives, without compromising the test objectives and responsibilities of participating test organizations. FOR OFFICIAL USE ONLY 14 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Finding Reliability of the AAG System Is Uncertain The Director, Operational Test and Evaluation (DOT&E) issued FY 2013 and FY 2014 annual reports that questioned the AAG system reliability. Additionally, DOT&E testified before Congress on the availability24 and reliability25 of the AAG system. Director, Operational Test and Evaluation Annual Reports (FOUO) In FY 2013 and FY 2014 the Director, Operational Test and Evaluation, reported that although AAG testing demonstrated the system should recover carrier aircrafts, reliability of the system is uncertain. DOT&E staff conducted an analysis of past aircraft carrier operations in major conflicts during an assessment of the CVN-78. The analysis concluded that the CVN-78 aircraft launch and recovery requirement is well above historical levels and that CVN‑78 is unlikely to achieve that requirement. DOT&E’s analyses also considered the operational impact to the CVN‑78’s mission because of the AAG’s poor reliability. DOT&E concluded that the number of CVN-78 sorties could be lowered to meet the Nimitz‑class carrier sorties requirement. Director, Operational Test and Evaluation Testimony (FOUO) If the reliability of the redesigned AAG system is not substantially better than prior test results, then CVN-78 likely will not be able to complete a normal day of flight operations and may frequently need to divert aircraft to other airfields.27 According to the AAG schedule, the first ARB (F/A-18 E/F) will be delivered in June 2016. Subsequent ARBs will cover the other aircraft in the CVN-78 air wing, with the final ARB scheduled for April 2017. Consequently, a delay of even a few months will affect initial operational test and evaluation. 24 25 Availability is the probability that the system is ready to perform its mission under various conditions when called upon to do so at random times. Reliability is a function based on the actual physical components in the design, generally defined as the probability that an item will not fail to perform its function when used under various conditions over a defined period. Reliability is an important factor in availability. Designing the system to be reliable and maintainable is the best way to minimize the cost to support and maximize the availability of the system. 27 Other airfields on land. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 15 FOR OFFICIAL USE ONLY Finding Program Officials Updated Reliability Prediction Model (FOUO) PMA-251 has not provided DOT&E with reliability data on the AAG system since December 2013. Performance testing of the AAG system resumed in June 2015 after the redesign of key components. PMA-251 revised the methodology used in the reliability prediction model to reflect the system architecture and incorporate lessons learned from testing of the Electromagnetic Aircraft Launch System. 28 PMA-251 will require more JCTS and RALS testing to collect data to update the operational availability prediction. The Defense Acquisition Guidebook 29 states that program managers and operational test agencies assess the reliability growth required for the system to achieve its reliability threshold during initial operational test and evaluation and report the assessment results to the milestone decision authority at Milestone C. 30 The Program Manager, Aircraft Launch and Recovery Equipment, should update the Advanced Arresting Gear Test and Evaluation Master Plan to revise the planned test strategy, test schedule, developmental and operational funding, and add measures to support the program’s reliability growth plan before the Acquisition IC Acquisition Program Baseline is finalized. Conclusion The Navy expected the AAG to recover carrier-based tail-hook equipped aircraft, while increasing the safety margin and decreasing the maintenance required to support the existing MK-7 arresting gear. However, key mechanical and electrical components required significant redesign to meet system requirements. Redesign of AAG system components delayed scheduled developmental testing, which led to re-baselining the program. As a result, PMA-251 reduced requirements for testing performance, and developmental testing scheduled to end in FY 2009 will continue through FY 2018. Software development is ongoing. Furthermore, aircraft compatibility testing may uncover additional challenges with the arrestment of other carrier-based aircraft, requiring further redesign and software rework. The program manager has not aligned the TEMP to the revised test strategy, test schedule, reliability, and funding. The test community uses the TEMP to verify that the AAG system meets requirements for effectiveness and suitability. In addition, 28 29 30 The Electromagnetic Aircraft Launch System is a catapult system that launches carrier-based aircraft. The Defense Acquisition Guidebook, September 16, 2013. Section 9.5.2 OT&E Planning. At Milestone C, the program manager requests approval to enter into the production and deployment phase. FOR OFFICIAL USE ONLY 16 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Finding the TEMP outlines the test resource requirements for the program. An updated test plan serves as a roadmap to plan and manage the test strategy and prioritize test events for the CVN-78 and future Ford-class aircraft carriers. Management Comments on the Finding The Assistant Secretary of the Navy for Research, Development, and Acquisition agreed that the system is not yet ready to test on an aircraft carrier and that the technology was not sufficiently mature for the planned use on CVN-78. However, the ASN (RDA) has proven the capability and safety of the system by arresting aircraft at the Runway Arrested Landing Site. Although testing has not yet occurred on an aircraft carrier, the land-based testing will result in an F/A-18 E/F Aircraft Recovery Bulletin, which will permit shipboard testing with aircraft. The ASN (RDA) agreed that the revised TEMP has not been approved; however, the only substantive change in the revised TEMP is to test at the Jet Car Track Site and the Runway Arrested Landing Site concurrently rather than in sequence, and the Navy has been executing to this revised strategy. The ASN (RDA) agreed that major AAG system components required redesign, which led to schedule delays, cost increases, and the program exceeding the Acquisition Category I threshold. The ASN (RDA) also agreed that until more reliability data is available, the reliability of the system is uncertain. Additionally, the delay in approving the revised TEMP had no direct impact on redesign, schedule, capability, or cost. Recommendations, Management Comments, and Our Response Recommendation 1 We recommend the Assistant Secretary of the Navy for Research, Development, and Acquisition perform cost-benefit analyses to determine whether the Advanced Arresting Gear is an affordable solution for Navy aircraft carriers before deciding to go forward with the system on future aircraft carriers. Assistant Secretary of the Navy for Research, Development, and Acquisition comments The Assistant Secretary of the Navy for Research, Development, and Acquisition agreed and stated that the Navy will make its determination by December 2016 as to whether AAG is an affordable solution for Navy carriers before deciding to go forward with AAG on future aircraft carriers. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 17 FOR OFFICIAL USE ONLY Finding Our Response ASN (RDA) addressed all specifics of the recommendation, and no additional comments are required. Recommendation 2 We recommend the Program Manager, Aircraft Launch and Recovery Equipment, update the Advanced Arresting Gear Test and Evaluation Master Plan to revise the planned test strategy, test schedule, developmental and operational funding, and add measures to support the program’s reliability growth plan before the Acquisition Category IC Acquisition Program Baseline is finalized. Program Manager, Aircraft Launch and Recovery Equipment comments Program Manager, Aircraft Launch and Recovery Equipment (PMA-251) agreed and stated the AAG program re-planning and baseline, which includes the updated TEMP, will be complete by December 2016. Our Response PMA-251 addressed all specifics of the recommendation, and no additional comments are required. FOR OFFICIAL USE ONLY 18 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Appendixes Appendix A Scope and Methodology We conducted this performance audit from May 2015 through May 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We interviewed key personnel and performed fieldwork at the following organizations: • Aircraft Launch and Recovery Equipment Program Office, PMA‑251 (Patuxent River, Maryland); • Naval Air Warfare Center Aircraft Division, Joint Base McGuire‑Dix‑Lakehurst (Lakehurst, New Jersey); • Office of the Director, Operational Test and Evaluation (Arlington, Virginia); • Assistant Secretary of the Navy Research, Development, and Acquisition (Arlington, Virginia); and • Program Executive Officer for Carriers (Navy Yard, Washington, D.C.). We collected, reviewed, and analyzed documents dated from 2003 through 2015. Key documents reviewed related to requirements determination, which included the Capabilities Development Document, acquisition program baseline, technology readiness assessment, and the acquisition strategy. Documents reviewed that related to testing included the TEMP and the test readiness review. The audit team did not validate the RDT&E or Procurement funding PMA-251 provided for FY 2003 through FY 2020. We reviewed program requirements and testing documents against DoD and Navy policies and guidance to determine whether the Navy effectively established requirements and updated AAG program documentation. The DoD and Navy policies included: • Defense Acquisition Guidebook, February 16, 2011; • DoD Directive 5000.01, “The Defense Acquisition System,” November 20, 2007; • DoD Instruction 5000.2, “Operation of the Defense Acquisition System,” May 12, 2003; FOR OFFICIAL USE ONLY DODIG-2016-107 │ 19 FOR OFFICIAL USE ONLY Appendixes • DoD Instruction 5000.02, “Operation of the Defense Acquisition System,” January 7, 2015; • Secretary of the Navy Instruction 5000.2E, “Department of the Navy Implementation and Operation of the Defense Acquisition System and the Joint Capabilities Integration and Development System,” September 1, 2011; and • Secretary of the Navy Manual 5000.2, “Department of the Navy Acquisition and Capabilities Guidebook,” May 9, 2012. Use of Computer-Processed Data We did not use computer-processed data to perform this audit. Use of Technical Assistance A general engineer from the Technical Assessment Division, DoD Office of Inspector General, assisted with the audit. The engineer assisted the team in evaluating and reviewing the AAG critical design review, technology readiness assessment, and systems engineering plan. Prior Coverage During the last 5 years, the Government Accountability Office (GAO) issued the following six reports discussing the AAG. Unrestricted GAO reports can be accessed over the Internet at http://www.gao.gov. GAO Report No. GAO-15-342SP, “Defense Acquisitions–Assessments of Selected Weapon Programs,” March 2015 Report No. GAO-15-188, “Defense Acquisitions–Better Approach Needed to Account for Number, Cost, and Performance of Non-Major Programs,” March 2015 Report No. GAO-15-22, “Ford-Class Aircraft Carrier-Congress Should Consider Revising Cost Cap Legislation to Include All Construction Costs,” November 2014 Report No. GAO-14-340SP, “Defense Acquisitions–Assessments of Selected Weapon Programs,” March 2014 Report No. GAO-13-396, “Ford-Class Carriers-Lead Ship Testing and Reliability Shortfalls Will Limit Initial Fleet Capabilities,” September 2013 Report No. GAO-13-294SP, “Defense Acquisitions–Assessments of Selected Weapon Programs,” March 2013 FOR OFFICIAL USE ONLY 20 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Appendixes Appendix B AAG Test Schedule FOR OFFICIAL USE ONLY DODIG-2016-107 │ 21 Management Comments FOR OFFICIAL USE ONLY Management Comments Assistant Secretary of the Navy for Research, Development, and Acquisition Comments FOR OFFICIAL USE ONLY 22 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Management Comments Assistant Secretary of the Navy for Research, Development, and Acquisition Comments (cont’d) DEPARTMENT OF THE NAVY RESPONSE TO DODIG DRAFT REPORT #D2015-D000AE-0191.000, “ADVANCED ARRESTING GEAR PROGRAM EXCEEDED COST AND SCHEDULE BASELINES”, DATED MAY 5, 2016 Finding: Advanced Arresting Gear Exceeded Cost and Schedule Baselines The Program Manager, Aircraft Launch and Recovery Equipment (PMA-251), did not effectively manage the acquisition to meet requirements and execute testing for the AAG program. Ten years after the program entered the engineering and manufacturing development phase, the Navy has not been able to prove the capability or safety of the system to a level that would permit actual testing of the system on an aircraft carrier because of hardware failures and software challenges. This occurred because the Navy pursued a technological solution for its Ford-class carriers that was not sufficiently mature for the planned use, resulting in hardware failures to mechanical and electrical components, and software modifications to accommodate these failures. In addition, the program manager has not revised the Test and Evaluation Master Plan (TEMP) to address significant changes to the test strategy and schedule. This occurred because redesign changes required significant revisions to AAG system key components, and those changes took priority over updating the TEMP. As a result, major AAG system components required costly redesign, which delayed developmental testing and will further postpone delivery of the full AAG system capability to the CVN-78 aircraft carrier. AAG hardware and software failures and test site preparation led to the AAG program exceeding the Acquisition Category I threshold for RDT&E costs. As of October 2015, the RDT&E costs for the AAG program total $743.5 million, which was $571.5 million above the planned costs in the 2005 Acquisition Program Baseline. Developmental testing originally scheduled to end in FY 2009 will continue through FY 2018, and reliability of the system is uncertain. NAVY RESPONSE: The Navy partially concurs. The Navy concurs that the system is not yet ready to test on an aircraft carrier and that the technology was not sufficiently mature for the planned use on CVN-78. The Navy has, however, proven the capability and safety of the system by actually arresting aircraft at the Runway Arrested Landing Site (RALS). As of May 24, 2016, the system has arrested 1253 deadloads at the Jet Car Track Site (JCTS) and 13 aircraft roll-ins at RALS. Although testing has not yet occurred on an aircraft carrier, the land-based testing will result in an F/A-18 E/F Aircraft Recovery Bulletin (ARB) which will permit shipboard testing with aircraft. Attachment FOR OFFICIAL USE ONLY DODIG-2016-107 │ 23 Management Comments FOR OFFICIAL USE ONLY Assistant Secretary of the Navy for Research, Development, and Acquisition Comments (cont’d) The Navy concurs that the revised TEMP has not been approved. However, the only substantive change in the revised TEMP is to test at JCTS and RALS in parallel rather than serially, and the Navy has been executing to this revised strategy. Specifically, the Program Manager (PM) manages to the contractors’ Integrated Master Schedule (IMS) which incorporates all planned testing with earned value to track performance. The PM enacted a schedule adjustment to bring JCTS and RALS events in parallel to meet the ship delivery timeline. The schedule adjustment will permit testing aboard the ship using the F/A-18E/F ARB provided by the RALS testing. The Navy concurs that major AAG system components required redesign which led to schedule delays, cost increases, and the program exceeding the Acquisition Category I threshold. The Navy also concurs that reliability of the system is uncertain until significantly more reliability data is obtained. However, the delay in approving the revised TEMP had no direct impact on redesign, schedule, capability, or cost. The PM plans to obtain approval of the revised TEMP along with the new Acquisition Program Baseline (APB). Recommendation 1: Assistant Secretary of the Navy for Research, Development, and Acquisition perform cost-benefit analyses to determine whether the Advanced Arresting Gear is an affordable solution for Navy aircraft carriers before deciding to go forward with the system on future aircraft carriers. ASN(RDA) RESPONSE: Assistant Secretary of the Navy for Research, Development, and Acquisition concurs. In February 2016, the Navy initiated a study to determine cost and schedule requirements necessary to install a three-wire NIMITZ-class Mk 7 arresting system on the next FORD-class aircraft carrier (CVN-79). Additionally, in July 2015, the Navy initiated a NIMITZ-class aircraft carrier Analysis of Alternatives (AoA) to evaluate the performance, operational capability, costs, and schedule of developing, installing, and supporting alternative systems or approaches to meet future air wing requirements on the existing NIMITZ-class aircraft carriers. The Navy will consider both of these analyses, the revised AAG APB, and the known life-cycle costs of the Mk 7 arresting system to determine whether AAG is an affordable solution for Navy carriers before deciding to go forward with AAG on future aircraft carriers. DATE COMPLETED/ESTIMATED COMPLETION DATE: The Navy will make its determination by December 2016. Recommendation 2: Program Manager, Aircraft Launch and Recovery Equipment, update the Advanced Arresting Gear Test and Evaluation Master Plan to revise the planned test strategy, test schedule, developmental and operational funding, and add measures to support the program’s reliability growth plan before the Acquisition Category IC Acquisition Program Baseline is finalized. 2 FOR OFFICIAL USE ONLY 24 │ DODIG-2016-107 Attachment FOR OFFICIAL USE ONLY Management Comments Assistant Secretary of the Navy for Research, Development, and Acquisition Comments (cont’d) NAVAIR RESPONSE: Program Manager, Aircraft Launch and Recovery Equipment concurs. A Schedule Risk Assessment (SRA) and an Estimate At Completion (EAC) are currently being conducted. The Test and Evaluation Master Plan (TEMP) will be updated based on the existing planned efforts required to certify the system and produce the necessary Aircraft Recovery Bulletins (ARB). The TEMP will then be aligned with the Component Cost Position, the funding controls established by the resource sponsor, and serve as the basis for the AAG Acquisition Program Baseline (APB). DATE COMPLETED/ESTIMATED COMPLETION DATE: The AAG program replanning and baseline, which includes an update of the TEMP, will be complete by December 2016. Additional Navy Comments: Page 10 10 10 Paragraph/Text: Change/Edit/Add Redesign of PCS paragraph doesn’t From FY13 to indicate resolution of the failure present, the redesigned inverter has been tested and the inverter failures have not been repeated. Software Challenges paragraph 1 Add (FOUO) to the discusses the aircraft settings file 1st paragraph which is considered critical program information Software Challenges paragraph 2 Add (FOUO) to the discusses an open lien 2nd paragraph 3 Rationale The JCTS system inverter redesign has successfully arrested deadloads without repeated failure in the inverter. System liens/deficiencies are considered FOUO information Attachment FOR OFFICIAL USE ONLY DODIG-2016-107 │ 25 FOR OFFICIAL USE ONLY Glossary Glossary Acquisition Program Baseline (APB): A document that identifies the threshold and objective values for the minimum number of cost, schedule, and performance attributes that describe the program over its life cycle. Cost values reflect the life-cycle cost estimate; scheduled dates include key activities such as milestones and the Initial Operational Capability; and performance attributes reflect the operational performance required for the fielded system. Aircraft Recovery Bulletin (ARB): The ARB provides standardized operating procedures and technical guidance, and is required to conduct AAG flight operations. Developmental Testing and Evaluation: 1) Any testing used to assist in the development and maturation of products, product elements, or manufacturing or support processes. 2) Any engineering-type test used to verify status of technical progress, minimize design risks, substantiate achievement of contract technical performance, and certify readiness for initial operational testing. Developmental tests generally require instrumentation and measurements completed by engineers, technicians, or soldier operator-maintainer test personnel in a controlled environment to facilitate failure analysis. Engineering and Manufacturing Development: Engineering and manufacturing development is the third phase of the acquisition life cycle. This phase consists of two efforts-Integrated System Design and System Capability and Manufacturing Process Demonstration and begins after Milestone B. It also contains a Post‑Critical Design Review Assessment at the conclusion of the Integrated Systems Design effort. Exit Criteria: Exit criteria are program-specific accomplishments that are required before a program can progress further in the current acquisition phase or transition to the next acquisition phase. Initial Operational Test and Evaluation: Dedicated Operational Test and Evaluation conducted on production, or production representative articles, to determine whether systems are operationally effective and suitable to support a Full-Rate Production decision. Joint Capabilities Integration and Development System: Supports the Chairman of the Joint Chiefs of Staff and the Joint Requirements Oversight Council in identifying, assessing, and prioritizing joint military capability requirements. FOR OFFICIAL USE ONLY 26 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Glossary Major Defense Acquisition Program: There are two ways an acquisition program becomes a major defense acquisition program. The first is if the program is designated one by the Under Secretary of Defense for Acquisition, Technology, and Logistics. The second is if the Under Secretary of Defense (Acquisition, Technology, and Logistics) estimates the program will require an eventual total expenditure for Research, Development, Test and Evaluation, including all planned increments, of more than $365 million in FY 2000 constant dollars, or for procurement, including all planned increments, of more than $2.19 billion in FY 2000 constant dollars. Milestone: A milestone is a recommendation made and approval sought regarding starting or continuing an acquisition program. For example, Milestone A approves entry into the Technology Development phase, Milestone B approves entry into the engineering and manufacturing development phase, and Milestone C approves entry into the production and deployment phase. Milestone Decision Authority: The Milestone Decision Authority is the designated individual with overall responsibility for a program. The Milestone Decision Authority has the authority to approve entry of an acquisition program in the next phase of the acquisition process and is accountable for cost, schedule, and performance reporting to higher authority, including congressional reporting. Operational Availability: One of the components of the Availability Key Performance Parameter is percentage of time that a system or group of systems within a unit are operationally capable of performing an assigned mission and can be expressed as decimal or percentage. Parameter: A determining factor or characteristic, usually related to performance in developing a system. Post-Shakedown Availability: The purpose of the post-shakedown availability is to accomplish the correction of new construction deficiencies found during the shakedown period, correction of other contractor and government responsible deficiencies, and accomplishment of other improvements or class items as authorized. Program Executive Officer: A military or civilian official who has responsibility for directing several Major Defense Acquisition Programs and for assigned major system and non-Major Defense Acquisition Programs. FOR OFFICIAL USE ONLY DODIG-2016-107 │ 27 FOR OFFICIAL USE ONLY Glossary Program Manager: Designated individual with responsibility for and authority to accomplish program objectives for development, production, and sustainment to meet the user’s operational needs. The program manager is accountable for credible cost, schedule, and performance reporting to the Milestone Decision Authority. Reliability: Reliability measures the probability that the system will perform without failure over a specified interval under specified conditions. Reliability must be sufficient to support the warfighting capability needed in its expected operating environment. Research, Development, Test, and Evaluation: Activities for new system development or expanding the performance of fielded systems. Test and Evaluation Master Plan: Test and Evaluation Master Plan documents the overall structure and objectives of the test and evaluation program. It provides a framework within which to generate detailed test and evaluation plans and documents schedule and resource implications associated with the test and evaluation program. In addition, the TEMP identifies the necessary developmental test and evaluation, operational test and evaluation, and live-fire test and evaluation activities. FOR OFFICIAL USE ONLY 28 │ DODIG-2016-107 FOR OFFICIAL USE ONLY Acronyms and Abbreviations Acronyms and Abbreviations AAG Advanced Arresting Gear APB Acquisition Program Baseline ARB Aircraft Recovery Bulletin ASN (RDA) Assistant Secretary of the Navy for Research, Development, and Acquisition DOT&E Director, Operational Test and Evaluation JCTS Jet Car Track Site MK-7 Mark-7 NAWCAD Naval Air Warfare Center Aircraft Division PMA-251 Aircraft Launch and Recovery Equipment Program Office RALS Runway Arrested Landing Site RDT&E Research, Development, Test, and Evaluation TEMP Test and Evaluation Master Plan FOR OFFICIAL USE ONLY DODIG-2016-107 │ 29 FOR OFFICIAL USE ONLY Whistleblower Protection U.S. Department of Defense The Whistleblower Protection Enhancement Act of 2012 requires the Inspector General to designate a Whistleblower Protection Ombudsman to educate agency employees about prohibitions on retaliation, and rights and remedies against retaliation for protected disclosures. The designated ombudsman is the DoD Hotline Director. For more information on your rights and remedies against retaliation, visit www.dodig.mil/programs/whistleblower. For more information about DoD IG reports or activities, please contact us: Congressional Liaison congressional@dodig.mil; 703.604.8324 Media Contact public.affairs@dodig.mil; 703.604.8324 For Report Notifications http://www.dodig.mil/pubs/email_update.cfm Twitter twitter.com/DoD_IG DoD Hotline dodig.mil/hotline FOR OFFICIAL USE ONLY FOR OFFICIAL USE ONLY D E PA R T M E N T O F D E F E N S E │ I N S P E C T O R G E N E R A L 4800 Mark Center Drive Alexandria, VA 22350-1500 www.dodig.mil Defense Hotline 1.800.424.9098 FOR OFFICIAL USE ONLY