FY14 DOD PROGRAMS F-35 Joint Strike Fighter (JSF) Executive Summary Test Planning, Activity, and Assessment • The program focused on completing F-35 Joint Strike Fighter (JSF) Block 2B development and flight testing in an effort to provide limited combat capability to the fielded early production aircraft and to support the Marine Corps plans for declaring Initial Operational Capability (IOC) in 2015. -- The test centers sustained flight operations at nearly the planned pace through the end of November, despite stoppages and restrictions placed on the test fleet of aircraft. -- Flights sciences testing for the F-35A lagged behind its test flight and test point goals for CY14 as the test centers prioritized resources to focus on Block 2B mission systems testing. Flight sciences testing for the F-35B and F-35C maintained overall test point productivity by accomplishing additional test points for Block 3F, while lagging behind planned progress for completing Block 2B. -- Test flights using the mission systems aircraft were ahead of the plan for the year, but test point productivity for Block 2B and Block 3i lagged behind the annual plan. • In spite of the focused effort, the program was not able to accomplish its goal of completing Block 2B flight testing by the end of October. -- Slower than planned progress in mission systems, weapons integration, and F-35B flight sciences testing delayed the completion of the testing required for Block 2B fleet release. The program now projects this to occur by the end of January 2015, instead of the end of October 2014 as was previously planned. -- Restrictions imposed on the test fleet as a result of the engine failure in June reduced test point availability and slowed progress in mission systems and flight sciences testing from July through November. For example, the effect on mission systems testing was approximately 17 percent loss of productivity in accomplishing test points, from 210 points accomplished per month prior to the engine restrictions to approximately 175 points per month. -- Discoveries of deficiencies continued to occur in later versions of Block 2B software, further slowing progress. For example, completion of weapons delivery accuracy events lagged the plans for CY14 and was put on hold in August when the program discovered a deficiency in the F-35 navigation system. -- Through the end of November, 10 of 15 weapon delivery events had been completed; all events were planned to be completed by the end of October. However, the program must transition development and flight test resources to Block 3 in order to preserve an opportunity to complete the System Design and Development phase as planned in 2018. Block 2B will finish later than planned, with deficiencies remaining that will affect operational units; fixes for these deficiencies will be deferred to Blocks 3i and 3F. • In the FY13 Annual Report, DOT&E estimated that the program would complete Block 2B testing between May and November 2015 (7 to 13 months late), depending on the level of growth experienced, while assuming the program would continue test point productivity equal to that of the preceding 12 months. Since the end of October 2013, the program has made several adjustments to reduce the delay estimated in the FY13 report: -- In February 2014, while finalizing the 2014 annual plan, the program consolidated test points from plans of earlier blocks of mission systems (Blocks 1A, 1B, and 2A) with those from the Block 2B test plan and decided to account for only those test points needed for Block 2B fleet release, eliminating approximately 840 points. All of these points were planned to be accomplished as of the DOT&E report. This reduction amounts to approximately four months of testing. F-35 JSF 39 FY14 DOD PROGRAMS -- F urther adjustments to the baseline number of test points needed for Block 2B fleet release were made in June 2014, resulting in additional reduction of points planned for the year. Although the program added points for new testing requirements (i.e., Manual Ground Collision Avoidance System), they also eliminated points that were assessed as no longer required. These adjustments resulted in the net reduction of 135 points. -- The program continued to experience an average test point growth rate throughout CY14 higher than planned (91 percent growth experienced through the end of November, 45 percent planned), but lower than experienced in CY13 (124 percent). -- The program realized a higher test point productivity rate per aircraft in CY14 than in CY13 (averaging 40 points per aircraft per month through the end of November, compared to 35). -- The program delayed plans to transition aircraft out of the Block 2B configuration to the Block 3i configuration, allowing more mission systems test aircraft to be available to contribute to Block 2B testing. At the time of this report, only AF-3 had been modified to the Block 3i configuration, among the six mission systems test aircraft assigned to the Edwards AFB test center, California, where the majority of the mission systems testing is accomplished. BF-5, a mission systems test aircraft assigned to the Patuxent River test center, Maryland, was modified into the Block 3i configuration in September and completed limited Block 3i testing prior to entering climatic testing later in the month. • Based on test point accomplishment rates experienced since October 2013, the program will complete Block 2B development in February 2015. -- ­ This estimate assumes no further growth in Block 2B testing (this is possible only if the current version entering test is the final Block 2B version) and productivity at the current rate. It further assumes all current Block 2B mission systems aircraft staying in the Block 2B configuration through the end of January 2015 (the program’s estimated completion date for Block 2B development), then one F-35B and one F-35C mission systems test aircraft converting to Block 3i while the other three stay in the Block 2B configuration until developmental testing is complete. Also, the operating restrictions stemming from the engine failure must be relieved for the test aircraft such that all blocked test points are made available. -- ­ Completion of Block 2B development by the end of January will, therefore, require a significant increase in test point productivity and/or elimination of additional test points. • In April, the program accepted a DOT&E recommendation that the Block 2B Operational Utility Evaluation (OUE), which was being planned for CY15, should not be conducted and that instead, resources should be focused on conducting limited assessments of Block 2B capability and re-allocated 40 F-35 JSF to assist in the completion of development and testing of Block 3i and Block 3F capabilities. -- This recommendation was based on DOT&E’s review of Block 2B progress and assessment of the program’s ability to start the Block 2B OUE as planned without creating a significant impact to Block 3F development. -- The Program Office, JSF Operational Test Team, and Service representatives then began working to “re-scope” use of operational test aircraft and operational test activities in lieu of the OUE—detailed planning is still under development. The scope of the operational test activities will be limited until the flight restrictions induced by the engine failure are removed from the operational test aircraft. Availability of the operational test aircraft will continue to be affected in CY15 and CY16 by the depot time required for modifications. F-35A Engine Failure • As a result of the engine failure that occurred in an F-35A in late June, the program imposed aircraft operating limitations (AOL) on all variants of F-35 aircraft at the flight test centers and operational/training bases. These AOLs were: -- Maximum speed of 1.6 Mach (0.9 Mach for production aircraft at operational/training bases), -- Maximum g-load of 3.2 g for test aircraft and 3.0 for production aircraft, -- Maneuvers limited to half-stick roll rate and 18 degrees angle of attack -- No rudder input, unless required for safe flight (production aircraft restriction only) -- Note: In some circumstances during flight test (but not in operational/training aircraft), exceedances were permitted and testing continued, controlled by the flight test team monitoring the aircraft, on an aircraft-by-aircraft basis (i.e., individual aircraft are cleared for specific test points). • Due to the AOL, numerous test points needed for the Block 2B fleet release and Marine Corps IOC were blocked and cannot be attempted until the restrictions are lifted. -- These test points include: ▪▪ Loads and buffet, Short Take-off and Vertical Landing (STOVL) envelope expansion, and propulsion testing for F-35B flight sciences ▪▪ Loads and buffet for F-35A flight sciences testing ▪▪ Manual ground collision avoidance system testing (for both aircraft). The manual ground collision avoidance system is a warning system that alerts the pilot that the state of aircraft attitude and altitude may be entering an unsafe condition (Service IOC requirement). -- There was also a requirement to inspect the engine with borescope equipment after no more than three flight hours; this creates additional down time and places stringent scheduling requirements, which negatively affects aircraft availability. ▪▪ Restrictions for test aircraft were gradually reduced between June and November, allowing access to more test points. The program developed a procedure to FY14 DOD PROGRAMS “rub-in” the seal in the stators of the engines in the test aircraft. Once this procedure was accomplished, restrictions were eased to allow greater g and angle of attack, but not to the full limits of the planned Block 2B envelope. ▪▪ The program began installing “pre-trenched” stators (where clearance between the stator and rotor has already been cut into the seal and no rub-in procedure is necessary) in the engines of the test aircraft in October, as they became available, to remove the restrictions associated with the engine failure. By the end of November, 6 of the 18 test aircraft had the pre-trenched stators installed. The program plans to have the engines in all developmental test aircraft modified by the end of February 2015. Also, the borescope inspection requirements were removed in November, with the latest revision of the list of restrictions. However, fielded production aircraft remained restricted at the time of this report. Mission Data Load Development and Testing • The F-35 relies on mission data loads – which are a compilation of the mission data files needed for operation of the sensors and other mission systems components – working in conjunction with the system software data load to drive sensor search parameters and to identify and correlate sensor detections of threat radar signals. The loads will be produced by a U.S. government lab, the U.S. Reprogramming Lab. -- The first two mission data loads support the Marine Corps IOC, planned for July 2015. Because the lab received its equipment late from the major contractor who produces the equipment, and with limited capability, the first two mission data loads will not be available until November 2015. -- Mission data loads undergo a three-phased lab development and test regimen, followed by flight test. The current plans are to certify the first two mission data loads in November 2015 after flight testing occurs between March and October 2015. Although this is later than desired by the program and the Marine Corps, truncating the mission data load development and conducting flight testing early on a limited open-air range for the purpose of releasing a mission data load in mid-2015 would create significant operational risk to fielded units, since the load will not have completed the planned lab test regimen and because the test infrastructure on the open-air range is capable of verifying only a small portion of the mission data. Weapons Integration • Progress in weapons integration, in particular the completion of planned Block 2B weapon delivery accuracy (WDA) events, has been less in 2014 compared to that planned by the program. The program planned to complete all 15 Block 2B WDA events by the end of October, but completed only 7. Through the end of November, the program completed 10 Block 2B WDA events and deferred 2 to Block 3F testing due to deficiencies and limitations in Block 2B capabilities. The remaining 3 Block 2B WDA events are scheduled to be completed by the end of January 2015. -- Multiple deficiencies in mission systems, aircraft grounding, and subsequent flight restrictions caused by the June engine failure all contributed to the limited progress. -- In addition, all WDA events were put on hold in August, when a deficiency in the aircraft’s navigation solution was discovered. Corrections to the deficiency were tested and confirmed in October, permitting Block 2B WDA events to restart in November. Suitability • Overall suitability continues to be less than desired by the Services, and relies heavily on contractor support and unacceptable workarounds, but has shown some improvement in CY14. -- Aircraft availability was flat over most of the past year, maintaining an average for the fleet of 37 percent for the 12-month rolling period ending in September – consistent with the availability reported in the FY13 DOT&E report of 37 percent for the 12-month period ending in October 2013. However, the program reported an improved availability in October 2014, reaching an average rate of 51 percent for the fleet of 90 aircraft and breaking 50 percent for the first time, but still short of the program objective of 60 percent set for the end of CY14. The bump in availability in October brought the fleet 12-month average to 39 percent. -- Measures of reliability and maintainability that have Operational Requirements Document (ORD) requirements have improved since last year, but all nine reliability measures (three for each variant) are still below program target values for the current stage of development. -- The reliability metric that has seen the most improvement since May 2013 is not an ORD requirement but a contract specification metric, mean flight hour between failures scored as “design controllable” (which are equipment failures due to design flaws). For this metric, the F-35B and F-35C are currently above (better than) program target values, and F-35A is slightly below (worse than) the target value but has been above the target value for several months over the last year. Live Fire Test and Evaluation (LFT&E) • The F-35 LFT&E program completed two major live fire test series using an F-35B variant full-scale structural test article. Preliminary evaluations are that the tests: -- Demonstrated the capabilities of multiple structural wing load paths and aft boom structure to mitigate threat-induced large scale structural failure. -- Confirmed the expected vulnerabilities of the fuel tank structure. F-35 JSF 41 FY14 DOD PROGRAMS due to ballistically-induced shorting of the 270 Volt and 28 Volt flight control electrical systems. Protection on the 28 Volt electrical system (designed for lightning protection) provides tolerance to such a single ballistic shorting event and is unlikely to result in a loss of aircraft. • The F-35 program continues to make progress in assessing the survivability of the F-35 to unconventional threats. Development of the chemical and biological agent protection and decontamination systems will be evaluated in the full-up system-level decontamination test planned for FY16. The Navy has been testing the vulnerability of the F-35B electrical and mission systems to electromagnetic pulse (EMP), and plans to complete this testing by 2QFY15. • The program is making advances in assessing the lethality of the 25 mm x 137 mm PGU-48 Frangible Armor Piercing (FAP) round, a designated round for the F-35A variant, and the PGU-32/U Semi-Armor Piercing High Explosive Incendiary-Tracer (SAPHEI-T) ammunition currently designated for the F-35B and F-35C variants. -- D emonstrated the expected cascading damage vulnerability to fuel ingestion, fuel and hydraulic fire, and hydrodynamic ram events. • Engine live fire tests in FY13 and prior live fire test data and analyses demonstrated vulnerability to engine fire, either caused by cascading effects or direct damage to engine fuel lines and fueldraulic components. Additional details and analyses of the uncontained F135 fan blade release and subsequent fuel fire in an F-35A at Eglin AFB in June are needed to support and update the existing engine vulnerability assessment. • The program demonstrated performance improvements of the redesigned fuel tank ullage inerting system in the F-35B ground-based fuel system simulator. However, aircraft ground and flight tests, designed to validate the fuel system simulator tests and aircraft system integration, revealed redesign deficiencies that require further hardware and software modifications. • Lockheed Martin provided test and analysis results to resolve the concern expressed in FY13 for the potential aircraft loss Actual versus Planned Test Metrics through November 2014 Test Flights All Testing Flight Sciences All Variants F-35B Only F-35A Only F-35C Only Mission Systems 2014 Actual 1,268 313 197 286 472 2014 Planned 1,209 296 262 261 390 Difference from Planned 4.9% 5.7% -24.8% 9.6% 21.0% Cumulative Actual 5,046 1,648 1,194 944 1,260 Cumulative Planned 4,674 1,471 1,205 894 1,104 Difference from Planned 8.0% 12.0% -0.9% 5.6% 14.1% Test Points All Testing All Variants 2014 Baseline Accomplished 7,055 Flight Sciences1 F-35B Only F-35A Only Mission Systems F-35C Only 2B 3F 2B 3F 2B 3F 1,070 846 546 708 768 1,453 Block 2B2 Block 3i Block 3F Other 1,126 177 0 361 2014 Baseline Planned 7,471 1,127 619 583 1,356 922 648 1,490 276 74 376 Difference from Planned -5.6% -5.1% 36.7% -6.3% -47.8% -16.7% 124.2% -24.4% -35.9% -100.0% -4.0% Added Points 1,756 0 Test Point Growth Rate Total Points Accomplished in 20143 119 236 329 1,021 51 0 24.9% 6.2% 18.8% 14.8% 90.7% 28.8% 0% 0% 8,811 2,035 1,490 2,550 2,147 228 0 361 Cumulative SDD Actual4 34,888 11,689 9,269 8,322 3,872 177 0 1,559 Cumulative SDD Planned 35,683 11,252 10,056 7,399 4,359 276 74 2,267 12.5% -11.2% -35.9% N/A -31.2% 529 523 3,811 1,870 Difference from Planned -2.2% Estimated Test Points Remaining 22,956 3.9% 77 7,013 -7.8% 54 4,049 150 4,880 1. Flight Sciences Test Points are shown separately for Block 2B and Block 3F. Flight envelopes differ in airspeed, maximum allowable g, and weapons carriage, depending on variant. 2. Includes Block 0.5, Block 1, and Block 2A quantities for Cumulative Actual and Cumulative Planned 3. Total Points Accomplished = 2014 Baseline Accomplished + Added Points 4. SDD – System Design and Development 42 F-35 JSF FY14 DOD PROGRAMS System • The F-35 Joint Strike Fighter (JSF) program is a tri-Service, multi-national, single seat, single-engine family of strike aircraft consisting of three variants: - F-35A Conventional Take-Off and Landing (CTOL) - F-35B Short Take-Off/Vertical-Landing (STOVL) - F-35C Aircraft Carrier Variant (CV) • It is designed to survive in an advanced threat (year 2015 and beyond) environment using numerous advanced capabilities. It is also designed to have improved lethality in this environment compared to legacy multi-role aircraft. • Using an Active Electronically Scanned Array radar and other sensors, the F-35 is intended to employ precision-guided bombs such as the Joint Direct Attack Munition (JDAM) and Joint Standoff Weapon, AIM-120C radar-guided Advanced Medium-Range Air-to-Air Missile, and AIM-9 infrared-guided short-range air-to-air missile. • The program provides mission capability in three increments: - Block 1 (initial training, two increments were fielded: Blocks 1A and 1B) Test Strategy, Planning, and Resourcing • In March, DOT&E recommended to the USD(AT&L) that the Block 2B Operational Utility Evaluation (OUE), which was being planned to occur in mid-2015 in accordance with the approved Test and Evaluation Master Plan (TEMP), should not be conducted. Instead, resources should be focused on conducting limited assessments of Block 2B capability and re‑allocated to assure the completion of development and testing of Block 3i and Block 3F capabilities. This recommendation was based on DOT&E’s review of Block 2B progress and assessment of the program’s ability to start the Block 2B OUE as planned without creating a significant impact to Block 3F development. - The factors that led to the DOT&E recommendation include: poor operational suitability, an inability to prepare pilots with adequate training and approved tactics on the planned schedule, and the deferral to Block 3 of operationally-relevant deficiencies that would affect performance. It was clear in March that aircraft availability for operational testing would be driven by the long timelines required to modify and retrofit the early production operational test aircraft to the Block 2B configuration, which would not be complete until mid‑2016. DOT&E assessed that delaying the Block 2B OUE until late 2016, as opposed to cancelling it, would have a negative impact on the program’s ability to complete development of the full Block 3F combat capability in a timely manner. - In April, in coordination with the Service Acquisition Executives and the JSF Program Executive Officer, the USD(AT&L) agreed with the DOT&E recommendation and approved revising the operational test period that - Block 2 (advanced training in Block 2A and limited combat in Block 2B ) - Block 3 (limited combat in Block 3i and full combat in Block 3F) • The F-35 is under development by a partnership of countries: the United States, Great Britain, Italy, the Netherlands, Turkey, Canada, Australia, Denmark, and Norway. Mission • A force equipped with F-35 units should permit the Combatant Commander to attack targets day or night, in all weather, and in highly-defended areas of joint operations. • F-35 will be used to attack fixed and mobile land targets, enemy surface units at-sea, and air threats, including advanced cruise missiles. Major Contractor Lockheed Martin, Aeronautics Division – Fort Worth, Texas was allocated for the Block 2B OUE in the TEMP into a re-scoped effort of assessing the limited Block 2B set of capabilities. The JSF Operational Test Team, JSF Program Office, and the Services’ operational test agencies began re-planning the Block 2B operational test period and activities. - By the middle of October, five of the six F-35A operational test aircraft assigned to the Edwards AFB, California, operational test squadron had been converted to the Block 2B configuration and loaded with a version of Block 2B software equivalent to the one being flown on the developmental test aircraft. The sixth F-35A operational test aircraft began an extended modification period at the depot in September and is scheduled to be returned to Edwards AFB in February 2015 in the Block 2B configuration. These operational test aircraft, although not in a full Block 2B operationally-representative configuration as would have been necessary to start the OUE, will be used to accomplish both developmental and operational testing events. They will be loaded with the latest version of Block 2B software as it becomes available and is determined airworthy for operational test purposes. - Program schedule pressures that caused DOT&E to recommend not completing the Block 2B OUE as planned increased throughout CY14. For example, Block 2B flight testing, which was scheduled to be complete in October 2014, is now projected by the Program Office to complete in January 2015. Aircraft depot modification plans are another example. The program developed plans to upgrade fielded production aircraft from Lots 3 through 5, which includes operational test aircraft planned F-35 JSF 43 FY14 DOD PROGRAMS for use in the OUE, to the full Block 2B configuration. These plans show that all of the operational test aircraft which were planned for the Block 2B OUE will not be in the full Block 2B configuration until September 2016, 21 months later than would have been needed to conduct the OUE. • DOT&E conditionally approved Revision 4 of the TEMP in March 2013, under the provision that the program revise the master schedule so that there was no overlap of spin-up training for IOT&E and the certification period needed for the Services’ airworthiness authorities to approve a flight clearance with the software to be used for IOT&E. Specifically, this would require the program to adjust the start of the spin-up training from February to July 2017, coinciding with an Operational Test Readiness Review. This adjustment also moved the start of IOT&E to January 2018, vice August 2017, and hence pushed the completion of IOT&E into FY19. In spite of the conditional approval, the program continues to show schedules that plan for the start of spin-up training in February 2017 and the start of IOT&E in August 2017. In addition to the justifications for adjusting the schedule that DOT&E outlined in the March 2013 TEMP conditional approval memo, the program has encountered more challenges to meeting the planned schedule to start IOT&E in August 2017 and completing System Design and Development (SDD) in 2018. These challenges include: - Block 3i flight testing began in late May 2014, five months later than the program’s baseline plan. - Block 3F flight testing was scheduled to start in November 2014 according to the program’s baseline plan; current program estimates show the testing starting no earlier than late February 2015, three months late. - Modification plans for the IOT&E aircraft will likely not have aircraft ready to begin the start of spin-up training in February 2017 as planned by the errant schedule submitted in the TEMP. To become Block 3F capable, the operational test aircraft require extensive modifications, including new processors, in addition to those needed for Block 2B capability. Block 3F modification plans are taking into consideration some modifications that already have engineering solutions and approved designs. Other modifications – although known to be required – are still in the formal change approval process leading to parts and modification kits being developed and procured from suppliers. Some of these latter modifications are currently not scheduled to be available until May 2017 for the F-35A and February 2018 for the F-35C, which is later than needed to support spin-up training for IOT&E. - There is carryover of incomplete work from Block 2B development into Block 3. In coordination with the Services, the program completed a review in June of 1,151 open deficiency reports identified during Block 2B development and earlier. Of these, 572 were rated as relevant to and affecting Block 2B capability; 579 were carried over for consideration for corrections in Block 3. 44 F-35 JSF - The program removed test points that were originally planned to be flown to support Block 2B fleet release (approximately 1,000 mission systems test points); some of these points may carry over and need to be flown during Block 3F development. - In order to account for these realities and reduce the overlap of spin-up training for IOT&E with final development activities (such as the activities that provide the certifications for use of the final configuration), the program master schedule should be adjusted to reflect these realities and depict the start of spin-up training for IOT&E no earlier than the Operational Test Readiness Review in November 2017, and the start of IOT&E for Block 3F to occur six months later, in May 2018 and completing in May 2019. If it becomes apparent that spin-up training entry criteria (e.g., providing properly configured production-representative aircraft in sufficient numbers) cannot be met on this timeline, then the schedule will have to be adjusted again. • This report reviews the program by analyzing the progress of testing and the capability delivered as a function of test results. The program plans a specific set of test points (discrete measurements of performance under specific test conditions) for accomplishment in a given calendar year. In this report, test points planned for a given calendar year are referred to as baseline test points. In addition to baseline test points, the program accomplishes test points added for discovery, regression of new software, and verification of fixes to deficiencies identified in flight test; these additional points are referred to as “growth” points in this report. Cumulative SDD test point data refer to the total progress towards completing development at the end of SDD. F-35A Engine Failure • An F-35A aircraft assigned to the training center at Eglin AFB, Florida experienced an engine failure on take-off on June 23, 2014. The aircraft was a Lot 4 production aircraft, delivered to Eglin AFB in June 2013, and had flown approximately 160 hours prior to the incident. • As a result of the engine failure, the Program Office and the Services initiated a series of actions that affected flight operations for both the fielded production aircraft and the test aircraft. - The Program Office instituted an operational pause to flight testing at the test centers on June 25, and the contractor suspended acceptance flight operations at the production plant. - A fleet-wide stop order was issued by the Program Office on July 4, which officially suspended flight operations and ground engine runs. This order also initiated requirements to visually inspect the affected engine components using special equipment called a borescope. - On July 8, the program began lifting restrictions by permitting engine runs up to 30 percent power for engines that had completed the borescope inspections. FY14 DOD PROGRAMS - On July 16, the program began permitting limited flight operations for F-35B and F-35C aircraft with stringent flight limitations and continued inspection requirements. - Aircraft operating limitations have been incrementally revised to permit flight testing to continue. By mid‑September, the flight sciences aircraft of each variant had been cleared to continue testing without engine‑imposed envelope restrictions. The rest of the test fleet continues to conduct flight testing, but under a restricted flight envelope. The program plans to have all engine‑imposed restrictions removed from the developmental test fleet by the end of February 2015, after modifications to the engines of each aircraft are complete. - On October 10, the program confirmed that excessive rubbing between the hard polyamide seal of the second stage stator and the titanium interface of the integrated blade third stage rotor led to the engine failure. This excessive rubbing occurred on a previous flight while maneuvering within the limited, cleared training envelope. Friction from the rubbing created excessively high temperatures within the titanium rotor, creating small cracks that eventually led to catastrophic failure of the rotor during the take-off on June 23. It is not clear what occurred differently than expected in the air vehicle and/or engine that caused the excessive rubbing. • Inspections of the engines on all variants led to discoveries on nine production and test aircraft requiring engine replacement. • As of July 23, restrictions on the flight test aircraft blocked 53 percent (1,357 of 2,575) of the remaining Block 2B test points; however, test points have incrementally become available as the flight restrictions were relaxed on some of the test aircraft beginning in September after the test centers complied with actions found necessary by the root cause analysis. • Resolution of the way forward with the engines in test and production aircraft was ongoing at the time of this report. - The program developed and tested an engine “rub-in” procedure. This procedure is designed to ensure the engines have sufficient clearance between the rotors and seals to prevent excessive rubbing during maneuvering. The rub-in process is accomplished through two flights during which a specific profile is flown to accomplish the procedure, followed by inspections. As flight test jets completed the rub-in procedure, they were cleared to accomplish some of the blocked test points and fly within an expanded, although still limited, flight envelope. - The program is developing an interim redesign of the seal, which will have grooves pre-cut in the polyamide material to provide clearance between the seal and the rotor and will prevent excessive rubbing during maneuvering. A prototype of this “pre-trenched” seal was flight tested in October and is being installed in the engines of each developmental test aircraft. - The program is working with the engine contractor to develop a new redesigned seal for production engines. Plans on a final design were not complete at the time of this report. F-35A Flight Sciences Flight Test Activity with AF-1, AF-2, and AF-4 Test Aircraft • F-35A flight sciences testing focused on: -- Completing the full Block 2B flight envelope -- High angle of attack testing (clean wing for Block 2B and with external stores for Block 3) -- Ground and flight testing of the redesigned fuel tank ullage inerting system (i.e., inerting of the space not occupied by fuel in a fuel tank), consisting of the On-Board Inert Gas Generation System (OBIGGS) and associated fuel pressurization and ventilation system -- Start of Block 3F loads and flying qualities testing, predominantly flying with externally-loaded air‑to‑ground and air-to-air weapons -- Regression testing of updated versions of vehicle systems software -- Testing of the aerodynamic loads in the gun bay. (Note:  Block 3F F-35A aircraft will have an internal gun; F-35B and F-35C aircraft will use a podded gun mounted on the center fuselage station.) • Restrictions imposed on the fleet from the June engine failure coupled with the focus on Block 2B mission systems testing hampered progress in F-35A flight sciences testing. • Excessive free-play in the rudder hinges on AF-2 required extended downtime for repair. These repairs occurred in July during the period of restrictions from the engine fire. F-35A Flight Sciences Assessment • Through the end of November, test point accomplishment in CY14 was 6 percent behind the plan for accomplishing Block 2B points and 48 percent behind for Block 3F. The test team flew 25 percent fewer test flights than planned for the year (197 flown; 262 planned). Prioritization of flight test resources to focus on mission systems flight testing for Block 2B at the Edwards AFB test center (where mission systems and F-35A flight sciences testing are conducted) reduced the opportunity for flight science testing to achieve planned progress in Block 3F testing. • The plan for Block 2B test points was adjusted in CY14, resulting in the net reduction of 343 of 926 (37 percent) of the original points planned for the year. The program designated these points as no longer required for Block 2B fleet release. • Restrictions imposed from the June engine failure initially blocked access to almost all (254 of 261) remaining Block 2B flight sciences test points. The program was able to relax the restrictions on an aircraft-by-aircraft basis beginning in September, providing access to some of the blocked test points; all points were available as of the end of October. The prioritization of mission systems testing coupled with the restrictions from the engine failure created a debt of flight sciences testing on the F-35A that will need to be overcome in CY15 and early CY16 for the program to maintain Block 3F flight envelope release schedule. F-35 JSF 45 FY14 DOD PROGRAMS • The program added 236 flight sciences test points through the end of November, equating to a growth rate of 19 percent, which is near the planned growth rate of 17 percent. • AF-4 underwent a modification from March through May, during which the redesigned fuel tank ullage inerting system was installed. This modification and testing is part of the effort to address deficiencies in lightning protection and vulnerability reduction to ballistic threats. Testing to assess on-the-ground inerting performance of the redesign and to validate modeling results was completed in May. Flight testing to assess the fuel system pressurization and ventilation capability of the redesign was mostly completed in June; dive test points were blocked by restrictions imposed by the engine failure. Further testing to assess corrections to the redesign is scheduled to occur in December 2014. • Discoveries in F-35A flight sciences testing: - Higher than expected wear in the rudder hinges of AF-2 was discovered during routine inspections, following flight testing in regions of the envelope where higher dynamic loads are exerted on the rudder surfaces. Replacement of the clevis of the middle rudder hinges was necessary, and additional inspections to check rudder free play are required. - AF-4 encountered a blown tire and damage to the main landing gear while conducting crosswind landing testing in February, requiring a two-week down period for repairs. - Ground testing on aircraft AF-4 revealed that pressure from the OBIGGS inadvertently pushes fuel between tanks. Per engineering directive, the test team removed and capped the inert air distribution lines that were causing the fuel transfer as a temporary measure to permit AF-4 to continue developmental testing of other (non-OBIGGS) test requirements. Further modifications to software and the addition of a control valve were made to AF-4 in November for testing planned for December 2014. - Inerting the aircraft on the ground with external nitrogen forces fuel to vent from the fuel tanks under certain fuel states. The procedure to purge the fuel system with external nitrogen was introduced with the redesigned ullage inerting system to provide lightning protection on the ground. The program plans to address this fuel venting by testing two additional check valves on AF-4 for incorporation into the final design. • Weight management of the F-35A is important for meeting air vehicle performance requirements and structural life expectations. These estimates are based on measured weights of components and subassemblies, calculated weights from approved design drawings released for build, and estimated weights of remaining components. These estimates are used to predict the weight of the first Lot 7 F-35A aircraft (AF-72), planned for delivery in 46 F-35 JSF August 2015, which will be the basis for evaluating contract specification compliance for aircraft weight. -- According to these reports, the program has reduced weight by 16 pounds in CY14 (from January to October estimate). The current estimate of 29,016 pounds is 355 pounds (1.2 percent) below the planned not-to-exceed weight of 29,371 pounds. -- The program has demonstrated positive weight management of the F-35A over the past 38 months, showing a net loss of 123 pounds in the estimates from August 2011 to October 2014. The program will need to ensure the actual aircraft weight meets predictions, as well as continue rigorous management of the actual aircraft weight beyond the technical performance measurements of contract specification in CY15 through the balance of SDD to avoid performance degradation that would affect operational capability. F-35B Flight Sciences Flight Test Activity with BF-1, BF-2, BF-3, BF-4, and BF-5 Test Aircraft • F-35B flight sciences focused on: -- Continued expansion of the Block 2B flight envelope, including weapons separation testing -- High angle of attack testing -- Wet runway testing (completed with BF-4 in May at Edwards AFB) -- Testing of landing control authority in crosswind conditions -- Testing with external air-to-air and air-to-ground weapons (Block 3F capability) -- STOVL mode flight operations -- Testing of fuel dump capability with a new valve and seals -- Ground and flight testing of the redesigned ullage inerting system -- Flight testing in support of expeditionary operations (i.e., landing on matted runways, AM-2 padding) -- Preparations for and conducting climatic testing on BF-5 in the climatic chamber F-35B Flight Sciences Assessment • Through the end of November, test point accomplishment for CY14 was 5 percent behind the plan for accomplishing Block 2B points and 37 percent ahead of the plan for Block 3F points. Test flights were slightly ahead of plan (313 flown; 296 planned). The test force maintained test point productivity by accomplishing test points from the Block 3F test plan for flying qualities, air data, propulsion, and loads in the STOVL mode and with external stores. The program projects the completion of Block 2B flight sciences testing to occur by the end of December 2014, two months later than planned. • This projection follows adjustments made by the Program Office to the plan for Block 2B test points in CY14, which resulted in the net reduction of 394 out of 1,545 (26 percent) FY14 DOD PROGRAMS of the points planned for the year. These points were reviewed by the contractor and the Program Office, and designated as no longer required for Block 2B fleet release and Marine Corps IOC. This reduction brought the total 2014 plan to 1,151 points, 1,127 of which were planned to be completed by the end of November. • Crosswind landing testing in the conventional landing mode (not vertical landing) was not completed; but sufficient testing was accomplished to clear landings up to 20 knots of crosswind, short of the ORD requirement of 25 knots of crosswind. • BF-4 was modified with the redesigned fuel tank inerting system late in CY13. Testing to assess ground inerting performance and validate results from the fuel system simulator – a full mock-up surrogate of the F-35B fuel system – was completed in December 2013. Further testing of the tank inerting system did not occur until September 2014, as other test requirements (i.e., wet runway testing) needed to be conducted with BF-4, and known deficiencies needed to be addressed with corrections to software. Flight testing of the tank inerting system is ongoing. Regression testing to verify correction of deficiencies in the redesign discovered from ground testing (on the aircraft and in the simulator) was conducted in early October and will continue in December after updated software is released to the test aircraft for flight testing. • Discoveries in F-35B flight sciences testing included: -- Early fuel dump testing in 2011 discovered that fuel does not completely eject overboard, but collects in the area between the flaperons and the aircraft structure and runs inboard toward the Integrated Power Package exhaust outlet, creating a potential fire hazard. Testing of a redesigned dump nozzle, improved seals for the flaperons, and heat-shrinkable tubing added to wiring harnesses for protection in the event of fuel wetting have all contributed to a new fuel dumping procedure. -- Inerting performance in certain fuel tanks during ground testing of the redesigned ullage inerting system did not meet the performance demonstrated during fuel system simulator testing. To address this discrepancy, an additional OBIGGS distribution line was installed on aircraft BF-4. The discovery affects all variants; retrofit kits have been developed for the F-35A and F-35C variants. -- The redesigned ullage inerting system has the potential to generate pressure spikes when pressure in the aerial refueling manifold is released into the fuel tanks. A blanking plate was installed on BF-4 to isolate the aerial refueling manifold from the OBIGGS as a temporary measure to allow it to ferry to Edwards AFB to conduct testing on wet runways. A software modification of the valve control logic was tested in late September, allowing removal of the blanking plate. -- The aircraft does not maintain residual inerting after flight for the required interval of 12 hours, which is a lightning protection requirement. Residual inerting is a result of the inert air produced by the OBIGGS remaining in the ullage area of the fuel tanks after a flight. The program is investigating a correction to this problem. If the residual inerting cannot be improved, aircraft maintainers will be required to purge fuel tanks with external nitrogen more frequently or alternative lightning protection strategies (e.g., lightning‑protected shelters, will have to be adopted. -- In heavy buffet conditions, which occur between 20 and 26 degrees angle of attack, faults occurred in the inertial measurement units (IMUs) in the aircraft that degraded the flight control system (two of three flight control channels become disabled), requiring a flight abort. This condition blocked 28 test points needed for the Block 2B fleet release. The program made adjustments to the flight control software, which were tested in late October and the test points were unblocked, enabling some testing in the heavy buffet conditions to continue. However, nine additional test points needed for the Block 2B fleet release remained blocked at the end of November because of high dynamic loads on the rudder at lower altitudes, in the same angle of attack range, and require additional analyses and mitigation to complete. • Weight management of the F-35B aircraft is critical to meeting the Key Performance Parameters (KPPs) in the ORD, including the vertical lift bring-back requirement. This KPP requires the F-35B to be able to fly an operationally representative profile and recover to the ship with the necessary fuel and balance of unexpended weapons (two 1,000-pound bombs and two AIM-120 missiles) to safely conduct a vertical landing. These estimates are based on measured weights of components and subassemblies, calculated weights from approved design drawings released for build, and estimated weights of remaining components. These estimates are used to predict the weight of the first Lot 7 F-35B aircraft (BF-44), planned for delivery in August 2015, which will be the basis for evaluating contract specification compliance for aircraft weight. -- Weight reports for the F-35B as of October show that the program added 18 pounds to the estimated weight in CY14 and a net addition of 82 pounds over the last 38 months (August 2011 to October 2014). The current estimate of 32,412 pounds is 337 pounds (1 percent) below the objective vertical lift bring-back not-to-exceed weight of 32,749 pounds. -- Managing weight growth for the F-35B will continue to be a challenge in light of the small weight margin available and the possibility for continued discovery through the remaining SDD phase, which extends two years past the delivery of the first Lot 7 aircraft, planned for August 2015. The program will need to ensure actual weights meet predictions. Known modifications and retrofits for production aircraft in Lots 2 through 6 will add weight to those aircraft, varying from 210 pounds for the Lot 3 aircraft to 17 pounds for the Lot 6 aircraft. In F-35 JSF 47 FY14 DOD PROGRAMS addition, the program is currently redesigning the FS496 bulkhead for Lot 9 production aircraft and later as a result of the failure of that bulkhead in the ground test article during durability testing. The effect of the redesigned bulkhead on the weight of the aircraft is not yet known. • The following table, first displayed in the FY11 Annual Report and updated each year, describes observed door and propulsion problems by component and identifies the production cut-in of the correction or update, if known. F-35B Door and Propulsion Problems Category Design Fix and Test Status Production Cut-In Inadequate life on door locks, excessive wear and fatigue due to the buffet environment, inadequate seal design. New designed doors are being installed on Low-Rate Initial Production (LRIP) aircraft as part of the on-going modification plan; 14 completed through the end of September. Fatigue testing of the doors started in November 2012 and completed the planned 2 lifetimes of testing at the end of September 2014. Inspections were ongoing as of the end of November, with no discoveries. Fix appears to resolve problem. BF-38 LRIP Lot 6 2014 Lift fan drive shaft is undergoing a second redesign. Original design was inadequate due to shaft stretch requirements to accommodate thermal growth, tolerances, and maneuver deflections. First redesign failed qualification testing. New design completed qualification testing and appears to reduce the problem. Full envelope requirements are currently being met on production aircraft with an interim design solution using spacers to lengthen the early production drive shaft. New design is dependent on updated propulsion software load planned to be available by Lot 9. BF-56 LRIP Lot 9 2016 Clutch Lift fan clutch has experienced higher than expected drag heating during conventional (up and away) flight during early testing. New clutch plate design, with more heat-tolerant material, is complete. Clutch plates are being thinned on Lot 5 and 6 aircraft, at the expense of reduced life (engagements) to the clutch, to prevent drag heating. Solutions appear to be effective; very few hot clutches are experienced in fleet wide operations now. Tail TBD Mid-LRIP Lot 8 2015 Propulsion Roll Post Nozzle Actuator Roll post nozzle bay temperatures exceed current actuator capability; insulation is needed to prevent possible actuator failure during vertical lift operations. Insulation between the roll post nozzle bay and the actuators is being installed in pre-Lot 6 aircraft to allow unrestricted operations, however the actuators must be replaced at 1,000 hour intervals. New actuators will be installed in Lot 6 aircraft and beyond, removing the requirements for the insulation and extending the service life to 4,000 hours. BF-38 LRIP Lot 6 2015 Propulsion Lift Fan Inter Stage Vanes (ISV) Vanes between stages of the lift fan experience excessive vibration/flutter during mode 4 flight when temperature is below 5oF or above 107oF degrees and speed is greater than 130 knots calibrated airspeed. Aircraft are restricted from mode 4 flight outside the temperature and speed restrictions noted. A unit level Time Compliant Technical Directive is being accomplished for 48 fielded lift fans to replace the ISVs with a new ISV made of more durable material tolerant over a greater temperature range, with production cut in on new Lift Fans. New vanes retrograded in fielded aircraft, incorporated in new production lift fans Structure Propulsion Propulsion Component Auxiliary Air Inlet Door (AAID) Drive Shaft Problem F-35C Flight Sciences Flight Test Activity with CF-1, CF-2, CF-3, and CF-5 Test Aircraft • F-35C flight sciences focused on: -- Structural survey testing of the newly designed arresting hook system (This testing was a pre-requisite for the first developmental testing period aboard an aircraft carrier, referred to as DT-1, which was conducted in November 2014.) -- Block 2B weapons envelope and loads testing -- Block 2B high angle of attack testing -- Testing with external air-to-air and air-to-ground weapons (Block 3F capability) -- Fuel dump testing • The program modified CF-3 and CF-5 with the new arresting hook system and modified nose landing gear, which was necessary to prepare for and accomplish the first set of ship trials, completed in November. 48 F-35 JSF F-35C Flight Sciences Assessment • Through the end of November, test point accomplishment for CY14 was 17 percent behind the plan for Block 2B points and 124 percent ahead for Block 3F points. Test flights were 10 percent ahead of the plan (286 flown; 261 planned). Similar to the F-35B, the test force has been able to maintain test point productivity by completing points from the Block 3F test plan, such as performance assessments with external weapons, which were completed earlier than planned. • Similar to the other variants, the program adjusted the plan for Block 2B test points, resulting in a net reduction of 81 of 1,003 test points (8 percent) planned for the year. These points were designated as no longer required for Block 2B fleet release. • Transonic Roll-Off (TRO) and airframe buffet continue to be a program concern. All three variants required FY14 DOD PROGRAMS modifications of the control laws to control the effects of transonic flight and buffet producing maneuvering. In anticipation of difficulty in these flight regimes, the ability to incorporate spoilers in F-35C aircraft was provided early in the program. F-35C handling characteristics in transonic and buffet‑producing regimes were in need of correction and worse than in other variants. Flight testing with the addition of spoilers is planned, but not yet started. • CF-8 (a mission systems test aircraft assigned to the Edwards AFB test force) was scheduled to undergo modifications to include the redesigned fuel tank inerting system in June 2014; however, the modification was delayed pending conversion of CF-8 to the Block 3i configuration. The program has scheduled the modifications for February 2015, with ground and flight testing to follow soon after. • Discoveries included: -- The test force flew test missions with CF-2 in December 2013 and January 2014 to assess and characterize the effects of buffet and TRO on the helmet-mounted displays and handling qualities while conducting tasks associated with operational maneuvering (basic offensive and defensive fighter maneuvers). Buffet affected display symbology, and would have the greatest impact in scenarios where a pilot was maneuvering to defeat a missile shot. -- Deficiencies in the nosewheel steering motor and the pitch pivot pin of the arresting hook system slowed testing (see ship integration section for details of the arresting hook system testing). • Weight management is important for meeting air vehicle performance requirements, including the KPP for recovery approach speed to the aircraft carrier, and structural life expectations. These estimates are based on measured weights of components and subassemblies, calculated weights from approved design drawings released for build, and estimated weights of remaining components. These estimates are used to project the weight of the first Lot 8 F-35C aircraft (CF-28), planned for delivery in April 2016, which will be the basis for evaluating contract specification compliance for aircraft weight. -- The weight reports show that the program has reduced weight by 62 pounds in CY14 (from January to October estimate). The current estimate of 34,519 pounds is 349 pounds (1 percent) below the planned not-to-exceed weight. -- The program has demonstrated positive weight management of the F-35C over the past 38 months, showing a net loss of 103 pounds in the estimates from August 2011 to October 2014. The program will need to ensure the actual aircraft weight meets predictions and continue rigorous management of the actual aircraft weight beyond the technical performance measurements of contract specification in CY16 through the balance of SDD to avoid performance degradation that would affect operational capability. Mission Systems Flight Test Activity with AF-3, AF-6, AF-7, BF-4, BF-5, BF-17, BF-18, CF-3, and CF-8 Flight Test Aircraft and Software Development Progress • Mission systems are developed, tested, and fielded in incremental blocks of capability. -- Block 1. The program designated Block 1 for initial training capability and allocated two increments: Block 1A for Lot 2 (12 aircraft) and Block 1B for Lot 3 aircraft (17 aircraft). No combat capability is available in either Block 1 increment. All Lot 2 aircraft have been converted to Block 1B; the U.S. Services currently have 26 Block 1B aircraft (13 F-35A in the Air Force and 13 F-35B in the Marine Corps). Additionally, two F-35B Block 1B aircraft have been accepted by the United Kingdom and one F-35A Block 1B aircraft by the Netherlands; these aircraft are currently assigned to the training center at Eglin AFB. -- Block 2A. The program designated Block 2A for advanced training capability and delivered aircraft in production Lots 4 and 5 in this configuration. No combat capability is available in Block 2A. The U.S. Services have 62 aircraft in the Block 2A configuration (32 F-35A in the Air Force, 19 F-35B in the Marine Corps, and 11 F-35C in the Navy). Additionally, one F-35B and one F-35A have been accepted by the United Kingdom and the Netherlands, respectively; both aircraft are assigned to the training center. -- Block 2B. The program designated Block 2B for initial, limited combat capability for selected internal weapons (AIM-120C, GBU-32/31, and GBU-12). This block is not associated with the delivery of any production aircraft. Block 2B software has been in flight test since February 2013. Once complete with flight test and certification, Block 2B software may be retrofitted onto aircraft from production Lots 2 through 5, provided the necessary hardware modifications have been completed as well. Block 2B is planned to be the Marine Corps IOC configuration. -- Block 3i. The program designated Block 3i for delivery of aircraft in production Lots 6 through 8, as these aircraft will be built with a set of upgraded integrated core processors (referred to as Technical Refresh 2, or TR2). The capabilities associated with Block 3i software will vary based on the production lot. Lot 6 aircraft are expected to be delivered with capabilities equivalent to Block 2A in Lot 5, aircraft in Lots 7 and 8 are planned to be delivered with capabilities equivalent to Block 2B. Block 3i software began flight testing in May 2014. The program delivered the first Block 3i aircraft, an F-35A, to Luke AFB, Arizona, in late October. Four more F-35A aircraft were delivered to Luke AFB and one F-35B to Marine Corps Air Station (MCAS) Beaufort, South Carolina, by the end of November. -- Block 3F. The program designated Block 3F as the full SDD capability for production Lot 9 and later. Although under development, flight testing with Block 3F software on the F-35 test aircraft has not started. The program F-35 JSF 49 FY14 DOD PROGRAMS • • • • • 50 plans to begin flight testing in early CY15. Aircraft from production Lots 2 through 5 will need to be modified, including the installation of TR2 processors, to have Block 3F capabilities. Mission systems testing focused on: -- Completing flight testing of Block 2B capabilities -- Start of flight testing of Block 3i software, which began in May -- Start of Generation III helmet-mounted display system (HMDS) testing -- Multi-ship data link performance (via the multi-platform advanced data link (MADL) system and Link 16) -- Radar performance -- Troubleshooting navigation solution problems, which caused a pause in weapon testing in August -- Manual Ground Collision Avoidance System testing, which was added by the program in CY14 as a Block 2B capability to be delivered with fleet release -- Flight testing six increments of Block 2B software and two increments of Block 3i software (note: the program plans to release another version of 3i software to flight test prior to the end of CY14) -- Block 3F software – first version began testing on the Cooperative Avionics Test Bed (first flight was on July 31) The six mission systems flight test aircraft assigned to the Edwards AFB test center flew an average rate of 7.0 flights per aircraft per month in CY14 through November, exceeding the planned rate of 5.4 by 30 percent, and flew 121 percent of the planned flights (472 sorties accomplished compared to 390 planned). The program prioritized flight test activity to attempt to complete Block 2B flight testing by the end of October 2014, per the approved baseline schedule. However, as of the end of November, 87 percent of the total Block 2B mission systems baseline test points were accomplished (3,654 of 4,183 total points accomplished, 529 points remaining). The test team accomplished 74 percent of the planned 2014 baseline mission systems test points from test plans for Blocks 2B and 3i by the end of November (1,303 baseline test points accomplished, 1,766 planned). The team also accomplished an additional 1,072 growth test points. These points were needed for regression testing of new revisions of Block 2B software, identifying and characterizing deficiencies in mission systems performance, verification of corrections of deficiencies, and other testing the program found necessary to add to the baseline test plans. Although the program plans for some growth points during development, the rate of growth experienced for CY14 through the end of November for Block 2B testing (91 percent) was higher than the planned rate of 45 percent used by the program for CY14. The growth rate for the limited amount of Block 3i testing was 29 percent. Five F-35A operational test aircraft (all of which include flight test instrumentation and recording equipment identical to SDD mission systems test aircraft) were modified and loaded with a developmental test version of Block 2B F-35 JSF software – one aircraft in July, two in August, one in September, and one in October. As a result of the decision to not conduct the Block 2B OUE, the program is able to use these aircraft to support the effort to complete Block 2B developmental testing. Depending on the availability of these aircraft after the Block 3F modifications plan is finalized, they will be available to support re-scoped Block 2B operational test activity. Mission Systems Assessment • Block 2B -- Although test flight sortie goals were exceeded, and over 75 percent of planned baseline test points were accomplished as of the end of November, delivery of Block 2B capability, and thus the ability to complete development by October, was hampered by several factors: ▪▪ The need to develop, release, and test unplanned versions of Block 2B software to improve stability and fix deficiencies. ▪▪ Discoveries continued to occur in later versions of software. ▪▪ Restrictions to flight test aircraft apart from those imposed due to the June engine failure reduced the accessible test points. »» For example, flight operations with AF-6 and AF-7 mission systems test aircraft were suspended temporarily on June 20 when the program issued a stop order on F-35A production aircraft until inspections were completed on the nacelle vent inlet tube. A crack in the tube was discovered on a production F-35A aircraft at Eglin AFB following an incident where ground crews observed fuel leaking from the tube during hot pit ground refueling operations on June 11 (AF-6 and AF-7 are Lot 1 production aircraft assigned to the Edwards AFB test center). »» Following the inspections, the program released an interim aircraft operating limitation restricting F-35A production aircraft to 3 g’s and no air refueling. This affects all fielded production aircraft as well, which carry these restrictions concurrent with the restrictions related to engine failure, until they are modified. These restrictions remained in place on AF-6 and AF-7 until the test center replaced the tubes. -- To date, performance of 2BS5 software, which began flight testing in June, has shown improvement in startup and inflight stability compared to earlier versions. However, fusion of information from own-ship sensors, as well as fusion of information from off-board sensors is still deficient. The Distributed Aperture System continues to exhibit high false-alarm rates and false target tracks, and poor stability performance, even in later versions of software. -- In June, the Program Office and the Services completed a review of nearly 1,500 deficiency reports accumulated since the beginning of testing to adjudicate the status FY14 DOD PROGRAMS of all capability deficiencies associated with Block 2B fleet release/Marine Corps IOC. The review showed that 1,151 reports were not yet fully resolved, 151 of which were assessed as “mission critical” with no acceptable workaround for Block 2B fleet release. The remaining development and flight test of Block 2B will determine the final status of these 151 mission critical deficiencies, whether they are corrected or will add to the incomplete development work deferred to Block 3F with the less critical flaws. -- Growth in mission systems test points (regression for new software versions, testing fixes) for CY14 through the end of November was at 91 percent; that is, for every Block 2B “baseline” test point accomplished in CY14, 0.91 “growth” points have been accomplished. Growth in test points for Block 2B has slowed later in CY14 as the program has deferred fixes of deficiencies to Block 3i or Block 3F, averaging 61 percent for the period August through November. This average rate of growth, although higher than the planning rate for the year, is less than that observed in CY13 (124 percent) at the time of reporting for the FY13 DOT&E Annual Report. -- The program is eliminating test points that are designed to characterize performance (i.e., in a greater envelope than a specific contract specification condition), reducing the number of test points needed to verify the final Block 2B capability for fleet release, and deferring fixes for deficiencies to Block 3. The program has also added points for the capability required by the Services to be included in Block 2B capabilities. Formal adjustments to the 2014 test plans through the end of October resulted in a net reduction of 135 Block 2B baseline test points. In November, the program considered making further adjustments to the plan in order to complete testing necessary to support Block 2B fleet release by the end of January 2015. After reviewing the remaining 529 baseline test points, the program deemed 139 as potentially no longer required and another 147 as optional, designating only 243 of the 529 remaining points as essential for completing testing to support Block 2B fleet release. Formal adjustments of the test plans were pending as of the completion of this report. These reductions in the 2014 plan are in addition to the removal of approximately 840 test points that occurred when the program consolidated test plans for software increments prior to Block 2B with the plan for 2014, all of which were planned to be flown prior to the 2014 plan. -- The program planned to complete Block 2B mission systems flight test in October, which did not occur. The completion date of Block 2B mission systems testing will depend, in part, on realizing further reductions to baseline test points and elimination of any remaining restrictions imposed on the fleet of test aircraft due to the engine failure. As of the end of November, 529 of 4,183 Block 2B baseline test points remained. Assuming the program would continue test point productivity equal to that realized in the preceding 12 months, the program will be able to complete the remaining 529 Block 2B test points by the end of February 2015. This estimate is based on the following assumptions: ▪▪ Modifications to upgrade any additional mission systems test aircraft from the Block 2B to Block 3i or Block 3F configuration (besides AF-3) occurs after January 2015, which is the program’s current estimate for completing Block 2B development. Starting in February, two of the seven remaining mission systems test aircraft upgrade to the Block 3i configuration, while the remaining mission systems test aircraft stay in the 2B configuration to complete testing. This schedule allows other mission systems test aircraft to be modified to support testing of the Block 3i and Block 3F mission systems software, the Generation III HMDS, and OBIGGS on the F-35C variant. ▪▪ The operating restrictions stemming from the engine failure do not restrict access to the remaining test points. These restrictions are lifted on each test aircraft after a “pre-trenched” stator is installed in the engine. Through the end of November, the engines in 6 of the 18 test aircraft had been modified with these stators and the program plans to have the entire test fleet modified by the end of February 2015. ▪▪ No additional growth is experienced in the remainder of Block 2B flight testing, and deficiencies not currently addressed by fixes included in the final test release of Block 2B software (version 2BS5.2) will be deferred to Block 3 or not addressed. • Block 3i -- Block 3i was not planned to incorporate any new capability or fixes from the Block 2B development /fleet release. The first increment of Block 3i capability, designated 3iR1, is the initial release to Lot 6 aircraft and will include only Block 2A capability (inherently less capable than the final Block 2B fleet release). Subsequent increments of Block 3i software will have additional capability. However, the prospects for Block 3i progress are dependent on completion of Block 2B development and flight test, which determines: ▪▪ When test aircraft are converted to Block 3i; two of seven mission systems aircraft – one at the Edwards test center and one at the Patuxent River, Maryland, test center – have been modified so far (flight testing can only occur on test aircraft upgraded with TR2 hardware). ▪▪ How much incomplete development work will be inherited by Block 3i due to deficiencies deferred from Block 2B. -- Though it eventually began in 2014, Block 3i flight test progress began late, and has progressed much slower than expected. As of the end of November 2014, the program had completed only 25 percent of the baseline Block 3i v F-35 JSF   51 FY14 DOD PROGRAMS test points, accomplishing 177 of 700 test points, which represented 64 percent of the plan for the year. ▪▪ The program temporarily modified two mission systems aircraft – CF-8 in October 2013 and AF-3 in November 2013 – with a portion of the TR2 hardware to attempt loading the first build of Block 3i software. The attempt on CF-8 failed, but the software was successfully loaded on AF-3, allowing the test center to complete ground software regression testing. AF-3 was returned to the Block 2B configuration to support testing until May 2014, when it underwent the full TR2 modification in preparation for Block 3i flight testing. ▪▪ In May, the first increment of flight test software (3iR1) was delivered to flight test approximately five months later than planned (December 2013 to May 2014). This version of the software is needed for delivery of Lot 6, TR2 equipped aircraft. The Edwards test center conducted flight testing of the Block 3i software on AF-3. The Patuxent River test center conducted one test flight of Block 3i software on BF-5, which is currently deployed to the climatic chamber for testing. No testing of Block 3i software has yet been accomplished on an F-35C test aircraft. As of the end of November, all remaining Block 3i test points were blocked, as the test centers were awaiting the next iteration of Block 3i software to proceed with flight testing. ▪▪ The test centers identified deficiencies in the 3iR1 software, five of which needed to be corrected before the software could be used in the Lot 6 production aircraft. These deficiencies were corrected and tested in the lab with an updated version of software. This final version of 3iR1 software was not flight tested at test centers, but tested by the contractor at the production facility, and is used to deliver Lot 6 aircraft. ▪▪ The second iteration of Block 3i software, 3iR4, included capability to test the new Generation III HMDS. The Edwards test center flew four test missions with 3iR4 on AF-3 in September, accomplishing regression test points and some initial test points from the Generation III HMDS test plan. This was the first testing of the new HMDS on F-35 test aircraft. The test team discovered deficiencies, particularly in the stability of the new display management computer for the helmet, and suspended further testing until software that fixes the deficiencies in the helmet system can be provided to the major contractor and included in an updated load of mission systems software. ▪▪ The third increment of Block 3i software, version 3iR5, will be used to provide production software for Lot 7 aircraft, the first lot to be delivered with the Generation III HMDS. The program plans for the production software to have the equivalent capabilities as Block 2B and plans to deliver 3iR5 software to flight test in January 2015. However, even if this occurs, since Block 2B development and flight testing were not completed as planned in October, the completion 52 F-35 JSF of Block 3i testing will be delayed if the equivalent capabilities from Block 2B development are to be realized in Block 3i. The program plans to convert four of the five Block 2B mission systems test aircraft at the Edwards test center to the Block 3i configuration in February 2015. Assuming this transition takes place, Block 3i flight testing could conclude by July 2015, two months later than the planned completion of May 2015. This assumes nominal growth of 66 percent is experienced during the rest of Block 3i development and flight testing, the program completes testing of the remaining baseline test points without reductions, and the program uses four of the six mission systems test aircraft at the Edwards test center for dedicated Block 3i testing. Of the two remaining mission systems test aircraft, one other test aircraft could be available for further Block 2B testing and one could be used to start Block 3F testing. Additional time will be needed to address corrections if additional deficiencies are identified in the Generation III HMDS and will add risk to the schedule. • Block 3F -- In order to manage and complete Block 3F development and flight testing as planned in late 2017, the program needs to complete Block 2B development and flight test as soon as possible and transition to Block 3. The program currently acknowledges four to six months “pressure” on the end of Block 3F development and test. The program needs to complete Block 2B development soon to focus resources (staffing, labs, flight test aircraft) on the development and testing of Block 3F, designated as “full warfighting capability.” -- The test centers and contractor began detailed test planning for Block 3F flight test. The draft test plan has nearly 6,000 test points. Plans completed after the 2012 re-baselining of the program showed the start of Block 3F flight testing in May 2014; however, current program plans are to start Block 3F flight test in March 2015, 10 months later than the 2012 baseline. Mission Data Load Development and Testing • The F-35 relies on mission data loads – which are a compilation of the mission data files needed for operation of the sensors and other mission systems components – working in conjunction with the system software data load to drive sensor search parameters and to identify and correlate sensor detections of threat radar signals. An initial set of files was produced by the contractor for developmental testing during SDD, but the operational mission data loads – one for each potential major area of operation – will be produced by a U.S. government lab, the U.S. Reprogramming Lab (USRL). These mission data loads will be used for operational testing and fielded aircraft, including the Marine Corps IOC aircraft. • In accordance with the approved mission data optimization operational test plan, mission data loads undergo a three‑phased lab development and test regimen, followed by FY14 DOD PROGRAMS flight test. The current plans are to certify the first two mission data loads, which are needed to support Marine Corps IOC, in November 2015 after flight testing occurs on operational test aircraft between March and October 2015. These plans provide the mission data load later than needed for the Marine Corps’ objective IOC date of July 2015. However, truncating the mission data load development and conducting open-air flight testing early on a limited open-air range for the purpose of releasing a mission data load in mid-2015 would create significant operational risk to fielded units, since the load will not have completed the planned lab testing and because the open-air range test infrastructure is capable of verifying only a small portion of the mission data. The program should complete lab testing of the mission data loads, as is planned in the mission data optimization operational test plan, prior to accomplishing the necessary flight testing to ensure the loads released to the fleet are optimized for performance. If mission data loads are released to operational units prior to the completion of the lab and flight testing required in the operational test plan, the risk to operational units must be clearly documented. • Several items are currently creating risk to the program’s ability to deliver certified mission data loads. Mission data lab equipment was held by the major contractor at their Fort Worth facility for three years past the planned delivery to the USRL to support mission systems software development for production aircraft, reducing productivity at the USRL. The USRL did not receive sufficient documentation of the equipment and software tools that were delivered by the program; this has hampered their training and slowed development. Contract issues had prevented USRL from direct communications with the subcontractor that designed both the electronic warfare system on the aircraft and the mission data programming tools. These communications were needed to understand undocumented lab and mission data file generation tool functions. The Program Office has taken steps to improve these communications. Other challenges that may affect on-time delivery of mission data include instability in the contractor-delivered mission data file generation tool, which creates the final mission data load, and slower than expected development of software analysis tools that optimize sensor performance. • Mission data load development and testing is a critical path to combat capability for Block 2B and Block 3F. Accuracy of threat identification and location depend on how well the mission data loads are optimized to perform in ambiguous operational environments. This is difficult work given a stable software capability in the platform, adequate lab equipment, and stable/well-understood mission data file generation tools – none of which are yet available in the program. • The current lab is essentially a copy of the mission systems integration lab used by the major contractor to integrate and test software. It is not adequate for development of mission data loads for use in operationally realistic conditions. As identified by DOT&E in early 2012, the program must plan and execute a significant upgrade to the lab in order for it to generate an operationally realistic signal environment for mission data load optimization. Though funding has been made available, plans for this upgrade, and integration with the Block 2B, Block 3i, and Block 3F mission data loads have not been finalized. Weapons Integration • Progress in weapons integration, in particular the completion of planned weapon delivery accuracy (WDA) events, has been very limited in 2014 compared to that planned by the program. Multiple deficiencies in mission systems, aircraft grounding, and subsequent flight restrictions caused by the June engine failure all contributed to the limited progress. • Each WDA event requires scenario dry-runs in preparation for the final end-to-end event to ensure the intended mission system functionality, as well as engineering and data analysis requirements (to support the test centers and weapon vendors) are available to complete the missile shot or bomb drop. Per the approved TEMP, these preparatory events, as well as the end-to-end events, are to be accomplished with full mission systems functionality, including operationally realistic fire control and sensor performance. • Mission systems developmental testing of system components required neither operation nor full functionality of subsystems that were not a part of the component under test. The individual mission system component tests were designed by the developmental teams to verify compliance with contract specification requirements rather than to test the full mission systems performance of the aircraft and complete the find‑fix‑ID-track-target-engage-assess kill chain for air-to-air and air-to-ground mission success. WDA events, however, were specifically designed to gather both the necessary weapons integration and fire-control characterization and performance using all the mission systems required to engage and kill targets. • Planning and scheduling of the WDA events assumed that all associated mission systems functionality would be mature by the WDA preparatory event dates. However, due to the limitations in progress in Block 2B mission systems, this has not occurred. - Deficiencies in the Block 2B mission systems software affecting the WDA events were identified in fusion, radar, passive sensors, identification friend-or-foe, electro-optical targeting system, and the aircraft navigation model. Deficiencies in the datalink systems also delayed completion of some events. Overall, these deficiencies have both delayed the WDA event schedule and compromised the requirement to execute the missions with fully functional and integrated mission systems. - The program had planned to complete all Block 2B WDA events by October 2014. This did not occur. Through the end of November, 10 of 15 live fire events had been completed, while the program planned to have all 15 completed by the end of October. In November, the F-35 JSF 53 FY14 DOD PROGRAMS program deferred two of the planned Block 2B WDA events to Block 3, due to deficiencies and limitations of capability in Block 2B mission systems. The adjacent table shows the planned date, completion or scheduled date, and weeks delayed as of the end of November for each of the WDA preparatory and end-to-end events. Events completed are shown with dates in bold font; events scheduled are shown with dates in italicized font. The program should complete the remaining three Block 2B WDA flight test events, using the currently planned scenarios, and ensuring full mission systems functionality is enabled in an operationally realistic manner. Weapons Delivery Accuracy (WDA) Progress Weapon AIM-120 WDA Number 102 Preparatory Events Planned Weeks Delayed Planned Completed/ Scheduled Weeks Delayed Sep 13 Sep 13 2 Oct 13 Oct 13 2 112 Sep 13 Sep 13 3 Oct 13 Nov 13 3 GBU-12 113 Sep 13 Oct 13 3 Oct 13 Oct 13 0 GBU-32 115 Sep 13 Nov 13 6 Nov 13 Dec 13 3 108 Oct 13 Dec 13 7 Dec 13 Feb 14 12 110 Oct 13 Aug 13 43 Dec 13 Nov 14 50 111 Dec 13 Deferred to Block 3F -- Jan 14 Deferred to Block 3F -- 106 Dec 13 Sep 14 40 Jan 14 Nov 14 43 45 Feb 14 Nov 14 41 30 Mar 14 Jan 15 44 12 May 14 Dec 14 30 17 Jun 14 Dec 14 26 -8 Aug 14 May 14 -10 Jan 14 -29 Sep 14 Mar 14 -27 Deferred to Block 3F -- Oct 14 Deferred to Block 3F -- AIM-120 May 14 GBU-31 114 Dec 13 F-35 JSF Jun 14 Oct 14 104 Feb 14 107 Mar 14 101 May 14 103 Jun 14 AIM-120 109 Jul 14 Static Structural and Durability Testing 105 Sep 14 • Structural durability testing of all variants using full scale test articles is ongoing, each having completed at least one full lifetime (8,000 equivalent flight hours, or EFH). All variants are scheduled to complete three full lifetimes of testing before the end of SDD; however, complete teardown, analyses, and Damage Assessment and • Damage Tolerance reporting is not scheduled to be completed until August 2019. The testing on all variants has led to discoveries requiring repairs and modification to production designs and retrofits to fielded aircraft. • F-35A durability test article (AJ-1) completed 11,000 EFH on September 13, which is 3,000 hours into the second lifetime. Testing restarted on October 29, after completing non-invasive inspections, which are required at 1,000 EFH intervals. - Cracking of the right hand side (RHS) Fuselage Station (FS) 402, discovered after the first lifetime of testing (8,000 EFH) at the end of CY12, required repairs to the test article, production redesign for production Lot 8 and later aircraft, and retrofitting a modification for production Lot 4 through 7 aircraft. - Discoveries from the second lifetime of testing, which started on December 13, 2013, include: ▪▪ Cracking of the left hand side (LHS) integrated power package shear web lug at FS503, found at 10,082 EFH ▪▪ Cracking of the LHS FS503 frame support, found at 10,162 EFH 54 End-to-End Event Completed/ Scheduled1 Aug 14 Sep 14 Jun 14 May 14 Sep 14 Mar 14 Apr 14 1. Some WDA events require more than one preparatory event. ▪▪ C racking in the LHS F2 fuel floor flange, found at 11,000 EFH - Disposition of these discoveries and repair plans were under consideration as of the time of this report. F-35B durability test article (BH-1) has been halted since September 2013, when the FS496 bulkhead severed at 9,056 EFH, transferred loads to an adjacent FS518 bulkhead, and caused cracking. Root cause analysis and corrective action – for repairing the bulkheads on the test article, modification for the fielded aircraft, and redesign for production Lot 8 (and subsequent lots) – have been ongoing throughout CY14. The program planned to restart testing in late September 2014, but repairs took longer than expected. Testing had not restarted as of the end of November. According to the Program Office, the effect on fielded aircraft will be limited life for FS496 (approximately 10 years of service life) until replaced or repaired. - Modifications to the test article include the addition of seven splice plates to repair cracks in the FS496 and FS518 bulkheads. - For retrofitting/modifying FS496 in production aircraft in Lots 1 through 8, the program is considering a number of fatigue mitigation steps, including relocating system attachment points, hardening the fastener holes through a FY14 DOD PROGRAMS cold working process, and the use of laser shock peening (LSP) to enhance fatigue life in sections of the bulkhead where tensile stresses are known to be concentrated. The objective of treating areas with LSP is to create compressive pre-stress states near surfaces where tensile stresses are expected to be high and hence reduce crack initiation. However, LSP has not been used on the type of aluminum alloy (AL-7085) used in manufacturing the FS496 bulkheads in the F-35B, and the ability to affect the structural life is not well understood. The program should require the contractor to conduct rigorous finite-element analyses to assess the benefit of LSP application. The main objectives are to assess the LSP effect in reducing tensile stress concentrations in critical areas and to assure limited increase of tensile stresses in the other areas. To date, the effect on AL-7085 fatigue properties due to LSP application are yet to be characterized, therefore a finite-element analysis using the existing AL-7085 fatigue property data is likely to over-estimate the effect of LSP in improving fatigue resistance, which should also be taken into account. - For aircraft in Lot 9 and beyond, the program is redesigning the five carry-through bulkheads in the F-35B (FS450, FS472, FS496, FS518, and FS556). The redesign will include LSP on two bulkheads, cold working of fastener holes on four, and increasing thickness in portions of all five bulkheads. The overall effect on aircraft weight increase is not yet known. - Because of the extensive repair required to the FS496 bulkhead, the certification path to full life will likely require additional follow-on testing. • F-35C durability test article (CJ-1) began second lifetime testing on April 2, and completed 2,312 EFH into the second lifetime in August (10,312 EFH total), followed by inspections. Testing resumed October 28, 2014. - Discoveries after the first lifetime of testing caused redesigns in the FS518 fairing support frame and FS402 upper inboard frame. Repairs and redesigns were completed at 8,869 EFH and 8,722 EFH, respectively. - Discoveries from the second lifetime of testing include cracking of outboard wing spar #5 and cracking on both the left and right hand sides of the FS575 center arch frame. Repairs to both were completed at 10,000 EFH prior to restart of testing. Modeling and Simulation Verification Simulation (VSim) • The Verification Simulation (VSim) is a man-in-the-loop, mission software-in-the-loop simulation developed to meet the OTAs’ requirements for Block 3F IOT&E, as well as to provide a venue for contract compliance verification for the Program Office. • At the beginning of CY14, the program planned to accredit the VSim for use in Block 2B contract compliance verification by the end of the year. However, lack of • • • • • progress on the Verification and Validation (V&V) process, and to a lesser extent the VSim development process, caused the program to charter an independent review of VSim. This review eventually led to cancellation of the contract verification portion of Block 2B VSim planned usage. For similar reasons, after the Block 2B OUE re-scoping effort began, the JSF Operational Test Team determined that VSim would likely not support planned Block 2B operational testing in 2015 and reduced the requirements for the simulation’s intended uses to support only tactics development and other activities that directly contribute to the fielding of Block 2B capabilities. About one-third of the validation evidence for Block 2B VSim was reviewed by the developmental and operational test stakeholders before the contractual use of VSim for Block 2B was cancelled. This review confirmed that additional time was needed before VSim V&V could potentially meet expectations. Collaborative replanning of Block 2B activities is not complete, but V&V reviews to support operational testing needs are now planned for early 2015, with accreditation of VSim for tactics development and other uses expected in October 2015. Exercising the V&V process for Block 2B VSim is critical to reducing risk for its use in Block 3F IOT&E. Rigorous validation will identify gaps in VSim performance, including threat modeling, in time to create the appropriate fixes for Block 3F. Creation of test and V&V procedures as well as V&V reports and accreditation documentation will provide a significantly better understanding of VSim status by the end of 2015. Rigorous validation depends on good source data, and the contractor and Program Office improved efforts to ensure VSim needs are met in the Block 3F flight test plan. Those plans are not finalized, but will certainly result in deficits as the enterprise-wide need for flight tests exceeds available resources. Success in validating Block 3F VSim will depend on bridging this gap with acceptable data sources. The contractor has increased resources on VSim V&V teams, and the quality of the V&V products is increasing. However, the rate of completing validation points (a comparison of VSim model performance to aircraft hardware performance under similar test conditions using data from flight test, avionics test bed, or labs), has been much slower than planned. This makes completing the validation reports, which analyze the points with respect to intended use, at risk to support even the reduced accreditation requirements for Block 2B. Additional resources may be required to complete the significant task of validating the complex federation of models in VSim in time for Block 3F IOT&E. Although the VSim validation process has improved, DOT&E has continued to highlight shortfalls in the test resources needed to gather key elements of data required for validation of the VSim for IOT&E, in particular for electronic warfare performance in the presence of advanced threats. These shortfalls are a function of limitations in the F-35 JSF 55 FY14 DOD PROGRAMS test assets currently available to represent threat systems. DOT&E has made formal recommendations to address the shortfalls and is pursuing solutions to make the assets available in time to prepare for IOT&E in a realistic threat environment. • Limiting VSim Block 2B validation, and use, to tactics development and evaluation will help the program progress towards V&V of Block 3F. Block 3F use of VSim for IOT&E is not optional; it is required for an adequate IOT&E. Training System • Pilot training continues at Eglin AFB, Florida, and expanded in September 2014 when additional F-35B training began at MCAS Beaufort, South Carolina. Additional F-35A pilot training is planned to start in May 2015 at Luke AFB, Arizona. Sixty-six student pilot training slots were available in FY14, but nine were not used due to reduced Service requirements. • The training center began transitioning from the Block 1B to the Block 2A training syllabus for all three variants in December 2013, and completed the transition in February 2014. The ability to train in and for adverse weather conditions was added to the Block 2A syllabus during CY14. The Block 2B syllabus is planned for delivery in mid-2015, and is planned to include limited combat capability. • Lot 5 deliveries to pilot training bases continued throughout 2014, including the first nine F-35A to Luke AFB, and an additional eight F-35A, one F-35B, and six F-35C aircraft to Eglin AFB. Lot 6 deliveries, which began in late October, included the first F-35B aircraft delivered directly to MCAS Beaufort where it joined other F-35B aircraft transferred from Eglin AFB. • All training to date has been in Block 2A-configured aircraft, which have envelope and other restrictions that preclude high performance training events. Because of this, all pilots attending Block 2A training complete only a portion of the planned syllabus before moving to their units. • The Training Management System (TMS) is a database that includes course material, syllabus flow, student records, and schedules for aircrew and maintainers. The academic center is using the TMS for instruction and tracking student progress. TMS functionality is relatively unchanged from that which existed during the 2012 training system OUE. For example, the TMS cannot yet be effectively used for scheduling, pilot qualification tracking, and the other typical unit functions. This year, the Program Office added funding to correct these deficiencies and improve the functionality for tracking operational unit “continuation training,” which includes monthly training requirements and pilot qualifications. Planned delivery is in the 2017 timeframe, and will also require Automatic Logistics Information System (ALIS) system-level architecture modifications to achieve full capability. Until then, flying units at both training and operational bases will most likely continue to use legacy scheduling and training databases, which causes double entry into databases and impedes program-level data analysis such as annual flying hour progress. 56 F-35 JSF • The training center continued to conduct maintenance training for experienced maintenance personnel for all F-35 variants during 2014. As of the end of October, more than 1,800 U.S. personnel and foreign partner students had completed training in one or more of the maintenance courses, including ALIS, to support fielded maintenance operations. For the 12-month period ending in October 2014, the contractor provided 1,018 training slots for maintenance courses, of which 701 were filled by U.S. or foreign partner students, equating to 69 percent training seat utilization rate. In addition, active duty personnel at the field units conducted training that is not included in these metrics. The Integrated Training Center at Eglin AFB currently offers 13 maintenance classes ranging from 3 to 13 weeks in length. Live Fire Test and Evaluation F-35B Full-Scale Structural System Vulnerability Assessment • The F-35 LFT&E Program completed the F-35B full-scale structural test series. The Navy’s Weapons Survivability Laboratory (WSL) in China Lake, California, completed 15 tests events using the BG:0001 test article. Preliminary review of the results indicates that: -- Anti-aircraft artillery (AAA) threat-induced damage stressed the critical wing structure members, but multiple structural load paths successfully limited the damage to expected areas around the impact points while preserving the static flight load carrying capabilities. Consistent with predictions, the tests demonstrated other cascading damage effects, including threat-induced fire and damage to adjacent fuselage fuel tanks. -- AAA and missile fragment-induced damage stressed the structural limitations of the forward fuselage fuel tanks (F-1 and F-2). Cascading effects from the F-1 tank damage included a large fuel release into the cockpit and damage to the pilot seat mounting structure. To mitigate the vulnerability to the pilot, the Program Office has recently altered the F-35B fuel burn strategy so that the F-1 tank behind the pilot empties sooner. Threat-induced damage in these fuel tank tests also caused large fuel discharge into the engine inlet, which would have likely caused engine failures due to fuel ingestion. The engine was not installed for these tests. -- The extent of AAA-induced structural damage to the wing leading edge flap and the horizontal tail is not flight critical from a structural tolerance perspective. The leading edge tests demonstrated the potential for sustained fire, which could have flight-critical cascading effects on the wing structure. -- The ballistic damage tolerance testing of propulsion system related structural components (variable area vane box nozzle, and hinges on the roll duct nozzle, lift fan, and auxiliary air inlet doors) revealed these components were nearly insensitive to expected threats. However, sustained fires were created in the shot into the variable area vane box nozzle due to leakage in the actuating hydraulics, and FY14 DOD PROGRAMS the shot into the roll duct nozzle door due to damage to the adjacent fuel tank. These fires would ultimately have led to cascading structural damage. -- Data support the evaluation of residual loading capabilities of the aft boom structure, including vertical tail and horizontal tail attachments, following a man‑portable air defense system impact and detonation. While having fuel in the aft-most F-5 fuel tank increased structural damage due to resultant hydrodynamic ram effects and fire, flight control surfaces remained attached. Further structural analysis of the damage effects is being completed to verify the structural integrity of the aft boom structure. F135 Engine • F135 live fire engine testing in FY13, engine vulnerability analysis in FY13, and uncontained engine debris damage analysis in FY03 demonstrated two primary threat-induced engine damage mechanisms: -- Penetration of the engine case and core that could cause blade removal, resulting in damage to turbomachinery leading to propulsion loss or fire -- Damage to external engine components (e.g., fuel lines, pumps, gearbox, etc.) leading to critical component failure and fire • Engine fuel ingestion testing in FY07 further demonstrated the potential of an engine stall providing a fire ignition source in the presence of additional fuel system damage. • The uncontained F135 fan blade release and subsequent fuel fire in an F-35A at Eglin AFB in June provides an additional data point that needs to be reviewed and analyzed to support the F-35 vulnerability assessment. Polyalphaolefin (PAO) Shut-Off Valve • The Program Office tasked Lockheed Martin to develop a technical solution for a PAO shut-off valve to meet criteria developed from live fire test results. This aggregate, 2-pound vulnerability reduction feature, if installed, would reduce the probability of pilot incapacitation, decrease overall F-35 vulnerability, and prevent the program from failing one of its vulnerability requirements. • The program has not provided any updates on the feasibility and effectiveness of the design, nor an official decision to reinstate this vulnerability reduction feature. Fuel Tank Ullage Inerting System • The program verified the ullage inerting design changes and demonstrated improved, inerting performance in the F-35B fuel system simulator (FSS) tests. A preliminary data review demonstrated that the system pressurized the fuel tank with nitrogen enriched air (NEA) while maintaining pressure differentials within design specifications during all mission profiles in the simulator, including rapid dives, but revealed the potential for pressure spikes from the aerial refueling manifold, as noted in the flight sciences section of this report. The Program Office will complete and document detailed data review and analyses to evaluate NEA distribution and inerting uniformity between different fuel tanks and within partitioned fuel tanks. • The program developed a computational model to predict inerting performance in the aircraft based on the F-35B simulator test results. Patuxent River Naval Air Station completed the ground inerting test on an actual F-35B aircraft to verify the model, but a detailed comparison to F-35B FSS has not yet been completed. The program will use this model, in conjunction with the completed F-35A ground tests and F-35C ground tests planned to start in February 2015, to assess the ullage inerting effectiveness for all three variants. The confidence in the final design and effectiveness will have to be reassessed after the deficiencies uncovered in the aircraft ground and flight tests have been fully resolved. • When effective, ullage inerting only protects the fuel tanks from lightning-induced damage. The program has made progress in completing lightning tolerance qualification testing for line-replaceable units needed to protect the remaining aircraft systems from lightning-induced currents. Lightning tolerance tests using electrical current injection tests are ongoing, and the program is expected to complete the tests by 2QFY15. Electrical System • DOT&E expressed a concern in FY13 for the potential loss of aircraft due to ballistically-induced shorting of power and control circuits in the F-35 flight control electrical systems. The F-35 is the first tactical fighter aircraft to incorporate an all-electric flight control system, using a 270 Volt power bus to power flight control actuator systems and a 28 Volt bus to control those actuators. The F-35 aircraft carries these voltages in wire bundles where they are in close proximity. Live fire tests of similar wire bundle configurations demonstrated the potential for arcing and direct shorts due to ballistic damage. • Lockheed Martin completed an electrical power systems report, which included a summary of development tests to demonstrate that transient-voltage suppression diodes installed throughout the 28 Volt systems shunt high voltage (including 270 Volt) to ground, preventing the high voltage from propagating to other flight-critical components. Some components might be damaged as a result of a short, but their redundant counterparts would be protected. Testing used direct injection of the high voltage, rather than shorting from ballistic damage, but the electrical effects would be the same. Vulnerability to Unconventional Threats • The full-up, system-level chemical-biological decontamination test on the BF-4 test article planned for 4QFY16 at Edwards AFB is supported by two risk-reduction events: -- The Limited Demonstration event conducted in 4QFY14 showed that the proposed decontamination shelter and liner design can sustain conditions of 160°F and 80 percent relative humidity. The high temperature alone is sufficient F-35 JSF 57 FY14 DOD PROGRAMS to decontaminate chemical agents. The combination Operational Suitability of high heat and humidity has been shown effective in • Overall suitability continues to be less than desired by decontaminating biological agents. Both chemical and the Services, and relies heavily on contractor support and biological decontamination techniques take 10 to 12 days to unacceptable workarounds, but has shown some improvement complete. in CY14. -- A System Integration Demonstration of the - Aircraft availability was flat over most of the past year, decontamination equipment and shelter was conducted maintaining an average for the fleet of 37 percent for the on an F-16 test article during 1QFY15 at Edwards AFB 12-month rolling period ending in September – consistent to simulate both hot air chemical and hot/humid air with the availability reported in the FY13 DOT&E biological decontamination operations. This testing will not report of 37 percent for the 12-month period ending demonstrate the decontamination system effectiveness in a in October 2013. However, the program reported an range of operationally realistic environments. improved availability in October 2014, reaching an average • The F-35 variant of the Joint Service Aircrew Mask rate of 51 percent for the fleet of 90 aircraft and breaking (JSAM‑JSF) successfully passed its Preliminary Design 50 percent for the first time, but still short of the program Review in 3QFY14. The Joint Program Executive Office objective of 60 percent set for the end of CY14. The for Chemical and Biological Defense and the F-35 Program bump in availability in October brought the fleet 12-month Office will have to integrate the JSAM-JSF with the average to 39 percent. Helmet‑Mounted Display, which is undergoing a challenging - Measures of reliability and maintainability that have ORD design process and consequently further aggravating this requirements have improved since last year, but all nine integration effort. reliability measures (three for each variant) are still below • Planned EMP testing will evaluate the aircraft to the threat program target values for the current stage of development. level defined in MIL-STD-2169B. Both horizontal and The reliability metric that has seen the most improvement vertical polarization testing, as well as active, passive, and since May 2013 is not an ORD requirement, but a contract direct drive testing are planned to assess effects and/or specification metric, mean flight hour between failure damage of the EMP induced currents and coupling to vehicle scored as “design controllable” (which are equipment and mission systems electronics. EMP testing on the F-35B failures due to design flaws). For this metric, the F-35B article was completed in 1QFY15; data analysis is ongoing. and F-35C are currently above program target values, Follow-on tests on other variants, including a test series and F-35A is slightly below the target value, but has been to evaluate any Block 3F hardware/software changes, are above the target value for several months over the last year. planned for FY16. F-35 Fleet Availability Gun Ammunition Lethality and Vulnerability • Aircraft availability is determined by measuring the percent • The program completed the ballistic impact response of time individual aircraft are in an “available” status, characterization of the PGU-47/U Armor Piercing Explosive aggregated over a reporting period (e.g., monthly). Aircraft (APEX) round for the partner F-35A variant using the AAA that are not available are assigned to one of three categories and fragment threats. Preliminary data analysis demonstrated of status: Not Mission Capable for Maintenance (NMC-M); no significant reactions or evidence of high pressures that Not Mission Capable for Supply (NMC-S); and in depot. could potentially induce sympathetic reactions from adjacent -- The program added this third category for tracking fleet rounds loaded on the aircraft. status in January 2014 as the number of aircraft entering • The program completed the terminal ballistic testing of the depot for modifications or receiving modifications the PGU-48 FAP round and the PGU‑32 round against or repair by a depot field team at the home station began a range of target-representative material plates and plate to increase. Prior to January 2014, these aircraft were arrays. Preliminary FAP test observations indicate lower assigned as Non-Possessed (NP) or Out Of Reporting than expected levels of fragmentation when passing through (OOR) for depot-level actions under an NMC-M status. multiple layer targets. PGU-32 test observations indicate -- The program established new goals for all three of these that the round detonates much closer to the impact point of unavailable statuses for 2014. The NMC-M goal is the first target plate than originally called out in ammunition 15 percent, NMC-S is 10 percent, and depot status is specification. The program will determine the impact of these 15 percent. These three non-available statuses sum to data on the ammunition lethality assessment. 40 percent, supporting the program’s availability goal • Ground-based lethality test planning is ongoing. All three of 60 percent for the fleet by the end of CY14. The goal rounds will be tested against a similar range of targets, of 60 percent is an interim program goal and does not including armored and technical vehicles, aircraft, and represent the availability needed for combat operations, personnel in the open. FY15 funds are in place for all tests nor the 80 percent needed to accomplish IOT&E in an except those against boat targets. operationally realistic manner. • Air-to-ground lethality tests will likely begin no earlier than • Aircraft monthly availability averaged 39 percent for the 1QFY16. Given the development test schedule of the APEX 12-month period ending October 2014 in the training and round, the existing flight test plan does not include this round. 58 F-35 JSF FY14 DOD PROGRAMS operational fleet, with no statistical trend of improvement for the first 11 months. In October 2014, availability jumped to a reported 51 percent (fleet size of 90 aircraft), a 12 percent increase from the previous month, and the largest month‑to‑month change since March 2013 (fleet size of 27 aircraft). Month-to-month variance in average availability should decrease as the fleet size increases. The improved availability was seen at most operating locations, and resulted from roughly equal improvements in the NMC-M and NMC-S rates. Historically NMC-M and NMC-S have tended to move in opposite directions; the improvement in one being negated by the increase in the other. • Aircraft availability rates by operating location for the 12-month period ending October 2014 are summarized in the table below. The first column indicates the average availability achieved for the whole period, while the maximum and minimum columns represent the range of monthly availabilities reported over the period. The number of aircraft assigned at the end of the reporting period is shown as an indicator of potential variance in the rates. Sites are arranged in order of when each site began operation of any variant of the F-35, and then arranged by variant for sites operating more than one variant. • F-35 Availability for 12-month Period ending October 20141 Operational Site Average Maximum Minimum Aircraft Assigned Total Fleet 39% 51% 35% 902 Eglin F-35A 39% 55% 32% 28 Eglin F-35B 41% 54% 25% 11 Eglin F-35C 50% 64% 24% 10 Yuma F-35B 33% 49% 24% 15 Edwards F-35A 43% 57% 19% 7 Nellis F-35A 28% 51% 2% 4 50% 58% 23% 10 37% 49% 4% 4 Luke F-35A3 Beaufort F-35B 4 • 1. Data do not include SDD aircraft. 2. Total includes 1 OT F-35B at Edwards that is not broken out in table 3. Luke F-35A data began in April 2014 4. Beaufort F-35 B data began in July 2014. • Sites that show extreme maximum or minimum availability values tend to have small fleet sizes; for example, Nellis AFB, Nevada, had only four F-35A aircraft for the majority of the reporting period. F-35B operations began at Edwards AFB, California, in October, when a single aircraft was transferred from Yuma MCAS. Availability of that aircraft is not broken out separately, but is included in the whole fleet calculation. • The NMC-M rate was relatively steady at an average of 26 percent for the 12-month period ending October 2014, nearly twice the goal for 2014, excluding the depot for this entire period. A substantial amount of NMC-M down time continues to be the result of field maintenance organizations • • waiting for technical dispositions or guidance from the contractor on how to address a maintenance issue that has grounded an aircraft. These Action Requests (ARs) are a result of incomplete or inadequate technical information in the field, in the form of Joint Technical Data (JTD). While JTD validation has progressed (see separate section below), the complexity of AR’s is increasing, leading to longer times to receive final resolution. Reducing the rate of ARs or decreasing the response time to the ARs will improve (lower) NMC-M rates. High Mean Times To Repair (MTTR), the average maintenance time to fix a single discrepancy, are experienced in all variants. This also contributes to the persistently high NMC-M rate. Over the same 12-month period, the NMC-S rate displayed an improving trend, peaking at 27 percent in November 2013, decreasing to rates in the high 10s to low 20s by mid-2014, and reaching a minimum of 15 percent in October. In 2013, the Program Office predicted that better contracting performance and the maturing supply system would result in improved supply support, which would in turn result in lower NMC-S rates by late 2014. Although the trend is favorable, the rate of improvement is not yet fast enough to allow the program to achieve their goal of 10 percent NMC-S by the end of 2014. If the current trend continues, the program could reach this target in early- to mid-2015. A large portion of the fleet began cycling through the depot for Block 2B modifications made necessary by concurrent development, exerting downward pressure on overall fleet availability. The program began reporting the percentage of the fleet in depot status starting in January 2014 at 13 percent. Since then, it has risen to as high as 18 percent in July 2014, and was at 11 percent by the end of October. Current plans show over 10 percent of the operational aircraft inventory will be in depot status for Block 2B modifications through at least mid-2015 (either at a dedicated facility or being worked on by a depot field team at the home station). If the Services elect to upgrade all early production aircraft to Block 3F capability, these aircraft will again be scheduled for depot-level modifications (operational test aircraft must be modified.) All necessary depot-level modifications are not yet identified for Block 3F, as testing and development are not complete. Therefore, the impact on availability due to Block 3F modifications in the 2016 through 2018 timeframe is unknown. Although depot modifications reduce overall fleet availability, they potentially improve availability once the aircraft is out of depot by replacing low reliability components with improved versions, such as the 270 Volt Battery Charger and Control Unit. Any increased availability from reliability improvements will take time to manifest in the fleet wide metrics, not showing more strongly until the majority of aircraft have been modified. Low availability rates, in part due to poor reliability, are preventing the fleet of fielded operational F-35 aircraft (all F-35 JSF 59 FY14 DOD PROGRAMS variants) from achieving planned, Service-funded flying hour goals. Original Service bed-down plans were based on F-35 squadrons ramping up to a steady state, fixed number of flight hours per tail per month, allowing for the projection of total fleet flight hours. -- In November 2013, a new “modelled achievable” flight hour projection was created since low availability was preventing the full use of bed-down plan flight hours. The revised model accounted for some actual fleet maintenance and supply data, and made assumptions about many factors affecting availability in the coming years to predict the number of flight hours the fleet could generate in future months. -- Through October 30, 2014, the fleet had flown approximately 72 percent of the modelled achievable hours because availability had not increased in accordance with assumptions. Planned versus achieved flight hours, for both the original plans and the modelled achievable, through October 30, 2014, by variant, for the fielded production aircraft are shown in the table below. • F-35 Fleet Planned vs. Achieved Flight Hours as Of October 30, 2014 Variant Original Bed-Down Plan Cumulative Flight Hours “Modelled Achievable” Cumulative Flight Hours Estimated Planned Achieved Percent Planned Estimated Planned Achieved Percent Planned F-35A 11,500 6,347 55% 9,000 6,347 71% F-35B 8,500 6,085 72% 7,500 6,085 81% F-35C 1,800 910 51% 1,600 910 57% Total 21,800 13,342 61% 18,600 13,342 72% F-35 Fleet Reliability • Aircraft reliability is assessed using a variety of metrics, each characterizing a unique aspect of overall weapon system reliability. -- Mean Flight Hours Between Critical Failure (MFHBCF) includes all failures that render the aircraft not safe to fly, and any equipment failures that would prevent the completion of a defined F-35 mission. It includes failures discovered in the air and on the ground. -- Mean Flight Hours Between Removal (MFHBR) gives an indication of the degree of necessary logistical support and is frequently used in determining associated costs. It includes any removal of an item from the aircraft for replacement with a new item from the supply chain. Not all removals are failures, and some failures can be fixed on the aircraft without a removal. For example, some removed items are later determined to have not failed when tested at the repair site. Other components can be removed due to excessive signs of wear before a failure, such as worn tires. -- Mean Flight Hours Between Maintenance Event, unscheduled (MFHBME) is useful primarily for evaluating maintenance workload. It includes all failures, whether inherent or induced by maintenance actions, that 60 F-35 JSF • • • • • led to maintenance and all unscheduled inspections and servicing actions. -- Mean Flight Hours Between Failure, Design Controllable (MFHBF_DC) includes failures of components due to design flaws under the purview of the contractor, such as the inability to withstand loads encountered in normal operation. Failures of Government Furnished Equipment (GFE) and failures induced by improper maintenance practices are not included. The F-35 program developed reliability growth projections for each variant throughout the development period as a function of accumulated flight hours. These projections are shown as growth curves, and were established to compare observed reliability with target numbers to meet the threshold requirement at maturity, defined by 75,000 flight hours for the F-35A and F-35B, and by 50,000 flight hours for the F-35C, and 200,000 cumulative fleet flight hours. In November 2013, the program discontinued reporting against these curves for all ORD reliability metrics, and retained only the curve for MFHBF_DC, which is the only reliability metric included in the JSF Contract Specification (JCS). The growth curves for the other metrics have been re-constructed analytically and are used in the tables below for comparison to achieved values, but are not provided by the Program Office. As of October 2014, the F-35, including operational and flight test aircraft, had accumulated approximately 22,000 flight hours, or slightly more than 11 percent of the total 200,000‑hour maturity mark defined in the ORD. Since May 2013, the program has reported Reliability and Maintainability (R&M) metrics on a three-month rolling window basis meaning, for example, the MFHBCF rate published for a month accounts only for the critical failures and flight hours of that month and the two previous months. Before May 2013, R&M metrics were reported on a cumulative basis. The switch to a three-month rolling window is intended to give a more accurate account of current, more production-representative aircraft performance, and eliminate the effect of early history when the SDD aircraft were very immature; however, this process can create significant month‑to-month variability in reported numbers. A comparison of current observed and projected interim goal MFHBCF rates, with associated flight hours, is shown in the first table on the following page. Threshold at maturity and the values in the FY13 DOT&E report are shown for reference as well. Similar tables comparing current observed and projected interim goals for MFHBR, MFHBME, and MFHBF_DC rates for all three variants are also provided. MFHBF_DC is a contract specification, and its JCS requirement value is shown in lieu of an ORD threshold. The large number of flight hours and events in each three‑month rolling window supporting the observed reliability metrics in the tables above provide statistical evidence that the program experienced reliability growth in FY14 DOD PROGRAMS • • • • all metrics and all variants F-35 Reliability: MFHBCF (hours) between August 2013 and ORD Threshold Values as of August 31, 2014 Values as of August 2013 August 2014. Interim Goal Observed Observed Observed The critical failure rates for Cumulative Cumulative Variant Flight to Meet ORD Value as MFHBCF MFHBCF MFHBCF Flight Flight all three variants were below (3 Mos. Rolling (3 Mos. Rolling Hours Threshold Percent of Hours Hours Window) Window) threshold values and below MFHBCF Goal projected interim goals. F-35A 75,000 20 8,834 14.9 8.2 55% 4,204 4.5 Due to the large variability F-35B 75,000 12 7,039 8.6 7.5 87% 3,286 3.0 in month-to-month reported F-35C 50,000 14 2,046 9.2 8.3 90% 903 2.7 values, however, the high apparent growth for both F-35 Reliability: MFHBR (hours) the F-35B and F-35C from ORD Threshold Values as of August 31, 2014 Values as of August 2013 the data point values above Interim Goal Observed Observed Observed may not be characteristic Cumulative Cumulative Variant Flight to Meet ORD Value as MFHBR MFHBR of the actual growth, with MFHBR Flight Flight (3 Mos. Rolling (3 Mos. Rolling Hours Threshold Percent of Hours Hours August 2013 being notably Window) Window) MFHBR Goal below average for those F-35A 75,000 6.5 8,834 4.8 3.1 65% 4,204 2.5 variants, and August 2014 F-35B 75,000 6.0 7,039 4.3 2.5 58% 3,286 1.4 being substantially above F-35C 50,000 6.0 2,046 3.9 2.3 59% 903 1.6 average. All variants are below F-35 Reliability: MFHBME (hours) their threshold values and projected interim goals for ORD Threshold Values as of August 31, 2014 Values as of August 2013 MFHBR and MFHBME. Interim Goal Observed Observed Observed Cumulative Cumulative Variant Design controllable failure Flight to Meet ORD Value as MFHBME MFHBME MFHBME Flight Flight Hours Threshold (3 Mos. Rolling Percent of (3 Mos. Rolling rate is the only metric where Hours Hours Window) Window) MFHBME Goal any variants exceed the F-35A 75,000 2.0 8,834 1.5 0.85 57% 4,204 0.78 interim goal; as shown in the table with the F-35B F-35B 75,000 1.5 7,039 1.1 0.96 87% 3,286 0.46 and F-35C. For all variants, F-35C 50,000 1.5 2,046 0.9 0.84 93% 903 0.35 the degree of improvement in MFHBF_DC by F-35 Reliability: MFHBF_DC (hours) August 2014, relative to JCS Requirement Values as of August 31, 2014 Values as of August 2013 the May 2013 value, is Interim Goal Observed Observed Observed greater than the degree of Cumulative Cumulative Variant Flight MFHBF_ to Meet JCS Value as MFHBF_DC MFHBF_DC Flight Flight improvement for all other Hours DC Requirement (3 Mos. Rolling Percent of (3 Mos. Rolling Hours Hours Window) Window) reliability metrics. This MFHBF_DC Goal indicates the improvement F-35A 75,000 6.0 8,834 4.2 4.0 95% 4,204 2.8 in the contract specification F-35B 75,000 4.0 7,039 2.7 3.5 130% 3,286 1.9 metric of MFHBF_DC is F-35C 50,000 4.0 2,046 2.4 3.6 150% 903 1.5 not translating into equally large improvement in the failures to a system that never fails. The closer the growth other reliability metrics, which are operational requirements. rate is to 1.0 the faster the growth, but the relationship DOT&E conducted an in-depth study of reliability growth between assessed growth rates is not linear, due to the in MFHBR and in MFHBME for the period from July 2012 logarithmic nature of the plot. For example a growth rate through October 2013. Reliability growth was modeled of 0.4 would indicate reliability growth much higher than using the Duane Postulate, which characterizes growth by a twice as fast as a growth rate of 0.2. single parametric growth rate. Mathematically, the Duane -- Only the F-35A and F-35B variants were investigated due Postulate assesses growth rate as the slope of the best fit line to a low number of flight hours on the F-35C. The study when the natural logarithm of the cumulative failure rate evaluated the current growth rate, then, using that rate, is plotted against the natural logarithm of cumulative flight projected the reliability metric to the value expected at hours. A growth rate of zero would indicate no growth, and maturity. a growth rate of 1.0 is the theoretical upper limit, indicating instantaneous growth from a system that exhibits some F-35 JSF 61 FY14 DOD PROGRAMS -- T he study also evaluated the growth rate needed to meet the ORD threshold value at maturity (75,000 hours each for the F-35A and F-35B) from the current observed value of the reliability metric. The results of the study are summarized in the following table. on a different part nearby, it is a judgment call whether that is an inherent design failure because the component could not withstand “normal” wear and tear in operational service, or if it’s an induced failure because the maintainer was “too rough.” ▪▪ Analysis on F-35A data including SDD Current Projected Projected Growth and LRIP aircraft from September 2012 to October Growth Rate Value at ORD Value as Rate Metric Variant 2013 April 2014 shows a generally increasing from Duane 75,000 Threshold % ORD Needed to Value number of failures categorized as induced each Postulate FH Threshold Meet ORD month over the entire period, but a generally F-35A 3.30 0.129 4.19 6.5 65% 0.232 MFHBR decreasing number of failures categorized as F-35B 1.87 0.210 4.05 6.0 68% 0.305 inherent for each month since April 2013. The F-35A 0.82 0.162 1.45 2.0 73% 0.241 decreasing inherent failure count per month is MFHBME notable, as during this period, the F-35A fleet F-35B 0.64 0.347 1.74 1.5 116% 0.312 size and total hours flown per month were -- For most of the measures, the F-35 must achieve a much increasing steadily. faster growth rate than currently exhibited in order to meet ▪▪ S ome of this is due to re-categorizing nut-plate failures. ORD requirements by maturity. Reliability growth rates Actual reliability growth can also explain some of this, are very sensitive when calculated early in a program, with as could poor training leading to bad troubleshooting only relatively low numbers of flight hours (i.e., less than and maintenance practices. Some of this could also 10,000), and can differ significantly either on the up or be due to re-categorizing failures previously scored down side from growth rates calculated once a program is as inherent failures as induced failures. For example, more mature. Program Office maintenance data records showed that MFHBME -- The above growth rates there were twice as many inherent failures as induced Aircraft Growth Rate were calculated with failures in September 2012, and there were many more F-15 0.14 around 4,700 flight inherent failures than induced for every subsequent hours for the F-35A, and month through May 2013. Then in June 2013, records F-16 0.14 3,800 for the F-35B. For F-22 (at 35,000 flight hours) showed that there were more than twice as many 0.22 comparison, observed induced failures than inherent failures, and induced B-1 0.13 MFHBME growth rates failures have always been much greater than inherent “Early” B-2 (at 5,000 flight hours) 0.24 for several historical failures for each month afterward. This sudden and “Late” B-2 0.13 aircraft are shown in the abrupt reversal of the relationship between induced and table to the right. inherent failures across the entire F-35A fleet suggests C-17 (at 15,000 flight hours) 0.35 -- The growth rates for the that scoring failures differently (induced vice inherent) F-35 to comply with ORD performance by maturity have may result in an increase in the design‑controllable been demonstrated in the past, but only on different type metric that is not manifested in other reliability aircraft and not on fighters. metrics. • The most recent 90-day rolling averages for MFHBF_DC -- Due to poorer than expected initial reliability of many show more growth in this metric than for any other reliability components, the program has started to re-design and metric for the period from May 2013 through August 2014. introduce new, improved versions of these parts. Once a The following contributed to the reported growth in new version of a component is designed, it is considered the production-representative version. However, failed MFHBF_DC. -- In June 2013, the program re-categorized nut plate failures, components may still be replaced by the old version of one of the most common failures in the aircraft, as induced the component in order to keep aircraft flying until the failures rather than inherent failures, removing them from new version is produced in enough quantity to proliferate the calculation of MFHBF_DC. Nut plates are bonded to 100 percent of the fleet and supply stock. During this to an aircraft structure and receive bolt-type fasteners to transition period, only failures of the new version of hold removable surface panels in place. One way nut the component are counted as relevant to the reliability plates can fail, for example, is when torquing a bolt down metrics, because the old version is no longer considered while replacing a removed panel, the nut plate dis-bonds production-representative. from the aircraft structure, preventing securing the surface ▪▪ T his creates a situation where not all failures are panel. counted in the calculation of mean flight hours between -- Distinguishing between inherent design failures and reliability events, but all flight hours are counted, induced failures can be subjective in certain cases. For and hence component and aircraft reliability are example, if a maintainer working on the aircraft bumps a reported higher than if all of the failures were counted. good component with a tool and breaks it while working The result is an increased estimation of reliability 62 F-35 JSF FY14 DOD PROGRAMS compared to an estimate using all failures, and is highest at the beginning of the transition period, especially if the initial batch of re-designed components is small. ▪▪ For example, as of September 2014, an improved horizontal tail actuator component had been introduced and installed on roughly 30 aircraft out of a fleet of nearly 100. Failures of the older component were not being counted in the metrics at all anymore, but flight hours from all 100 aircraft were counted. This calculation could result in the reported reliability of that component being increased by up to a factor of three compared to reliability if all of the horizontal tail actuator failures were counted. There are hundreds of components on the aircraft, so a single component’s increased estimate of reliability may have little influence on overall observed aircraft reliability. However, since multiple components are being upgraded simultaneously due to the unprecedented and highly concurrent nature of the F-35 program, the cumulative effect on the overall observed aircraft reliability of the increased estimate of reliability from all of these components may be significant. -- Tire assemblies on all F-35 variants do not last as long as expected and require very frequent replacement. However, only when a tire failure is experienced on landing is it counted as a design controllable failure. The vast majority of tires are replaced when worn beyond limits, and in these cases they are scored as a “no-defect.” Thus, most tire replacements show up in the MFHBR and MFHBME metrics, but not in MFHBF_DC or MFHBCF, even though the aircraft is down for unsafe tires. The program is seeking redesigned tires for all variants to reduce maintenance down time for tire replacements. • A number of components have demonstrated reliability much lower than predicted by engineering analysis, which has driven down the overall system reliability and/or led to long wait times for re-supply. High driver components affecting low availability and reliability include the following, grouped by components common to all variants followed by components failing more frequently on a particular variant or completely unique to it, as shown below: High Driver Components Affecting Low Availability and Reliability Common to All Variants F-35A F-35B F-35C • Avionics Processors • Main Landing Gear Tires • Thermal Management System • Ejection Seat Assembly • Panoramic Cockpit Display Electronics Unit • Low Observable Cure Parameters • Helmet Display Unit • Seat Survival Kit • Igniter-Spark, Turbine Engine • On-Board Oxygen Generating System Additional High Drivers by Variant • Exhaust Nozzle Assembly • Exhaust Nozzle ConvergingDiverging Link • Upper Lift Fan Door Actuator1 • 270 Volt DC Battery • Data Transfer Cartridge • Solenoid Operated Fuel Valve 1. Unique to the F-35B Maintainability • The amount of time spent on maintenance for all variants exceeds that required for mature aircraft. Two measures used to gauge this time are Mean Corrective Maintenance Time for Critical Failures (MCMTCF) and Mean Time To Repair (MTTR) for all unscheduled maintenance. MCMTCF measures active maintenance time to correct only the subset of failures that prevent the JSF from being able to perform a defined mission, and indicates how long it takes, on average, to return an aircraft to mission capable status. MTTR measures the average active maintenance time for all unscheduled maintenance actions, and is a general indicator of ease and timeliness of repair. Both measures include active touch labor time and cure times for coatings, sealants, paints, etc., but do not include logistics delay times such as how long it takes to receive shipment of a replacement part. • The tables below compare measured MCMTCF and MTTR values for the three-month period ending August 2014 to the ORD threshold and the percentage of the value to the threshold for all three variants. The tables also show the value reported in the FY13 DOT&E Annual Report for reference. For the F-35A and F-35C, MCMTCF increased (worsened) over the last year while MCMTCF for the F-35B showed slight improvement. For all variants, MTTR showed improvement over the last year. Both maintainability measures for all variants are well above (worse than) the ORD threshold value required at maturity. F-35 Maintainability: MCMTCF (hours) Variant ORD Threshold Values as of August 31, 2014 (3 Mos. Rolling Window) Observed Value as Percent of Threshold (3 Mos. Rolling Window) Values as of August 2013 F-35A 4.0 15.6 390% 12.1 F-35B 4.5 15.2 338% 15.5 F-35C 4.0 11.2 280% 9.6 F-35 Maintainability: MTTR (hours) Variant ORD Threshold Values as of August 31, 2014 (3 Mos. Rolling Window) Observed Value as Percent of Threshold (3 Mos. Rolling Window) Values as of August 2013 F-35A 2.5 8.6 344% 9.2 F-35B 3.0 7.5 250% 8.9 F-35C 2.5 6.6 264% 7.7 • More in depth trend analysis between May 2013 and August 2014 shows that the MTTR for the F-35A and F-35C variants have been decreasing slowly, while the MTTR for the F-35B has been growing slightly, with all exhibiting high month-to-month variability. Over the same period, the MCMTCF values for the F-35B and F-35C were increasing slightly and flat for the F-35A, but again with very high monthly variability. • Several factors likely contribute to extensive maintenance time, especially long cure times for Low Observable repair materials. The Program Office is addressing this issue with F-35 JSF 63 FY14 DOD PROGRAMS new materials that can cure in 12 hours vice 48 for example, but some of these materials may require freezer storage, making re-supply and shelf life verification in the field or at an austere operating location more difficult. -- The immaturity of the system overall, including training system immaturity, lack of maintainer experience on such a new aircraft, and incompletely written and verified, or poorly written, JTD may all also contribute to protracted maintenance times. -- Additionally, design factors of the aircraft itself make affecting certain repairs difficult and time-consuming. Field maintainers have reported poor cable routing behind panels that interferes with required maintenance, and awkward placement of some components, which makes removing and replacing them slow, and increases the chances they will induce a failure in a nearby component working with tools in confined spaces. -- Scoring also affects higher than expected MTTR values. Discrepancies for which maintainers have to attempt multiple solutions before finding a true fix are being re-scored as a single event, while in the past they were documented as multiple repair attempts, each with its own MTTR. The individual MTTRs for these attempted repairs are now rolled up into the single, re-scored event. Improved diagnostics and training can reduce MTTR by pointing maintainers to the true root cause of discrepancies more quickly. Autonomic Logistics Information System (ALIS) • The program develops and fields ALIS in increments similar to the mission systems capability in the air vehicle. Overall, ALIS is behind schedule, has several capabilities delayed or deferred to later builds, and has been fielded with deficiencies. The program does not have a dedicated end-to‑end developmental testing venue for ALIS and has relied on feedback from the field locations for identifying deficiencies. Though some of the early deficiencies have been addressed, ALIS continues to be cumbersome to use and inefficient, and requires the use of workarounds for deficiencies awaiting correction. The program has tested ALIS software versions at the Edwards flight test center, including a formal Logistics Test and Evaluation (LT&E) of ALIS software versions 1.0.3 and 2.0.0. These formal test periods had limitations, however, as the ALIS that supports the developmental test aircraft is different than the production ALIS hardware at fielded units. As a result, the program has begun limited testing of software updates at fielded operational sites and will expand this testing in CY15. The program should ensure adequate testing of ALIS software upgrades on operationally-representative hardware is complete prior to fielding to operational units. -- In the last year, the Program Office adjusted the schedule and incremental development plans for ALIS hardware and software capability releases three times. These adjustments were necessary to align ALIS capabilities with 64 F-35 JSF Service requirements to support planned IOC declaration dates. ▪▪ In December 2013, the program re-planned the schedule and capability release of ALIS 2.0.0, the next version to be fielded, moving the initial release from November 2014 to January 2015. ▪▪ In February 2014, the program adjusted the schedule and release plans for the follow-on version of ALIS, version 2.0.1. The schedule for fielding was adjusted by three months (from March 2015 to June 2015) and the life limited parts management (LLPM) module was deferred to later increments of ALIS. Because of delays in development, the LLPM capability was split into two increments (initial and final); the initial increment will be fielded with ALIS 2.0.2 and aligned to support Air Force IOC plans, and the final increment of LLPM will be fielded in ALIS 2.0.3. ▪▪ In November 2014, the program adjusted the schedule and release plans again, moving the final increment of the LLPM to ALIS 3.0.0 and accelerating the integration of an upgraded processor from ALIS 3.0.0 to ALIS 2.0.2, eliminating the need for ALIS release 2.0.3. The content previously planned for ALIS 3.0.0 will be renamed 3.0.1. The program’s planned release dates are July 2017 for ALIS 3.0.0 and July 2018 for ALIS 3.0.1. ▪▪ A Windows server update has moved forward to an earlier ALIS release, from ALIS 3.0.0 to 2.0.1, which the program plans to field in June 2015. • During CY14, the program accomplished the following with ALIS software development and fielding: -- The program completed the migration of operational units from older versions to ALIS 1.0.3 (the current fielded version) in January 2014 as planned, followed by an updated version in February 2014 (version 1.0.3A3.3.1), which included limited fixes for deficiencies identified during testing in late CY12 and early CY13. ALIS 1.0.3A.3.3.1 has reduced screen refresh and load times compared to 1.0.3, and reduced the number of nuisance/false health reporting codes; however, time‑consuming workarounds are required to determine and update the readiness of aircraft to fly missions. The following are examples of workarounds. ▪▪ Additional steps required to process aircraft health information to be compatible with the Exceedance Management System, which is not integrated into ALIS. ▪▪ Manual entry of information into ALIS to track consumables such as oil usage. ▪▪ Frequent submission of formal ARs to Lockheed Martin for assistance, because troubleshooting functionality is incomplete. ▪▪ Manual correlation of health reporting codes between ALIS domains. FY14 DOD PROGRAMS -- I n future versions of ALIS, the program plans to address the above workarounds and include three key requirements identified by the Services as needed for IOC: ▪▪ Integration with a new deployable ALIS standard operating unit (SOU) hardware (SOU V2, described below) ▪▪ Support of detached, sub-squadron operations at deployment locations away from the main operating base ▪▪ Distributed maintenance operations allowing supervisors to verify completion of maintenance operations from various locations at the main or deployed operating base (i.e., dynamic routing). -- The next major increment of ALIS software, version 2.0.0, began testing with the mission systems developmental test aircraft at Edwards AFB in September 2014. The program plans to field version 2.0.0 starting in January 2015. The ALIS 2.0.0 upgrade includes integrated exceedance management, Windows 7, recording of structural health data for use in the future development of prognostic health capabilities, and continued optimization efforts with improvements to data structures and database tuning. ▪▪ Testing of the screen refresh times for ALIS 2.0.0 in a laboratory environment has shown improvement compared to those observed with ALIS 1.0.3A3.3.1. For example, in a simulated environment supporting 28 aircraft, squadron health management debrief time decreased from 101 seconds to less than 5 seconds after implementation of several cycles of improvements. Actual fielded performance is unknown. ▪▪ Preliminary results from the LT&E of ALIS 2.0.0 show that multiple deficiencies from past evaluations remain unresolved, and the system demonstrated deficiencies in new capabilities. Although results have not been finalized with a deficiency review board, the initial LT&E report indicates: »» A critical deficiency noted in the LT&E of ALIS 1.0.3 for the failure of the manual control override to work correctly, which results in the incorrect reporting of the air vehicle status as not mission capable in the squadron health management function of ALIS, has not been corrected in ALIS 2.0.0. »» ALIS 2.0.0 demonstrated 4 additional critical deficiencies and 53 serious deficiencies. »» Exceedance management has been integrated into ALIS 2.0.0 but exhibited processing delays. »» The test site was unable to complete testing of all ALIS 2.0.0 functionality because the site lacks a squadron operating unit and instead relied on data transfers between Edwards AFB and Fort Worth, Texas. The test team recommended that the remaining tests be conducted at an operating location with representative hardware. -- A LIS 2.0.0 will provide the basis for incremental builds (versions 2.0.1 and 2.0.2), which are intended to be fielded in support of Marine Corps IOC and the Air Force IOC declarations, respectively. ▪▪ The program plans to deliver ALIS 2.0.1 to the flight test center in February 2015, conduct a formal LT&E, in preparation for fielding in July 2015, which is the current objective date for Marine Corps IOC. ALIS 2.0.1 software will align with a new hardware release (SOU version 2) that will improve deployability and will include fault isolation improvements and a Windows server update. ▪▪ To support the Marine Corps preparation for IOC, the program plans to release ALIS 2.0.1 in May 2015 to Yuma MCAS, Arizona, simultaneous with the planned delivery of the deployable ALIS hardware system for limited validation and verification testing of the software prior to release to the rest of the fielded units. Though the current ALIS release schedule leaves no margin for delay to meeting the Marine Corps IOC objective date in July, fielding ALIS 2.0.1 before formal testing and fix verification is complete may result in the continued need for workarounds to support field operations. -- The program has scheduled ALIS 2.0.2 fielding, which is required to meet Air Force IOC requirements, for December 2015. It will provide a sub-squadron reporting capability that allows air vehicle status reporting of deployed assets back to the parent SOU, and adds dynamic routing, which allows delivery of messages and data via alternate network paths. ALIS 2.0.2 will also reduce the need for propulsion contractor support by integrating the first portion of a required LLPM capability. -- ALIS 3.0.0 will complete the majority of the ALIS development effort. The schedule, which is pending approval, shows a fielding date of July 2017. This version of ALIS will include a complete LLPM capability and eliminate the need for propulsion contractor support. • The following sections describe progress in the development and fielding of ALIS hardware and alignment with ALIS software capabilities described earlier: -- The program continued to field ALIS hardware components at new locations during CY14 as the global sustainment bed-down and F-35 basing continued to be activated. The table on the following page shows ALIS components, location, and sustainment function for new components fielded in CY14. -- In order to reduce post-flight data download times, the program added and fielded a new piece of hardware, the Portable Maintenance Device (PMD) reader, to operational units beginning in July 2014. The PMD reader is designed to accelerate the download of unclassified maintenance data from the aircraft without the need for a secure facility. The PMD reader permits maintenance personnel to download maintenance data only, vice waiting for full portable memory device download from the aircraft to be processed in a secure facility via the Ground Data Security F-35 JSF 65 FY14 DOD PROGRAMS Assembly Receptacle (GDR). ALIS hARDWARE fIELDED IN FY14 Testing of the PMD could not be done Component Location Function at the flight test center because the One per country to provide in-country architecture of the ALIS supporting and software and data distribution, enable the developmental testing aircraft is Central Point of Entry Eglin AFB interoperability with government systems not production-representative. The at national level, and enable ALIS data fielded PMD readers have functioned connectivity between bases. as intended. Maintenance downloads Beaufort Academic Training Facility generally take less than 5 minutes Italian FACO Supports squadron-level F-35 operations, Italian Pilot Training Center using a PMD reader, while the Standard Operating including maintenance, supply chain Australian Pilot Training Center procedure using the ground data Unit (SOU) management, flight operations, training, and Luke AFB Pilot Training Center mission planning. receptacle – which downloads all Nellis AFB 57th Fighter Wing data recorded on the PMD – usually Netherlands SOU (at Edwards AFB) takes an hour, delaying access to Nellis AFB Centralizes base supply for bases operating Base Kit maintenance information. Edwards AFB with several squadrons. -- SOU Version 1 (SOU V1), the Similar to a squadron kit but permanently LHD Ship Kit USS Wasp current ALIS unit-level hardware installed shipboard. configuration, failed to meet the Short of a full squadron kit but contains deployability requirement in the sufficient hardware to support four aircraft. Deployment Kit Luke AFB Pilot Training Center Will become a squadron kit upon delivery of ORD due to its size, bulk, and remaining hardware. weight. The program is developing Hill AFB Similar to a base kit but geared to support a deployable version of the SOU, Depot Kit MCAS Cherry Point depot operations. deemed SOU Version 2 (SOU V2). It will support Block 2B, Block 3i, Management (PHM) System as it matures. PHM has and Block 3F aircraft, and is needed for service IOC three major components: fault and failure management dates. It will be incrementally developed and fielded with (diagnostic capability), life and usage management increasing capability over the next several years. (prognostic capability), and data management, all of which ▪▪ The first increment of SOU V2, a modularized and will be an integral part of ALIS. Currently PHM has no man-portable design for easier deployability, will first be prognostic capability, while diagnostic and data management made available to Marine Corps for IOC in 2015. This functionality remain immature. The program plans to first increment aligns SOU V2 hardware and ALIS 2.0.1 include the first set of prognostic algorithms in ALIS 2.0.2. software release. The program plans to conduct limited - Diagnostic capability is designed to enable maintenance by validation and verification testing of the ALIS 2.0.1 detecting true faults within the air vehicle and accurately software on the SOU V2 once delivered to Yuma MCAS isolating those faults to a line-replaceable component. To (planned for May 2015), and prior to fielding it to other date, the diagnostic functional capability has demonstrated units in July. low detection rates, poor accuracy, and high false alarm ▪▪ The second increment of SOU V2 went on contract in rates. The table on the following page shows metrics of August 2014. This increment will address Air Force diagnostic capability, the ORD threshold requirement at hardware requirements for sub-squadron reporting maturity (200,000 hours), and demonstrated performance capabilities and inter-squadron unit connectivity and as of May 2014. For comparison, demonstrated will align with release of ALIS software version 2.0.2. performance from May 2013 is also shown. While It is scheduled to begin testing at the flight test centers in detection and isolation performance metrics improved July 2015. between May 2013 and May 2014, mean flight hours ▪▪ The third increment of SOU V2, which also went between false alarm performance decreased (worsened). on contract in August 2014, will address hardware As a result, fielded operations have had to rely on manual requirements for decentralized maintenance, which will workarounds, such as maintainer-initiated built-in tests, allow maintenance personnel to manage tasks with or extra scheduled inspections, and reliance on contractor without connectivity to the main SOU and allow for a support personnel, for more accurate diagnostics of Portable Maintenance Aid-only detachment; it will align system faults. Although these workarounds have aided with ALIS 3.0.0. troubleshooting, they increase the maintenance man-hours • ALIS was designed to provide the analytical tools and per flight hour and reduce sortie generation rates. algorithms to assess air vehicle health management using health reporting codes (HRCs) collected during flight. These functions will enable the Prognostic Health and 66 F-35 JSF FY14 DOD PROGRAMS mETRICS OF dIAGNOSTIC cAPABILITY diagnostic Measure Threshold Requirement Demonstrated Performance (May 2013) Demonstrated Performance (May 2014) Fault Detection Coverage (percent mission critical failures detectable by PHM) 98 74 81 Fault Detection Rate (percent correct detections for detectable failures) 98 73 81 Fault Isolation Rate (percentage): Electronic Fault to One LRC 90 77 72 Fault Isolation Rate (percentage): Non-Electronic Fault to One LRC 70 70 79 Fault Isolation Rate (percentage): Non-Electronic Fault to 3 or Fewer LRC 90 80 85 Mean Flight Hours Between False Alarm 50 0.59 0.23 Mean Flight Hours Between Flight Safety Critical False Alarm 450 69 170 Joint Technical Data (JTD) • Lack of verified JTD modules continues to challenge fielded operations, requiring workarounds such as ARs to the contractor for engineering dispositions on required maintenance actions. Also, maintenance personnel in the fielded units are finding that verified JTD may not be adequate to complete maintenance actions efficiently, such as an engine removal and replacement and maintenance built-in test troubleshooting. • JTD modules are first identified as needed in the field, then developed by the contractor, and finally verified before being provided to the operating locations. Entire JTD packages (i.e., all JTD modules bundled together) are periodically distributed to field locations using ALIS, and then downloaded at the units to the Portable Maintenance Aids. -- The current process is cumbersome, as all modules are distributed together, including modules with no changes or updates, along with new modules and those with updates. ALIS 2.0.0 should allow the program to deliver partial JTD builds (i.e., changes and amendments to existing JTD). -- The total number of data modules identified continues to grow as the program matures and additional JTD deliveries are added in LRIP contracts. According to Program Office schedules, the development of identified JTD modules for each variant of air vehicle and for propulsion is on track to meet Service milestones. Air vehicle JTD includes time compliance technical data, depot-level technical data, air vehicle diagnostics and troubleshooting procedures, complete structural field repair series data, aircraft battle damage assessment and repair, and maintenance training equipment. Propulsion JTD development is nearly complete and on schedule. Development of Support Equipment (SE) JTD lags the Program Office schedule by 9 percent (approximately 200 modules out of 2,150 identified), primarily due to the lack of delivery of fault isolation engineering source data. • Verification of air vehicle JTD modules is behind and has been slowed by the program’s dependence on production aircraft to conduct opportunistic aircraft verification events. Because priority is given to the flight schedule, verification events are not scheduled and require support from the field to complete JTD verification. The program has identified more air vehicle JTD modules during the last year, hence the percentage of JTD modules verified has not increased substantially compared to what was reported in DOT&E’s FY13 Annual Report. To reduce the number of unverified JTD modules at Marine Corps IOC declaration, the program should provide dedicated time on fielded aircraft for F-35B JTD verification teams. -- SE JTD verification occurs as modules are developed and released in ALIS, so it lags the schedule by a similar amount as module development. SE assets at the training units at Eglin AFB are the primary source for SE verification. -- The program placed Supportable Low Observable (SLO) JTD verification on contract in March 2014, with most verification performed using desktop analysis. SLO JTD verification for the F-35B is nearly complete. Since many of the SLO modules for the F-35A and F-35C variants are similar to those for the F-35B, the program expects the verification of SLO modules for those variants to proceed on schedule. SLO JTD verification data were not available at the time of this report; progress in identification and development of SLO modules is reported separately in the table below. • As of the end of October, the program had verified approximately 84 percent of the identified air vehicle JTD modules for the F-35A, 74 percent for the F-35B, and 62 percent for the F-35C. The table on the following page shows the number of JTD modules identified, developed, and verified for the air vehicle (by variant), pilot flight equipment (PFE), and SE. Overall, 67 percent of the air vehicle, PFE, and SE identified modules have been verified. Propulsion JTD and SLO JTD are tracked and addressed separately. • Propulsion JTD are current as of April 2014. More current information was not available for this report. Propulsion JTD development and verification has proceeded on schedule and the Program Office considers completion by the end of SDD as low risk. F-35 JSF 67 FY14 DOD PROGRAMS • SLO JTD are current as of the end of October 2014. SLO JTD are tracked under a separate contract with a period of performance of February 2014 through April 2016. The Program Office did not have data showing verification of SLO JTD modules in time for this report. • When verified JTD are not available or not adequate to troubleshoot or resolve a problem with an aircraft, maintenance personnel submit ARs to the contractor. These ARs are categorized as critical (Category 1) or severe (Category 2) and sub-categorized as high, medium, or low. -- The contractor prioritizes and responds to ARs through an engineering team (referred to as the Lightning Support Team), which is composed of Service and contractor personnel. -- As of October 15, 2014, 24 Category 1 ARs remained open while 617 Category 2 ARs were open. The number of open Category 1 ARs has remained relatively flat over the last year, while the number of open Category 2 ARs has decreased by two-thirds since January 2014. F-35 Joint Technical Data Development and Verification Status Air Vehicle, Pilot Flight Equipment (PFE), and Support Equipment (SE) (as of October 2014) Module Type Modules Identified Modules Developed Percent Identified Modules Developed Modules Verified Percent Identified Modules Verified F-35A1 Unit-level 4,658 4,198 90.1% 3,893 83.6% F-35B1 Unit-level 5,783 5,221 90.3% 4,300 74.4% F-35C Unit-level 4,799 3,764 78.4% 2,949 61.5% Unit-level 322 201 62.4% 142 44.1% 1 Common2 (all variants) PFE Common 337 254 75.4% 250 74.2% SE Common 2,150 945 44.0% 604 28.1% 18,049 14,583 80.8% 12,138 67.3% TOTAL 1. Includes field- and depot-level JTD for Operations and Maintenance, on- and off-equipment JTD, and structured field repairs. 2. Includes aircraft JTD for general repairs, sealants, bonding, structured field repairs, and non-destructive investigation. F-35 Joint Technical Data Development and Verification Status Propulsion (as of October 2014) Percent Percent Air-Ship Integration and Ship Suitability Testing Module Modules Modules Identified Modules Identified F-35B Type Identified Developed Modules Verified Modules • The program previously completed two test Developed Verified periods on the USS Wasp with developmental Engine and Propulsion 3,123 2,988 95.7% 2,840 90.9% test aircraft, one in October 2011 and one in Lift Fan August 2013. These periods assessed handling F-35 Joint Technical Data Development and Verification Status qualities for take-off and landing operations at Supportable Low Observable (SLO) sea, and were used to develop an initial flight (as of October 2014) operating envelope for ship operations. ALIS Percent Percent was not deployed to the ship, and very limited Module Modules Modules Identified Modules Identified maintenance operations were conducted (routine Type Identified Developed Modules Verified Modules pre- and post-flight inspections, refueling Developed Verified operations, etc.). F-35A 676 180 26.6% • The Marine Corps began making plans to conduct F-35B 550 52 9.5% N/A N/A another test period on the USS Wasp in May 2015 SLO F-35C 547 52 9.5% to assess ship integration and suitability issues, Common 2 0 0.0% using non-instrumented production aircraft and a non-deployable version of ALIS (SOU V1) installed on the vessel. This deployment was originally a -- Maintenance will be mostly military, but with contractor part of the Block 2B OUE; however, it is being re-scoped to logistics support in line with expected 2015 shore-based support plans for the Marine Corps IOC later in 2015. operations, such as contractors for propulsion data -- Up to six production aircraft are planned to be used for downloads after each flight. Maintenance will be limited the deployment. These aircraft are not instrumented (as to that required for basic flight operations, staging test aircraft are) and will allow the USS Wasp to operate necessary support equipment for engine and lift fan its radars and communications systems in a representative removals only to check if space permits, and loading and manner since there is no concern with electromagnetic downloading demonstrations of inert ordnance on the interference with flight test instrumentation. flight deck. -- The flight operations will not be representative of combat -- These limitations are not representative of combat operations, unless the flight clearance and associated deployment operations. certifications enabling the deployment include clearances • The Marine Corps and Naval Air Systems Command began for weapons carriage and employment. These clearances exploring issues that would arise with employing more than are expected at fleet release, which the program plans to six F-35B aircraft per Air Combat Element (ACE) on L-class occur in July 2015, after the deployment. 68 F-35 JSF FY14 DOD PROGRAMS ships. ACE represents the mix of fixed- and rotary-wing aircraft assigned to the ship to conduct flight operations in support of Marine Corps combat objectives. These “heavy” ACEs could include up to 20 F-35Bs, or 12 or 16 F-35Bs plus MV-22Bs, depending on the specific L-class vessel. Through these exercises, they identified issues, many which will apply to standard-sized ACE operations as well. These issues include: -- The currently-planned service maintenance concept, where few components will be repaired underway but must be sent for repair back to a depot facility or to the Original Equipment Manufacturer (OEM) may not be achievable for initial fielding. The program is conducting a Level Of Repair Analysis (LORA) to assess the feasibility of repairing components at the Intermediate level, including onboard CVN and L-class ships. -- More than six F-35Bs in the ACE will require a more robust engine repair and resupply process than for the standard, six F-35B ACE. The Services are still investigating the best method for F135 engine re‑supply at sea. Work continues on the heavy underway replenishment station and a re-designed engine storage container that can survive a drop of 18 inches while protecting the engine and weighing low enough to be transferred across the wire between the supply ship and the L-class or CVN ship. Adequate storage is needed for the engines, spare parts, and lift fans, as well as workspace for engine module maintenance within the small engine shops on L-class vessels. -- Moving an engine container, including placing an engine in or taking one out of the container, requires a 20,000 pound-class forklift and cannot be concurrent with flight ops since this item is required to be on the flight deck for crash and salvage purposes while flying. Engines can be moved around on a transportation trailer once removed from the container to enable engine maintenance in the hangar bay during flight operations. -- Adequate Special Access Program Facilities (SAPF) are required for flight planning and debriefing aboard the ship. Current modification plans for L-class vessels are expected to meet the requirements for a six F-35B ACE, but would be inadequate for an operation with more aircraft. -- Unlike many legacy aircraft, the F-35B needs external air conditioning when on battery power or an external power source. Cold fueling operations, when the engine is not turned on, will thus need an air conditioning cart. For many more F-35B’s in the ACE, the logistics footprint will have to increase significantly to include more air conditioning units as many aircraft are refueled cold to achieve efficient operations. F-35C • The program began testing the redesigned arresting hook system on aircraft CF-3 at Patuxent River Naval Air Station in February 2014. This test aircraft is modified with unique instrumentation to monitor loads on the arresting hook system and the nose landing gear for catapult launches and arrested landings. The structural survey testing was a pre-requisite for initial carrier sea trials. -- Testing encountered significant delays, as numerous deficiencies were discovered, some requiring new engineering designs. Testing was planned to be completed in July, to support deployment to a CVN for the first set of sea trials. The following problems caused delays: ▪▪ In February, a hydraulic leak in the nose landing gear steering motor, caused by over-pressurization, required a redesigned valve and halted testing for 10 weeks. ▪▪ Excessive galling of the arresting hook pitch pivot pin, which required a redesigned pin made of titanium and additional inspections after each landing. ▪▪ Damage to the nose landing gear shock strut, which required down time for repair -- The structural testing was partially completed in two phases, all on CF-3. ▪▪ Phase one completed September 10, 2014, and consisted of 24 test points needed to clear a monitored envelope for carrier landings. Completion of phase one was necessary for CF-3 to conduct landings on a CVN in November. ▪▪ Phase two consists of 20 additional test points to clear an unmonitored envelope for carrier landings. Completion of phase two testing would allow non-loads instrumented test aircraft to conduct landings on a CVN. Phase two work was ceased on September 25, with 17 of 20 phase two test points completed, but the program waived the remaining three test points to allow CF-5 to participate in DT-1. • Carrier-based ship suitability testing is divided into three phases. -- The first phase, DT-1, consisted of initial sea trials to examine the compatibility of F-35C with a CVN class ship and to assess initial carrier take-off and landing envelopes with steady deck conditions. DT-1 was conducted November 3 – 15, 2014; it was initially scheduled to begin in July. ▪▪ Testers accomplished 100 percent of the threshold test points and 88 percent of the objective points during deployment, completing 33 test flights (39.2 flight hours) and 124 arrested landings, of 124 attempts, including one night flight with two catapult launches and two arrested landings. The results of the test were still in analysis at the time of this report. ▪▪ No other aircraft deployed to the carrier, except transient aircraft needed for logistical support. All landings were flown without the aid of the Joint Precision Approach Landing System, which is planned for integration on the F-35C in Block 3F. No ALIS equipment was installed on the carrier. The test team created a network connection from the ship to the major contractor in Fort Worth to process necessary maintenance actions. F-35 JSF 69 FY14 DOD PROGRAMS -- T he second and third phases, DT-2 and DT-3, consist of ship-borne operations with an expanded envelope (e.g., nighttime approaches, higher sea states than observed in DT-1, if available, and asymmetrical external stores loading). DT-2, which is currently planned for August 2015, will expand the carrier operating envelope. The third set of sea trials is planned for CY16. • The Navy is working on the following air-ship integration issues, primarily for carriers. Each of the following integration issues also applies to F-35B on L-class ships, with the exception of Jet Blast Deflectors (JBDs): -- Due to the higher temperature of F-35 engine exhaust compared to legacy aircraft, carrier JBDs need at least two modifications. A cooling water orifice modification enables basic operations, but additional side panel cooling must be added for higher afterburner thrust catapult launches. The Navy is accomplishing these full modifications on at least some JBDs on USS Abraham Lincoln (CVN-72) in preparation for IOT&E and on USS George Bush (CVN‑77) for developmental testing, and performed the basic orifice modification on USS Nimitz (CVN-68) for the November DT-1. -- The Lockheed Martin-developed F-35 ejection seat dolly failed Critical Design Review. The F-35 ejection seat has a higher center of gravity than legacy seats due to supports for the helmet-mounted display, and in the shipboard environment needs to be securely tied down in case of rolling motion from high sea states. The Navy is investigating developing less expensive adapters to the current ejection seat dolly, and determining what seat shop modifications (if any) will be required to safely tie down the dolly when a seat is installed. -- Two separate methods for shipboard aircraft firefighting for the F-35 with ordnance in the weapon bays are being developed, one for doors open and one for doors closed. Each will consist of an adapter that can fit to the nozzle of a standard hose. The open door adapter will also attach to a 24-foot aircraft tow bar so firefighters can slide it underneath the aircraft and spray cooling water up into the bay. ▪▪ Testing of a prototype open bay adapter was conducted in October and included use on an AV-8B hulk, propane fires, and JP-8 pool fires, as well as assessing ordnance cooling effectiveness. Mobility tests of the rig were also performed on CVN and L-class non-skid, asphalt, grass, dirt, and rough terrain. All tests indicate that the adapter provides sufficient access to the bay for water spray, and featured sufficient ease of use to place the adapter where needed quickly in all environments. ▪▪ The closed door adapter will consist of a penetrating device to punch through the fuselage’s carbon fiber skin, secure in place, and hold when water pressure is applied so deck personnel can then back away from the fire. The Navy also plans to test closed bay door firefighting testing of on-aircraft lithium ion battery fires. 70 F-35 JSF -- W ork on noise abatement during launch and recovery continues. The Navy is installing sound dampening material in the highest noise level areas for flight operations on the USS Abraham Lincoln (CVN-72) during its nuclear refueling and overhaul, and the Office of Naval Research (ONR) will analyze effectiveness compared to untreated ships. This effort will not involve treatment of all work areas, however, and may not be sufficient to allow conversational-level speech in every mission planning space during flight operations. -- The need for improved flight deck hearing protection is not limited to the F-35, as the F-35 and F/A-18E/F Super Hornet produce similar maximum ground noise in afterburner (149 decibels for the F-35 and 150 decibels for the Super Hornet). ▪▪ Based on an assumed F-35 noise environment of 149 decibels when in maximum thrust where personnel are normally located, 53 decibels of attenuation is required to enable 38 minutes of exposure to this worst‑case noise per day before long-term hearing loss ensues. This is estimated to be equivalent to 60 launches and 60 recoveries. ▪▪ Current expected performance for triple hearing protection only reaches into the mid 40’s decibels of attenuation though, which enables less than 10 minutes exposure to maximum noise before the daily limit is reached. Workarounds may include re-positioning launch crew personnel and tighter administrative controls for exposure times. -- The unique Integrated Power Package (IPP), and high‑speed/low-thrust engine turn capability for maintenance on the F-35, may introduce new concerns for hangar bay maintenance. The Navy plans to investigate the impact of IPP exhaust emissions on hangar bay atmosphere, exhaust temperature, and the noise environment produced, to determine acceptable hangar bay maintenance practices. No IPP or engine turns were conducted during the DT-1 sea trials. Progress in Plans for Modification of LRIP Aircraft • Modification of production aircraft is a major endeavor for the program, driven by the large degree of concurrency between development and production, and is a burden independent of the progress made in developmental testing. Modifications are dependent on the production, procurement, and installation of modification kits, completed either at the aircraft depot locations or at the field units. The program will need to provide operationally representative Block 3F operational test aircraft for an adequate IOT&E. • During CY14, the Program Office and Services continued planning for modification of early production aircraft to attain planned service life and the final SDD Block 3F capability, including the production aircraft that will be used to conduct operational testing. Planning had previously focused on modifying aircraft in preparation for the Block 2B OUE FY14 DOD PROGRAMS and Marine Corps IOC, planned by the program to occur in mid‑2015. This created challenges in obtaining long-lead items and dock availability at aircraft depots, and maintaining adequate aircraft availability to maintain pilot currency while eventually modifying all operational test aircraft into a production-representative Block 2B configuration. However, the decision to not conduct the Block 2B OUE allowed the program to focus on Marine Corps IOC aircraft requirements, while attempting to create a more efficient modification plan for operational test aircraft to achieve the Block 3F configuration. - The Program Office has prioritized Block 2B associated modification for Marine Corps F-35B IOC aircraft over operational test aircraft. Because manufacturers could not meet the schedule demand for modification kits, not all of the operational test aircraft will be in the Block 2B configuration by early 2015 when the planned start of spin-up training for the OUE would have occurred, as was noted in the FY13 DOT&E Annual Report. - Modification planning has also included early plans to ensure operational test aircraft scheduled for IOT&E will be representative of the Block 3F configuration. However, these plans show that the program is likely to face the same scheduling and parts shortage problems encountered in planning for Block 2B modifications of the operational test aircraft. • Upgrading aircraft to a Block 2B capability requires the following major modifications: mission systems modifications; structural life limited parts (SLLP), referred to as Group 1 modifications; F-35B Mode 4 operations modifications, which include a modification to the three Bearing Swivel Module (3BSM) to allow F-35B aircraft to conduct unrestricted Mode 4 operations; OBIGGS; and upgrades to ALIS and the training systems to fully support Block 2B-capable aircraft. - The program maintains a modification and retrofit database that tracks modifications required by each aircraft, production break in of modifications, limitations to the aircraft in performance envelope and service life, requirements for additional inspections until modifications are completed, and operational test requirements and concerns. - The program uses this database to develop and update a complex flow plan of aircraft and engines through depot‑level modifications, modifications completed by deployed depot field teams, and those completed by unit-level maintainers. - The current depot flow plan shows that none of the operational test aircraft would become fully Block 2B-capable by January 2015, and only 7 of 14 will complete the necessary modifications by July 2015, which was the planned start date of the Block 2B OUE. Block 2B modifications would finally be complete on all operational test aircraft in September 2016. • Program Office modification planning for Block 3F IOT&E has begun and shows some of the same scheduling pressures as have been observed for Block 2B; however, these would have been much worse if the OUE were conducted. The depot flow plan includes a seven-month placeholder to complete all modifications to bring each operational test aircraft to a Block 3F configuration, though the span of time required to complete these modifications, including the next increment of structural modifications (SLLP Group 2), is unknown. Additions to modification packages are possible as the potential for discoveries in flight test still exists. Although the program has prioritized for modification the aircraft planned to be used for IOT&E, the Air Force plans for at least 12 F-35A aircraft to be available for IOC declaration in 2016. These Air Force IOC aircraft will be in the Block 3i configuration from production Lot 6 or later, and may require a post-production OBIGGS modification, which could compete for resources with the aircraft scheduled for IOT&E. • Management of the SLLP Group 2 modifications will need to be handled carefully as the program and Services prepare for IOT&E. If the program does not schedule SLLP Group 2 modifications to operational test aircraft until after IOT&E is completed, 495 flight hours must remain before reaching that life limit so aircraft can fully participate in IOT&E, per the approved TEMP. Recommendations • Status of Previous Recommendations. The program and Services have been addressing the redesign and testing of the OBIGGS system, but performance assessment has not yet been completed. The Program Office addressed the vulnerability of the electrical power system to ballistic threats. The remaining recommendations concerning the reinstatement of the PAO shut-off valve, reinstatement of the dry-bay fire extinguisher system, design and reinstatement of fueldraulic shut-off system, improvement of the Integrated Caution and Warning system to provide the pilot with necessary vulnerability information, and a higher resolution estimate of time remaining for controlled flight after a ballistic damage event are all outstanding. • FY14 Recommendations. The program should: 1. Update program schedules to reflect the start of spin-up training for IOT&E to occur no earlier than the operational test readiness review planned for November 2017, and the associated start of IOT&E six months later, in May 2018. 2. The program should complete lab testing of the mission data loads, as is planned in the mission data optimization operational test plan, prior to accomplishing the necessary flight testing to ensure the loads released to the fleet are optimized for performance. If mission data loads are released to operational units prior to the completion of the lab and flight testing required in the operational test plan, the risk to operational units must be clearly documented. 3. The program should complete the remaining three Block 2B weapon delivery accuracy (WDA) flight test events using the currently planned scenarios and ensuring full mission F-35 JSF 71 FY14 DOD PROGRAMS systems functionality is enabled in an operationally realistic manner. 4. The program should require the contractor to conduct rigorous finite-element analyses to assess the benefit of LSP application for the F-35B durability test article and for the F-35B FS496 bulkhead redesign. 5. The program should provide adequate resourcing to support the extensive validation and verification requirements for the Block 3 VSim in time for IOT&E, including the data needed from flight test or other test venues. 72 F-35 JSF 6. To accelerate verification of JTD modules, the program should provide dedicated time on fielded aircraft for F-35B JTD verification teams. 7. Extend the full-up system-level decontamination test to demonstrate the decontamination system effectiveness in a range of operationally realistic environments. 8. The program should ensure adequate testing of ALIS software upgrades on operationally-representative hardware is complete prior to fielding to operational units.