FY16 DOD PROGRAMS

F-35 Joint Strike Fighter
Executive Summary
Test Strategy, Planning, Activity, and Assessment
• The Joint Strike Fighter (JSF) Program Office (JPO)
acknowledged in 2016 that schedule pressure exists for
completing System Development and Demonstration (SDD)
and starting Initial Operational Test and Evaluation (IOT&E)
by August 2017, the planned date in JPO’s Integrated Master
Schedule. In an effort to stay on schedule, JPO plans to
reduce or truncate planned developmental testing (DT) in
an effort to minimize delays and close out SDD as soon as
possible. However, even with this risky, schedule-driven
approach, multiple problems and delays make it clear that
the program will not be able to start IOT&E with full combat
capability until late CY18 or early CY19, at the soonest.
These problems include:
- Continued schedule delays in completing Block 3F
mission systems development and flight testing, which
DOT&E estimates will likely complete in July 2018
- Delayed and incomplete Block 3F DT Weapons Delivery
Accuracy (WDA) events and ongoing weapons integration
issues
- Continued delays in completing flight sciences test points,
particularly those needed to clear the full F-35B Block 3F
flight envelope, resulting in a phased release of Block 3F
envelope across the variants, with the full Block 3F
envelope for F-35B not being released until mid-CY18
- Further delays in completing gun testing for all three
variants and recently discovered gunsight deficiencies
- Late availability of verified, validated and tested Block 3F
Mission Data Loads (MDLs) for planned IOT&E and
aircraft delivery dates; DOT&E estimates the first
validated MDLs will not be available until June 2018
- Continued shortfalls and delays with the Autonomic
Logistics Information System (ALIS) and late delivery of
ALIS version 3.0, the final planned version for SDD, at
risk of slipping from early CY18 into mid-CY18
- Significant, well-documented deficiencies; for hundreds
of these, the program has no plan to adequately fix and
verify with flight test within SDD; although it is common
for programs to have unresolved deficiencies after
development, the program must assess and mitigate the
cumulative effects of these remaining deficiencies on F-35
effectiveness and suitability prior to finalizing and fielding
Block 3F
- Overall ineffective operational performance with multiple
key Block 3F capabilities delivered to date, relative to
planned IOT&E scenarios which are based on various
fielded threat laydowns
- Continued low aircraft availability and no indications
of significant improvement, especially for the early
production lot IOT&E aircraft

- Insufficient progress in verification of Joint Technical
Data, particularly those for troubleshooting aircraft fault
codes and for support equipment
- Delays in completing the required extensive and
time-consuming modifications to the fleet of operational
test aircraft which, if not mitigated with an executable plan
and contract, could significantly delay the start of IOT&E
- Insufficient progress in the following areas which are
required for IOT&E:
▪ Development, integration, and testing of the Air-to-Air
Range Infrastructure instrumentation into the F-35
aircraft
▪ Flight testing to certify the Data Acquisition, Recording,
and Telemetry pod throughout the full flight envelope
▪ Development of other models, including the Fusion
Simulation Model, Virtual Threat Insertion table, and
the Logistics Composite Model
- Delays in providing training simulators in the Block 3F
configuration to the initial training centers and operational
locations
• Based on these ongoing problems and delays, and including
the required time for IOT&E spin-up, the program will not
be ready to start IOT&E until late CY18, at the soonest, or
more likely early CY19. In fact, IOT&E could be delayed
to as late as CY20, depending on the completion of required
modifications to the IOT&E aircraft.
Progress in Developmental Testing
• Mission Systems Testing
- The program continues to pursue a cost- and
schedule-driven plan to delete planned mission systems
DT points by using other test data for meeting test point
objectives in order to accelerate SDD close-out. This
plan, if not properly executed with applicable data,

F-35 JSF

47

 FY16 DOD PROGRAMS
sufficient analytical rigor and statistical confidence, would
shift significant risk to operational test (OT), Follow-on
Modernization (FoM) and the warfighter.
- This risky approach would also discard carefully planned
build-up test content in the Test and Evaluation Master
Plan (TEMP) and the Block 3F Joint Test Plan (JTP),
content the program fully agreed was required when
those documents were signed. The program plans to
“quarantine” JTP build-up test points, which are planned
to be flown by the test centers, and instead skip ahead
to complex graduation-level Mission Effectiveness Risk
Reduction test points, recently devised to quickly sample
full Block 3F performance. Then, if any of the Block 3F
functionality appears to work correctly during the complex
test points, the program would delete the applicable
underlying build-up test points for those capabilities and
designate them as “no longer required.” However, the
program must ensure the substitute data are applicable and
provide sufficient statistical confidence that the test point
objectives had been met prior to deleting any underlying
build-up test points. While this approach may provide a
quick sampling assessment of Block 3F capabilities, there
are substantial risks. The multiple recent software versions
for flight test may prevent the program from using data
from older versions of software to count for baseline test
point deletions because it may no longer be representative
of Block 3F. The limited availability and high cost of
Western Test Range periods, combined with high re-fly
rates for test missions completed on the range, make it
difficult for the program to efficiently conduct this testing.
Finally, the most complex capabilities in Block 3F have
only recently reached the level of maturity to allow them
to be tested, and they are also some of the most difficult
test points to execute (i.e., full Block 3F capabilities and
flight envelope).
- Historical experience indicates this approach, if not
properly executed, may delay problem discoveries and
increase the risk to completing SDD and increase the risk
of failure in IOT&E (as well as, much more importantly, in
combat). In fact, the program needs to allocate additional
test points – which are not in its current plans – for
characterization, root cause investigations, and correction
of a large number of the open high-priority deficiencies
and technical debt described later in this report. The
completion of the planned baseline test points from
the Block 3F JTP, along with correction or mitigation
of significant deficiencies, is necessary to ensure full
Block 3F capabilities are adequately tested and verified
before IOT&E and, more importantly, before they are
fielded for use in combat.
- Until recently, the Program Office estimated that mission
systems flight testing will complete in October 2017. It
now acknowledges the risk that this testing may extend
into early CY18.
▪ The October 2017 estimate was based on an inflated
test point accomplishment rate and optimistically low

48

F-35 JSF

regression and re-fly rates. The estimate also assumed
that the Block 3FR6 software, delivered to flight test
in December 2016, would have the maturity necessary
to complete the remaining test points and meet
specification requirements without requiring additional
versions of software to address shortfalls in capability.
However, this is highly unlikely, since several essential
capabilities – including aimed gunshots and Air-to-Air
Range Infrastructure – had not yet been flight tested
or did not yet work properly when Block 3FR6 was
released.
▪ The Services have designated 276 deficiencies in combat
performance as “critical to correct” in Block 3F, but less
than half of the critical deficiencies were addressed with
attempted corrections in 3FR6.
▪ Independent estimates from other Pentagon staff
agencies vary from March 2018 to July 2018 to
complete mission systems testing – all based on the
current number of test points remaining and actual
historic regression and re-fly rates from the flight test
program. Even these estimates are optimistic in that
they account for only currently planned testing, which
does not yet include the activities needed to correct the
Services’ remaining high-priority deficiencies.
• Flight sciences testing continues to be a source of significant
discovery, another indication that the program is not nearing
completion of development and readiness for IOT&E. For
example:
- Fatigue and migration of the attachment bushing in the
joint between the vertical tail and the aircraft structure are
occurring much earlier than planned in both the F-35A
and F-35B, even with a newly designed joint developed to
address shortfalls in the original design.
- Excessive and premature wear on the hook point of the
arresting gear on the F-35A, occuring as soon as after only
one use, has caused the program to consider developing a
more robust redesign.
- Higher than predicted air flow temperatures were measured
in the engine nacelle bay during flight testing in portions
of the flight envelope under high dynamic pressure on both
the F-35A and F-35C; thermal stress analyses are required
to determine if airspeed restrictions will be needed in this
portion of the flight envelope.
- Overheating of the horizontal tail continued to cause
damage, as was experienced on BF-3, one of the
F-35B flight sciences test aircraft, while accelerating in
afterburner to Mach 1.5 for a loads test point. The left
horizontal inboard fairing surface reached temperatures
that exceeded the design limit by a significant amount.
Post-flight inspections revealed de-bonding due to heat
damage on the trailing edge of the horizontal tail surface
and on the horizontal tail rear spar.
- Vertical oscillations during F-35C catapult launches were
reported by pilots as excessive, violent, and therefore a
safety concern during this critical phase of flight. The
program is still investigating alternatives to address this

 FY16 DOD PROGRAMS
deficiency, which makes a solution in time for IOT&E and
Navy fielding unlikely.
Mission Data Load Development and Testing
• Mission data files, which comprise MDLs, are essential to
enable F-35 mission systems to function properly. Block 3F
upgrades to the U.S. Reprogramming Laboratory (USRL) –
where mission data files are developed, tested and validated
for operational use – are late to meet the needs for Block
3F production aircraft and IOT&E. These upgrades to the
Block 3F configuration, including the associated mission
data file generation tools, are necessary to enable the USRL
to begin Block 3F mission data file development. In spite
of the importance of the mission data to both IOT&E and
to combat, the Program Office and Lockheed Martin have
failed to manage, contract, and deliver the necessary USRL
upgrades to the point that fully validated Block 3F MDLs
will not be ready for IOT&E until June 2018, at the earliest.
• Operational units are also affected by the capability shortfalls
in the USRL to create, test and field MDLs. The complete
set of Block 2B and Block 3i MDLs developed for overseas
areas of responsibility (AORs) have yet to undergo the full
set of lab and flight tests necessary to validate and verify
these MDLs for operational use. Because of the delays
in upgrading the USRL to the Block 3F configuration, the
Services will likely not have Block 3F MDLs for overseas
AORs until late 2018 or early 2019.
• In addition to the late Block 3F USRL upgrades, the required
signal generators for the USRL – with more high-fidelity
channels to simulate modern fielded threats – have not yet
been placed on contract. As a result, the Block 3F MDLs
will not be tested and optimized to ensure the F-35 will
be capable of detecting, locating, and identifying modern
fielded threats until 2020, per a recent program schedule.
The program is developing multiple laboratories in order
to produce MDLs tailored for partner nation-unique
requirements, some of which will have more high-fidelity
signal generator channels earlier than the USRL. The
program is considering using one of these other laboratories
for Block 3F MDL development and testing; however,
the MDL that will be used for IOT&E must be developed,
verified, validated, and tested using operationally
representative procedures, like the MDLs that will be
developed for the operational aircraft in the USRL.
Weapons Integration and Demonstration Events
• Block 3F weapons delivery accuracy (WDA) events are
not complete. These events, required by the TEMP, are key
developmental test activities necessary to ensure the full
fire-control capabilities support the “find, fix, track, target,
engage, assess” kill chain. As of the end of November,
only 5 of the 26 events (excluding the gun events) had been
completed and fully analyzed. Several WDAs have revealed
deficiencies and limitations to weapons employment (e.g.,
AIM-9X seeker status tone problems and out-of-date launch
zones for AIM-120 missiles). An additional 11 WDAs had
occurred, but analyses were ongoing. Of the 10 remaining

WDAs that had not been completed, 4 were still blocked
due to open deficiencies that must be corrected before the
WDA can be attempted. However, the program did not
have time to fix the deficiencies, complete the remaining
WDAs and analyze them before finalizing Block 3FR6 in
late November for flight testing to begin in December 2016.
For example, recent F-35C flight testing to prepare for a
weapons event with the C-1 version of the Joint Stand-Off
Weapon (JSOW-C1) discovered weapon integration,
Pilot Vehicle Interface (PVI) and mission planning problems
that will prevent full Block 3F combat capability from
being delivered, if not corrected. These discoveries were
made too late to be included in the Block 3FR6 software,
the final planned increment of capability delivered to flight
test for SDD. Also, multiple changes are being made late
in Block 3F development to mission systems fire control
software to correct problems with the British AIM-132
Advanced Short-Range Air-to-Air Missile (ASRAAM)
missile and Paveway IV bomb, changes which could affect
the U.S. AIM-9X air-to-air missile and GBU-31 laser-guided
bomb capabilities, and may require regression testing of the
U.S. weapons.
• Block 3F adds gun capability for all variants. The F-35A
gun is internal; the F-35B and F-35C each use a gun pod.
Ground firing tests have been completed on all variants;
only on the F-35A has initial flight testing of the gun been
accomplished. Early testing of the air-to-ground and
air-to-air symbology have led to discovery of deficiencies in
the gunsight and strafing symbology displayed in the pilot’s
helmet – deficiencies which may need to be addressed before
accuracy testing of the gun, aimed by the HMDS, can be
completed. Because of the late testing of the gun and the
likelihood of additional discoveries, the program’s ability
to deliver gun capability with Block 3F before IOT&E is at
risk, especially for the F-35B and F-35C.
Pilot Escape System
• The program completed pilot escape system qualification
testing in September 2016, which included a set of
modifications designed to reduce risk to pilots weighing less
than 136 pounds.
- Modifications include:
▪ Reduction in the weight of the pilot’s Generation III
Helmet Mounted Display System (HMDS), referred to
as the Gen III Lite HMDS
▪ Installation of a switch on the ejection seat which
allows lighter-weight pilots to select a slight delay in the
activation of the main parachute
▪ Addition of a Head Support Panel (HSP) between the
risers of the parachute.
- These modifications to the pilot escape system were
needed after testing in CY15 showed that the risk of
serious injury or death is greater for lighter-weight pilots.
Because of the risk, the Services decided to restrict pilots
weighing less than 136 pounds from flying the F-35.

F-35 JSF

49

 FY16 DOD PROGRAMS
• Twenty-two qualification test cases were completed
between October 2015 and September 2016, with variations
in manikin weight, speed, altitude, helmet size and
configuration, and seat switch setting. Data from tests
showed that the HSP significantly reduced neck loads
under conditions that forced the head backwards, inducing
a rearward neck rotation, during the ejection sequence.
Data also showed that the seat switch reduced the “opening
shock” by slightly delaying the main parachute for lighterweight pilots at speeds greater than 160 knots. The extent
to which the risk has been reduced for lighter-weight pilots
(i.e., less than 136 pounds) by the modifications to the
escape system and helmet is still to be determined by a
safety analysis of the test data. If the Services accept the
risk associated with the modifications to the escape system
for the lighter-weight pilots, restrictions will likely remain in
effect until aircraft have the modified seat and the HSPs, and
until the lighter-weight Gen III Lite helmets are procured and
delivered to the applicable pilots.
• Based on schedules for planned seat modifications,
production cut-in of the modified seat, and the planned
delivery of the Gen III Lite HMDS, the Air Force may be
able to reopen F-35 pilot training to lighter-weight pilots
(i.e., below 136 pounds) in early 2018. DOT&E is not aware
of the plans for the Marine Corps and the U.S. Navy to open
F-35 pilot training to the lighter-weight pilots.
• Part of the weight reduction to the Gen III Lite HMDS
involved removing one of the two installed visors (one
dark, one clear). As a result, pilots that will need to use
both visors during a mission (e.g., during transitions from
daytime to nighttime) will have to store the second visor in
the cockpit. However, there currently is not enough storage
space in the cockpit for the spare visor, so the program is
working a solution to address this problem.
• The program has yet to complete the additional testing and
analysis needed to determine the risk of pilots being harmed
by the Transparency Removal System (which shatters the
canopy first, allowing the seat and pilot to leave the aircraft)
during off-nominal ejections in other than ideal, stable
conditions (such as after battle damage or during out-ofcontrol situations). Although the program completed an
off-nominal rocket sled test with the Transparency Removal
System in CY12, several aspects of the escape system have
changed since then (including significant changes to the
helmet) which warrant additional testing and analyses.
Joint Simulation Environment (JSE)
• JSE is a man-in-the-loop, F-35 mission systems software-inthe-loop simulation being developed to meet the operational
test requirements for Block 3F IOT&E. However, multiple
aspects of the JSE development effort continue to fall
significantly behind schedule. The Program Office has been
negotiating with the contractor to receive the F-35 aircraft
and sensor models, referred to as “F-35 In A Box (IAB),”
but very limited progress was made in CY16. Also, delays
with security clearances for new personnel limited progress

50

F-35 JSF

on several aspects of the development and validation effort.
Although the Naval Air Systems Command (NAVAIR)
government team has begun installing hardware on their
planned timeline (facilities, cockpits, etc.), the team’s
progress in integrating the many different models (i.e.,
multi-spectral environment, threats, weapons) with F-35 IAB
has been severely limited, and the verification, validation and
accreditation of these models within JSE for use in IOT&E,
have effectively stalled. The F-35 program’s JSE schedule
indicates that it plans to provide a fully accredited simulation
for IOT&E use in May 2019; a schedule that carries high
risk of further slips without resolving these issues, and is
not credible. Without a high-fidelity simulation, the F-35
IOT&E will not be able to test the F-35’s full capabilities
against the full range of required threats and scenarios.
However, for the reasons above, it is now clear that the JSE
will not be available and accredited in time to support the
Block 3F IOT&E. Therefore, the recently approved IOT&E
detailed test design assumes only open-air flight testing will
be possible and attempts to mitigate the lack of an adequate
simulation environment as much as possible. In the unlikely
event the JSE is ready and accredited in time for IOT&E, the
test design has JSE scenarios that would be conducted.
Live Fire Test and Evaluation (LFT&E)
• The F-35 LFT&E program completed one major live fire test
series using an F-35C variant full-scale structural test article
(CG:0001). Preliminary test data analyses:
- Demonstrated the tolerance of the vertical tail attachments
to high-explosive incendiary (HEI) projectile threats
- Confirmed the tolerance of the aft boom structures to
Man-Portable Air Defense System (MANPADS) threats
- Demonstrated vulnerabilities to MANPADS-generated
fires in engine systems and aft fuel tanks. The data
will support a detailed assessment in 2017 of these
contributions to overall F-35 vulnerability.
• The test plan to assess chemical and biological
decontamination of pilot protective equipment is not
adequate; no plans have been made to test either the Gen II
or the Gen III HMDS. The Program Office is on track
to evaluate the chemical and biological agent protection
and decontamination systems in the full-up system-level
decontamination testing in FY17.
• The Navy conducted vulnerability testing of the F-35B
electrical and mission systems to electromagnetic pulses
(EMP).
• The 780th Test Squadron at Eglin AFB, Florida completed
ground-based lethality tests of the PGU-47/U Armor
Piercing High Explosive Incendiary with Tracer (APHEI-T)
round, also known as the Armor Piercing with Explosive
(APEX), against armored and technical vehicles, aircraft, and
personnel-in-the-open targets.
Suitability
• The operational suitability of all variants continues to be
less than desired by the Services. Operational and training

 FY16 DOD PROGRAMS
units must rely on contractor support and workarounds that
would be challenging to employ during combat operations.
In the past year some metrics of suitability performance have
shown improvement, while others have been flat or declined.
- Most metrics still remain below interim goals to achieve
acceptable suitability by the time the fleet accrues 200,000
flight hours, the benchmark set by the program and defined
in the Operational Requirements Document (ORD)
for the aircraft to meet reliability and maintainability
requirements.
- Reliability growth has stagnated and, as a result, it is
highly unlikely that the program will achieve the ORD
threshold requirements at maturity for the majority
of reliability metrics, most notably Mean Flight
Hours Between Critical Failures, without redesigning
components of the aircraft.
Autonomic Logistics Information System
• The program failed to release any new ALIS capability
in 2016, but did release two updates to the currently fielded
ALIS 2.0.1 software to address deficiencies and usability
shortfalls. The program planned to test and field ALIS 2.0.2,
including integration of propulsion data management, in
the summer of 2016, to support the Air Force declaration
of Initial Operational Capability; however, delays in
development and integration have pushed the testing and
fielding into 2017.
• Because of the delays with ALIS 2.0.2, Lockheed Martin
shifted personnel to support that product line development.
This caused delays in the development schedule of ALIS 3.0,
the last major SDD software release. The program
acknowledged in August 2016 that it could not execute the
ALIS 3.0 schedule and developed plans to restructure this
ALIS release and the remaining planned ALIS capabilities
into multiple releases, including some that will occur after
SDD completion.
- The program’s restructuring of the ALIS capability
delivery plan divided the planned capabilities and security
updates for ALIS into four more versions: one version for
SDD (ALIS 3.0), with what the Program Office considered
to be needed for IOT&E, and three additional software
releases intended to be fielded at 6-month intervals after
SDD completion, with the remaining content originally
planned for ALIS 3.0.
- The program plans to release software maintenance
updates midway between each of these four software
releases to address deficiencies and usability problems, but
these releases will not include new capabilities.
• The Air Force completed its first deployment of F-35A
aircraft using the modularized version of the ALIS squadron
hardware, called the Standard Operating Unit Version 2
(SOU v2), and software release 2.0.1 to Mountain Home
AFB, Idaho in February 2016. Difficulties integrating the
SOU v2 into the base network interfered with connectivity
between the SOU v2 and the Mountain Home-provided
workstations, but did not affect connectivity of the SOU v2

with the main Autonomic Logistics Operating Unit (ALOU)
in Fort Worth, Texas.
Air-Ship Integration and Ship Suitability
• The program completed the last two ship integration DT
periods in 2016 – both referred to as “DT-III” – one with
the F-35B in November aboard the amphibious assault ship
USS America, and one with the F-35C in August aboard the
aircraft carrier USS George Washington. Test objectives
included expanding the flight clearances for shipboard
operations with carriage of external weapons, night
operations, and Joint Precision Approach Landing System
(JPALS) integration testing. For both periods, operational
and test units accompanied the deployment to develop
concepts of operations for at-sea periods.
• The specialized secure space set aside for F-35-specific
mission planning and the required Offboard Mission
Support (OMS) workstations is likely unsuitable for regular
Air Combat Element (ACE) operations on the Landing
Helicopter Dock (LHD) and Landing Helicopter Assault
(LHA)-class assault ships with the standard complement
of six F-35B aircraft, let alone F-35B Heavy ACE
configurations with more aircraft. Similarly, for F-35C
operations onboard CVN, adequate secure spaces will be
needed to ensure planning and debriefing timelines support
carrier operations.
• The F-35C DT-III included external stores, including bombs,
but only pylons with no AIM-9X missiles on the outboard
stations (stations 1 and 11) due to the F-35C wingtip
structural deficiency. The U.S. Navy directed a proof-ofconcept demonstration of an F-35C engine change while
underway, a process that took several days to complete.
ALIS was not installed on USS George Washington, so
reach-back via satellite link to the shore-based ALIS unit was
required, similar to previous F-35C test periods at sea, but
connectivity proved troublesome.
• The F-35B DT-III deployment included an engine
installation due to required maintenance, along with a lift fan
change proof-of-concept demonstration. The Marine Corps
deployed with an operational SOU v2 on USS America.
Cybersecurity Testing
• The JSF Operational Test Team (JOTT) continued to conduct
cybersecurity testing on F-35 systems, in partnership with
certified cybersecurity test organizations and personnel, and
in accordance with the cybersecurity strategy approved by
DOT&E in February 2015. In 2016, the JOTT conducted
adversarial assessments (AA) of the ALIS 2.0.1 SOU, also
known as the Squadron Kit, at Marine Corps Air Station
(MCAS) Yuma, Arizona, and the Central Point of Entry
(CPE) at Eglin AFB, Florida, completing testing that began
in the Fall of 2015. They also completed cooperative
vulnerability and penetration assessments (CVPA) of the
mission systems ALOU at Edwards AFB, California, used
to support developmental testing, and the operational ALOU
in Fort Worth, Texas. The JOTT, with support from the

F-35 JSF

51

 FY16 DOD PROGRAMS
Air Force Research Laboratory (AFRL) also completed a
limited cybersecurity assessment of the F-35 air vehicle
in September 2016, on an F-35A aircraft assigned to the
operational test squadron at Edwards AFB. These tests were
not conducted concurrently as originally planned, so end-toend testing of ALIS, from the ALOU to the air vehicle, has
not yet been accomplished. An AA of the operational ALOU
was scheduled for early December 2016, which would
complete a full assessment (CVPA and AA) of each ALIS
2.0.1 component.
• The cybersecurity testing in 2016 showed that the program
has addressed some of the vulnerabilities identified during
earlier testing periods; however, much more testing is needed
to assess the cybersecurity structure of the air vehicle and
supporting logistics infrastructure system (i.e., ALOU, CPE,
Squadron Kit) and to determine whether, and to what extent,
vulnerabilities may have led to compromises of F-35 data.
The scope of the cybersecurity testing must also expand to
include other systems required to support the fielded aircraft,
including the Multifunction Analyzer Transmitter Receiver
Interface Exerciser (MATRIX) system which is used by
contractor maintenance technicians, the USRL, avionics
integration labs, the OMS and training simulators.
Follow-on Modernization
• The program continued making plans for Follow-on
Modernization (FoM) for all variants, also referred to as
Block 4, which is on DOT&E oversight. The program
intends to award the contract for the modernization effort
in 2QCY18 with developmental flight testing beginning
in 3QCY19. Four increments of capability are planned,
Blocks 4.1 through 4.4. Blocks 4.1 and 4.3 will provide
software-only updates; Blocks 4.2 and 4.4 will include
significant avionics hardware changes as well as software
updates. Improved Technical Refresh 3 (TR3) processors
with open architecture, designed to make adding, upgrading
and replacing components easier, are planned to be added in
Block 4.2.
• The program’s plans for FoM are not executable for
a number of reasons including, but not limited to the
following:
- Too much technical content for the production-scheduledriven developmental timeline
- Overlapping increments without enough time for
corrections to deficiencies from OT to be included in the
next increment
- High risk due to excessive technical debt and deficiencies
from the balance of SDD and IOT&E being carried
forward into FoM because the program does not have a
plan or funding to resolve key deficiencies from SDD prior
to attempting to add the planned Block 4.1 capabilities
- Inadequate test infrastructure (aircraft, laboratories,
personnel) to meet the testing demands of the capabilities

52

F-35 JSF

planned and the multiple configurations (i.e., TR2, TR3,
and Foreign Military Sales)
- Insufficient resources for conducting realistic operational
testing of each increment
System
• The F-35 Joint Strike Fighter (JSF) program is a tri-Service,
multi-national, single-seat, single-engine family of strike
aircraft consisting of three variants:
- F-35A Conventional Take-Off and Landing (CTOL)
- F-35B Short Take-Off/Vertical-Landing (STOVL)
- F-35C Aircraft Carrier Variant (CV).
• The F-35 is designed to survive in an advanced threat
environment (year 2015 and beyond) using numerous
advanced capabilities. It is also designed to have improved
lethality in this environment compared to legacy multi-role
aircraft.
• Using an active electronically scanned array (AESA) radar and
other sensors, the F-35 with Block 3F is intended to employ
precision-guided weapons, such as the GBU-12 Laser-Guided
Bomb (LGB), GBU-31/32 Joint Direct Attack Munition
(JDAM), GBU-39 Small Diameter Bomb (SDB), Navy Joint
Stand-Off Weapon (JSOW)-C1, and air-to-air missiles such
as AIM-120C Advanced Medium-Range Air-to-Air Missile
(AMRAAM), and AIM-9X infrared guided short-range
air-to-air missile.
• The SDD program was designed to provide mission capability
in three increments:
- Block 1 (initial training; two increments were fielded:
Blocks 1A and 1B)
- Block 2 (advanced training in Block 2A and limited combat
capability in Block 2B)
- Block 3 (limited combat capability in Block 3i and full
SDD warfighting capability in Block 3F)
• The F-35 is under development by a partnership of countries:
the United States, Great Britain, Italy, the Netherlands, Turkey,
Canada, Australia, Denmark, and Norway.
Mission
• The Combatant Commander will employ units equipped with
F-35 aircraft in joint operations to attack targets during day or
night, in all weather conditions, and in heavily defended areas.
• The F-35 will be used to attack fixed and mobile land targets,
surface units at sea, and air threats, including advanced aircraft
and cruise missiles.
Major Contractor
Lockheed Martin, Aeronautics Division – Fort Worth, Texas

 FY16 DOD PROGRAMS
Test Strategy, Planning, and Resourcing
• Preparations for IOT&E. In 2016, the JPO acknowledged
schedule pressure for starting IOT&E in August 2017, as
planned in the Integrated Master Schedule created in 2012.
Due to multiple problems and further delays, the program will
not be able to start IOT&E until late CY18, at the earliest, and
more likely early CY19, but it could be as late as CY20 before
required modifications are completed to IOT&E aircraft. The
issues that will not allow IOT&E to start as planned include:
- Continued schedule delays in completing Block 3F mission
systems development and flight testing
▪ The program’s plan to deliver the “Full SDD Warfighting
Capability” version of Block 3F software – now referred
to as version 3FR6 – was significantly delayed. It was
planned for release to flight test in February 2016,
according to the program’s latest mission systems
software and capability release schedule, but did not
begin flight test until early December 2016 (10 months
late). However, during this time, the program released
several “Quick Reaction Cycle” (QRC) versions of
software to quickly resolve deficiencies that were
preventing the completion of key test points, like
weapons deliveries. Due to these delays, along with the
recently acknowledged SDD funding shortfall, software
versions 3FR7 and 3FR8 have fallen off the program’s
schedule. However, ongoing delays in maturing some
of the capabilities and new problem discoveries continue
to prevent testing of some planned Block 3F capabilities
and will almost certainly require additional unplanned
releases of Block 3F software.
▪ DOT&E estimates that mission systems flight testing will
not complete prior to July 2018, based on the number of
Block 3F baseline mission systems test points to go, the
monthly average mission systems test point completion
rate observed for CY16 to date, and the average
regression, discovery and developmental test point rate
of 63 percent experienced so far in CY16. This estimate
also includes a decrement of 11 percent for test points to
be designated “no longer required,” the percentage used
by the Program Office to account for efficiency in CY16
planning of test point accomplishment objectives.
- Delayed and incomplete Block 3F developmental testing
Weapons Delivery Accuracy (WDA) events and ongoing
weapons integration issues
▪ WDA events – key developmental test activities
necessary to ensure the full fire-control capabilities
work together to properly support the “find, fix, track,
target, engage, assess” kill chain – are not complete. As
of the end of November, only 5 of the 26 WDA events
(excluding gun events) had been completed and fully
analyzed.
▪ Several WDAs have revealed deficiencies and limitations
to weapons employment (e.g., AIM-9X seeker status
tone problems and out-of-date launch zones for AIM-120
missiles). An additional 11 WDAs had occurred, but
analyses are ongoing. Of the 10 remaining WDAs,

4 were still blocked due to open deficiencies that must
be corrected before the WDA can be attempted, but the
program did not have time to complete and analyze them
before finalizing Block 3FR6.
- Continued delays in completing flight sciences test points,
particularly those needed to provide the F-35B Block 3F
flight envelope for operational use
▪ Through the end of November, flight sciences testing on
all variants was behind the plan for the year. Although
the program planned to complete Block 3F testing on
the F-35A in October, testing continued into December,
with weapons separations and regression testing of new
software to be completed.
▪ Flight sciences test point completion for CY16 was
5 percent behind for the F-35B and 23 percent behind
for the F-35C as of the end of November. The program
plans to complete Block 3F flight sciences testing
in August 2017 with the F-35C and by the end of
October 2017 with the F-35B, the latter being 10 months
later than planned in the program’s Integrated Master
Schedule.
▪ Due to the delays with completing flight sciences testing,
the program plans a phased release of the Block 3F
envelope across all three variants, with the full Block 3F
envelope for the F-35B not being released until midCY18.
- Further delays in completing gun testing for all three
variants and recently discovered gunsight deficiencies
▪ Block 3F adds gun capability for all three variants. The
F-35A gun is internal; the F-35B and F-35C each use a
gun pod. Differences in mounting make the gun pods
unique to a specific variant, i.e., a gun pod designated
for an F-35B cannot be mounted on an F-35C aircraft.
Flight sciences testing of the gun has occurred with the
F-35A; discoveries required control law changes to the
flight control software and delayed the start of mission
systems gun testing on the F-35A from September 2016
to December 2016. Although the F-35B and F-35C have
completed ground firings of their gun pods, airborne
flight sciences gun testing (i.e., airborne firing) for the
F-35B and F-35C has yet to be accomplished.
▪ Besides the ongoing delays with software and gun
modifications, both DT and OT pilots have reported
concerns from preliminary test flights that the air-toground gun strafing symbology, displayed in the helmet,
is currently operationally unusable and potentially unsafe
to complete the planned testing due to a combination of
symbol clutter obscuring the target, difficulty reading
key information, and pipper stability. Also, for air-to-air
employment, the pipper symbology is very unstable
while tracking a target aircraft; however, the funnel
version of the air-to-air gunsight appears to be more
stable in early testing.
▪ Fixing these deficiencies may require changes to the
mission systems software that controls symbology

F-35 JSF

53

 FY16 DOD PROGRAMS
to the helmet, or the radar software, even though the
program recently released the final planned version of
flight test software, Block 3FR6. Plans to begin flight
testing of aimed gunshots, integrated with mission
systems, which requires aiming with the helmet, on the
F-35A were planned for fall of 2016, but had slipped to
December 2016, at the soonest, before this new problem
with the gun symbology was discovered.
▪ F-35B ground test firing of its gun pod was accomplished
in July 2016 and flight testing is planned to begin in
January 2017; the F-35C conducted first ground firing
in November 2016; flight testing is planned to begin in
March 2017.
- Late availability of verified, validated and tested Block 3F
MDLs
▪ Failure by the program to plan for, procure, and provide
the necessary Block 3F upgrades and the associated
Mission Data File Generation (MDFG) tools to the USRL
has caused delays in developing, testing, and verifying
mission data loads for IOT&E.
▪ If Block 3F MDFG tools are delivered in early CY17,
verified, validated and tested MDLs will not be available
for IOT&E until June 2018 (15 months later) at the
soonest, which is late to need for both IOT&E and
fielding of Block 3F.
▪ In collaboration with partner nations, the program is
developing multiple laboratories to produce MDLs
tailored for country-unique requirements. Although
these other laboratories may provide additional capacity
for developing and testing MDLs, the MDL that will be
used for IOT&E must be developed, verified, validated,
and tested using operationally representative procedures
involving the USRL.
- Continued shortfalls and delays with ALIS and late
delivery of ALIS software version 3.0, the final planned
version for SDD, which is at risk of slipping from
early-CY18 into mid-CY18
▪ The program has failed to deliver increments of ALIS
capability as planned. No new capability has completed
testing in 2016, although the program had planned to
field ALIS 2.0.2, with the propulsion integration module
included, by August 2016 to support the Air Force IOC
declaration, but continued problems caused this to slip
into early CY17.
▪ The program restructured the ALIS capabilities delivery
plan in 2016 and moved content planned for ALIS
3.0 – the last version to be developed during SDD – to
post-SDD ALIS development and fielding. Despite the
delays and deferred content, IOT&E will still evaluate
the suitability of the F-35 with ALIS in operationally
realistic conditions.
- Significant, well-documented deficiencies resulting in
overall ineffective operational performance of Block 3F,
hundreds of which will not be adequately addressed with
fixes and corrections verified with flight testing within
SDD

54

F-35 JSF

▪ The program, Services, JOTT, and DT and OT pilots
recently conducted a review of the status and priority
of open deficiency reports (DRs). This review was a
follow-on from a review in the spring of 2016, where
the stakeholders reviewed all the open DRs and created
a rank-ordered list of 263 priority deficiencies to be
addressed by the program. The review team later pared
the list down to 176 priority DRs, with 12 being brought
forward to the JPO’s Configuration Steering Board
(CSB); 7 for decision and 5 for CSB awareness. In the
review in the fall of 2016, the stakeholders reviewed
the approximately 1,200 open deficiencies, including
the original 176 priority DRs, plus 231 new DRs since
Feb 2016, minus 55 that had been corrected, to create an
updated DR list. This time, however, the team prioritized
the open DRs into one of 4 priorities: priority 1 DRs
are “service critical,” and the Services will not field
the aircraft unless these DRs are fixed; priority 2 have
significant impact that may, when combined with other
DRs, lead to mission failure; priority 3 carry medium
impact and should be addressed by the program, but
maybe not within SDD; and priority 4 have low impact.
The review team identified 72 DRs as priority 1 and
204 DRs as priority 2, for a total of 276 DRs to address
within SDD or risk fielding deficiencies that could lead to
operational mission failures during IOT&E or combat.
▪ While these deficiencies must be addressed to some
degree during the remaining time in development,
the final planned software load, Block 3FR6, which
started flight test in December 2016, only included
attempted fixes for less than half of the 276 priority
1 and 2 DRs. Corrections to these deficiencies will
need to be developed, tested in the labs (if possible)
and then flight tested, since the labs have proven to not
be an adequate test venue for verifying corrections to
deficiencies identified during flight testing. However,
the current schedule-driven program plans to close out
SDD testing in 2017 do not include enough time to fix
these key deficiencies, nor time to verify corrections in
flight test. There is risk in attempting to verify DR fixes
only in the lab because the labs proved to not always
be representative of the actual aircraft for detecting
problems or verifying fixes for stability problems. The
labs are also not able to adequately replicate the demands
on the mission systems like open air testing does, such as
infrared and radar background clutter and terrain-driven
multipath reflections of radio-frequency emissions from
threat emitters, so most fixes to deficiencies will require
flight testing.
- Overall ineffective operational performance with multiple
key Block 3F capabilities to date
▪ Three independent assessments conducted during the
past 6 months rate the F-35 as red or unacceptable
(not all assessments used the same scoring criteria) in
most critical combat mission areas: The Air Force’s
IOC Readiness Assessment (IRA) of Block 3i, an OT

 FY16 DOD PROGRAMS
community assessment of Block 3FR5.03 based on
observing develomental testing, and an assessment by the
JOTT of the capability of Block 3FR5.05 to perform the
planned mission trials in the IOT&E, based on observing
and assisting with DT.
▪ In July, the Air Force completed their IRA report. The
assessment was based on a limited series of events
conducted with six Block 3i-configured aircraft,
including test missions in Close Air Support (CAS), Air
Interdiction (AI), and Suppression/Destruction of Enemy
Air Defenses (SEAD/DEAD). The assessment noted
unacceptable problems in fusion and electronic warfare
and, concerning the CAS mission, determined that the
Block 3i F-35A does not yet demonstrate equivalent CAS
capabilities to those of fourth generation aircraft.
▪ In August, an F-35 OT pilot from Edwards AFB,
California, briefed the results of an OT community
assessment of F-35 mission capability with Block
3FR5.03, based on observing developmental flight
test missions and results to date. This OT assessment
rated all IOT&E mission areas as “red,” including
CAS, SEAD/DEAD, Offensive Counter Air (OCA)
and Defensive Counter Air (DCA), AI, and Surface
Warfare (SuW). Several DT Integrated Product Team
representatives also briefed the status of different F-35
mission systems capabilities, most of which were rated
“red,” and not meeting the entrance criteria to enter
the “graduation level” mission effectiveness testing.
Trend items from both the OT and IPT briefings were
limitations and problems with multiple Block 3F system
modes and capabilities, including Electro-Optical
Targeting System (EOTS), Distributed Aperture System
(DAS), radar, electronic warfare, avionics fusion,
identification capabilities, navigation accuracy, GPS,
datalinks, weapons integration and mission planning.
▪ In November 2016, the JOTT provided an assessment of
a later version of Block 3F software – version 3FR5.05
– based on observing and assisting with F-35 DT flight
operations and maintenance. The JOTT assessment
made top-level, initial predictions of expected IOT&E
results of the F-35 with Block 3FR5.05 against planned
scenarios and realistic threats. For mission effectiveness,
the assessment predicted severe or substantial operational
impacts across all the planned IOT&E missions (similar
to the list of missions above) due to observed shortfalls
in capabilities, with the exception of the Reconnaissance
mission area, which predicted minimal operational
impact. Unlike the other assessments, the JOTT also
assessed suitability, predicting mixed operational impacts
due to shortfalls for deployability (from minimal to
severe), severe impacts for mission generation, and
substantial impacts for training and logistics support.
- Continued low aircraft availability, especially for the early
production lot IOT&E aircraft. The program has still
not been able to improve aircraft availability, in spite of
reliability and maintainability initiatives, to the goal of
60 percent, which is well short of the 80 percent necessary

-

-

-

-

to conduct an efficient IOT&E and to support sustained
combat operations. As a result, IOT&E will likely take
longer than currently planned and suitability, along with
fielded operations, will be adversely affected.
Late delivery of the JSE, a man-in-the-loop simulator
expected for IOT&E, which required the test team to create
a test design that attempts to mitigate the high likelihood
that it will not be available. Some IOT&E measures of
effectiveness will not be fully resolved without a verified,
validated and accredited simulator to evaluate the F-35 in an
operationally realistic, dense threat environment.
Progress in verification of Joint Technical Data (JTD) is
behind plans to complete within SDD, particularly those
for troubleshooting aircraft fault codes and for support
equipment. As of September 2016, the program had verified
approximately 83 percent of all JTD modules, but just over
50 percent of those associated with support equipment.
While symptomatic of an immature system, the lack of
verified JTD makes the completion of aircraft maintenance
more difficult and forces maintainers to rely more heavily
on submitting electronic requests to the contractor for help
or to seek assistance from contractor representatives at field
locations.
▪ The program has made significant progress in verifying
JTD for sustaining the aircraft’s low observable signature,
primarily by completing verifications on an F-35A
damaged in 2014 by an engine fire
▪ All Block 3F JTD must be written and verified prior to the
start of IOT&E
Delays in completing the extensive and time-consuming
modifications required to the fleet of operational test aircraft
which, if not mitigated with an executable plan and contract,
could significantly delay the start of IOT&E.
▪ The program is developing and working plans with
Lockheed Martin and the Services to provide productionrepresentative operational test aircraft, with the necessary
instrumentation, to start IOT&E. Although it was part of
the agreed-to entrance criteria for IOT&E, the program
currently does not have an adequate plan to provide test
aircraft that meet the TEMP criteria for entering IOT&E
until late-2018, at the earliest, and possibly as late as 2020.
Extensive modifications are required on all of the TEMPdesignated OT aircraft; 155 different modifications (known
to date) are necessary between all variants and all lots of
aircraft (Lots 3 through 5) to bring the IOT&E aircraft
to the required production-representative configuration,
although no single aircraft requires all 155 modifications.
Additional discoveries and modifications are likely as the
program finishes SDD.
▪ The Program Office and the Services are considering using
later lot aircraft with an alternate instrumentation package.
However, to date, no analyses of the adequacy of the
alternate instrumentation has been completed; nor is there
a contract to design, build and test alternative packages.
Insufficient progress in the development and testing of
modeling, simulations, and instrumentation required for
IOT&E.

F-35 JSF

55

 FY16 DOD PROGRAMS
▪ Flight testing to allow the Data Acquisition Recording
and Telemetry (DART) pod to be used throughout the
full Block 3F flight envelope during IOT&E, including
during simulated weapons releases when the weapons
bay doors will cycle open, has not yet been planned, put
on contract or completed. The DART pod is required for
collecting data during IOT&E.
▪ Flight testing of the Air-to-Air Range Infrastructure
(AARI) – as integrated with the F-35 and required for
adequacy of the open air flight test trials – has not yet
been completed. AARI is used to support battle-shaping
of air-to-air engagements by modeling weapon fly-outs
and accounting for endgame effects to remove aircraft
“shot down” by another aircraft or ground threat.
The program must begin testing AARI and allow for
corrections of deficiencies during flight testing, to ensure
AARI is adequate for IOT&E.
▪ Integration of AARI and associated range simulators
with the F-35 to indicate inbound missiles on cockpit
displays is required for an adequate evaluation of
open air missions. Within the aircraft, the Embedded
Training (ET) function is intended to support live/virtual/
constructive training using a mixture of real and virtual
entities (e.g., missiles, ground systems, and aircraft). To
avoid intermingling data from real and virtual entities,
as it may cause issues within the F-35, the contractor
developed a separate model, the Fusion Simulation
Model (FSM), to emulate fusion functionality for virtual
entities within ET. The current FSM implementation has
significant deficiencies that make the model so inaccurate
that some required capabilities may not be usable for
IOT&E. Although a properly functioning FSM is
required for IOT&E, the program had not yet completed
contract actions for fixes to correct the FSM deficiencies
within SDD and prior to IOT&E, but was apparently
developing plans and intended to award contract actions
for at least some of the work on FSM by the end of
January 2017.
▪ Virtual Threat Insertion (VTI) is a function inside of FSM
that correlates virtual threat parametric data supplied by
AARI with data from tables embedded within the FSM
to provide cockpit display indications to the pilot for
threat activity (i.e., a surface-to-air missile launched).
The reference tables for VTI are incomplete and do
not include all threats planned for use in IOT&E. The
program was also apparently planning to update the VTI
tables, but this was also not yet on contract.
▪ The Logistics Composite Model (LCOM), which will
be used to support assessments of suitability measures
including sortie generation rate and logistics footprint
– two key performance parameters in the ORD – is still
under development. Seven versions of the model will
be needed to cover the three variants as well as partnerunique and shipborne operations.
- The program is behind in developing and fielding training
simulators, referred to as F-35 Full Mission Simulators

56

F-35 JSF

(FMS), to train pilots, both at the integrated training
centers for initial F-35 pilot training and at the operational
locations. The FMS is a multi-ship, man-in-the-loop, F-35
mission systems software-in-the-loop simulation using
virtual threats, it is used to train both U.S. and partner
pilots.
▪ In 2014, the program moved simulator development
from Akron, Ohio to Orlando, Florida. As a result of the
move, the program lost experienced personnel, suffered
from shortfalls in required staffing, and fell behind in
meeting the hardware and software demands of the
rapidly growing pilot training requirements.
▪ In March 2016, following an inspection of the Block 2B
FMS, evaluators reported 203 test discrepancies;
173 remained open, 4 were canceled, 2 were pending
corrections, and 24 had been closed and corrections
included in the next build of FMS for Block 3i.
▪ The Block 3i FMS is behind the planned schedule
for fielding. The first Block 3i FMS is scheduled for
delivery to Marine Corps Air Station Iwakuni, Japan, in
December 2016, followed by two more FMS delivered to
partner countries.
▪ Because of delays in delivering the Block 3i FMS, the
Block 3F FMS is even further behind schedule. Although
earlier plans included delivering the Block 3F FMS in
CY17, the program is now replanning the schedule.
▪ Since the FMS runs F-35 mission systems software, it
requires Block 3F mission data files, integrated with
virtual threats, to build the threat environment simulation
(TES). It currently takes up to 20 months for the
program to build the TES after new mission data files
are available, hence pilots will not have Block 3F FMS,
with the USRL-produced mission data files, available
for training prior to IOT&E. Alternatively, the program
may elect to use the contractor-developed DT mission
data files for the Block 3F FMS. However, doing so
would make the training in the FMS not operationally
representative, as those mission data files do not
accurately portray the TES to the pilot. Without an
adequate Block 3F FMS, the OT pilots will have to rely
on the available Block 3F OT aircraft for training.
• The JOTT completed detailed test designs for accomplishing
IOT&E. DOT&E approved the designs in August 2016. The
test designs include comparisons of the F-35 with the A-10
in the Close Air Support role, the F-16C (Block 50) in the
Suppression/Destruction of Enemy Air Defenses (SEAD/
DEAD) mission area, and the F-18E/F in the air-to-surface
strike mission area. The JOTT has begun detailed test
planning based on these designs, and will provide these plans
to DOT&E for approval, prior to the start of IOT&E.
• Block Buy. The program and Services continue to pursue
a “Block Buy” for production lots 12 through 14. This
multi-year procurement scheme is based on a partial group
of the partner nations, designated as “Full Participants,”
funding a 2 percent Economic Order Quantity (EOQ) in FY17
and another 2 percent EOQ in FY18. Other partner nations,

 FY16 DOD PROGRAMS
designated “Partial Participants,” would procure Lot 12 as a
single year lot procurement, then commit to procuring Lots 13
and 14 as a part of the Block Buy and provide funding of
4 percent EOQ in FY18. Similar to the Partial Participants, the
Services would procure Lot 12 as a single year procurement
and fund 4 percent EOQ in FY18, but maintain the options
for single year procurements in Lots 13 and 14. Altogether,
452 F-35 aircraft would be procured under the Block Buy
scheme, on top of the 490 aircraft (346 for the U.S. Services)
previously procured in lots 1-11, all purchased without the
informed results of an IOT&E. As reported in the FY15
DOT&E Annual Report, many questions remain on the
prudence of committing to the multi-year procurement of a
Block Buy scheme prior to the completion of IOT&E:
- Is the F-35 program sufficiently mature to commit to the
Block Buy with the ongoing rate of discovery while in
development?
- Is it appropriate to commit to a Block Buy given that
essentially all the aircraft procured thus far require
modifications to be used in combat? The Services will
have accepted delivery of 346 aircraft through Lot 11,
before the additional aircraft are purchased via the Block
Buy scheme.
- Would committing to a Block Buy prior to the completion
of IOT&E provide the contractor with needed incentives
to fix the problems already discovered, as well as those
certain to be discovered during IOT&E?
- Would the Block Buy be consistent with the “fly before
you buy” approach to acquisition advocated by the
Administration, as well as with the rationale for the
operational testing requirements specified in title 10,
U.S. Code, or would it be considered a “full rate” decision
before IOT&E is completed and reported to Congress, not
consistent with the law?
• Follow-on Modernization (FoM). The program continued
making plans for all variants for FoM, also referred to as
Block 4, which is on DOT&E oversight. The program intends
to award the contract for the modernization effort in 2QCY18
with developmental flight testing beginning 3QCY19. Four
increments of capability are planned, Blocks 4.1 through
4.4. Blocks 4.1 and 4.3 will provide software-only updates,
Blocks 4.2 and 4.4 will add hardware as well as software
updates. Improved Technical Refresh 3 (TR3) processors are
planned to be added in Block 4.2. However, the plans for FoM
are not executable for a number of reasons including, but not
limited to, the following:
- Too much technical content for the allocated developmental
timeline. Experience with the F-22 modernization program
indicates the planned 18- to 24-month cycle for FoM is
insufficient for the large number of planned additional
capabilities; the F-22 increments had less content plus
software maintenance releases between new capability
releases.
- High risk of carrying excessive technical debt and
deficiencies from Block 3F and the balance of SDD into
FoM. The planned 4-year gap between the planned final

release of Blocks 3F in 2017 and Block 4.1 in 2021 lacks
resources (i.e., funding and time) for a bridge software
maintenance release to reduce technical debt and verify
Block 3F IOT&E corrections of deficiencies. Although
the unresolved technical debt is an SDD shortfall, it sets
up FoM to fail due to unrealistic planning and inadequate
resourcing.
- Insufficient time for conducting adequate operational
testing for each increment.
▪ The current plan for F-35 Block 4.2 only has 18 months
for DT flight test and 6 months for OT&E, despite
containing substantially more new capabilities and
weapons than F-22 Block 3.2B.
▪ For comparison, the F-22 Block 3.2B program planned
approximately two years for DT flight test and one
year of OT&E spin-up and flight test; F-22 Blocks 3.1,
3.2A and 3.2B have suffered delays and problems
accomplishing testing due to inadequate test resources
and schedule.
- Inadequate test infrastructure (aircraft, laboratories,
personnel) to meet the testing demands of the capabilities
planned.
▪ The current end-of-SDD developmental test aircraft
drawdown plan is still being developed. However, any
plan that significantly reduces the F-35 test force in 2017
and 2018 – precisely when the program needs this test
force to finish the delayed SDD Block 3F Joint Test Plan
(JTP) and correct remaining deficiencies with additional
Block 3F updates in preparation for IOT&E – would
result in shortfalls of the necessary resources to provide
full Block 3F capability.
▪ A robust test force will also be required to be available
through 2020 to correct the inevitable new discoveries
from IOT&E and produce a final Block 3F software
release that provides a stable foundation for adding the
new Block 4.1 capabilities.
▪ The program plans to award contracts to start
simultaneous development of Blocks 4.1 and 4.2 in 2018,
well prior to completion of IOT&E and having a full
understanding of the deficiencies that will emerge from
IOT&E; without any budget or time to fix deficiencies
from earlier development.
▪ The requirement to integrate and test multiple
configurations simultaneously (TR2 and TR3) will
require additional time, test aircraft, and lab resources; a
problem that must be addressed as the program considers
plans for the fleet of test aircraft for FoM.
▪ As of the writing of this report, the program’s published
FoM plan would have reduced test infrastructure
from 18 DT aircraft and 1,768 personnel, which are
still heavily tasked to complete ongoing Block 3F
development, to just 9 aircraft and approximately
600 personnel to support FoM. Clearly, this plan is
grossly inadequate. However, the program and Services
were in the process of replanning the test infrastructure
for FoM and had not yet provided the results.

F-35 JSF

57

 FY16 DOD PROGRAMS
▪ Both the Air Force and the Navy conducted independent
studies in 2016 to determine what infrastructure and test
periods for FoM would be adequate. Neither report had
been released as of the time of this report. DOT&E has
requested to see the preliminary results of the Air Force
study, but the Air Force has refused to provide them, citing
the fact that the results are not final and the report is in
draft.
- Significant technical and schedule risk due to Block 4.1
adding new capabilities to the already-stretched TR2
avionics hardware, along with Block 4.2 attempting to
simultaneously migrate to a new open-architecture TR3
processor while adding many significant new capabilities.
▪ For Block 4.1, the program plans to add multiple new
capabilities to the TR2 avionics hardware, even though
this architecture already has memory and processing
limitations running the full Block 3F capabilities, resulting
in avionics stability issues and capability limitations.
▪ For Block 4.2, the program plans to simultaneously add
multiple significant new software capabilities while
migrating to a new avionics hardware configuration,
including a new open-architecture TR3 processor and new
electronic warfare (EW) hardware. This will be far more
challenging than the program’s problematic re-hosting of
Block 2B software, designed to run on TR1 processors, on
to TR2 processors to create Block 3i. Although no new
capabilities were added in Block 3i, significant avionics
stability issues were manifested due to technical debt and
differences with the new architecture.
▪ The program claims the new F-35 Block 4.2 software,
which will be designed to run on new TR3 processors,
will also be backward-compatible to run in the hundreds
of early production aircraft with TR2 processors, but has
not yet presented a plan to demonstrate this. Based on the
current TR2 architecture capacity limitations with Block
3F, this claim is unlikely to be realized.
▪ Instead of adding lab capacity to support testing of
processor loads with the additional mission systems
capabilities, the program plans to reduce the lab
infrastructure supporting development. The program has
already retired the Cooperative Avionics Test Bed aircraft
– a decision that has increased the burden on flight testing
with F-35 aircraft.
▪ Current JPO projections for modifying aircraft with TR2
processors to the TR3 processor configuration extend into
the 2030s. As a result, up to three configurations of test
aircraft and labs may be needed if the program requires
more advanced processors than the TR3 planned for
Block 4 (i.e., the next Block upgrade requiring even more
processing capacity driving the need for new processors).
▪ The program also does not yet have an executable plan
to provide a mission data reprogramming lab in the TR3
configuration in time to support Block 4.2 OT and fielding.
- Attempting to proceed with the current unrealistic plans
for FoM would be to completely ignore the costly lessons
learned from Block 2B, 3i and 3F development, as well as

58

F-35 JSF

those from the F-22 program. As learned from the F-22
Blocks 3.1, 3.2A and 3.2B, an overly aggressive plan with
inadequate resources ultimately takes longer, costs more
and delays needed capabilities for the warfighter.
• This report includes assessments of the progress of testing to
date, including developmental and operational testing intended
to verify performance prior to the start of IOT&E. Test flights
and test points are summarized in two tables on the next page.
- For developmental flight testing, the program creates
test plans by identifying specific test points (discrete
measurements of performance under specific flight
test conditions) for accomplishment, in order to assess
the compliance of delivered capabilities with contract
specifications.
▪ Baseline test points refer to points in the test plans that
must be accomplished in order to evaluate if performance
meets contract specifications.
▪ Non-baseline test points are accomplished for various
reasons. Program plans include a budget for some of
these points within the capacity of flight test execution.
The following describes non-baseline test points.
» Development points are test points required to
“build up” to, or prepare for, the conditions needed
for assessing specification compliance (included in
non-baseline budgeted planning in CY16).
» Regression points are test points flown to ensure
that new software does not introduce shortfalls in
performance for requirements that had previously
been verified using previous software (included in
non-baseline budgeted planning in CY16).
» Discovery points are test points flown to investigate
root causes of newly discovered deficiencies or to
characterize deficiencies so that the program can
design fixes for them (not included in planning in
CY16).
▪ As the program developed plans for allocating test
resources against test points in CY16, the program
included a larger budget for non-baseline test points
(development and regression points) for mission
systems testing, as the plans for the year included
multiple versions of software, requiring regression and
developmental test points be completed. For CY16
mission systems testing, planners budgeted an additional
69 percent of the number of planned baseline test points
for non-baseline test purposes (e.g., development and
regression points), the largest margin planned for a CY
to date. This large margin was planned because the
program anticipated the test centers would need points
for building up to the baseline points that would be flown
for specification compliance as well as for completing
regression of multiple versions of Block 3F software. In
this report, growth in test points refers to points flown
over and above the planned amount of baseline and
budgeted non-baseline points (e.g., discovery points and
any other added testing not originally included in the
formal test plan).

 FY16 DOD PROGRAMS
▪ The continued need to budget for non-baseline test
points in the CY16 plan is a result of the limited
maturity of capabilities in the early versions of
mission systems software. Although the program
planned to complete developmental flight testing in
January 2017, according to their Integrated Master
Schedule, developed after the program was restructured
in 2010, delays in issuing mature software to flight
test made it clear that regression and development test
points would still be needed throughout CY16.

▪ Cumulative SDD test point data in this report refer to the
total progress towards completing development at the end
of SDD.
- Limited operational testing was also conducted throughout
the year to support assessments of weapon capability,
deployment demonstrations, shipborne testing, and the Air
Force’s IOC declaration; results of these limited tests are
used to support assessments throughout this report.

TEST FLIGHTS (AS OF NOVEMBER 30, 2016)
All Testing

Flight Sciences

All Variants

F-35A

F-35B

Mission
Systems

F-35C

2016 Planned

1,221

151

359

237

474

2016 Actual

1,362

226

386

271

479

+11.5%

+49.7%

+7.5%

+14.3%

+1.1%

7,624

1,587

2,242

1,469

2,326

Difference from Planned
Cumulative Planned
Cumulative Actual

7,853

1,697

2,318

1,479

2,359

Difference from Planned

+3.0%

+6.9%

+3.4%

+0.7%

+1.4%

Prior to CY16 Planned

6,403

1,436

1,883

1,232

1,852

Prior to CY16 Actual

6,492

1,471

1,932

1,209

1,880

TEST POINTS (AS OF NOVEMBER 30, 2016)
All
Testing
F-35A
All
Variants

Mission Systems1

Flight Sciences
F-35B

Block 3F
Baseline

Budgeted
NonBaseline2

F-35C

Block 3F
Baseline

Budgeted
NonBaseline2

Block 3F
Baseline

Budgeted
NonBaseline2

Block 3F

Budgeted
NonBaseline2

Other3

2016 Test Points Planned (by type)

8,774

1,205

159

1,876

115

1,695

146

1,189

1,534

855

2016 Test Points Accomplished (by type)

7,838

1,303

156

1,783

115

1,304

136

975

1,534

532

-10.7%

+8.1%

-1.9%

-5.0%

0.0%

-23.1%

-6.8%

-18.0%

0.0%

-37.8%

Difference from Planned
Points Added Beyond Budgeted Non-Baseline
(Growth Points)

304

0

54

0

250

Test Point Growth Percentage (Growth
Points/Test Points Accomplished)

3.9%

0.0%

3.0%

0.0%

25.6%

Total Points (by type) Accomplished in 20164

8,142

1,459

1,952

1,440

3,291

Cumulative Data
Cumulative System Design and Development
(SDD) Planned Baseline

51,060

12,225

15,994

12,604

10,237

Cumulative SDD Actual Baseline

50,278

12,327

15,970

12,279

9,702

Difference from Planned

-1.5%

+0.8%

-0.2%

-2.6%

-5.2%

Est. Baseline Test Points Remaining

6,649

100

1,726

1,178

3,645

Est. Non-Baseline Test Points Remaining

2,502

12

136

73

2,281

1. Mission Systems Test Points for CY16 are shown only for Block 3F. Testing conducted to support Block 2B and Block 3i Mission Systems are discussed separately in the text. Cumulative
numbers include all previous Mission Systems activity.
2. These points account for planned development and regression test points built into the 2016 plan; additional points are considered “growth.” The total number of regression, development
and discovery points completed is the sum of budgeted non-baseline test points accomplished plus points added beyond budgeted non-baseline.
3. Represents mission systems activity not directly associated with Block capability (e.g., radar cross section characterization testing, test points to validate simulator).
4. Total Points Accomplished = 2016 Baseline Accomplished + Added Points

F-35 JSF

59

 FY16 DOD PROGRAMS
Developmental Testing: F-35A Flight Sciences
Flight Test Activity with AF-1, AF-2, and AF-4 Test Aircraft
• F-35A flight sciences testing focused on:
- Clearing the F-35A Block 3F flight envelope (i.e., to
Mach 1.6, 700 knots, and 9.0 g) for loads, flutter, and
weapons environment
- Testing of the internal gun
- Flight envelope clearance for external weapons required
for full Block 3F weapons capability
- Weapons separation testing of the AIM-9X missile
(external only), GBU-12 bomb (external carriage added for
Block 3F)
- High energy braking, high sink rate landings, and arresting
gear engagements
- AF-4 completed all flight testing for which it had been
slated, in July, and transitioned to chemical and biological
testing in August
F-35A Flight Sciences Assessment
• The program planned to complete F-35A flight sciences
testing by the end of October 2016; however, additional
testing for weapons environment and regression of
new software forced testing to continue into at least
December 2016. The program was able to complete baseline
test points to clear the aircraft structure for Block 3F
envelope (up to 9 g, 1.6M and 700 knots), completing flutter
testing on AF-2 on September 29 and loads testing on AF-1
on November 4, 2016. Through the end of November,
the test team flew 50 percent more flights than planned
(226 flown versus 151 planned) and accomplished 8 percent
more baseline test points than planned for the year (1,303 test
points accomplished versus 1,205 planned). These additional
baseline test points were added by the program throughout
the year and represent testing not originally budgeted for
when the CY16 plans were made. The test team also flew an
additional 156 test points for regression of new air vehicle
software, all of which were within the budgeted non-baseline
test points allocated for the year. As of the end of November
the program had approximately 100 baseline test points
remaining to complete F-35A flight sciences testing for
Block 3F.
• The following discoveries were made during F-35A flight
sciences testing:
- Failure of the attachment joint, as indicated by the
migration of the bushing in the joint, between the
vertical tail and the airframe structure is occurring much
earlier than planned, even with a newly designed joint
developed to address shortfalls in the original design.
In October 2010, the F-35A full scale durability test
article, AJ-1, showed wear in the bushing of this joint
after 1,784 test hours, which indicated that the joint will
fall short of the 8,000 hours of service life required by
the JSF contract specification. The program developed
a redesigned joint and began installing them on the
production line with Lot 6 aircraft, which began delivery
in October 2014. Subsequently, in July 2015, when

60

F-35 JSF

-

-

-

-

inspections showed bushing migrations and significant
damage to the right and left side attachment joints in
BF-3, one of the F-35B flight sciences developmental test
aircraft, the joint was repaired and the bushing replaced to
replicate the redesigned joint. In August, 2016, inspections
of the joints in AF-2, one of the F-35A flight sciences
developmental test aircraft, showed similar bushing
migration requiring repair and bushing replacement in
accordance with the redesign. On September 1, 2016,
inspections of the vertical tail on BF-3 showed that the
newly designed joint had failed, after only 250 hours
of flight testing since the new joint had been installed,
requiring another repair and replacement. BF-3 completed
repairs and returned to flight on November 10, 2016.
Vibrations induced by the gun during firing are excessive
and caused the 270 volts DC battery to fail. The program
began qualification testing of a redesigned battery in 2015,
but cracks in the casing discovered after the first series
of testing required additional redesigning of the battery.
Requalification of a newly designed battery has not yet
occurred as of the writing of this report.
Limitations to the carriage and employment envelope of
the AIM-120 missile above 550 knots may be required
due to excessive vibrations on the missiles and bombs in
the weapons bay. Analyses of flight test data and ground
vibration test data are ongoing (this applies to all variants).
Excessive and premature wear on the hook point of
the arresting gear has caused the program to consider
a more robust redesign. In fact, the hook point has
required replacement after only one engagement in some
instances; the longest a hook point has lasted to date is
five arrestments. This fails to meet the minimum service
life of 15 arrestments. Additionally, failure of the hook
point of the arresting gear on AF-4 occurred in July during
testing of high speed engagements. However, this appears
to be due to a malfunction of the Mobile Aircraft Arresting
System (MAAS), which holds the arresting cable in place
on both sides of the runway. The MAAS is designed to
allow the arresting cable to slide across the hook upon
engagement until the right and left sides are in equilibrium
before the braking action to slow the aircraft takes place
(this helps steer the aircraft toward the center of the
runway during the engagement). For unknown reasons,
only one side of the MAAS released the cable, resulting
in the hook point becoming abraded by the arresting cable
and failing 1.5 seconds after engagement.
Block 3F envelope testing required an inflight structural
temperature assessment, which yielded higher than
predicted air flow temperatures in the engine nacelle bay
in high-speed portions of the flight envelope under high
dynamic pressures. This resulted in higher than expected
nacelle structural temperatures on both the F-35A and
F-35C aircraft. Thermal stress analyses of the affected
parts are necessary before the program can provide the full

 FY16 DOD PROGRAMS
Block 3F flight envelope for fleet release. The outcome
may result in restricting fielded operational aircraft to
600 knots airspeed below 5,000 feet altitude or a structural
change; this will be determined when the Services review
the analyses and issue the military flight release, which
certifies the operational flight envelope.
- All F-35 variants display objectionable or unacceptable
flying qualities at transonic speeds, where aerodynamic
forces on the aircraft are rapidly changing. Particularly,
under elevated “g” conditions, when wing loading causes
the effects to be more pronounced, pilots have reported
the flying qualities as “unacceptable.” The program
adjusted control laws that govern flight control responses
in an updated version of software released to flight test in
March 2016. Results from flight testing of the software
changes have not yet been released. Although the elevated
g “dig-in” apparently affects all three variants, the program
does not plan to develop any additional control law
changes to mitigate these responses to aerodynamic effects
in the transonic region. In operational fleet aircraft, g limit
exceedances are annunciated to the pilot and, in peacetime,
result in subsequent restricted maneuvering, mission
termination, and a straight-in approach and landing to
recover the aircraft. The aircraft is then down for some
time for maintenance inspections and potential repairs.
Also, the probability and long-term structural effects of
the g exceedances should be assessed by the program and
mitigated, if necessary.
- Foam insulation around the polyalphaolefin (PAO)
coolant tubes that pass through wing and main body
fuel tanks in F-35A aircraft was found to be failing after
exposure to fuel. The discovery was made on a fielded
production F-35A aircraft (AF-101) as it was undergoing
depot-level modifications for fuel valves in August 2016.
The program determined the cause was a failure of the
manufacturing process with the sealant coating on the
insulation designed to protect the insulation from being
exposed to fuel. Instead, the sealant was permeable to
fuel, permitting the insulation to absorb fuel and expand,
forcing cracking and failure of the sealant coatings and
eventual breakdown and flaking of the insulation. This
affected a total of 57 F-35A aircraft; 42 in the production
process and 15 fielded aircraft. The Air Force temporarily
grounded the 15 fielded aircraft, 10 of which were
designated as Initial Operational Capability aircraft. The
program quickly developed inspections and implemented
procedures to mitigate the insulation problems for fielded
aircraft and those too far in the production line to have the
fuel lines replaced with proper insulation. The procedures
vary depending on whether fuel has entered the tank
with the PAO lines. For aircraft in which the fuel tanks
have contained fuel, the procedures involve accessing the
affected fuel tanks, removing the defective insulation,
installing blocking screens to prevent debris from leaving
the tank (and possibly contaminating other tanks, clogging
valves or affecting fuel pump operation). For the aircraft

in the production line that have not yet had fuel in the
tanks, the insulation will be removed from the PAO tubes,
but screens will not be added to the tank. The program
does not plan to re-insulate the PAO tubes, as the Block 3F
avionics – which are cooled by the PAO – apparently have
adequate thermal margin to tolerate the loss of insulation
on the tubes. The program must ensure that deployed
operating locations with high ambient temperatures – such
as those in Southwest Asia – are able to provide the
cooling effect necessary to prevent avionics overheat
conditions, especially for heat-soaked aircraft with hot
fuel tanks and during extended ground operations. The
program will need to conduct another assessment for
Block 4 avionics, and any new processors, to ensure the
thermal margin with that hardware configuration is still
adequate.
- An Air Force F-35A aircraft assigned to Luke AFB,
Arizona, experienced a tailpipe fire during engine
start while deployed to Mountain Home AFB, Idaho
in September 2016, causing significant damage to the
aircraft. The incident is under investigation.
- The program designed and fielded an electrical Engine
Ice Protection System (EIPS) to protect the engine from
ice damage when exposed to icing conditions during
ground operations and in flight. Although it was qualified
during SDD engine ground tests, no SDD aircraft have the
system installed in the engine. The program fielded the
system with later-lot production aircraft, but deficiencies
in the system caused electrical shorting and damage to
the composite blades (referred to as the Fan Inlet Variable
Vanes) on the front of several engines. To prevent further
damage to engines in the field, the program has disabled
EIPS and is changing the technical orders to require
pilots to shut down the aircraft if icing conditions are
encountered on the ground. DOT&E is not aware of any
corrections to the EIPS planned during SDD.
- The program completed the final weight assessment of the
F-35A air vehicle for contract specification compliance in
April 2015 with the weighing of AF-72, a Lot 7 aircraft.
The actual empty aircraft weight was 28,999 pounds, 372
pounds below the planned not-to-exceed weight of 29,371
pounds. The actual weights of production aircraft since
then have been stable, with no significant weight growth
observed. Weight estimates for production Lots 10 and
later indicate an expected weight growth of between 120
and 140 pounds, primarily due to new electronic warfare
(EW) avionics. Weight management of the F-35A is
important for meeting performance requirements and
structural life expectations. The program will need to
continue disciplined management of the actual aircraft
weight beyond the contract specification as further
discoveries during the remainder of SDD may add
weight and result in performance degradation that would
adversely affect operational capability.

F-35 JSF

61

 FY16 DOD PROGRAMS
Developmental Testing: F-35B Flight Sciences
Flight Test Activity with BF-1, BF-2, BF-3, BF-4, and BF-5 Test
Aircraft
• F-35B flight sciences focused on:
- Clearing the F-35B Block 3F flight envelope (i.e., to Mach
1.6, 630 knots, and 7.0 g)
- High angle-of-attack testing with external stores
- Air refueling with the British KC-30A Voyager and Air
Force KC-10 aircraft
- Mode 4 (i.e., flight with the lift fan engaged to support
short takeoff and vertical landing operations) envelope
expansion
- Weapons separation testing of the AIM-9X missile
(external only), GBU-12 bomb (external carriage added for
Block 3F); Paveway IV bomb (internal and external) for
the United Kingdom, AIM-132 missile (external only) for
the United Kingdom
- Ground gun fire testing with the F-35B gun pod;
accomplished on BF-1 in July
F-35B Flight Sciences Assessment
• Through the end of November, the test team flew 8 percent
more flights than planned (386 flown versus 359 planned),
yet accomplished 5 percent less than the planned Block 3F
baseline test points (1,783 points accomplished versus 1,876
planned). The team flew an additional 169 test points for
regression of new air vehicle software, 115 of which were
the budgeted non-baseline points planned for CY16 and 54
points representing growth.
• The following details discoveries in F-35B flight sciences
testing:
- Limitations to the carriage and employment envelope of
the AIM-120 missile above 550 knots may be required
due to excessive vibrations induced on the missiles and
bombs in the weapons bay. Analyses of flight test data and
ground vibration test data are ongoing (this applies to all
variants).
- All F-35 variants display objectionable or unacceptable
flying qualities at transonic speeds, where aerodynamic
forces on the aircraft are rapidly changing. Particularly,
under elevated “g” conditions, when wing loading causes
the effects to be more pronounced, pilots have reported
the flying qualities as “unacceptable.” The program
adjusted control laws that govern flight control responses
in an updated version of software released to flight
test in March 2016. In the F-35B, an uncommanded
aircraft g “dig-in” that exceeds design limits has been
observed while performing elevated-g maneuvers in the
transonic region between 0.9M and 1.05M. Significant
g exceedances (up to 7.7 g; a 0.7 g exceedance) have
occurred when pilots were attempting to sustain 6.5 g or
greater in this region. Based on flight test data, the F-35B
responses to transonic aerodynamic effects between 0.9M
and 1.05M during rolling or elevated-g maneuvering cause
uncommanded excursions that exceed the designed g limit
as well. Although the elevated g “dig-in” apparently

62

F-35 JSF

affects all three variants, the program does not plan to
develop any additional control law changes to mitigate
these responses to aerodynamic effects in the transonic
region. In operational fleet aircraft, g limit exceedances
are annunciated to the pilot, and in peacetime, result in
subsequent restricted maneuvering, mission termination,
and a straight-in approach and landing to recover the
aircraft. The aircraft is then down for some time for
maintenance inspections and potential repairs. Also,
the probability and long-term structural effects of the
g exceedances should be assessed by the program and
mitigated, if necessary.
- Horizontal tail overheating was experienced on BF-3
during loads testing while accelerating to 1.5M for a loads
test point. The left horizontal inboard fairing surface
reached temperatures that exceeded the design limit by
a significant amount. Post-flight inspections revealed
de-bonding on the trailing edge of the horizontal tail
surface and heat damage was noted on the horizontal
tail rear spar. Hardness checks on the rear spar were
performed and were determined to be within the
acceptable range. It is not yet known whether the program
or the Services will impose airspeed or afterburner time
restrictions in the Block 3F envelope due to horizontal tail
overheating.
- Failure of the attachment joint, as indicated by the
migration of the bushing in the joint, between the vertical
tail and the airframe structure, is occurring much earlier
than planned, even with a newly designed joint developed
to address shortfalls in the original design. In October
2010, the F-35A full scale durability test article, AJ-1,
showed wear in the bushing of this joint after 1,784 test
hours, which indicated that the joint will fall short of the
8,000 hours of service life required by the JSF contract
specification. The program developed a redesigned joint
and began installing them on the production line with
Lot 6 aircraft, which began delivery in October 2014.
Subsequently, in July 2015, when inspections showed
bushing migrations and significant damage to the right and
left side attachment joints in BF-3, one of the F-35B flight
sciences developmental test aircraft, the joint was repaired
and the bushing replaced, to replicate the redesigned joint.
In August 2016, inspections of the joints in AF-2, one
of the F-35A flight sciences developmental test aircraft,
showed similar bushing migration requiring repair and
bushing replacement in accordance with the redesign. On
September 1, 2016, inspections of the vertical tail on BF-3
showed that the newly designed joint had failed, after
only 250 hours of flight testing since the new joint had
been installed, requiring another repair and replacement.
BF-3 completed repairs and returned to flight on
November 10, 2016.
- An F-35B assigned to Marine Corps Air Station Beaufort,
South Carolina, experienced a fire within the weapons
bay during a training mission in late October 2016. The

 FY16 DOD PROGRAMS
incident, although still under investigation, resulted in a
Class A mishap (involves loss of life or damage of more
than $2 Million). The Marine Corps did not ground any of
the training fleet as a result of the incident.
- The program designed and fielded an electrical Engine Ice
Protection System (EIPS) to protect the engine and lift fan
from ice damage when exposed to icing conditions during
ground operations and in flight. Although it was qualified
during SDD engine ground tests, no SDD aircraft have the
system installed in the engine. The program fielded the
system with later-lot production aircraft, but deficiencies
in the system caused electrical shorting and damage to
the composite blades (referred to as the Fan Inlet Variable
Vanes) on the front of the several engines. To prevent
further damage to engines in the field, the program has
disabled EIPS and is changing the technical orders to
require pilots to shut down the aircraft if icing conditions
are encountered on the ground. DOT&E is not aware of
any corrections to the EIPS planned during SDD.
- Weight management of the F-35B aircraft is critical to
meeting the Key Performance Parameters (KPPs) in the
Operational Requirements Document (ORD), including
the Vertical Landing Bring-Back (VLBB) requirement,
which will be evaluated during IOT&E. This KPP requires
the F-35B to be able to fly an operationally representative
profile and recover to the ship with the necessary fuel and
balance of unexpended weapons (two 1,000-pound bombs
and two AIM-120 missiles) to safely conduct a vertical
landing.
▪ The program completed the final weight assessment
of the F-35B air vehicle for contract specification
compliance in May 2015 with the weighing of BF-44, a
Lot 7 production aircraft. Actual empty aircraft weight
was 32,442 pounds, only 135 pounds below the planned
not-to-exceed weight of 32,577 pounds and 307 pounds
(less than 1 percent) below the objective VLBB
not-to-exceed weight of 32,749 pounds.
▪ The actual weights of production aircraft through Lot 8
have increased slightly, with the latest Lot 8 aircraft
weighing approximately 30 pounds heavier than BF-44.
Weight estimates for Lot 10 aircraft and later project
weight growth of an additional 90 pounds, primarily due
to additional EW equipment.
▪ Known modifications to the 14 Lot 2 through 4 F-35B
aircraft, required to bring those aircraft to the Block 3F
configuration, are expected to potentially add an
additional 350 pounds, which will push their weight
above the objective not-to-exceed weight to meet the
VLBB KPP. This KPP will be evaluated during IOT&E
with an F-35B OT aircraft.
▪ Estimates for FoM weight growth include an additional
250 pounds, which will exceed the vertical landing
structural limit not-to-exceed weight of 33,029 pounds
for the Lot 2 through Lot 4 aircraft. This additional
weight may prevent these aircraft from being upgraded
to the Block 4 configuration.

Developmental Testing: F-35C Flight Sciences
Flight Test Activity with CF-1, CF-2, CF-3, and CF-5 Test Aircraft
• F-35C flight sciences focused on:
- Clearing the F-35C Block 3F flight envelope (i.e., to Mach
1.6, 700 knots, and 7.5 g)
- Air refueling with F/A-18, KC-10, and KC-135 aircraft
- Weapons separation testing of the AIM-9X missile
(external only), Joint Standoff Weapon (JSOW, internal
only), GBU-12 bomb (external carriage added for
Block 3F)
- Shore-based ship suitability testing with external stores,
in preparation for shipborne trials that were conducted in
August
- High angle-of-attack testing with external stores
- Testing of the Joint Precision Approach and Landing
System (JPALS)
- Ground gun fire testing with the F-35C gun pod;
accomplished on CF-3 in November
F-35C Flight Sciences Assessment
• Through the end of November, the test team flew 14 percent
more than planned flights (271 flown versus 237 planned),
but accomplished 23 percent less than the planned Block 3F
baseline test points (1,304 points accomplished versus 1,695
planned). The team flew an additional 136 test points for
regression of new software, all of which were accounted for
in the budgeted non-baseline points planned for the year.
• The following details discoveries in F-35C flight sciences
testing:
- Flight testing of structural loads with the AIM-9X
air-to-air missile, which will be carried on external
pylons outboard of the wing fold in the F-35C, shows
exceedances above the wing structural design limit
during flight in regions of aircraft buffet (increased
angle-of-attack) and during landings. To address these
deficiencies, the program is developing a more robust
outer wing design, which is scheduled for flight testing in
early CY17. Without the redesigned outer wing structure,
the F-35C will have a restricted flight envelope for missile
carriage and employment, which will be detrimental to
maneuvering, close-in engagements.
- Limitations to the carriage and employment envelope of
the AIM-120 missile above 550 knots may be required due
to excessive vibrations induced on the missiles and bombs
due to the acoustics in the weapons bay. Analyses of flight
test data and ground vibration test data are ongoing (this
applies to all variants).
- All F-35 variants display objectionable or unacceptable
flying qualities at transonic speeds, where aerodynamic
forces on the aircraft are rapidly changing. Particularly,
under elevated “g” conditions, when wing loading causes
the effects to be more pronounced, pilots have reported
the flying qualities as “unacceptable.” The program
adjusted control laws that govern flight control responses
in an updated version of software released to flight test
in March 2016. In the F-35C, like the other variants, an

F-35 JSF

63

 FY16 DOD PROGRAMS
uncommanded aircraft g “dig-in” that exceeds design
limits has been observed while performing testing of
elevated-g maneuvers in the transonic region of the flight
envelope. While attempting to sustain a maximum g
(7.5g) turn, an F-35C test aircraft experienced 8.2 g – an
exceedance of 0.7 g. The program does not plan to
develop any additional control law changes to address
the flying quality. Similar to the other variants, an over-g
condition requires the pilot to terminate the mission (in
peacetime) and recover the aircraft with a straight-in
approach and landing with minimal maneuvering. The
aircraft is then down for some time for maintenance
inspections and potential repairs. Also, the probability and
long-term structural effects of the g exceedances should be
assessed by the program and mitigated, if necessary.
- Weapons environment testing showed that the
aircraft experienced transient rolling conditions while
asymmetrically opening and closing the weapon bay
doors (WBD). The flight control laws were designed
to compensate for the doors opening and closing
asymmetrically. The program corrected the on-board
aerodynamic models in two vehicle systems software
updates (versions R31.1 and R35.1) to reduce the roll
transients. These corrections resolved the transients for
the subsonic and transonic flight regimes, but not for
supersonic regimes. The operational impact of these
transients will be assessed during IOT&E.
- Block 3F envelope testing required an inflight structural
temperature assessment, which yielded higher than
predicted air flow temperatures in the engine nacelle
bay in high-speed portions of the flight envelope under
high dynamic pressures. This resulted in higher nacelle
structural temperatures on both the F-35A and F-35C
aircraft. Thermal stress analyses of the affected parts are
necessary before the program can provide the full Block
3F flight envelope for fleet release. The outcome may
result in restricting fielded operational aircraft to 600 knots
airspeed below 5,000 feet altitude, or a structural change;
this will be determined when the Services review the
analyses and issue the military flight releases, which will
certify the operational flight envelope.
- As reported in previous DOT&E Annual Reports, the
F-35C experiences buffet and transonic roll off (TRO),
an uncommanded roll, at transonic Mach numbers and
elevated angles of attack. It is caused by the impact of
airflow separating from the leading edge of the wing that
“buffets” aft areas of the wing and aircraft during basic
fighter maneuvering. The TRO and buffet occur in areas
of the maneuvering envelope that cannot be sustained
for long periods of time, as energy depletes quickly and
airspeed transitions out of the flight region where these
conditions manifest. However fleeting, these areas of the
envelope are used for critical maneuvers. Operational
testing of the F-35C during IOT&E will assess the effect
of TRO and buffet on overall mission effectiveness.

64

F-35 JSF

- Due to the stiffness of the landing gear struts,
particularly the nose gear, taxiing in the F-35C results
in excessive jarring of the aircraft and often requires
pilots to stop taxiing if they need to make changes using
the touchscreens on the cockpit displays or to write
information on their kneeboard. Currently, the program
has no plans to correct the deficiency of excessive jarring
during F-35C taxi operations.
- Excessive vertical oscillations during catapult launches
make the F-35C operationally unsuitable for carrier
operations, according to fleet pilots who conducted
training onboard USS George Washington during the latest
set of ship trials. Although numerous deficiencies have
been written against the F-35C catapult launch – starting
with the initial set of F-35C ship trials (DT-I) in
November 2014 – the deficiencies were considered
acceptable for continuing developmental testing. Fleet
pilots reported that the oscillations were so severe that
they could not read flight critical data, an unacceptable
and unsafe situation during a critical phase of flight. Most
of the pilots locked their harness during the catapult shot
which made emergency switches hard to reach, again
creating, in their opinion, an unacceptable and unsafe
situation. The U.S. Navy has informed the Program
Office that it considers this deficiency to be a “must fix”
deficiency. The program should address the deficiency
of excessive vertical oscillations during catapult launches
within SDD to ensure catapult operations can be conducted
safely during IOT&E and during operational carrier
deployments.
- Overheating of the Electro-Hydraulic Actuator System
(EHAS) occurs under normal maneuvering in the F-35C.
The EHAS actuators move the flight surfaces and are
cooled by airflow across the control surfaces. Pilots are
alerted in the cockpit of an overheat condition and must
then minimize maneuvering and attempt to cool the
EHAS by climbing, if practical, to an altitude with lower
temperatures to enhance cooling. Recovery and landing
must be completed as soon as possible, terminating the
mission.
- The program designed and fielded an electrical Engine Ice
Protection System (EIPS) to protect the engine from ice
damage when exposed to icing conditions during ground
operations and in flight. Although it was qualified during
SDD engine ground tests, no SDD aircraft have the system
installed in the engine. The program fielded the system
with later-lot production aircraft, but deficiencies in the
system have caused electrical shorting and damage to the
composite blades (referred to as the Fan Inlet Variable
Vanes) on the front of the engine. To prevent further
damage to engines in the field, the program has disabled
EIPS and is changing the technical orders to require
pilots to shut down the aircraft if icing conditions are
encountered on the ground. DOT&E is not aware of any
corrections to the EIPS planned during SDD.

 FY16 DOD PROGRAMS
- Weight management of the F-35C is important for meeting
air vehicle performance requirements, including the KPP
for recovery approach speed to the aircraft carrier, and
structural life expectations. The program completed
the final weight assessment of the F-35C air vehicle for
contract specification compliance in May 2016 with the
weighing of CF-28, a Lot 8 aircraft. The actual empty
aircraft weight was 34,581 pounds, 287 pounds below
the planned not-to-exceed weight of 34,868 pounds. The
weights of the other three Lot 8 production aircraft have
been consistent with that of CF-28. Weight estimates for
production Lots 11 and later indicate an expected weight
growth of approximately 160 pounds. The program
will need to continue rigorous management of the actual
aircraft weight through the balance of SDD to avoid
performance degradation that would affect operational
capability.
Developmental Testing: Mission Systems
• Mission systems are developed, tested, and fielded in
incremental blocks of capability.
- Block 1. The program designated Block 1 for initial
training capability in two increments: Block 1A for Lot 2
(12 aircraft) and Block 1B for Lot 3 aircraft (17 aircraft).
No combat capability was available in either Block 1
increment. The Services have upgraded all of these
aircraft to the Block 2B configuration through a series of
modifications and retrofits. Additional modifications will
be required to configure these aircraft in the Block 3F
configuration.
- Block 2A. The program designated Block 2A for
advanced training capability and delivered aircraft in
production Lots 4 and 5 in this configuration. No combat
capability was available in Block 2A. The Services
accepted 62 aircraft in the Block 2A configuration
(32 F-35A aircraft in the Air Force, 19 F-35B aircraft in
the Marine Corps, and 11 F-35C aircraft in the Navy).
Similar to the Block 1A and Block 1B aircraft, the
Services have upgraded all of the Block 2A aircraft to the
Block 2B configuration with modifications and retrofits,
although fewer modifications were required. Additional
modifications will be required to fully configure these
aircraft in the Block 3F configuration.
- Block 2B. The program designated Block 2B for initial,
limited combat capability with selected internal weapons
(AIM-120C, GBU-31/32 JDAM, and GBU-12). This
block is not associated with the delivery of any lot of
production aircraft, but with an upgrade of mission
systems software capability for aircraft delivered through
Lot 5 in earlier Block configurations. Block 2B is the
software that the Marine Corps accepted for the F-35B IOC
configuration. Corrections to some deficiencies identified
during Block 2B and Block 3i mission systems testing
have been included in the latest production release of
Block 2B software – version 2BR5.3 – fielded in May 2016
after airworthiness testing in April. The Services began

converting aircraft from these earlier production lots to the
Block 3i configuration by replacing the older Technical
Refresh 1 (TR1) integrated core processor with newer
Technical Refresh 2 (TR2) processors this year. As of the
end of November, 1 F-35A (AF-31) and 1 F-35B (BF-19)
had completed the TR2 modifications, both of which are
instrumented operational test aircraft. The Marine Corps
declared IOC with Block 2B-capable aircraft in July 2015.
- Block 3i. The program designated Block 3i for delivery
of aircraft in production Lots 6 through 8, as these aircraft
include a set of upgraded TR2 integrated core processors.
The program delivered Lot 6 aircraft with a Block 3i
version that included capabilities equivalent to Block 2A
in Lot 5. Lot 7 aircraft were delivered with capabilities
equivalent to Block 2B, as are Lot 8 aircraft currently.
Block 3i software began flight testing in May 2014 and
completed baseline testing in October 2015, eight months
later than planned in the Integrated Master Schedule (IMS).
Because of software immaturity and instability during
startup and in flight, the program paused flight testing of
Block 3F software in February 2016 (software version
3FR5) and returned to Block 3i development and flight
testing to address poor mission systems stability. After
completing flight testing in April of another build of Block
3i software, version 3iR6.21, that version was fielded to
the operational units with improved stability performance,
which was similar to that seen in the latest build of Block
2B software. By the end of November, the program had
delivered 51 F-35A aircraft to the Air Force, 17 F-35B
aircraft to the Marine Corps, and 13 F-35C to the Navy
in the Block 3i configuration in Lots 6, 7 and 8. The
Air Force declared IOC with Block 3i-capable aircraft in
August 2016.
- Block 3F. The program designated Block 3F as the full
SDD warfighting capability for production Lot 9 and later.
Block 3F expands the flight envelope for all variants and
includes additional weapons, external carriage of weapons,
and the gun. Flight testing with Block 3F software on the
F-35 test aircraft first began in March 2015. Flight testing
of Block 3F mission systems software, version 3FR5, was
paused in February 2016 when the program discovered
that it was too unstable for productive flight testing. The
program elected to reload a previous version of Block 3F
software – version 3FR4 – on the mission systems flight
test aircraft, to allow limited testing to proceed. After
improving the flight stability of the Block 3i software, the
program applied the corrections to deficiencies causing
instabilities to the Block 3FR5 software and delivered
another version to flight test – version 3FR5.02 – in March,
to continue Block 3F testing. The program restarted
Block 3F testing in earnest in May with Block 3FR5.03
and released several more Quick Reaction Cycle (QRC)
versions, Blocks 3FR5.04 through 3FR5.07, through
November 2016 in attempts to quickly address key
deficiencies that were blocking test points. The program
delivered the final planned version of Block 3F software –

F-35 JSF

65

 FY16 DOD PROGRAMS
3FR6 – to flight testing in December 2016. The program
will then determine, with testing in early 2017, if additional
QRC patches will be adequate to meet specifications,
or if another full release of Block 3F software (e.g.,
3FR7) will be required. Of note, all of the aircraft from
earlier production lots, i.e., Lots 2 through 5 will need
to be modified, including structural modifications and
the installation of TR2 processors, to have full Block 3F
capabilities. The program plans to begin delivering Lot 9
aircraft in early CY17. The Program Office has agreed to
allow the initial Lot 9 aircraft to be delivered with Block
3i software. These provisional acceptances may continue
until August 2017, when the program plans to have Block
3FP8 – the first version of Block 3F production software –
for delivery of the remainder of Lot 9 and later aircraft.
- Block 4. The program has designated the first release
of added capabilities following completion of SDD
as Block 4, with four distinct increments (Blocks 4.1,
4.2, 4.3, and 4.4). Current program schedules plan for
testing of Block 4.1 to begin at the end of CY19 with
subsequent increments following at 2-year intervals.
Hardware upgrades are planned in Blocks 4.2 and 4.4,
and will include the next upgrade in processors with
open-architecture Technical Refresh 3 (TR3) processors.
Production cut-in for initial Block 4.1 capabilities is
planned with Lot 13, beginning delivery in 2021, and
Lot 15 for Block 4.2. The post-SDD development
program is referred to as Follow-on Modernization (FoM).
However, for reasons discussed elsewhere in this report,
the program’s initial FoM plan is not executable and is
being re-planned by the program and stakeholders.
Flight Test Activity with AF-3, AF-6, AF-7, BF-4, BF-5, BF-17,
BF-18, CF-3, CF-5, and CF-8 Flight Test Aircraft and Software
Development Progress
• Mission systems testing focused on:
- Attempting to resolve software stability problems with
Block 2B and Block 3i mission systems
- Block 3F mission systems development and testing
- Initial integration testing of the U.S. Navy Joint Standoff
Weapon, version C1 (JSOW-C1)
- Completing weapons separation testing for the Small
Diameter Bomb (SDB) version I (SDB-I), which requires
mission systems-capable aircraft for interfacing with the
SDB
- Weapons integration and testing of the United Kingdom
Paveway IV bomb and Advanced Short-Range Air-to-Air
Missile (ASRAAM); determining root cause and options
to fix ASRAAM integration deficiencies
- On-Board Inert Gas Generation System (OBIGGS) testing
on CF-8, the only F-35C test aircraft modified with the
necessary hardware to complete testing
- Regression testing of Block 2B software on operational
test aircraft (AF-21, AF-23, BF-16 and BF-20), since the
developmental test aircraft had all already been converted
to the Block 3i or Block 3F configuration

66

F-35 JSF

- Joint Precision Approach and Landing System (JPALS)
testing with CF-5
- Testing of the Gen III Helmet Mounted Display System
(HMDS) illumination settings during the third F-35C
developmental test period at sea, designed to correct
excessive “green glow” during night operations onboard
the carrier
- The six mission systems developmental flight test aircraft
assigned to the Edwards AFB test center flew an average
rate of 6.9 flights per aircraft, per month in CY16 through
November, slightly above the planned rate of 6.7 for the
year, and flew slightly more than the planned number of
flights (479 flights accomplished versus 474 planned).
Mission Systems Assessment
• Block 2B
- Although the program completed Block 2B mission
systems testing in 2015 and provided a fleet release
version of the software to the fielded units, deficiencies
remained and were carried forward into Block 3i. This
schedule-driven decision to pass deficiencies forward
had consequences. The many deficiencies, including
instabilities in both Block 3i and Block 3F mission
systems software, led the program to return to Block 3i
development to make corrections. When the revised
Block 3i software, Block 3iR6.21, demonstrated improved
inflight stability, the program developed and tested another
version of Block 2B software – version 2BS5.3 – with
the corrections to the stability deficiencies included. This
version was released to fielded units in May 2016 for the
F-35A and F-35B, and in August 2016 for the F-35C; the
program expects to complete retrofit of all fielded aircraft
in the Block 2B configuration with the Block 2BS5.3
software by the end of January 2017.
- Because the test center aircraft had all been upgraded to
the Block 3i/3F configuration (i.e., with the newer TR2
processors), flight testing of the Block 2BS5.3 software
occurred on OT aircraft assigned to the OT squadron at
Edwards AFB, California.
• Block 3i
- Block 3i began with the schedule-driven decision to rehost
the immature Block 2B software and capabilities into
new TR2 avionics processors. Because of the extreme
overlap of development and production, combined with
delays in software development, the program was forced
to create a Block 3i capability to support delivery of Lot
6 and later aircraft, as they were being delivered with the
new processors. Although the program originally intended
that Block 3i would not inherit technical problems from
earlier blocks, this is what occurred, resulting in severe
problems with Blocks 3i and 3F software that needed to be
addressed, affecting both Block 2B and Block 3i fielded
aircraft, and stalling the progress of mission systems
testing early in CY16.
- When Block 3i developmental flight testing began in
May 2014, six months later than planned in the program’s

 FY16 DOD PROGRAMS
Integrated Master Schedule (IMS), the combination of
rehosted, immature software and new processors resulted
in severe avionics stability problems that were significantly
worse than those in Block 2B. Continued delays in
completing Block 2B software development and testing
in support of the Marine Corps IOC, which was a priority
over Block 3i development for the program and the test
centers, combined with the severe stability problems with
the early versions of Block 3i software, caused several
pauses in early Block 3i flight testing. Block 3i flight
testing resumed again in March 2015 and was considered
to be complete in October 2015, eight months later than
planned in the IMS. Despite the continued problems
with avionics stability, sensor fusion, and other inherited
issues from Block 2B, the program terminated Block 3i
developmental flight testing in October 2015, and released
Block 3i software to the fielded units. This decision was
made in an attempt to meet the program’s unrealistic
schedule for completing development and flight testing of
Block 3F mission systems.
- The program created an initial version of Block 3F
software by adding the final required capabilities and
weapons to the problematic Block 3i software. However,
productive and efficient flight testing was not possible
due to inherited instabilities and other deficiencies. The
Air Force insisted on fixes for seven (five identified
in 2014 and two more in 2015) of the most severe
deficiencies inherited from Block 2B as a prerequisite
to use the final Block 3i capability in the Air Force IOC
aircraft. Consequently, in February 2016, the program
decided to return to Block 3i development and testing
in another attempt to fix key unresolved software
deficiencies, including the avionics instabilities troubling
both Block 3i and Block 3F. A new version of mission
systems software, Block 3iR6.21, was quickly developed
and tested, and showed improvement to several of the
“must fix” deficiencies identified by the Air Force and the
inflight stability problems, so it was released to the fielded
aircraft in late May 2016. Data collected on start-up and
inflight stability of the Block 3iR6.21 mission systems
software showed that both have improved over earlier
versions of Block 3i, and are approximately equivalent to
the final version of Block 2B software. Based on flights
conducted with the production software through the
end of October 2016, the Air Force reported that, of the
seven “must fix” deficiencies, five had been corrected,
one was partially corrected, but needed full Block 3F set
of capabilities to ensure full implementation, and one –
associated with extended post-mission download times
from the aircraft’s portable memory device (PMD) – was
awaiting fielding of an upgraded ground data receptacle
(see more detail in the ALIS section below).
• Block 3F
- Block 3F flight testing began in March 2015, six months
later than the date planned in the IMS.

- The emphasis on, and return to, Block 3i testing in March
and April 2016 contributed in part to the program’s
inability to progress with Block 3F flight testing at the
planned rate. As of the end of November, a total of 975
Block 3F baseline test points had been completed in CY16,
compared to 1,189 planned (82 percent of planned). An
additional 1,784 development and regression points were
flown, 1,534 of which were accounted for in the budgeted
non-baseline points for the year and 250 representing
growth.
- The lag in completing baseline test points – which are used
to verify capability – is also due to the program delivering
Block 3F software to flight test that was not mature enough
to meet specification compliance, or because deficiencies
prevent the specification from being met. In an attempt
to address the deficiencies and the lack of maturity in the
software, the program began developing and delivering
QRC versions of software to flight test. These software
versions are built, lab tested, and delivered to flight test
on a shorter timeline than the originally planned series of
software versions for Block 3F.
- Delays in starting Block 3F testing, pausing to redo
Block 3i work, and the immaturity of the Block 3F
software delivered to flight test have all contributed to the
program being well behind the plan to complete Block 3F
flight testing by the end of July 2017, the forecasted
completion date according to the program’s most recent
Mission Systems Software and Capability Release
Schedule. Instead, DOT&E estimates the program will
likely not finish Block 3F development and flight testing
prior to July 2018, based on the following:
▪ Continuing a 6.5 test point per flight accomplishment
rate, which is the CY16 rate observed through the end of
November.
▪ Continuing a flight rate of 6.9 flights per aircraft per
month, as was achieved through the end of November.
▪ Completing all of the baseline test points
(3,645 remaining as of the end of November) and
experiencing a regression, development and discovery
test point work load of 63 percent (historical average,
but well below the rate of 83 percent experienced in
CY16 through November).
- The program plans to truncate the planned testing by
eliminating test points, instead using alternative test
points or old data, in order to meet schedule deadlines
with the expectation of finishing SDD, getting to IOT&E,
and starting full-rate production. While this approach
may provide a quick sampling assessment of Block 3F
capabilities, there are substantial risks. The multiple
recent software versions for flight test may prevent the
program from using data from older versions of software
to count for baseline test point deletions because it may no
longer be representative of Block 3F. Limited availability
and high cost of range periods, combined with high re-fly
rates for test missions completed on the Western Test

F-35 JSF

67

 FY16 DOD PROGRAMS
Range, make it difficult for the program to efficiently
conduct this testing. Finally, the most complex capabilities
in Block 3F have only recently reached the level of
maturity to allow them to be tested, and they are also
some of the most difficult test points to execute (i.e., full
Block 3F capabilities and flight envelope). Such a risky
course of action, if not properly executed with applicable
data, sufficient analytical rigor and statistical confidence,
would likely result in failures in IOT&E causing the need
for additional follow-on operational testing, and, most
importantly, deliver Block 3F to the field with severe
shortfalls in capability – capability that the Department
must have if the F-35 is ever needed in combat against
current threats. In fact, the plan to eliminate or replace test
points is at a point in the development program where the
most difficult, yet some of the most important capabilities,
have just started to reach maturity to begin flight testing.
The program should complete testing of all necessary
Block 3F baseline test points, as defined in the Joint
Test Plans; if the program attempts to use test data from
previous testing or added complex test points to sign off
some of these test points, the program must ensure the data
are applicable and provide sufficient statistical confidence
prior to deleting any underlying build-up test points.
Additionally, the program should consider adding another
full version of Block 3F software to develop and deliver to
flight test in order to address more deficiencies.
- Deficiencies in performance and significant operational
shortfalls must be resolved if the program is to deliver
the expected full Block 3F capability by the end of
SDD. Based on operational test pilot observations of
developmental test missions flown in June and July 2016,
an assessment of the operational utility of Block 3FR5.03
software to support planned IOT&E missions, including
Close Air Support, Suppression/Destruction of Enemy
Air Defenses, Offensive and Defense Counter-Air,
Air Interdiction, and Surface Warfare, rated each of
the mission areas “red” and unacceptable overall.
Additionally, the JOTT provided an assessment of the
Block 3F capabilities, based on observing and assisting
with F-35 developmental testing with Block 3FR5.05
software, which began flight testing in August. The team’s
assessment made top-level, initial predictions of expected
IOT&E results of the F-35 for each of the mission areas.
The team predicted severe or substantial operational
impacts across all the planned IOT&E missions, similar
to the list of missions above, due to shortfalls and
deficiencies, with the exception of the Reconnaissance
mission area, which predicted minimal operational impact.
The program should ensure adequate resources remain
available (personnel, labs, flight test aircraft) through
the completion of IOT&E to develop, test and verify
corrections to deficiencies identified during flight testing
that may cause operational mission failures during IOT&E
or in combat.

68

F-35 JSF

- The program plans to provide full Block 3F capability,
as defined in the TEMP, with the first Lot 10 aircraft
delivery in January 2018. In fact, as required by the
National Defense Authorization Act (NDAA) for FY16,
the Secretary of the Air Force certified to Congress in
September 2016 that these aircraft will have full combat
capability, as determined as of the date of the enactment
of the NDAA, with Block 3F hardware, software, and
weapons carriage. However, for many reasons, it is clear
that the Lot 10 aircraft will not initially have full Block 3F
capability. These reasons include, but are not limited to,
the following:
▪ Envelope limitations will likely restrict carriage and
employment of the AIM-120 missile and bombs well
into 2018, if not later.
▪ The full set of geographically specific area of
responsibility MDLs will not be complete, i.e.,
developed, tested and verified, until 2019, at the soonest,
due to the program’s failure to provide the necessary
equipment and software tools for the USRL.
▪ Even after they are delivered, the initial set of MDLs
will not be tested and optimized to deal with the full set
of threats present in operational test, let alone in actual
combat, which is part of full combat capability.
▪ The program currently has more than 270 Block 3F
unresolved high-priority (Priority 1 and Priority 2, out of
a 4-priority categorization) performance deficiencies, the
majority of which cannot be addressed and verified prior
to the Lot 10 aircraft deliveries; less than half of these
deficiencies were being actively worked in Block 3F.
▪ The program currently has 17 known and acknowledged
failures to meet the contract specification requirements,
all of which the program is reportedly planning to get
relief from the SDD contract due to lack of time and
funding.
▪ Dozens of contract specification requirements are
projected to be open into FY18; these shortfalls in
meeting the contract specifications will translate into
limitations or reductions to full Block 3F capability.
▪ Estimates to complete Block 3F mission systems that
extend into the summer of 2018 have been put forth
not just from DOT&E, but also from other independent
Department agencies (e.g., CAPE), affirming that
delivery of full capability in January 2018 will be nearly
impossible to achieve, unless testing is prematurely
terminated, which would increase the likelihood that the
full Block 3F capabilities will not be adequately tested
and priority deficiencies fixed.
▪ Deficiencies continue to be discovered at a rate of about
20 per month, and many more will undoubtedly be
discovered before and during IOT&E.
▪ ALIS version 3.0, which is necessary to provide full
combat capability, will not be fielded until mid-2018,
and a number of capabilities that had previously been
designated as required for ALIS 3.0 are now being

 FY16 DOD PROGRAMS
deferred to later versions of ALIS (i.e., after summer of
2018).
▪ The Department has chosen to not fund the program
to the CAPE estimate that the completion of Block 3F
mission systems testing will last until mid-2018, a time
span which is much later than, and at a cost that is at
least double, the Program Office’s latest unrealistic
estimate to complete SDD. This guarantees the program
will attempt a premature resource- and schedule-driven
shutdown of mission systems testing which will
increase the risk of mission failures during IOT&E and,
more importantly, if the F-35 is used in combat.
▪ Finally, rigorous operational testing in IOT&E, which
provides the most credible means to predict combat
performance in advance of actual combat, will not be
completed until at best the end of 2019 – and more
likely later.
Assessment of Block 2B and 3i “Initial Warfighting” Fielded
Capability
• Using aircraft in the Block 2B configuration, both the
Air Force, with the F-35A, and the Marine Corps, with
the F-35B, have flown simulated combat missions during
training or in support of training exercises. These training
missions have highlighted numerous shortfalls in Block 2B
capability.
- Unlike legacy aircraft, Block 2B aircraft will need
to make substantial use of voice communications to
receive targeting information and clearance to conduct
an attack during Close Air Support (CAS) missions due
to the combined effects of digital data communications
deficiencies, lack of infrared pointer capability, limited
ability to detect infrared pointer indications from a
controller (which may be improved in the Generation
III Helmet Mounted Display System (Gen III HDMS)),
and inability to confirm coordinates loaded to GPS-aided
weapons. Each of these shortfalls limit effectiveness and
increase the risk of fratricide in combat.
- Many pilots assess and report that the Electro-Optical
Targeting System (EOTS) on the F-35 is inferior to those
currently on legacy systems, in terms of providing the
pilot with an ability to discern target features and identify
targets at tactically useful ranges, along with maintaining
target identification and laser designation throughout the
attack. Environmental effects, such as high humidity,
often forced pilots to fly closer to the target than desired
in order to discern target features and then engage for
weapon employment, much closer than needed with
legacy systems, potentially alerting the enemy, exposing
the F-35 to threats around the target area or requiring
delays to regain adequate spacing to set up an attack.
However, due to design limitations, there are no significant
improvements to EOTS planned for Block 3F.
- When F-35 aircraft are employed at night in combat, pilots
are restricted from using the current limited night vision
camera in the Generation II helmet with Block 2B aircraft.
This restriction does not apply to pilots equipped with the

Generation III helmet, which is fielded with the Block 3i
aircraft. In general, if used in combat, pilots flying
Block 2B aircraft would operate much like early fourth
generation aircraft using cockpit panel displays, with the
Distributed Aperture System providing limited situational
awareness of the horizon, and heads-up display symbology
projected on the helmet.
• Because Block 3i is an interim capability based on Block 2B,
it inherited numerous limitations that will reduce operational
effectiveness and require workarounds if F-35 in the Block 3i
configuration are used in combat. The Air Force conducted
an IOC Readiness Assessment (IRA), using F-35A aircraft
with four different versions of Block 3i mission systems
software. Based on observations from fielded units and
from the Air Force’s IRA, the following mission areas
will be affected by limitations, which may affect overall
effectiveness:
- Close Air Support (CAS). In many ways, the F-35 in
the Block 3i configuration does not yet demonstrate
CAS capabilities equivalent to those of fourth generation
aircraft. The F-35A in the Block 3i configuration has
numerous limitations that make it less effective overall
in the CAS mission role than most currently fielded
fighter aircraft like the F-15E, F-16, F-18 and A-10 in a
permissive or low-threat environment, which is where
CAS is normally conducted. These limitations, consistent
with observations made by the Air Force in its IRA report,
include:
▪ The limited weapons load of two bombs (along with two
missiles for self-defense) constrains the effectiveness of
the Block 3i F-35 for many CAS missions. Compared
to a legacy fighter with multiple weapons on racks, and
multiple weapons types per aircraft, the limited Block 3i
load means that only a limited number and type of
targets can be effectively attacked.
▪ No gun capability. An aircraft-mounted gun is a key
weapon for some CAS scenarios when a bomb cannot
be used due to collateral damage concerns or when
the enemy is “dangerously close” to friendly troops.
The gun can also be an effective weapon for attacking
moving targets. However, even though an internal gun
is installed in the Block 3i F-35A, it cannot be used
until significant modifications to both the gun system
and aircraft are completed, and a version of Block 3F
software is tested and delivered to fielded aircraft. Gun
weapons delivery accuracy (WDA) testing, aimed by the
HMDS, with the required modifications and software,
has slipped from September 2016 to early 2017. Initial
build-up testing for the gun WDA was being planned for
December 2016 at the time of writing this report.
▪ Limited capability to engage moving targets. Even
though the Block 3i F-35A does not have a functioning
gun, it can carry the GBU-12 laser guided bomb which
has limited moving target capability. However, Block
3i (and Block 3F because it is currently not planned
to be addressed) does not have an automated targeting

F-35 JSF

69

 FY16 DOD PROGRAMS
function with lead-laser guidance (i.e., automatically
computing and positioning the laser spot proportionately
in front of the moving target to increase the likelihood
of hitting the target) to engage moving targets with the
GBU-12, like most legacy aircraft have that currently
fly CAS missions. Instead, F-35 pilots can only use
basic rules-of-thumb when attempting to engage moving
targets with the GBU-12, resulting in very limited
effectiveness. Also, limitations with cockpit controls
and displays have caused the pilots to primarily use
two-ship “buddy lasing” for GBU-12 employment,
which is not always possible during extended CAS
engagements when one of the aircraft has to leave to
refuel on a tanker. To meet the ORD requirement for
engaging moving targets, the Air Force is considering
integrating the GBU-49, a fielded weapon that has
similar size, weight and interfaces as the GBU-12, or a
similar weapon that does not require lead-laser guidance,
in Block 3F. Otherwise, the program plans to develop
and field lead-laser guidance in Block 4.2, which
would be delivered in CY22, at the earliest. However,
because of the similarities, the GBU-49 could be quickly
integrated with Block 3F to provide a robust moving
target capability for the F-35 much earlier.
▪ Voice communications are sometimes required to
validate digital communications. Problems with
Variable Message Format (VMF) and Link-16 datalink
messaging – including dropped or hidden information
or incorrect formats – sometimes require pilots to
use workarounds by validating or “reading back”
information over the radio that prevent them from
conducting digital (only) CAS, a capability that is
common in most legacy CAS aircraft. Recent use
of VMF digital communications during weapons
demonstration events by the operational test teams
has been more successful; however, data analyses are
ongoing.
▪ Limited night vision capability. Although Lot 7 and
later aircraft are fielded with the Gen III HMDS, which
has shown improvement to the deficiencies with the
earlier Gen II HMDS, limitations with night vision
capability remain. Pilots using the Gen III helmet for
night operations report that visual acuity is still less than
that of the night vision goggles used in legacy aircraft,
which makes identification of targets and detecting
markers more difficult, if not impossible. Also, “green
glow” – a condition where light leakage around the
edge of the display during low-light conditions makes
reading the projected information difficult – is improved
over the Gen II HMDS, but is still a concern during low
ambient illumination conditions. The program currently
has two open “Category 1 High” deficiency reports for
“green glow,” with the most significant safety concerns
pertaining to nighttime carrier operations.
▪ Lack of target marking capability – a key capability
for both Forward Air Controller-Airborne (FAC-A)

70

F-35 JSF

and CAS missions. Legacy CAS platforms can mark
targets with rockets, flares, and/or infrared (IR) pointers,
none of which are currently available on the F-35. The
F-35 has a laser designator as part of its Electro-Optical
Targeting System (EOTS), but the laser is used for
targeting from ownship when using the GBU-12 laser
guided bomb or to “buddy-guide” a weapon from
another aircraft. This limitation is not planned to be
fixed during SDD.
- Other mission areas. In addition to the Block 3i
limitations listed above that affect the CAS mission area,
the following inherent Block 3i limitations will also affect
the capability of the F-35 in other mission areas:
▪ Poor ability to accurately locate (i.e., determine
geographic location with precision needed for weapons
employment) and identify threat emitters.
▪ No standoff weapon. With only direct attack bombs,
the F-35 in the Block 3i configuration will be forced
to fly much closer to engage ground targets and,
depending on the threat level of enemy air defenses and
acceptable mission risk, it may be limited to engaging
ground targets that are defended by only short-range air
defenses, or by none at all.
▪ The limited weapons loadout of the Block 3i F-35 makes
effective attack of many expected types of targets in a
typical theater a challenge. For example, unlike legacy
aircraft, the Block 3i F-35 has no mixed weapons load
capability, which limits flexibility to attack targets with
appropriately matched weapons. Block 3i F-35 aircraft
can only employ two internally carried bombs, and
although internal carriage reduces the susceptibility of
the F-35 relative to legacy aircraft, by virtue of the low
observability it provides, it does not provide the ability
to attack more than one or two targets.
▪ Pilots report that inadequacies in Pilot Vehicle Interfaces
(PVI) in general, and deficiencies in the Tactical
Situation Display (TSD) in particular, which displays
the results of sensor fusion and is designed to provide
increased situation awareness, continue to degrade
battlespace awareness and increase pilot workload.
Workarounds to these deficiencies are time-consuming
for the pilot and detract from efficient and effective
mission execution.
- Block 3i has significant deficiencies that must still be
addressed, despite the additional software release to the
field, Block 3iP6.21, in May 2016. In addition to the
limitations listed above, Block 3i also has hundreds of
other deficiencies, the most significant of which must be
fixed in Block 3F to realize the full warfighting capability
required of the F-35. These deficiencies include, but are
not limited to, the following:
▪ Avionics sensor fusion performance is still unacceptable.
» Air tracks often split erroneously or multiple false
tracks on a single target are created when all sensors
contribute to the fusion solution. The workaround
during early developmental testing was to turn off

 FY16 DOD PROGRAMS
some of the sensors to ensure multiple tracks did
not form, which is unacceptable for combat and
violates the basic principle of fusing contributions
from multiple sensors into an accurate track and clear
display to gain situational awareness and to identify
and engage enemy targets.
» Similarly, multiple false ground tracks often are
displayed when only one threat emitter is operating.
In addition, tracks that “time out” and drop from the
display cannot be recalled, which can cause pilots
to lose tactical battlefield awareness on enemy air
defense radars that turn on only intermittently, as is
typical of missile engagement radars.
» Sharing erroneous tracks over the Multifunction
Advanced Data Link (MADL) between aircraft in
the F-35 formation multiplies the problems described
above.
» The Air Force IOC Readiness Assessment (IRA)
report also identified deficiencies with fusion in
Block 3i.
▪ Electronic warfare (EW) capabilities, including
electronic attack (EA), are inconsistent and, in some
cases, not effective against required threats.
» Although the details of the deficiencies are classified,
effective EW capabilities are vital to enable the F-35
to conduct Suppression/Destruction of Enemy Air
Defenses (SEAD/DEAD) and other missions against
fielded threats.
» The Air Force IRA report also identified significant
EW deficiencies in Block 3i.
▪ Datalinks do not work properly. Messages sent across
the MADL are often dropped or pass inaccurate offboard
inter-flight fusion tracks based on false or split air tracks
and inaccurate ground target identification and positions.
▪ Reduced on-station time and greater reliance on tanker
aircraft. Although this limitation is not unique to the
Block 2B or Block 3i configuration, the F-35 has high
fuel burn rates and slow air refueling rates that extend
air refueling times and decrease overall on-station time,
which may reduce overall mission effectiveness.
- The program was able to improve stability of the mission
systems software to support the Air Force’s plan to declare
IOC. The Program Office reported improvements in Mean
Flight Hours Between Instability Events (MFHBIE) for
both start-up and in-flight of Block 2B and Block 3i. The
latest inflight stability metrics from the Program Office
are provided in the table to the right. Note that “2BS”
versions of software refer to Block 2B versions delivered
to flight test. For Block 3i, the program adopted a naming
convention where a “P” version refers to software released
for production aircraft and an “R” version is for flight
testing. An “R” version of software has additional coding
that permits data to be collected from data buses on the
aircraft and stored on the DART pod or transmitted to
ground stations for recording or playback. For IOT&E,
since data will be collected with the instrumentation

packages on the OT aircraft, IOT&E will be flown with an
“R” version of software where selected data and messages
can be directed for recording for post-flight analyses.
- The operational effect of mission systems software
instabilities on the F-35 will not be well understood
before the completion of formal operational testing. One
of the objectives of the Air Force IRA was to examine
the frequency and effect of these instability events. The
Air Force defined and scored instability events during
the IRA in the same way as the Program Office and the
contractor for comparison purposes and observed similar
trends. An instability event is generally the initial failure,
or the primary system failure, and does not account for
subsequent failures of the same system or failures of
subsystems. In addition, the Air Force collected data on
instability occurrences, which includes a broader set of
instabilities. An instability occurrence accounts for all
failures of systems and associated subsystem failures,
when each of the failures could have affected the mission
capability of the aircraft. The Air Force collected data on
instability occurrences with F-35A aircraft flying the most
current Block 3i software and counted 25 occurrences
in 34.1 flight hours, resulting in a Mean Flight Hours
Between Instability Occurrences of 1.4 hours. During
IOT&E, all relevant stability events and occurrences,
on the ground or in the air, which impact mission
effectiveness or suitability, including repeat events (unless
attributed to a hardware failure) will be counted to assess
overall mission effect. Similar to the table below, stability
data from IOT&E will be compared with data from fielded
aircraft with the “P” version of Block 3F software to assess
any differences.
• The Air Force IRA test team at Nellis AFB flew a total of 18
mission scenarios (72 aircraft sorties) covering the mission
sets of CAS, Air Interdiction (AI), and SEAD/DEAD.
The missions were flown over the Western Test Ranges
from March 1 through April 29, 2016. Additionally, the
assessment included observations from an Air Force-led
deployment to Mountain Home AFB, Idaho, with six F-35A
MISSION SYSTEMS SOFTWARE INFLIGHT STABILITY METRICS
(DATA AS OF NOVEMBER 27, 2016)
Software
Release

Number of
Inflight
Stability Events

Cumulative
Flight Hours

Mean Flight
Hours Between
Instability Events

2BS5.2

31

224.8

7.3

2BS5.3

1

28.5

Insufficient data

3iP6.21

13

349.5

26.9

3iR6.21 (Edwards
OT Aircraft)

6

75.8

12.6

3FR5*

222

950.1

4.3

* 3FR5 metrics are a summation of 8 versions of software used in flight testing: 3FR5,
3FR5.02, 3FR5.03, 3FR5.03QRC, 3FR5.04QRC, 3FR5.05, 3FR5.06, and 3FR5.07

F-35 JSF

71

 FY16 DOD PROGRAMS
aircraft from Edwards, supported by an ALIS SOU v2 with
software 2.0.1. Although the Air Force has determined that
the F-35A with Block 3i mission systems software provides
“basic” capabilities for IOC, many significant limitations
and deficiencies remain. In comparison to a dedicated
operational test and evaluation, this was a brief, but
revealing assessment of mission capability. However, until a
full operational test and evaluation of the F-35 is completed,
we will have low confidence that we understand all of the
limitations in the system.
- The detailed results of the IRA, as reported by the Air
Force, are consistent with the assessments in this Annual
Report.
- Inflight stability of the Block 3i mission systems was
assessed to be back to a level comparable to that in
Block 2B, as measured by the number of inflight
instability events per flight hour.
- If used in combat, F-35 aircraft will need support to locate
and avoid modern threat ground radars, acquire targets,
and engage formations of enemy fighter aircraft, due to
unresolved performance deficiencies and limited weapons
carriage available (i.e., two bombs and two air-to-air
missiles).
- Unresolved Block 3i deficiencies in fusion, EW, and
weapons employment continue to result in ambiguous
threat displays, limited ability to effectively respond to
threats, and, in some cases, a requirement for offboard
sources to provide accurate coordinates for precision
attack.
- Concerning the CAS mission area, the team concluded that
the Block 3i F-35A does not yet demonstrate equivalent
CAS capabilities to those of fourth generation aircraft.
Mission Data Load Development and Testing
• F-35 effectiveness in combat relies on mission data loads
(MDL) – which are a compilation of the mission data files
needed for operation of the sensors and other mission
systems – working in conjunction with the system software
data load to drive sensor search parameters so that the F-35
can identify and correlate sensor detections, such as threat
and friendly radar signals. The contractor team produced
an initial set of mission data files for developmental testing
during SDD, but the operational MDLs – one for each
potential major geographic area of operation – are being
created, tested, and verified by a U.S. government lab, the
U.S. Reprogramming Lab (USRL), located at Eglin AFB,
Florida, which is operated by government personnel from the
Services. The Air Force is the lead Service. These MDLs
will be used for operational testing and fielded aircraft,
including the Marine Corps and Air Force IOC aircraft. The
testing of the USRL MDLs is an operational test activity,
as was arranged by the Program Office after the restructure
that occurred in 2010. The Department must have a
reprogramming lab that is capable of rapidly creating, testing
and optimizing MDLs, and verifying their functionality
under stressing conditions representative of real-world

72

F-35 JSF

scenarios, to ensure the proper functioning of F-35 mission
systems and the aircraft’s operational effectiveness in both
combat and the IOT&E of the F-35 with Block 3F.
• Despite the critical requirement for developing and fielding
F-35 MDLs, significant ongoing software and hardware
deficiencies in the USRL have yet to be addressed,
which continue to prevent efficient creating, testing, and
optimization of the MDLs for operational aircraft fielded in
the Block 2B and Block 3i configuration, and are preventing
the development of MDLs for Block 3F.
- The current reprogramming hardware and software tools
are so cumbersome that it takes months for the USRL
to create, test, optimize, and verify a new MDL. This
time-consuming process was still not complete for the
complete set of Block 3i AOR-specific MDLs.
- The program has mismanaged sustainment and upgrades
of the USRL to the point that it currently does not have the
ability to start creating MDFs for Block 3F and will not
have that capability until February 2017, at the earliest.
Once the USRL can start creating Block 3F MDFs, it will
take approximately 15 months to deliver a verified MDL for
IOT&E and for fielded Block 3F aircraft.
- The program plans to start delivering production aircraft
in the Block 3F configuration in May 2017. Because the
USRL will not be able to develop, test, and validate a
Block 3F MDL until mid-2018, the Services will have to
field Block 3F-capable aircraft with either Block 3i, or with
a Block 3F test MDL provided by the contractor; however,
either course of action will likely restrict these fielded
Block 3F aircraft from use in combat.
• Additionally, the Program Office and Lockheed Martin have
failed to complete necessary contracting actions to address
current shortfalls in signal generation capability within
the USRL, including the key hardware upgrades needed to
create, test, and verify Block 3F MDFs to detect and identify
emissions from currently fielded threat systems in scenarios
with realistic threat densities. This failure occurred in spite
of the requirement being clearly identified in 2012 and
the Department programming $45 Million in the FY13-16
budgets to address it. The JPO sponsored a gap analysis
study of USRL capabilities to determine the lab upgrade
requirements at the engineering level before beginning
contracting actions. When completed in 2014, the study
concluded that between 16 and 20 upgraded radio frequency
(RF) signal generator channels would be needed for the USRL
to adequately create and test MDFs in the USRL for the
fielded threats examined in the study, using realistic scenarios
and threat densities. After receiving a proposal for the
upgrades from the contractor priced at over $200 Million in
May 2016, the JPO requested a new proposal, reportedly with
options only for up to 12 upgraded signal generator channels,
which the contractor indicated would not be answered until
July 2017. Furthermore, once on contract, it would then take
approximately 3 years after ordering the equipment for it
to be delivered and installed, which will be late to need for

 FY16 DOD PROGRAMS
both IOT&E and fielding of Block 3F aircraft. As a result,
even though the USRL will eventually have the capability
to create MDLs for Block 3F in 2017, it still will not have
the required signal generators to test and optimize the MDLs
to ensure adequate performance against currently fielded
threats.
• To provide the necessary and adequate Block 3F
mission data development capabilities for the USRL, the
Program Office must immediately fund and expedite the
contracting actions for the necessary hardware and software
modifications, including an adequate number of additional
RF signal generator channels and the other required
hardware and software tools. Unless these actions are taken
immediately, the USRL will not be configured to create,
test, and verify Block 3F MDLs for aircraft for current
threat systems and threat scenarios until sometime in 2020,
placing the operational aircraft at risk in combat against
fielded threats and the program at risk of failing IOT&E.
The program is working to find alternative facilities with
the required signal generators to mitigate this lab capability
shortfall for Block 3F.
• Significant additional investments are also required
within 2-3 years to further upgrade the USRL to support
F-35 Block 4 Follow-on Modernization (FoM) MDL
development. Block 4.2 is currently planned to include
new Technical Refresh 3 (TR3) processors and other new
hardware which, due to the overlapping Block 4 increments,
will require the USRL, or an additional reprogramming lab,
to have two different avionics configurations simultaneously
– a TR2 line for Blocks 3F and 4.1, plus a TR3 line for
Block 4.2 and later. Although the Block 4 hardware
upgrades in the USRL will need to begin soon to be ready in
time, the reprogramming requirements for Block 4 have yet
to be fully defined. The Program Office must expeditiously
undertake the development of those requirements and plan
for adequate time and resources within the DOD budget
cycle, in order to ensure the USRL is able to meet Block 4
MDL requirements.
• The USRL, with JOTT observers, held an “Urgent
Reprogramming Exercise (URE)” from April 20 to July 25,
2016. This type of exercise is intended to test the USRL’s
ability to respond to an urgent request from a Service to
modify the mission data in response to a new threat or
new mode of an existing threat. Due to USRL’s ongoing
production efforts, the URE was conducted concurrently
with the lab’s effort to produce an operational MDL, which
is why the exercise period was several months, instead of
a few days. The JOTT and USRL carefully tracked hours
that were specific to the URE as they occurred and surveyed
USRL personnel to identify process issues. The total hours
recorded were double the Air Force standard for rapidly
reprogramming a mature system. The JOTT identified
several key process problems, many of which are described
above, including the lack of necessary hardware, analysis
tools that were not built for operational use, and missing
capabilities, like the ability to quickly determine ambiguities

in the mission data. These problems must be corrected in
order to bring the USRL’s ability to react to new threats up to
the identified standards routinely achieved on legacy aircraft.
• In addition to the above deficiencies that involve overall
laboratory capability and tools to develop MDLs, there are
also deficiencies in the program’s sustainment efforts to
ensure a high state of readiness, particularly if the Services
have an urgent reprogramming requirement at any time.
To meet these tasks, the USRL must have all necessary
equipment in a functioning status, similar to aircraft
availability. Inadequacies in the current level of sustainment
include, but are not limited to:
- Insufficient number of Field Service Engineers (FSE) to
assist in maintenance and operation of the lab equipment,
which include both specialized equipment and aircraft
mission equipment
- Inadequate or insufficient training for most laboratory
personnel, which is hindered by the insufficient number of
FSEs
- No engineering drawings or JTD for many critical
components, making troubleshooting of failures of those
components difficult and lengthening the time required to
return the laboratory to full operational status
- Insufficient spare parts for many critical components
- Low supply priority, equivalent to that of a unit in training,
resulting in long delays to receive required parts
- Missing part numbers for many components, forcing
USRL personnel to submit an Action Request (AR) first to
determine the part number before a replacement part can
be ordered through supply.
Weapons Integration and Demonstration Events
Block 3F Developmental Testing
• After the release of Block 3iP6.21 software in May 2016,
the program focused on completing development of Block
3F capabilities, including weapons envelope and integration
testing. To provide an operational employment flight
envelope, the program accomplished flight sciences testing
of external weapons carriage and employment, as well as
integrating bombs (SDB-I, JSOW C-1, and PW-IV) and
missiles (AIM-9X and AIM-132 ASRAAM) not previously
integrated on the F-35 in Block 2B or 3i.
• The TEMP requires 26 Block 3F weapons delivery
accuracy (WDA) events be completed as part of the Block
3F developmental testing effort. These WDAs are key
developmental test activities necessary to ensure the full
Block 3F fire-control capabilities support the “find, fix,
track, target, engage, assess” kill chain. As of the end of
November, only 5 of the 26 events (excluding the gun
events) had been completed and fully analyzed. Several
WDAs have revealed deficiencies and limitations to weapons
employment. An additional 11 WDAs have occurred, but
analyses are ongoing. Of the 10 remaining WDAs, 4 are
still blocked due to open deficiencies that must be corrected
before the WDA can be attempted. The program should
correct deficiencies that are preventing completion of all of

F-35 JSF

73

 FY16 DOD PROGRAMS
the TEMP-required Block 3F WDA events and ensure they
are completed prior to finishing SDD.
• Discoveries from the Block 3F WDA events include:
- AIM-9X and AIM-132 ASRAAM seeker status tone
problems
- Out-of-date launch zones for AIM-120 missiles
- Pilot Vehicle Interface (PVI) and mission planning
problems with the U.S. Navy’s JSOW-C1 missile that, if
not corrected, may cause significant weapon employment
limitations in the fleet’s ability to attack moving ship
targets and enable flexible engagement of land-based
targets of opportunity
- Ongoing radar and fusion deficiencies affecting air-to-air
target track stability and accuracy, which could cause
reduced missile lethality
- Multiple hung stores, which typically result in an inflight
emergency, occurred with the AIM-9X due to mission
systems software and weapon integration deficiencies
- Problems with integrating the British AIM-132 ASRAAM
missile and Paveway IV bomb; changes to address these

74

F-35 JSF

problems could have unintentionally affected the U.S.
AIM-9X and laser-guided bomb capabilities, which may
require regression testing of these U.S. weapons.
• In an effort to efficiently accomplish the WDA events, the
program dedicated several test aircraft to a WDA surge
period during June through August. Although the program
had planned to begin WDA events as early as February 2016,
the first live weapons event did not occur until July. Delays
in starting the Block 3F WDAs were caused by immature
software and deficiencies affecting weapons employment.
The following table lists the Block 3F WDA events, software
versions, scheduled and completion dates, overall results
and assessments for each completed live fire event through
the end of November. Many of the events were originally
blocked from completion due to software deficiencies that
had to be addressed using QRC versions of software in order
to allow the weapons events to proceed.

 FY16 DOD PROGRAMS
Block 3F Developmental Testing Weapons Events Accomplished Through November 2016
WDA Number

Weapon Event

Software
Configuration

301

AMRAAM

3FR5.03

Scheduled Date
Completion Date
Feb 16
Jul 16
Feb 16

302

AMRAAM with
AIM-9X

3FR5.03
Jul 16

Result

Assessment

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

Initial data analysis indicates that there was an
inflight issue that may have affected targeting
accuracy. Analysis in process to determine the
root cause and impact(s).

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

Initial data review indicated that the AIM-9X
tones were not as expected and there was no
missile post-launch timer indication to the pilot.

Partially successful
accomplishment; shot
captured key radar capability
data but failed primary test
objective; shot required
control room intervention.

Known issues with outdated F-35 AMRAAM
Attack Model in mission systems software
resulted in no shoot cues or dynamic launch
zone displayed to pilot requiring the control
room to provide a “shoot” call to the pilot. Initial
data review indicates that there was also no
post-launch timer indication to the pilot. Also,
weapon quality track was erratic pre- and
post-launch. More detailed analyses are pending,
following data to be provided by the missile
vendor.

Partially successful
accomplishment; shot
required control room
intervention.

The cockpit indication was a guidance failure
on the missiles and required control room
intervention to confirm the shot parameters and
direct the pilot to shoot. More detailed analyses
are pending, following data to be provided by
the missile vendor.

Successful accomplishment
of event.

All weapons initially appear to have functioned
successfully. Analysis ongoing.

Pending Data Review; shot
required control room
intervention.

Unsuccessful; also the pilot indications in the
cockpit indicted a guidance fail resulting in
control room intervention to accomplish the
shot. More detailed analyses are pending,
following data to be provided by the missile
vendor.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

Inflight weapon failed on first missile attempt
(built-in test failure and no missile tone to the
pilot); back-up missile functioned as expected.
Deficiency report was written on missile tone
anomalies.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

Initial data review indicates that the missile
tones were not correct, no dynamic launch
zone indication in Dogfight mode and the
gun symbology occluded the target in the
helmet-mounted display. More detailed analyses
on radar track accuracy and radar ranging
accuracy following data to be provided by the
missile vendor.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

The test team planned to use a known
workaround for minor Launch Acceptability
Region (LAR) inaccuracy due to an outdated
LAR model in mission systems software. Pilot
released the bomb using a “rule of thumb”
guidance to determine “in-zone.” JDAM LAR
model update in mission systems software is
required.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

The test team planned to use a known
workaround for a minor LAR inaccuracy due
to an outdated LAR model in mission systems
software. Pilot released the bomb using a “rule
of thumb” guidance to determine “in-zone.”
Post-mission initial data review indicates that the
target elevation values available to the pilot were
not consistent between the mission planned
terrain elevation, the displayed elevation on the
cockpit displays, and the value loaded into the
JDAM in the transfer alignment.

Feb 16

303

AMRAAM fired
with target
off-boresight

3FR5.03
Aug 16

Jun 16
307

2 X AMRAAM

3FR5.03
Aug 16

308

2 X SDB-I
(GBU-39) and 1 X
AMRAAM

Jun 16
3FR5.06
Nov 16
Apr 16

311

2 X AMRAAM

3FR5.03
Jul 16

316

AIM-9X fired
against a nonmaneuvering
target

Feb 16
3FR5.03
Jul 16
Jun 16

317

320

AIM-9X fired
against a
maneuvering
target

JDAM (GBU-31)
delivered against
a single target
using Synthetic
Aperture Radar
(SAR) map
coordinates

3FR5.03
Aug 16

Feb 16

3FR5.03
Jul 16

Apr 16

321

JDAM (GBU-31)
delivered against
a single target
using Bombon-Coordinate
employment

3FR5.03
Jul 16

F-35 JSF

75

 FY16 DOD PROGRAMS
Block 3F Developmental Testing Weapons Events Accomplished Through November 2016 (CONTINUED)
WDA Number

Weapon Event

Software
Configuration

Scheduled Date
Completion Date

Result

Assessment

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

The test team planned to use a known
workaround for a minor LAR inaccuracy due to an
outdated LAR model in mission systems software.
Pilot released the bomb using a “rule of thumb”
guidance to determine “in-zone.” Pilot released
weapons on rule-of-thumb with minor impact
for this DT scenario and Service representatives
have stated that the rule-of-thumb workaround
may be adequate for operations. Post mission
data analysis showed a SAR map coordinate
inaccuracy, but within the Circular Error Probable
(CEP) of the weapon.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

Weapons impacted as expected with the
selections made by the pilot and with accurate
PVI indications. Dual voltage bomb rack unit
(BRU) functioned properly with no power
distribution issues.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

The test team used a planned workaround for
BRU-61; using the new dual-voltage BRU in
single-voltage mode due to a mission systems
software limitation.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

The test team used a U.S. non-operationally
representative BRU-61, one with only a single
voltage unit, to complete this WDA event. This
older BRU-61 is representative for partner
operations.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

Weapons integration deficiencies were identified
during this event and deficiency reports
completed.

Successful accomplishment
of event and sufficient
data collected for weapons
integration analyses.

The test team used a U.S. non-operationally
representative BRU-61, one with only a single
voltage unit, to complete this WDA event. This
older BRU-61 is representative for partner
operations. Awaiting data delivery for detailed
analysis.

Jun 16

322

323

JDAM (GBU-31) X
2 Ripple release
on two targets

3FR5.03
Aug 16

Jul 16

JDAM (GBU-31)
Pattern on
target (multiple
weapons)

3FR5.05

SBD-I (GBU-39) X
2 on two targets

3FR5.03

Oct 16
May 16

324

Aug 16
Feb 16

325

SDB-I (GBU-39)
Single release

3FR5.03

UK Paveway IV
bomb

3FR5.05

Jul 16

Jul 16
328

SDB Seps

SDB-I (GBU-39)
multiple ripple
release for
flight sciences
separation
test points,
completed on
mission systems
aircraft.

Oct 16
Feb 16

3FR5.03
Jul 16

• The remaining 10 events are planned to be completed over
the next several months, as the program provides versions of
Block 3F software with necessary deficiency fixes to allow
the rest of the events to proceed. The remaining events are
complex multi-weapon, multi-target, and advanced threat
presentations. Whether all WDAs will be completed with
the final planned increment of Block 3F software – version
3FR6 – released in December is still to be determined, but
several key deficiency fixes related to weapons employment
are apparently not included and the probability of additional
discoveries during the remaining weapons test events is high,
based on results to date.
Gun Testing
• All three variants add gun capability with Block 3F. The
F-35A gun is internal; the F-35B and F-35C each use a gun
pod. Differences in the outer mold-line faring mounting
make the gun pods unique to a specific variant, i.e., a gun
pod designated for an F-35B cannot be mounted on an F-35C
aircraft.

76

F-35 JSF

• Flight sciences testing of the F-35A internal gun was
completed in May 2016. The first firing of the gun in flight
occurred October 30, 2015, and the entire flight sciences test
effort consisted of 11 flights over the 7-month period. Testing
revealed that the small doors that open when the gun is fired
induce a yaw (i.e., sideslip), resulting in gun aiming errors that
exceed accuracy specifications. As a result, software changes
to the flight control laws were needed to enable adjustments,
which are still to be determined by flight testing, to cancel
out the yaw when the gun doors are open. These control law
changes, and the resulting regression testing, delayed the start
of gun accuracy flight testing on mission systems test aircraft
until December 2016, at the earliest. Since no missionsystems-capable developmental test aircraft were built with
an internal gun, the program modified one of the operational
test F-35A aircraft (AF-31) to conduct the needed gun testing
events. Until testing is completed on AF-31, it is unknown if
the F-35 gun system, aimed by the Gen III HMDS, will meet
accuracy requirements for effective air-to-air and air-toground gun employment.

 FY16 DOD PROGRAMS
• The program has conducted ground testing of the F-35B
gun pod and plans to start airborne testing in January 2017.
Initial ground firing of the F-35C gun pod occurred in
mid-November 2016 and airborne gun testing is planned
to start in March 2017. New discoveries, as well as
determining the amount of adjustment to the flight control
laws to counter the pitching moments induced by firing the
gun pod, are likely.
• Accuracy testing of the gun with the HMDS has not yet been
completed and continues to be delayed as new discoveries
are made. Hence, the effectiveness of the gun, aimed
via the gunsight in the HMDS, is still unproven for both
air-to-air and air-to-ground gun employment. The effects
of the canopy transparency on gun aiming – i.e., the pilot
aiming the gun via the HMDS gunsight looking through the
thick canopy material, associated distortions, and attempted
software-programmed corrections – are not yet characterized.
• Although aimed firing of the gun had yet to occur, both
DT and OT pilots have flown with the air-to-ground gun
strafing symbology displayed in the helmet and reported
concerns that it is currently operationally unusable and
potentially unsafe to complete the planned aimed gun fire
testing. These deficiencies may cause further delays to
the start of gun accuracy flight testing. Also, testing of the
air-to-air symbology by both DT and OT pilots revealed that
the gunsight is very unstable when tracking a target aircraft.
Fixing these deficiencies may require changes to the mission
systems software that controls symbology to the helmet, or to
the radar software, as the program is working to finalize the
last version of Block 3F. Plans to begin aimed flight testing

of the gun on the F-35A were planned for this fall, but will
likely not start until December 2016, at the earliest.
• Because of the late testing of the gun and likelihood of
additional discoveries, the program’s ability to deliver gun
capability with Block 3F before IOT&E is at risk, especially
for the F-35B and F-35C, which have not yet fired the gun in
flight.
Weapons Demonstration Events by the Operational Test Teams
• The JOTT and the associated Service operational test
squadrons (VMX-1, 31TES, and 422TES) assigned
to Edwards AFB, California, and Nellis AFB, Nevada
accomplished 6 air-to-air missile events, 19 GBU-31/32
JDAM air-to-ground events, and 28 GBU-12 laser guided
bomb events during 2016. For one of these events, the
team accomplished one combined AMRAAM missile with
one GBU-12 laser guided bomb event, as described in the
AMRAAM Air-to-Air Missile Event Table on the following
page. These weapon delivery events were accomplished
on range complexes at the Naval Weapons Center China
Lake, California; Marine Corps Air Station Yuma, Arizona;
and Eglin AFB, Florida. All of the OT weapon events were
planned and accomplished in operationally representative
scenario profiles constructed to evaluate the F-35’s ability
to find-fix-track-target-engage-assess airborne and fixed and
moving ground targets.
• The following tables and accompanying assessments show
the weapon events, aircraft Block configuration, date
accomplished, and results.

F-35 JSF

77

 FY16 DOD PROGRAMS
AMRAAM Air-to-Air Missile Events Accomplished by Operational Test Teams
Event
Identifier

Event Description

Aircraft Block
Software
Configuration

Date
Accomplished

Results

WDA-108

Cruise Missile Defense

3iR6.01

May 16

This event was a re-shoot of a developmental test event. The reshoot
was required by the operational test community because of control
room workarounds needed during the DT event. The OT profile was
successful.

OT 2.1

2 F-35 aircraft in MADL network attacking
one F-16 drone target with jamming

2BR5.3

Aug 16

Profile did not meet test objectives due to issues with the target
presentation. Data analysis in progress.

OT 2.2

2 F-35 aircraft in MADL network defending
against an off-boresight attacker

2BR5.3

Aug 16

Partially successful. Missile guided to objective target, however
secondary objective compromised due to issues with the target
presentation. Data analysis in progress.

OT 2.3

2 F-35 aircraft in MADL network vs 2
jamming equipped F-16 drones

2BR5.3

Aug 16

Profile did not meet test objectives due to issues with the target
presentation. Data analysis in progress.

OT 2.4

F-35 combined Air-to-Air AMRAAM and
GBU-12 Air-to-Ground profile

2BR5.3

Aug 16

Primary test objective to confirm ability of the F-35 to support a laser
guided bomb to impact while simultaneously supporting a missile
inflight was successful. Secondary objective was unsuccessful due to
issues with the target presentation.

MAWTS-2

2 F-35 aircraft attacking a high closure rate
supersonic target

2BR5.3

Aug 16

This profile was a USMC engagement scenario to support ongoing
tactics development. Profile objective was successful

Air-to-Air General Observations
• The operational test teams completed the missile profiles in
accordance with the DOT&E-approved test plan; however,
some weapons integration objectives were not successful
due to the drone target presentation failures (details are
classified). The failures in the drone target presentations
prevented either the primary or secondary test objectives to
verify the F-35’s capability to complete the find-fix-tracktarget-engage-assess fire control thread. The test team is
conducting data analyses to determine whether engineering
characterization runs or re-shooting of the profiles are
required.

• Although four of the five missile events fell short of
addressing all of the specific data objectives, they were
successful in identifying key deficiencies in the ability
of the aircraft to support selected missile functionality,
stores management system anomalies, and the instability
of the shoot cues provided to the pilot to support missile
employment. Data analyses to identify root cause for all the
noted deficiencies are ongoing and the operational test team
will recommend specific mission systems software fixes to
address the noted deficiencies.

GBU-31/32 Joint Direct Attack Munition (JDAM) and GBU-12 Laser Guided Bomb (LGB) Air-to-Ground Event Summary
Weapon Type

Number of Weapons Events

GBU-12 LGB

28 Laser Guided Bomb (LGB) Events*

GBU-31 or
GBU-32 JDAM

F-35
Variant****

Date
Accomplished

21 F-35A
7 F-35B

15 GBU-31 (BLU-109) Events (8 inert/7 live)**

F-35A

3 inert GBU-32 (Mk-83) Events**

F-35B

Results
22 successful/6 partially successful*** events.

Jan to July
2016

10 successful/5 partially successful***
2 successful/1 partially successful***

*GBU-12 OT events were conducted against an operationally representative mix of fixed and moving targets; self-, airborne buddy-, and ground tactical control party
target-lasing; target cueing via voice, VMF digital, and F-35 shoot-list sharing via MADL.
**JDAM GBU-31/32 events were accomplished against an operationally representative mix of fixed target coordinates consisting of: pre-planned targeted coordinates,
F-35 self-targeting using SAR map and EOTS derived coordinates, and target cueing via voice, VMF digital, and F-35 shoot-list sharing via MADL.
***Air-to-Ground fully successful missions achieved weapon miss distances within expected mean radial error. Partially successful missions were cases where the
weapon was employed but with larger miss distances and observed mission systems issues described below.
****Mission Systems software for all variants was 2BS5.2 or 2BS5.3

78

F-35 JSF

 FY16 DOD PROGRAMS
Air-to-Ground General Observations
• Although initial observations from weapons integration
can be characterized in general, detailed data analyses are
ongoing to determine precise mean radial error results for
both the LGB and JDAM weapons delivery events, and
to identify root causes for the observed mission systems
deficiencies and weapon delivery issues.
• The JDAM predictive launch acceptability region (LAR)
and dynamic launch zone (DLZ) information were
consistently in error compared to the expected pilot drop
cues calculated from both the JDAM truth model and initial
DT characterizations. In the majority of the OT JDAM
drops, there were wide discrepancies between the LAR
presentations to the pilot via the HMDS, the corresponding
presentations on the in-cockpit controls and displays, and
the actual JDAM in-weapon LAR. In a number of cases,
the mission systems bombing cues available to the pilot via
the Tactical Situation Display on the Panoramic Cockpit
Display were in conflict with the HMDS shoot cues and the
DLZ. This inconsistency is both confusing to the pilot and
can result in erratic and inaccurate weapon impact relative
to the target desired impact point. Also, the tactical displays
available to the pilot did not allow the pilot to confirm
the actual target coordinates passed to the weapon. This
confirmation of the in-weapon target coordinates is usually
required by rules of engagement (ROE) in operational areas
in order to enable positive target information confirmation to
the ground controllers prior to clearance to drop any weapon.
The F-35 in the Block 2B or Block 3i configuration is not
currently able to comply with these ROE.
• In general, pilots were able to use the F-35 Synthetic
Aperture Radar (SAR) mapping function to derive
weapons quality coordinates, which are adequate to deliver
ordinance on target. Pilots were also able to share the
SAR-map-derived coordinates between flight members to
validate and confirm target positions and coordinates prior to
releasing weapons.
• The EOTS was not able to provide the pilot with sufficient
resolution at tactical employment ranges to enable a positive
ID on the intended target. However, the EOTS generally
was able to track targets, both moving and stationary, but
only after the target identification was confirmed by an
external source or multiple sources. However, there are still
significant tracking limitations, as evidenced by a new, open
Category 1-High deficiency titled “EOTS TFLIR Tracker
Unable to Point or Area Track.” The EOTS system also was
able to generate accurate weapon quality coordinates when
cued to the correct target.
• The lack of any lead-point-compute or lead-laser guidance
in the F-35 EOTS system required rule-of-thumb pilot
techniques to provide limited capability with the GBU-12
on moving targets. The OT moving target attacks were
generally successful; however, the successes relied on high
levels of pilot experience and were not enabled by the F-35
mission systems. While the rule-of-thumb procedures
allowed the technical requirements of the weapons delivery

event to be met, they did not allow the pilot to maintain
positive target ID using the PVI procedures to designate,
track, and employ the weapon for the full attack timeline.
Most importantly, these procedures would likely not
have met the current positive target ID requirements for
operational employment rules of engagement. Due to these
limitations, which threaten the effectiveness of the F-35
to engage moving targets, the program and Services are
exploring other options to meet this ORD requirement. One
option, which is being considered by the Air Force, is to
integrate the GBU-49, a fielded weapon that has similar size,
weight, and interfaces as the GBU-12, or a similar weapon
that does not require lead-laser guidance, in Block 3F.
Otherwise, the program plans to develop and field lead-laser
guidance in Block 4.2, which would be delivered in CY22, at
the earliest. However, because of the similarities, the GBU49 could be quickly integrated with Block 3F to provide a
robust moving target capability for the F-35 much earlier.
• Pilots were able to use the digital Variable Message Format
(VMF) system to communicate between F-35 aircraft
and tactical ground controllers. The VMF links and data
provided the expected data to both the pilot and the ground
parties. In previous developmental testing, the VMF
has exhibited significant issues with both reliability and
accuracy; however, in the OT events the system was both
reliable and accurate. Data analysis is ongoing to determine
the differences between the uses of VMF in developmental
testing compared to the operational weapons test events. The
ground parties used in the operational testing were equipped
with the most up-to-date software, firmware, and hardware
and were staffed by fully qualified ground controllers.
• Pilots experienced multiple inflight failures of the Fuselage
Remote Interface Unit (FRIU), an electronic component that
provides the interface between the aircraft avionics and all
weapon stations, which often disrupted the ground attack
profile. The failures resulted in degraded weapons at critical
phases of the target attack profile and required the pilots
to abort the attack, reset the FRIU to regain control and
communications with the weapon, and then recommit to a
follow-on target attack. Such target attack interruptions are
unacceptable for combat operations.
• Pilots consistently rated the Offboard Mission Support
(OMS) mission planning system as cumbersome, unusable,
and inadequate for operational use. As a result, the time
required for operational planners to build a mission plan
is excessive and cannot support current planning cycle
requirements for multiple aircraft combat missions.
Additionally, the post-mission download times are too long
to support operational debriefing requirements.
Pilot Escape System
• Testing of the pilot escape system in CY15 showed that the
risk of serious injury or death is greater for lighter-weight
pilots, which led to the decision by the Services to restrict
pilots weighing less than 136 pounds from flying the F-35.

F-35 JSF

79

 FY16 DOD PROGRAMS
In an effort to reduce this risk, the program developed three
modifications associated with the escape system and began
testing them in late CY15 and throughout CY16. These
modifications include:
- Reduction in the weight of the pilot’s Generation III helmet
(the new helmet is called Gen III Lite) to reduce the effect
of forces on the pilot’s neck during the ejection sequence.
- Installation of a switch in the seat that allows
lighter-weight pilots to select a slightly delayed activation
of the main parachute. This delay allows the drogue chute,
which deploys almost immediately during the ejection
sequence, to further slow and align the pilot before the
main parachute deploys. This delay is designed to reduce
the severity of loads on the neck experienced during
opening shock.
- The addition of a Head Support Panel (HSP) between the
risers of the parachute designed to prevent the pilot’s neck
from “snapping back” through the risers during the opening
of the main parachute.
• Concerned with the problems with the escape system and
the possibility of more discoveries, the U.S. Air Force asked
the JPO in June 2016 to gather and provide information on
potential costs and challenges to changing ejection seats
from the Martin Baker US16E seat currently installed in all
F-35 variants to the United Technologies ACES 5 seat as an
alternative for the F-35A.
• After prototypes of the design changes were available,
twenty-two qualification test cases were completed between
October 2015 and September 2016, with variations in manikin
weight, speed, altitude, helmet size and configuration, and the
seat switch settings. Seven of the tests were accomplished
with the lightweight (103 lbs) manikin. Data from these
tests showed that the HSP significantly reduced neck loads
under conditions that forced the head backwards, inducing a
rearward neck rotation, during the ejection sequence. Data
also showed that the seat switch delay reduced the opening
shock from the main parachute for lighter-weight pilots at
speeds greater than 160 knots. Results of the additional tests
were provided to the Services in late CY16 to update their risk
assessments associated with ejections. Despite the improved
results, the extent to which risks have been reduced to lighterweight pilots (i.e., less than 136 pounds) by the modifications
to the escape system and helmet is still to be determined by
these analyses. If the Services accept the risk associated with
the modifications to the escape system for pilots weighing
less than 136 pounds, restrictions will likely remain in effect
until aircraft have the modified seat with the switch and
HSP installed, and the Gen III Lite helmets are procured and
delivered to the applicable pilots in the fleet.
• The program plans to start retrofitting fielded F-35s with
the modifications to the ejection seats in February 2017 and
delivering aircraft with the upgraded seat in Lot 10, starting in
January 2018. The Gen III Lite helmets will be included with
the Lot 10 aircraft delivery, and will be delivered starting in
November 2017. If these delivery timelines are met, the Air

80

F-35 JSF

Force may open F-35 pilot training to lighter-weight pilots
(i.e., below 136 pounds) as early as December 2017.
• Part of the weight reduction to the Gen III Lite HMDS
involved removing one of the two visors (one dark, one clear).
As a result, pilots that will need to use both visors during a
mission (e.g., during transitions from daytime to nighttime),
will have to store the second visor in the cockpit. However,
there currently is not adequate storage space in the cockpit for
the visor; the program is working a solution to address this
problem.
• The program has yet to complete additional testing and
analysis needed to determine the risk of pilots being harmed by
the Transparency Removal System (which shatters the canopy
first, allowing the seat and pilot to leave the aircraft) during
ejections in other than ideal, stable conditions (such as after
battle damage or during out-of-control situations). Although
the program completed an off-nominal rocket sled test with the
Transparency Removal System in CY12, several aspects of the
escape system have changed since then, including significant
changes to the helmet, which warrant additional testing
and analyses. DOT&E recommends the program complete
these tests, in a variety of off-nominal conditions, as soon as
possible, so that the Services can better assess risk associated
with ejections under these conditions.
Static Structural and Durability Testing
• Structural durability testing of all variants using full-scale test
articles continues, with plans for each variant to complete three
full lifetimes (one lifetime is 8,000 equivalent flight hours, or
EFH). Although all variants are scheduled to complete testing
before the end of SDD, the complete teardown, analyses,
and damage assessment and damage tolerance reporting is
not scheduled to be completed until August 2019. Testing
on all variants has led to discoveries requiring repairs and
modification to production designs and retrofits to fielded
aircraft.
• F-35A durability test article (AJ-1) completed the second
lifetime of testing, or 16,000 EFH in October 2015. After
completing second lifetime inspections, third lifetime testing
began on March 11, 2016. As of November 16, 2016,
20,000 EFH, or 50 percent of the third lifetime had been
completed. Third lifetime testing is projected to complete in
December 2017.
• F-35B durability test article (BH-1) completed 14,051 EFH
by November 17, 2016, which is 6,051 hours (76 percent) into
the second lifetime. Due to the amount of modifications and
repairs to bulkheads and other structures in the current F-35B
ground test article, it may not be adequate to continue testing
and a new one may be needed and durability testing repeated
to ensure adequate lifetime testing is completed. The program
needs to conduct an assessment to determine the extent to
which the results of further durability testing are representative
of production aircraft and if necessary procure another test
article for the third life testing.

 FY16 DOD PROGRAMS
- Two main wing carry-through bulkheads, FS496 and
FS472, are no longer considered production-representative
due to the extensive repairs that have been required. The
program plans to continue durability testing, repairing
the bulkheads as necessary, through the second lifetime
(i.e., 8,001 through 16,000 EFH), which is projected to be
complete in February 2017.
- Prior to CY16, testing was halted on September 29,
2013, at 9,056 EFH, when the FS496 bulkhead severed,
transferred loads to, and caused cracking in the adjacent
three bulkheads (FS518, FS472, and FS450). The repairs
and an adequacy review of the repairs to support further
testing were completed on December 17, 2014, when the
program determined that the test article could continue
testing. Testing restarted on January 19, 2015, after a
16-month delay.
- The program determined that several of the cracks
discovered from the September 2013 pause at 9,056 EFH
were initiated at etch pits. These etch pits are created by
the etching process required prior to anodizing the surface
of the structural components; anodizing is required for
corrosion protection. Since the cracks were not expected,
the program determined that the etch pits were more
detrimental to fatigue life than the original material design
suggested. The program is currently developing an
analysis path forward to determine the effect on the overall
fatigue life.
- After the durability test completed 11,915 EFH on August
13, 2015, the load cycling was stopped to allow removal
and replacement of the FS496 bulkhead outer segments
(both left- and right-hand sides), removal and replacement
of the left-hand-side aft fuselage close-out frame, repairs
to the engine thrust mount shear webs, installation of
fasteners at the FS518 frame, maintenance of the righthand-side EHAS panel, repairs to the right-hand-side of the
mid-fairing longeron, and repairs to the FS556 upper arch.
The entire repair activity took about 9 months, with an
85-EFH testing effort conducted in early March 2016 that
reached 12,000 EFH.
- Testing resumed in early May 2016, reached 13,000 EFH
in mid-June 2016, and then stopped for another month to
repair the FS472 lower flange.
- Testing resumed in mid-July. At 13,086 EFH, cracks
were discovered on the forward fuselage including FS236
bulkhead, left-hand-side FS223 frame, and right-hand-side
FS191 upper frame.
- Testing continued with buffet loads until it reached 13,980
EFH before stopped to implement fuselage repairs in
August 2016.
- Testing resumed on September 17 and had reached
14,051 EFH on November 17, 2016.
• F-35C durability test article (CJ-1) completed the second
lifetime of testing, or 16,000 EFH on October 29, 2016.
The third lifetime testing is scheduled to begin in late
December 2016.

- In October 2015 with 13,731 EFH accomplished, cracks
were discovered on the left-hand side and right-hand
side of one wing front spar and one left-hand-side wing
forward root rib; this discovery was considered significant
because wing spar and wing root rib are primary structural
components and the cracks were not predicted by the
finite element model (FEM) used in the design of these
components. The repairs took over 3 months before the test
resumed in early February 2016.
- On February 9, 2016, with 13,827 EFH accomplished, a
crack was found on the left-hand-side inverter/converter/
controller and power distribution center/inverter bay floor.
Testing continued with catapult and trap load cycling.
- In late February 2016 with 13,931 EFH accomplished,
cracks were found on the left- and right-hand sides of the
FS496 bulkhead flanges, which were deemed significant.
The repairs took another 3 months to complete before the
test resumed in May 2016.
- In August 2016 with 14,831 EFH accomplished, small
cracks were found on the right-hand-side armpit (below
wing root) and were quickly repaired with a simple blend.
- In August 2016 with 14,892 EFH accomplished, cracks
were found on the FS518 lower frame and some nearby
broken fasteners. A weld repair for the titanium frame
was completed. Further investigation revealed cracks
on the right- and left-hand-side wing rear spars. While a
repair disposition was being developed, the durability test
resumed with loading only for catapult takeoffs and carrier
trap landings.
• The program plans to use Laser Shock Peening (LSP), a
mechanical process designed to add compressive residual
stresses in the materials, in an attempt to extend the lifetime
of the FS496 and FS472 bulkheads in the F-35B. The
first production line cut-in of LSP will start with Lot 11
F-35B aircraft. Earlier Lot F-35B aircraft will undergo
LSP processing as part of a depot modification. Testing is
proceeding in three phases: first, coupon-level testing to
optimize LSP parameters; second, element-level testing to
validate LSP parameters and quantify life improvement; and
third, testing of production and retrofit representative articles
to verify the service life improvements. All three phases are
in progress, with full qualification testing scheduled to be
completed in August 2017. As of December 1, 2016, 122 of
211 durability tests had been conducted with results within
expectations, which is a 58 percent completion.
Joint Simulation Environment (JSE)
• The JSE is a man-in-the-loop, mission systems software-inthe-loop simulation developed to meet the operational test
requirements for Block 3F IOT&E. The Program Office made
the decision in September 2015 to stop development on the
contractor’s effort to build a similar system, the Verification
Simulation (VSim), instead tasking the Naval Air Systems
Command (NAVAIR) to lead the building of a governmentowned Joint Simulation Environment (JSE), with the

F-35 JSF

81

 FY16 DOD PROGRAMS
contractor providing only the F-35 aircraft and sensor models.
However, negotiations for the F-35 models have not yet been
successful, which has prevented NAVAIR from fully defining
the simulation’s architecture and environment (the virtual
software environment in which aircraft, sensor, and threat
models interact with one another).
• While the Program Office continued to negotiate with
the contractor, and had success in meeting the hardware
requirements (facilities, cockpits, etc.), the lack of definition
of the simulation environment makes any integration schedule
not credible. In the next year, the program must acquire the
F-35 models, integrate them into an as-yet undefined and
undeveloped battlespace environment, complete development
of several dozen threat aircraft and surface system models,
ensure that aircraft sensor models correctly perceive the threat
system models, and validate the entire simulation. Previous
efforts of this magnitude have taken several years, so it is
unlikely that NAVAIR will complete the project as planned
in time to support IOT&E. Current Program Office estimates
are that JSE will deliver late to need in May 2019, but before
the end of IOT&E. Verification, Validation, and Accreditation
(VV&A) activities remained effectively stalled in 2016 and are
also a very high risk to timely completion of the simulation.
• Without a high-fidelity simulation, the F-35 IOT&E will not be
able to test the F-35’s full capabilities against the full range of
required threats and scenarios. Nonetheless, because aircraft
continue to be produced in substantial quantities (essentially
all of which require modifications and retrofits before being
used in combat), the IOT&E must be conducted without
waiting for the JSE, to demonstrate F-35 combat effectiveness
under the most realistic conditions that can be obtained in
flight testing, once the aircraft hardware and software meet
the IOT&E entrance criteria, which is expected to occur long
before the completion and successful VV&A of JSE. It is
now clear that the JSE will not be available and accredited in
time to support the Block 3F IOT&E. The currently approved
IOT&E detailed test design, which was developed entirely
around open-air flight testing, mitigates the lack of an adequate
simulation environment as much as possible.
Live Fire Test and Evaluation (LFT&E)
F-35C Full-Scale Aft Fuselage and Empennage Structure Test
• The F-35 LFT&E program completed the F-35C full-scale
aft fuselage and empennage structure tests. The Navy’s
Weapons Survivability Laboratory in China Lake, California,
accomplished three test events using the CG:0001 full scale
structural test article. The tests evaluated the ability of the
vertical tail and aft boom structure to withstand damage from
high-explosive incendiary (HEI) projectile and simulated
Man-Portable Air Defense System (MANPADS) threats. A
preliminary review of the test results indicates that:
- The F-35 vertical tail is capable of withstanding an HEI
projectile impact. The threat can target and fail one
attachment lug but the remaining lugs demonstrated their
ability to handle normal flight loads after the impact.
However, the pilot receives no alerts from the Integrated

82

F-35 JSF

Caution, Advisory and Warning (ICAW) system from this
type of structural damage, so there is a potential that a
damaged vertical tail could fail without warning the pilot
if the pilot demands higher than normal flight loads on the
vertical tail after the damage occurs.
- Two MANPADS shots were completed against the aft
boom structures, which support the horizontal and vertical
tails. Combined with results from earlier tests on an
F-35A and F-35B test articles, these tests showed that the
structures are sufficiently robust against these threats to
retain all control surfaces. Although damage to a single
control surface actuator is possible, earlier flight control
tests showed sufficient controllability within a limited
flight envelope to allow controlled flight back to a safe
area where the pilot could eject.
- The MANPADS tests demonstrated the potential for
damage to the fueldraulics system – the engine fuel-based
hydraulics system – which can result in a sustained
fire leading to further damage to the aircraft and a pilot
ejection over enemy territory. The data will be used
to support an assessment in 2017 that will determine
the contribution of this issue to the overall aircraft
vulnerability.
- While extended fires occurred in the MANPADS tests,
there has been no effort expended to determine what
catastrophic damage might result and the timeframe for
that to occur. Current procedures are for an immediate
ejection upon determination of a sustained fire. However,
if the time-to-failure could be established for this sort of
fire, it might allow the pilot time to depart a combat area
and eject somewhere relatively safe. Further analysis of
these test results and the related issue are needed.
PAO Shut-Off Valve
• The program has not provided an official decision to
reinstate this vulnerability reduction feature. There has
been no activity on the development of the PAO-shut-off
valve technical solution to meet criteria developed from
2011 live fire test results. As stated in several previous
reports, this aggregate, 2-pound vulnerability reduction
feature, if installed, would reduce the probability of pilot
incapacitation, decrease overall F-35 vulnerability, and
prevent the program from failing one of its vulnerability
requirements.
Vulnerability to Unconventional Threats
• The full-up, system-level chemical-biological
decontamination test on an SDD aircraft, which began
4QFY16 and is scheduled to end in 2QFY17 at Edwards
AFB, was supported by two risk-reduction events:
- A System Integration Demonstration of the proposed
decontamination equipment and shelter was conducted on
an F-16 test article during 1QFY15 at Edwards AFB to
simulate both hot air chemical and hot/humid air biological
decontamination operations. Extensive condensation
inside the shelter and on the test article during the

 FY16 DOD PROGRAMS
hot/humid air biological decontamination event indicated
the need for process and shelter modifications.
- A 2QFY16 event demonstrated that a modified system
process and a better insulated shelter can maintain
adequate temperature and humidity control inside the
shelter, even in a cold-weather environment.
• The test plan to assess chemical and biological
decontamination of pilot protective equipment is not
adequate. Compatibility testing of protective ensembles
and masks has shown that the materials survive exposure
to chemical agents and decontamination materials and
processes, but the program has neither tested nor provided
plans for testing the HMDS currently being fielded.
Gen II HMDS compatibilities were determined by analysis,
comparing HMDS materials with those in an extensive DOD
aerospace materials database. A similar analysis is planned
for the Gen III HMDS design. However, even if material
compatibilities were understood, there are no plans to
demonstrate a process that could adequately decontaminate
either HMDS from chemical and biological agents.
• The Joint Program Executive Office for Chemical and
Biological Defense approved initial production of the F-35
variant of the Joint Service Aircrew Mask (JSAM-JSF)
during 1QFY16. This office and the F-35 Joint Program
Office are integrating the JSAM-JSF with the HMDS, which
is undergoing Safety of Flight testing.
• The Navy evaluated an F-35B aircraft to the EMP threat
level defined in Military-Standard-2169B. Follow-on tests
on other variants of the aircraft, including a test series to
evaluate any Block 3F hardware/software changes, are
planned for FY16-17.
Gun Ammunition Lethality and Vulnerability
• The 780th Test Squadron at Eglin AFB, Florida, completed
the ground-based lethality test of the PGU-47/U Armor
Piercing High Explosive Incendiary with Tracer (APHEI-T)
round (also known as Armor Piercing with Explosive
(APEX)) against armored and technical vehicles, aircraft,
and personnel-in-the-open targets. Ground-based lethality
tests for the APEX correlated well with pre-test predictions
for the round penetrations, but potential problems were
discovered with fuze functioning when impacting rolled
homogeneous armor at high obliquity. Nammo, the
Norwegian manufacturer, conducted additional testing to
identify the cause of the dudded rounds during the ground
tests and subsequently modified the fuze design to increase
reliability. The program will determine the effect of the
ground-based lethality test data on the ammunition lethality
assessment.
• Per the current mission systems software schedule, the
weapons integration characterization of the gun and sight
systems will not be ready for the air-to-ground gun strafe
lethality tests until December 2016, at the earliest. Strafing
targets will include a small boat, light armored vehicle,
technical vehicle (pickup truck), and plywood mannequins
for each round type tested.

Operational Suitability
• The operational suitability of all variants continues to be less
than desired by the Services. Operational and training units
must rely on contractor support and workarounds that would
be challenging to employ during combat operations. In the
past year some metrics of suitability performance have shown
improvement, while others have been flat or declined. Most
metrics still remain below interim goals to achieve acceptable
suitability by the time the fleet accrues 200,000 flight hours,
the benchmark set by the program and defined in the
Operational Requirements Document (ORD) for the aircraft to
meet reliability and maintainability requirements. This level
of maturity is further stipulated as 75,000 flight hours for the
F-35A, 75,000 flight hours for the F-35B, and 50,000 flight
hours for the F-35C.
• Reliability growth has stagnated, so it is highly unlikely that
the program will achieve the ORD threshold requirements at
maturity for the majority of reliability metrics, most notably
the Mean Flight Hours Between Critical Failures, without
redesigning aircraft components.
• Aircraft fleet-wide availability averaged 52 percent for 12
months ending October 2016, compared to the modest goal of
60 percent. It is important to note that the expected combat
sortie rates will require significantly greater availability than
60 percent; therefore, if the F-35 is to replace legacy aircraft
for combat taskings, availability will likely need to improve to
near 80 percent.
• Monthly availability had been averaging in the mid-30s to
low-40s percent for the 2-year period ending September 2014.
Monthly availability then increased rapidly and significantly
from October to December, peaking at 56 percent in December
2014. However, since then it has remained flat, centering
around the low-50s percent with no strong improving trend
over time.
• Only two out of nine reliability metrics that have ORD
requirement thresholds have improved since last year’s report.
All nine are below the interim goals that were set to determine
if the metrics will meet the thresholds by maturity. None are
within 5 percent of their interim goal, whereas previously,
several of these metrics were reported as being above or
within 5 percent of their interim goal. In particular, reliability
metrics related to critical failures have decreased over the
past year. This decrease in reliability correlates with the
simultaneously observed decline in the Fully Mission Capable
(FMC) rate for all variants, which measures the percentage of
aircraft not in depot status that are able to fly all defined F-35
missions. The fleet-wide FMC rate peaked in December 2014
at 62 percent and has fallen steadily since then to 21 percent in
October 2016.
• In addition to the nine ORD metrics, there are three contract
specification metrics, Mean Flight Hours Between Failure
scored as “design controllable,” or DC, one for each
variant. DC failures are equipment failures due to design
flaws considered to be the fault of the contractor, such as
components not withstanding stresses expected to be found

F-35 JSF

83

 FY16 DOD PROGRAMS
in the normal operational environment. It does not include
failures caused by improper maintenance, or caused by
circumstances unique to flight test. This metric exhibited the
highest rate of the growth in the past and, for this metric, all
variants are currently above program target values for this stage
in development. However, since May 2015, DC reliability has
generally decreased or remained flat as well.
• Although most measures of reliability have not improved
significantly over the past year, three of six measures of
maintainability have improved slightly. Maintainability metrics
record the amount of time required to troubleshoot and repair
faults on the aircraft. Additionally, the number of flight hours
each aircraft flies per month, known as the utilization rate, has
also increased marginally.
• F-35 aircraft spent 9 percent more time down for maintenance
than intended (fleet average of 16.4 percent compared
to 15 percent goal), and waited for parts from supply for
71 percent longer than the program targeted (fleet average
of 17 percent compared to goal of 10 percent). At any given
time, from 10 to 20 percent of aircraft were in a depot facility
or depot status at the home base for major rework or planned
upgrades. Of the remaining aircraft not in any depot status, on
average less than a third were able to fly all missions of even
a limited capability set that is associated with the Block 2B or
Block 3i aircraft.
• Accurate suitability measures rely on adjudicated data from
fielded operating units. A Joint Reliability and Maintainability
Evaluation Team (JRMET), composed of representatives
from the Program Office, the JOTT, the contractor (Lockheed
Martin), and Pratt and Whitney (for engine records), reviews
maintenance data to ensure consistency and accuracy for
reporting measures; government representatives chair the
team. However, the Lockheed Martin database that stores
the maintenance data, known as the Failure Reporting and
Corrective Action System (FRACAS), was not in compliance
with U.S. Cyber Command information assurance policies
implemented in August 2015 through late summer of 2016.
Because of this non-compliance, government personnel were
not able to access the database via government networks,
preventing the JRMET from holding regularly scheduled
reviews of maintenance records for nearly a year, other than
a few ad hoc reviews. Regular JRMET meetings resumed
in September 2016, but the program is currently working
through reviewing a large backlog of un-adjudicated field
data. The program restarted publishing monthly reliability and
maintainability (R&M) status reports from adjudicated data in
October 2016, after roughly a year-long hiatus.
F-35 Fleet Availability
• Aircraft availability is determined by measuring the percent of
time individual aircraft are in an available status, aggregated
over a reporting period (e.g., monthly). The program assigns
aircraft that are not available to one of three categories of
status: Not Mission Capable for Maintenance (NMC-M); Not
Mission Capable for Supply (NMC-S); and depot status.

84

F-35 JSF

- Program goals for these not-available categories have
remained unchanged since 2014, at 15 percent for
NMC-M, 10 percent for NMC-S, and 15 percent of
the fleet in depot status. Depot status is primarily for
completing the modifications required to bring currently
fielded aircraft in compliance with their expected
airframe structural lifespans of 8,000 flight hours and to
incorporate additional mission capability. The majority
of aircraft in depot status are located at dedicated depot
facilities for scheduled modification periods that can
last several months, and they are not assigned as a part
of the operational or training fleet during this time. A
small portion of depot activity can occur in the field
when depot field teams conduct a modification at a main
operating base, or affect repairs beyond the capability of
the local maintenance unit. Similar to being at a depot
facility, aircraft are temporarily assigned to depot status
during these periods and are not considered a part of the
operational or training fleet.
- These three not-available category goals sum to
40 percent, resulting in a fleet-wide availability goal of
60 percent for 2016.
- In addition to these overall program goals, the program
has implemented a Performance Based Logistics (PBL)
construct with Lockheed Martin that ties contract
incentive awards to a slightly different set of tailored fleet
performance targets. These tailored targets prioritize
improvement efforts for Marine Corps F-35B performance
as the first branch to declare Initial Operational Capability
(IOC), and also because the F-35B variant has shown the
lowest overall availability performance. Current PBLbased goals are 53 percent availability, 35 percent FMC,
and 70 percent mission effectiveness rates for the F-35B
training and operational fleets assigned to Marine Corps
Air Station (MCAS) Beaufort and MCAS Yuma. The
majority of the incentive structure is tied to these goals.
To ensure Lockheed Martin continues to try to improve
performance across the board, a smaller portion of the
incentive fee is tied to overall fleet performance metrics
of 60 percent F-35A, 50 percent F-35B, and 60 percent
F-35C availability, regardless of operating site.
• Aircraft monthly availability averaged 52 percent for the
12-month period ending October 2016 in the training and
operational fleets, with a maximum availability of 55 percent
in May 2016 and a minimum availability of 44 percent in
October 2016. This is only a minor improvement over the
average 51 percent monthly availability reported in the
FY15 DOT&E Annual Report for the 12 months ending
October 2015. Further, some groups of aircraft continue to
experience minimum availability well below 50 percent.
- In no month did the overall fleet exceed its goal of
60 percent availability. Only the F-35C variant exceeded
the 60 percent goal, in 6 of 12 months, with a maximum
availability of 71 percent in April 2016. The F-35A and
F-35B variants never exceeded 60 percent, but the F-35A

 FY16 DOD PROGRAMS
achieved 59 percent in May 2016 and the F-35B reached a
maximum 50 percent in January, April, and July 2016.
- The table below summarizes aircraft availability by
operating location for the 12-month period ending
October 2016. The first column indicates the average
availability achieved for the whole period, while the
maximum and minimum columns represent the range
of monthly availabilities reported over the period. The
number of aircraft assigned at the end of the reporting
period is shown as an indicator of potential variance in
availability. Sites are arranged in order of when each
site began operation of any variant of the F-35, and then
arranged by variant for sites operating more than one
variant. The Marine Corps terminated F-35B operations
at Eglin AFB in February 2015, so there were no F-35Bs
at that site for the 12-month period of this report; thus,
that entry, previously reported in the FY15 DOT&E
Annual Report, has been removed. The Navy operational
test squadron at Edwards AFB received its first F-35C in
August 2016, the only new operating site to stand up since
the FY15 DOT&E Annual Report.
- Trend analysis of monthly fleet availability from
August 2012 through October 2016 showed a weak rate
of improvement of approximately 5 percent growth per
year over this period. This is consistent with the growth
rate reported in the DOT&E FY15 Annual Report – but,
again, the growth was neither steady nor continuous. The
majority of this growth still results from a concentrated
increase in availability that occurred during the months
of September 2014 through December 2014. Analysis of
availability from January 2015 through October 2016, the
time period after this concentrated increase, shows a more
modest less than 1 percent annual growth rate, which is in
better agreement with recent observations.
- The combined fleet of designated, instrumented OT
aircraft currently at Edwards AFB, which was built in
F-35 AVAILABILITY FOR 12-MONTH PERIOD ENDING OCTOBER 20161
Operational
Site

Average

Maximum

Minimum

Aircraft
Assigned2

Whole Fleet

52%

55%

44%

178

Eglin F-35A

38%

49%

32%

25

Eglin F-35C

60%

71%

54%

21

Yuma F-35B

55%

62%

40%

19

Edwards F-35A

53%

74%

40%

8

Edwards F-35B

46%

64%

30%

7

3

Edwards F-35C

27%

40%

4%

2

Nellis F-35A

50%

62%

42%

13

Luke F-35A

61%

68%

44%

44

Beaufort F-35B

43%

53%

33%

24

Hill F-35A

57%

80%

22%

15

1. Data do not include SDD aircraft.
2. Aircraft assigned at the end of October 2016.
3. Edwards AFB F-35C operations began August 2016.

Lots 3 to 5, averaged 48 percent availability from January
to October 2016. Seventeen instrumented OT aircraft
were assigned to Edwards AFB as of October 2016. This
is well-short of the target of 80 percent that will be needed
to conduct an efficient IOT&E, or combat operations.
• Due to concurrent development and production, which
resulted in delivering operational aircraft before the program
has completed development and finalized the aircraft
design, the Services must send the current fleet of F-35
aircraft to depot facilities. This is to receive modifications
that have been designed since the aircraft were originally
manufactured and are now required for full capability. Some
of these modifications are driven by faults in the original
design that were not discovered until after production had
started, such as major structural components that do not
meet the requirements for the intended lifespan, and others
are driven by the continuing improvement of the design of
combat capabilities that were known to be lacking when the
aircraft were first built. These modifications are a result of
the concurrency of production and development and cause
the program to expend resources to send aircraft for major
re-work, often multiple times, to keep up with the aircraft
design as it progresses. Since SDD will continue at least
to the middle of 2018, and by then the program will have
delivered nearly 200 aircraft to the Services in other than
the 3F configuration, the depot modification program and its
associated concurrency burden will be with the Services for
years to come.
- Sending aircraft to depot facilities for several months
at a time to bring them up to Block 3i capability from
Block 2B (i.e., upgrading avionics processors) and to meet
life limit requirements, and eventually to the Block 3F
configuration, reduces the number of aircraft at field sites
and thus decreases fleet availability. For the 12-month
period ending October 2016, the proportion of the fleet in
depot status averaged 15 percent, compared to 16 percent
for the 12-month period ending October 2015 stated in
the DOT&E FY15 Annual Report. The proportion of
aircraft in depot status was relatively flat over the majority
of this period with little overall trend, ranging between a
maximum monthly value of 22 percent and a minimum
value of 11 percent. The maximum value of 22 percent
occurred in October 2016, and was partly driven by
one-time repairs to shedding foam insulation around
PAO lines in the fuel tanks for 15 fielded F-35A aircraft.
DOT&E expects this rise in the depot rate to be a one-time
occurrence, and not indicative of a general trend.
- There is evidence from Program Office reports, however,
that later production lot aircraft achieve higher availability
rates than earlier lots. For example, for the period from
October 2015 to September 2016, accounting for 30 Lot 4
aircraft of all variants, each variant averaged a monthly
availability between 43 and 44 percent. For the same time
period and accounting for 33 Lot 7 aircraft of all variants,
each variant averaged a monthly availability between
64 and 68 percent, which was a statistically significant

F-35 JSF

85

 FY16 DOD PROGRAMS
increase. However, a significant amount of this increase
in availability can be attributed to the newer lot aircraft
requiring fewer depot modifications. Over this period
the Lot 4 aircraft averaged a monthly depot rate between
19 and 26 percent, depending on variant, whereas the
Lot 7 aircraft averaged a monthly depot rate between
0 and 6 percent, considering variant.
- Projections of depot rates beyond 2016 are difficult, since
testing and development are ongoing and discoveries
continue, including the need for redesigned outer wing
structure on the F-35C to accommodate AIM-9X missile
carriage. This structural modification was installed on an
F-35C developmental test aircraft for testing in late 2016.
Also, the program does not yet know the full suite of
modifications that will be necessary to bring currently
produced aircraft up to the final Block 3F configuration.
However, as the program continues to ramp up production
rates, the later lot aircraft, which generally require fewer
modifications, will comprise a larger proportion of the
fleet and may exert a downward influence on the depot
percentage rate.
• To examine the suitability performance of fielded aircraft,
regardless of how many are in the depot, the program reports
on the Mission Capable (MC) and Fully Mission Capable
(FMC) rates for the F-35 fleet. The MC rate represents the
proportion of the fleet that is not in depot status and that is
ready to fly any type of mission (as opposed to all mission
types). This rate includes aircraft that are only capable of
flying training flights, however, and not necessarily a combat
mission. The FMC rate calculates only the proportion
of aircraft not in depot status that are capable of flying
all assigned missions and can give a better view into the
potential combat capability available in the fielded units.
- F-35 aircraft averaged a 62 percent MC rate for the
12-month window ending in October 2016 considering
all variants, a slight decrease from the 65 percent reported
in the FY15 DOT&E Annual Report. The rate showed
little change over time, ranging from a minimum value of
57 percent to a maximum value of 66 percent
for the whole fleet, and was relatively consistent
across variants as well. The F-35A achieved
the highest variant-specific rate at 64 percent,
Lot
followed by 63 percent for the F-35C, and
59 percent for the F-35B.
2/3
- The FMC rate continued to exhibit a steady
4
decline first observed in 2015, and averaged
5
only 29 percent over the period, compared
to 46 percent reported in the FY15 DOT&E
6
Annual Report. The rate started at 32 percent in
7
November 2015, which was close to the peak of
8
33 percent in April 2016, but generally dropped

86

F-35 JSF

month over month to a minimum value of 21 percent by
October 2016. The FMC rate has not been consistent
across variants. The F-35A fleet achieved the highest
average FMC rate for the period at 37 percent, followed by
the F-35C at 24 percent. The F-35B fleet exhibited only
a 14 percent average FMC rate, however. Failures in the
Distributed Aperture System (DAS), electronic warfare
(EW) system, and Electro-Optical Targeting System
(EOTS) were the highest drivers pushing aircraft into
Partial Mission Capable (PMC) status.
- Analysis of the MC rate of each production lot reveals
that later lot aircraft have a greater MC rate than earlier
lot aircraft; the difference is less pronounced than the
comparison of availability, but still significant. The
30 Lot 4 aircraft averaged between 52 and 61 percent MC
over this period by variant, compared to 68 to 73 percent
for the Lot 7 aircraft by variant.
- The OT fleet at Edwards AFB averaged an MC rate of
53 percent from January to October 2016.
• The first table below shows F-35 MC and FMC rates for
the total fleet and each variant for the 12-month period
ending October 2016, including the average, maximum, and
minimum monthly values observed. The second table shows
F-35 availability and MC rates by production lot and by
variant for the 12-month period ending September 2016.

F-35 MC AND FMC RATES BY VARIANT FOR 12-MONTH PERIOD
ENDING OCTOBER 2016
Variant

MC

FMC

Avg.

Max

Min

Avg.

Max

Min

Fleet

62%

66%

57%

29%

33%

21%

F-35A

64%

70%

55%

37%

42%

27%

F-35B

59%

65%

53%

14%

17%

10%

F-35C

63%

73%

55%

24%

44%

13%

F-35 AVAILABILITY AND MISSION CAPABLE RATES BY LOT
(OCTOBER 2015 TO SEPTEMBER 2016)
No. of Aircraft

Availability

Mission Capable

F-35A

F-35B

F-35C

Total

F-35A

F-35B

F-35C

F-35A

F-35B

F-35C

14

13

-

27

33%

37%

N/A

57%

54%

N/A

10

17

3

30

44%

44%

43%

61%

59%

52%

22

3

7

32

51%

50%

57%

62%

52%

60%

23

6

7

36

62%

60%

67%

63%

66%

68%

22

7

4

33

67%

64%

68%

73%

68%

68%

14

3

3

20

49%

65%

79%

68%

65%

80%

 FY16 DOD PROGRAMS
• The monthly NMC-M rate averaged 16 percent over the period
and was relatively stable, with a minimum value of 14 percent
and a maximum value of 20 percent. This rate achieved the
program goal of 15 percent, or lower, in 4 of the 12 months of
the period. It also shows a slight decreasing (improving) trend
over time that indicates with further improvement it may be
possible to achieve and sustain program targets within the next
calendar year.
- Completing directed modifications or upgrades on
still-possessed aircraft in the field also affects the NMC-M
rate. In such cases, squadron-level maintainers, instead of
the depot or contractor field teams, are tasked to complete
Time Compliance Technical Directives (TCTDs). The
“time compliance” limits for these directives vary,
normally allowing the aircraft to be operated for a certain
period of time without the modification. This permits
maintenance personnel to do the work at an opportune time,
without taking the aircraft off the flight schedule to do so,
such as by combining the TCTD with other maintenance
activities. While maintainers accomplish these TCTDs, the
aircraft are designated as NMC-M status, and not in depot
status. Incorporating these TCTDs will drive the NMC-M
rate up (worse) until these remaining modifications are
completed. Publishing and fielding new TCTDs is
expected for a program under development and is needed
to see improvement in reliability and maintainability;
however, they inherently add to the maintenance burden in
the fielded operational units.
• The NMC-S rate averaged 17 percent and showed no
significant trend over the period. In no month did the
rate achieve the program goal of 10 percent or less, with
a minimum value of 14 percent and a maximum value of
20 percent.
- Several factors have contributed to the NMC-S rate
underperforming relative to its goal more than either the
NMC-M or depot not-available categories. First, the
program originally funded spares to a 20 percent NMC-S
rate. To determine the quantity and type of spares needed
to achieve this, the program used incorrect engineering
predictions that overestimated component reliability (fleet
data were not available when this modeling was done early
in the program). Actual mean time between failures for
many components is lower than the forecasted values used
in the spares model. Second, contracting for spares has
often been late to need to support the first aircraft delivery
for several of the initial production lots. Third, the program
has been late to stand up organic depot capabilities to repair
existing parts that have failed but can be refurbished instead
of being replaced with new parts. Such a capability would
reduce the strain on suppliers to produce more spare parts.
- The lack of spares available in the supply system is
driving operating units to take good parts from one NMC
aircraft and install them in other aircraft down for those
parts, bringing the latter back to available status. This
process, known as cannibalization, is performed by units
when supply cannot provide needed parts in a timely

manner. Cannibalization results in a significant increase
in maintenance man-hours compared to replacing a bad
part with a new or repaired part. For the 12-month period
ending in October 2016, the monthly cannibalization rate
averaged 9.8 cannibalization actions for every 100 sorties
against a program goal of no more than 8 actions
for every 100 sorties. The fleet met this goal in only
1 month, performing 6.2 cannibalizations per 100 sorties
in December 2015, but analysis over this period does
not demonstrate a statistically significant trend in the
cannibalization rate.
- Modifying aircraft also has an effect on the NMC-S rate
as the Services can cannibalize parts from aircraft in the
depots to support field units when replacement parts are
not otherwise available from normal supply channels or
stocks of spare parts on base. With the large number of
aircraft in depot status, the program may have been able to
improve the NMC-S rate by using depot cannibalizations,
instead of procuring more spare parts, or reducing the
failure rate of parts installed in aircraft, or improving
how quickly failed parts are repaired and returned to
circulation. If the Services endeavor to bring all of the
early lot aircraft into the Block 3F configuration, the
program will continue to have an extensive modification
program for several years. While this will continue to
provide opportunities for depot cannibalizations during
that time, once the Block 3F modifications are complete,
there will be fewer aircraft in the depot serving as spare
parts sources and more in the field requiring parts support.
If demand for spare parts remains high, this will put
pressure on the supply system to keep up with demand
without depot cannibalization as a source.
- While the fleet was much closer to achieving the NMC-M
goal than the NMC-S goal, these two rates are not
necessarily completely independent. Specifically, poor
diagnostics or difficult-to-conduct troubleshooting – issues
that are maintainability problems at root cause – can
drive the NMC-S rate up as well. For example, if
troubleshooting efforts initially isolate faults to incorrect
parts, units may inadvertently take good parts off the
aircraft, return them to the supply system for depot or
manufacturer checks, and demand replacement parts,
unnecessarily straining the supply system for repair actions
that will not resolve the fault. Units will report aircraft
in NMC-S status until these replacement parts arrive.
Once the unit receives and installs these parts, it would
discover that the original problem remains, and return the
aircraft to NMC-M status until further troubleshooting
hopefully isolates the correct part. Thus, actions to reduce
higher-than-targeted NMC-S rates may include improving
the accuracy of diagnostics and troubleshooting procedures
as well as increasing the availability of spare parts.
• The following table summarizes depot, NMC-M, and
NMC-S rates for the total F-35 fleet and each variant for
the 12-month period ending October 2016, including the
average, maximum, and minimum monthly values observed.

F-35 JSF

87

 FY16 DOD PROGRAMS
F-35 DEPOT, NMC-M, AND NMC-S RATES BY VARIANT FOR 12-MONTH
PERIOD ENDING OCTOBER 2016
Variant

Depot

NMC-M

NMC-S

(Goal of 15% or less)

(Goal of 15% or less)

(Goal of 10% or less)

Avg.

Max

Min

Avg.

Max

Min

Avg.

Max

Min

Fleet

15%

22%

11%

16%

20%

14%

17%

20%

14%

F-35A

14%

27%

8%

17%

24%

12%

17%

21%

12%

F-35B

20%

25%

14%

17%

25%

11%

16%

20%

13%

F-35C

6%

15%

2%

14%

20%

9%

20%

27%

13%

• Low availability is preventing the fleet of fielded operational
F-35 aircraft from achieving the originally planned, Servicefunded flying hour goals. The original Service beddown
plans were based on F-35 squadrons ramping up to a steady
state, fixed number of flight hours per tail per month,
allowing for the projection of total fleet flight hours.
- Since poor availability in the field has shown that these
original plans were unexecutable, the Program Office has
since produced modeled-achievable projections of total
fleet flight hours, basing these projections on demonstrated
fleet reliability and maintainability data, as well as
expectations for future improvements. The most current
modeled-achievable projection is from March 2016.
- Through November 21, 2016, the fleet had flown
approximately 91 percent of the modeled-achievable
hours. This is an improvement since November 2015, the
date used in the FY15 DOT&E Annual Report, when the
fleet had flown 82 percent of modeled-achievable hours;
however, recent updates to the model revised the projected
hours downward. The completion of actual flight hours
against modeled-achievable flight hours was consistent
across all three variants, with each variant completing
between 90 or 96 percent of its variant-specific projection.
By comparison, the fleet had flown only 72 percent of
the original beddown plan hours, with wide discrepancy
between variants. The F-35A had flown 82 percent of its
original beddown plan hours, while the F-35C had flown
only 49 percent, for example.
- The following table shows the planned versus achieved
flight hours by variant for both the original plans and the
modeled-achievable projections for the fielded production
aircraft through November 21, 2016.

F-35 FLEET PLANNED VS. ACHIEVED FLIGHT HOURS
AS OF NOVEMBER 21, 2016

Variant

F-35A

Original Beddown Plan
Cumulative Flight Hours

“Modeled Achievable”
Cumulative Flight Hours

Est.
Planned

Achieved

Percent
Planned

Est.
Modeled

Achieved

Percent
Planned

41,000

33,754

82%

36,788

33,754

92%

F-35B

29,000

19,644

68%

21,935

19,644

90%

F-35C

12,500

6,070

49%

6,348

6,070

96%

Total

82,500

59,469

72%

65,071

59,469

91%

88

F-35 JSF

F-35 Fleet Reliability
• Aircraft reliability assessments include a variety of metrics,
each characterizing a unique aspect of overall weapon
system reliability.
- Mean Flight Hours Between Critical Failures (MFHBCF)
includes all failures that render the aircraft not safe to
fly, and any equipment failures that would prevent the
completion of a defined F-35 mission. It includes failures
discovered in the air and on the ground.
- Mean Flight Hours Between Removal (MFHBR) gives
an indication of the degree of necessary logistical support
and is frequently used in determining associated costs.
It includes any removal of an item from the aircraft for
replacement. Not all removals are failures, and some
failures can be fixed on the aircraft without a removal. For
example, some removed items are later determined to have
not failed when tested at the repair site. Other components
can be removed due to excessive signs of wear before a
failure, such as worn tires.
- Mean Flight Hours Between Maintenance Event
Unscheduled (MFHBME_Unsch) is a useful reliability
metric for evaluating maintenance workload due to
unplanned maintenance. Maintenance events are either
scheduled (e.g., inspections, planned removals for part
life) or unscheduled (e.g., maintenance to remedy failures,
troubleshooting false alarms from fault reporting or defects
reported but within limits, unplanned servicing, removals
for worn parts— such as tires). One can also calculate the
mean flight hours between scheduled maintenance events,
or total events including both scheduled and unscheduled.
However, for this report, all MFHBME_Unsch metrics
refer to the mean flight hours between unscheduled
maintenance events only, as it is an indicator of aircraft
reliability and the only metric with an ORD requirement
for mean flight hours between maintenance event.
- Mean Flight Hours Between Failures, Design Controllable
(MFHBF_DC) includes failures of components due to
design flaws under the purview of the contractor, such
as the inability to withstand loads encountered in normal
operation. Failures induced by improper maintenance
practices are not included.
• The F-35 program developed reliability growth projection
curves for each variant throughout the development
period as a function of accumulated flight hours. These
projections were established to compare observed reliability
with target numbers to meet the threshold requirement at
maturity, defined by 75,000 flight hours for the F-35A and
F-35B, and by 50,000 flight hours for the F-35C, for a total
200,000 cumulative fleet flight hours. In November 2013,
the program discontinued reporting against these curves for
all ORD reliability metrics, and retained only the curve for
MFHBF_DC, which is the only reliability metric included in
the JSF Contract Specification (JCS). DOT&E reconstructed
the growth curves for the other metrics analytically for this
report. The following discussion and tables compare the

 FY16 DOD PROGRAMS

•

•

•

•

•

3-month reliability metrics to the growth goals required to be
on track to meet threshold requirements at maturity.
- As of the end of July 2016, the F-35 fleet, including
operational and flight test aircraft, had accumulated
nearly 60,300 flight hours, or approximately 30 percent
of the total 200,000-hour maturity mark defined in the
ORD. Unlike the above table, which accounts only for
fielded production aircraft, the flight test aircraft are
included in the fleet hours which count toward reliability
growth and maturity. By variant, the F-35A had flown
approximately 32,400 hours, or just over 43 percent of
its individual 75,000-hour maturity mark; the F-35B had
flown approximately 20,300 hours, or 27 percent of its
maturity mark; and the F-35C had flown approximately
7,600 hours, or 15 percent of its maturity mark.
The program reports reliability and maintainability metrics
on a 3-month rolling window basis. This means, for
example, the MFHBR rate published for a month accounts
only for the removals and flight hours of that month and
the two previous months. This rolling 3-month window
provides enough time to average out variability often seen
in month-to-month reports, while providing a short enough
period to distinguish current trends.
The first table, below, compares the most recently reported
and projected interim goal MFHBCF values, with associated
flight hours. It shows the ORD threshold requirement at
maturity and the values for May 2015, the month used in the
FY15 DOT&E Annual Report, for reference as well.
The three similar tables on the next page compare the most
recently reported and projected interim goals for MFHBR,
MFHBME_Unsch, and MFHBF_DC rates for all three
variants. MFHBF_DC is contract specification, and its JCS
requirement is shown in lieu of an ORD threshold.
Note that data more current than July 2016 were not
available at the time of this report due to the backlog of
maintenance events awaiting JRMET review as a result
of the Lockheed Martin database (FRACAS) not being
compliant with all applicable DOD information assurance
policies mandated by U.S. Cyber Command.
Reliability values decreased (worsened) for 8 of 12 metrics
between the May 2015 and the July 2016 values. All
three MFHBCF metrics decreased between May 2015
and July 2016, and usually showed the greatest degree of
reduction compared to the other reliability metrics. This

aligns with the declining FMC rates for all variants. Of the
remaining metrics, F-35A MFHBR and MFHBME_Unsch,
and F-35A and F-35B MFHBF_DC, improved slightly.
A more in-depth trend analysis over the 12-month period
showed that all three variants exhibited declining MFHBCF;
F-35B and F-35C MFHBR and MFHBME_Unsch were either
flat or decreasing slowly; and MFHBF_DC for all variants
were also either flat or decreasing. Only F-35A MFHBR and
MFHBME_Unsch increased over this period.
• All nine of the ORD metrics are below interim program
goals based on their planned reliability growth curves to
meet threshold values by maturity. Furthermore, none of
the ORD metrics are within 5 percent of their interim goals.
Of the ORD metrics, F-35B MFHBME, at 86 percent, was
the closest to its interim goal, while F-35C MFHBCF, at 39
percent, was the farthest. All of the JCS metrics, which are
the MFHBF_DC for each variant, are above their growth
curve interim values, ranging from 12 percent above for
the F-35A to 28 percent above for the F-35B. This pattern
indicates that the performance of the contract specification
reliability metrics exceeding their interim values is not
translating into the ORD reliability metrics showing the same
improvement, which are operational requirements that will be
evaluated during IOT&E.
• The fact that all the contract specification metrics are above
their growth curve does not necessarily imply that the F-35
will deliver desired reliability in the field, especially in light
of the fact that all ORD requirements are below their growth
curves. The ORD requirements reflect how the aircraft will
perform in combat, while the JCS metrics are limited to
failures that are definitively the fault of component design.
However, several situations can divorce improvement in the
JCS metrics to similar improvements in the ORD metrics or
availability. For example, components that are easily broken
during maintenance, such as nutplates, may not be scored
as design-controllable failures, but repairing and replacing
these fragile components will adversely affect the ORD
reliability metrics. Likewise, when old versions of redesigned
components fail in the field, depending on circumstances,
these failures may not be reported in the reliability metrics,
but the effect on downing the aircraft will always be reflected
in the availability metrics.
• The effect of lower (poorer) MFHBCF values is reduced
aircraft fully mission capable, mission capable, and

F-35 RELIABILITY: MFHBCF (HOURS)
ORD Threshold

Values as of July 31, 2016

Flight
Hours

MFHBCF

Cumulative Flight
Hours

Interim Goal
to Meet ORD
Threshold
MFHBCF

F-35A

75,000

20

32,358

F-35B

75,000

12

F-35C

50,000

14

Variant

Values as of May 2015*
Observed Value as
Percent of Goal

Cumulative Flight
Hours

Observed MFHBCF

(3 Mos. Rolling
Window)

17.8

8.0

45%

15,845

8.8

20,256

10.0

4.6

46%

11,089

7.2

7,648

10.9

4.2

39%

3,835

7.5

Observed MFHBCF

(3 Mos. Rolling
Window)

* The JPO revised past R&M metrics based on applying the current JRMET scoring rules to past data. As a result, values reported for May 2015 in this report may be
different than the values for the same month in the FY15 DOT&E Annual Report. See the Reliability Growth section below for more details.

F-35 JSF

89

 FY16 DOD PROGRAMS
F-35 RELIABILITY: MFHBR (HOURS)
ORD Threshold

Values as of July 31, 2016

Values as of May 2015

Flight
Hours

Cumulative Flight
Hours

Interim Goal
to Meet ORD
Threshold MFHBR

Observed MFHBR
(3 Mos. Rolling
Window)

Observed Value as
Percent of Goal

Cumulative Flight
Hours

Observed MFHBR

MFHBR

F-35A

75,000

6.5

32,358

5.8

4.7

81%

15,845

4.4

F-35B

75,000

6.0

20,256

5.0

2.8

56%

11,089

4.0

F-35C

50,000

6.0

7,648

4.7

2.3

49%

3,835

3.9

Variant

(3 Mos. Rolling
Window)

F-35 RELIABILITY: MFHBME_Unsch (HOURS)
ORD Threshold

Values as of July 31, 2016

Flight
Hours

MFHBME_
Unsch

Cumulative Flight
Hours

Interim Goal
to Meet ORD
Threshold
MFHBME_Unsch

F-35A

75,000

2.0

32,358

1.77

F-35B

75,000

1.5

20,256

1.25

F-35C

50,000

1.5

7,648

1.13

Variant

Observed
MFHBME_Unsch

1.36

Values as of May 2015
Observed
MFHBME_Unsch

Observed Value as
Percent of Goal

Cumulative Flight
Hours

77%

15,845

1.08

86%

11,089

1.10

0.74

65%

3,835

0.98

(3 Mos. Rolling
Window)

(3 Mos. Rolling
Window)

1.13

F-35 RELIABILITY: MFHBF_DC (HOURS)
JCS Requirement

Values as of July 31, 2016

Flight
Hours

MFHBF_
DC

Cumulative Flight
Hours

Interim Goal
to Meet JCS
Requirement
MFHBF_DC

(3 Mos. Rolling
Window)

F-35A

75,000

6.0

32,358

5.2

F-35B

75,000

4.0

20,256

F-35C

50,000

4.0

7,648

Variant

Cumulative Flight
Hours

5.8

112%

15,845

5.4

3.2

4.1

128%

11,089

3.6

2.9

3.3

114%

3,835

4.2

Reliability Growth
• In the fall of 2016, the Program Office revised reliability
and maintainability (R&M) metrics that had been previously
reported by applying new or updated JRMET scoring rules
that had been created or modified at different times over the
course of system development, and agreed to by the JRMET
members, to historical maintenance event data. Scoring
rules determine such criteria as when a maintenance event is
considered relevant and should be included in R&M metrics,
when an event is not relevant and will not be included in
metrics, such as failures in test-specific instrumentation that
will not be installed in operational aircraft, and when an
event is chargeable to the design-controllable metric as being
the fault of the design as opposed to induced by improper
maintenance. There are many detailed scoring rules to
ensure similar maintenance situations are scored consistently.
As the JRMET developed new scoring rules and changed
some existing ones, the program realized that previously
reported metrics needed to be revised – scored by the new

F-35 JSF

Observed
MFHBF_DC

Observed Value as
Percent of Goal

availability rates. MFHBR values lagging behind planned
growth targets drive a higher demand for spare parts from
the supply system than originally envisioned. When
MFHBME_Unsch values are below expectation, there is a
higher demand for maintenance manpower than anticipated.

90

Observed
MFHBF_DC

Values as of May 2015

(3 Mos. Rolling
Window)

rule set – in order to ensure current R&M metrics could be
compared more accurately with past R&M performance.
The effects on each reliability metric of this revision were
mixed, with 7 of 12 of the May 2015 metrics being revised
downward (worsening), and the remaining 5 increasing
compared to their originally reported values; however,
4 of these improved metrics decreased, or worsened, by
July 2016. Note the values in the tables above reflect the
JPO revised past R&M metrics based on applying the
current JRMET scoring rules to past data. As a result, values
reported for May 2015 in this report may be different than
the values for the same month in the FY15 DOT&E Annual
Report.
• In the two prior Annual Reports, DOT&E reported the
results of reliability growth analysis based on the Duane
Postulate, using R&M data provided by the Program
Office, to determine the rate of growth for MFHBR and
MFHBME_Unsch. In 2016, DOT&E conducted an updated
analysis of reliability growth using the more refined U.S.
Army Materiel Systems Analysis Activity (AMSAA)-Crow
model, examining data from the start of the program to
July 2016. The AMSAA-Crow model characterizes growth
by a single growth parameter, using a method that is similar
to the Duane Postulate. A growth rate between zero and
one implies improvement in reliability, a growth rate of zero

 FY16 DOD PROGRAMS
implies no growth, and a growth rate less than zero implies
reliability decay. Since it is logarithimic, a growth rate
of 0.40 represents much faster than twice the growth of a
rate of 0.20.
• Unlike the Duane Postulate, the AMSAA-Crow model
enables the determination of statistical confidence intervals
on its estimated growth rate based on the underlying
mathematics in the model. Further, the expected growth rate
is determined by Maximum Likelihood Estimator (MLE)
methods, rather than linear regression as in the Duane
Postulate, allowing for the quantity of data to have an effect
on the growth parameter estimate.
- Previous DOT&E Annual Report reliability growth
analyses included only the F-35A and F-35B variants,
and only for the MFHBR and MFHBME metrics, due to
a small amount of hours on the F-35C, and fewer critical
failures than removals and unscheduled maintenance
events. For this year’s updated analysis, sufficient data for
the MFHBCF metric and the F-35C variant were available
for these metrics and estimates to be included.
- The first table below shows the most likely growth rate
and 95 percent upper and lower confidence bound growth
rates, providing a range of likely values for the actual
growth rate, for all three variants and all three ORD
reliability metrics. It also includes the projected values
of these three metrics for each variant based on the most
likely, upper, and lower bound growth rates at maturity;
i.e., 75,000 flight hours for the F-35A and F-35B and
50,000 flight hours for the F-35C.

Metric

MFHBCF

MFHBR

MFHBME
_Unsch

July 2016
Growth Rates

Variant

Projections at Maturity

Most
Likely

Lower
Bound

Upper
Bound

Most
Likely

Lower
Bound

Upper
Bound

F-35A

0.137

0.109

0.164

9.6

9.0

10.2

ORD
Threshold

20.0

F-35B

-0.051

-0.089

-0.014

N/A *

12.0

F-35C

-0.107

-0.180

-0.039

N/A *

14.0

F-35A

0.192

0.173

0.211

6.1

5.8

6.4

6.5

F-35B

0.126

0.103

0.148

4.1

3.9

4.4

6.0

F-35C

-0.068

-0.119

-0.020

F-35A

0.170

0.161

0.179

1.38

N/A *
1.35

1.41

6.0
2.0

F-35B

0.359

0.351

0.367

2.01

1.96

2.08

1.5

F-35C

0.189

0.174

0.205

1.26

1.20

1.33

1.5

* No estimates for projections at maturity were made for metrics with negative growth rates.

Aircraft

MFHBME_Unsch Growth Rate

F-15

0.14

F-16

0.14

F-22 (at 35,000 flight hours)

0.22

B-1

0.13

“Early” B-2 (at 5,000 flight hours)

0.24

“Late” B-2

0.13

C-17 (at 15,000 flight hours)

0.35

- The growth rates listed in the first table were calculated
with approximately 32,400 hours for the F-35A,
20,300 hours for the F-35B, and 7,600 hours for
the F-35C. For comparison, historically observed
MFHBME_Unsch growth rates for several currently
fielded aircraft are shown in the second table. Analogous
rates for MFHBR and MFHBCF are not available.
• The updated reliability growth analysis through July 2016,
using the AMSAA-Crow model, accounts for the recent
tapering off of reliability growth better than the Duane
Postulate. As a result, most of the growth rates in the table
above are lower than those reported in prior DOT&E Annual
Reports. For the nine ORD metrics, the current growth
analysis predicts that only one will meet or surpass the
ORD threshold value at maturity, F-35B MFHBME_Unsch.
As the analysis showed no growth for F-35B and F-35C
MFHBCF, and F-35C MFHBR, no projections out to
maturity were made for those metrics and current estimates
do not meet threshold requirements.
- Comparing the currently exhibited MFHBME_Unsch
growth rates to historical aircraft shows that from program
initiation to July 2016, F-35 reliability has improved faster
than average for all variants. However, F-35 reliability
remains below program interim goals for its current stage
of development in all cases, and is not projected to achieve
threshold values by maturity in most cases, due to very
low initial reliability at the start of the program, well
below the assumed initial reliability values that informed
program interim goals.
- Although there were approximately 7,600 hours on the
F-35C fleet for this year’s analysis, usually enough time
to establish a growth trend, the lack of evidential growth
in the MFHBCF and MFHBR metrics may be explained
by the fact that the F-35C fleet has only recently begun
to send aircraft to the depot for modifications. Also,
the F-35C fleet has the least hardware improvements
incorporated relative to the F-35A and F-35B fleets. The
relatively strong growth in the MFHBME metric, by
contrast, can be partly explained for all variants by a
reduction in false alarms from the aircraft Prognostics and
Health Management (PHM) system, driving fewer overall
unscheduled maintenance actions, in addition to the natural
learning curve process.
• Based on current reliability trends, projections to maturity
may not be appropriate. Reliability growth projection
methodologies often assume that a system is in a single
phase of testing, characterized by a nearly constant operating
mode and environment, and gets reliability improvements
incorporated while the system is under test. For most of
the F-35 program, these conditions have held sufficiently
true such that reliability growth displayed consistent
behavior; however, with the release of Block 2B capabilities,
including increased flight envelope, beginning in 2015,
both the operating mode and environment apparently
changed enough to constitute a new phase for the purpose of
analyzing reliability growth. Programs with multiple phases

F-35 JSF

91

 FY16 DOD PROGRAMS
of development, where each phase is defined by different
environments or operational usage, normally generate
separate reliability planning curves (used to determine
interim goals during that phase) and separate reliability
growth tracking curves for each phase, as a single curve is
not sufficient to mathematically represent reliability growth
behavior across multiple phases. Because the reliability
projections are based on data that span the periods of time,
both before and after the Block 2B fleet release, they may not
best capture reliability trends.
- For programs with multiple phases, it is common for
reliability to decrease or level off at the start of a new phase
when the system is subjected to a more stressing operating
mode or environment that exposes new failure modes. As a
result, reliability growth can come to a halt or even decline;
however, after a while, growth may resume as the program
starts to implement reliability improvements for these new
failure modes.
- Reliability growth may resume as a result of ongoing
program reliability improvement initiatives, continuing
to send aircraft through the depot modifications program,
replacing lower reliability components with higher
reliability versions via TCTDs, and other reliability
initiatives. However, DOT&E also expects that the
Block 3F envelope and capabilities release, incrementally
released between CY17 and CY18, will reveal new failure
modes (e.g., new weapons, higher airspeeds and g with
Block 3F envelope) that will limit the overall effect of these
reliability improvement initiatives.
- Despite the difficulty projecting accurate reliability values
at maturity, given the phased introduction of F-35 block
capabilities, DOT&E does not expect any variant to achieve
interim threshold goals for MFHBCF by the start of
IOT&E, considering the recent decline in this metric over
the past year. In fact, indications are that for each variant,
this metric is the furthest from its current interim goal.
• Failing to grow reliability sufficiently by the start of IOT&E
will make achieving the necessary 80 percent availability
to accomplish all mission trials within the planned time
span very difficult. Further, a failure to achieve adequate
MFHBCF reliability in particular will impede the ability of
the Operational Test Squadrons (OTS) to generate multiple
four-ship formations with all required mission systems
functional, a necessary condition for a set of the planned
mission trials.
• A number of components have demonstrated reliability much
lower than predicted by engineering analysis. This drives
down the overall system reliability and can lead to long wait
times for resupply as the field demands more spare parts
than the program planned to provide. Aircraft availability is
also negatively affected by longer-than-predicted component
repair times. The table at top right shows some of the
high-driver components affecting low availability and
reliability, grouped by components common to all variants,
followed by components failing more frequently on a
particular variant or which are completely unique to it.

92

F-35 JSF

HIGH-DRIVER COMPONENTS AFFECTING LOW AVAILABILITY
AND RELIABILITY
Variant

F-35A

F-35B

F-35C

Common to All Variants
• Avionics Processors
• Low Observable
Maintenance
• Shock Struts
• Cold Air Duct
• IPP Vent Fan Controller
• Main Landing Gear Tires
• Nutplates
• On-Board Oxygen
Generating System

Additional High Drivers by Variant
• Horizontal Tail Actuation
• Vertical Tail Bulb Seal
• Electronic Warfare Receiver
• Fuel System Components and
Mods
• Flexible Linear Shaped Charge
• Main Landing Gear Retract
Actuator *
• Nose Landing Gear Steering
Motor *
* Unique to the F-35C
IPP – Integrated Power Package

- The composition of the list of some of the high-driver
components has changed as the program has progressed
and either fielded more reliable components, or new
failures have occurred to displace previous high drivers.
For example, compared to the list reported in previous
DOT&E Annual Reports, the 270V DC battery and
associated components, the F-35B Upper Lift Fan Door
Actuator, and the exhaust nozzle assembly components
used on the F-35A and F-35C, are no longer high drivers.
Improving aircraft availability can be realized by more
than just improving the reliability of components and
restocking supply with improved, redesigned parts;
updating JTD and improving repair procedures can
contribute to increased aircraft availability as well.
However, in the current reporting period, overall reliability
has not increased and new components have become
high drivers, such as the Electronic Warfare Receiver and
the Vertical Tail Bulb Seal. Note also that the program
released Block 2B capabilities and flight envelope to the
fleet in the period of this report. As the flight envelope
is expanding and the fleet uses more mission system
capabilities, new failure modes will likely emerge
to dampen the overall effect of individual reliability
improvements, consistent with recent trends observed in
reliability growth analysis.
Maintainability
• The amount of time needed to repair aircraft and return
them to flying status remains higher than the requirement
for the system when mature, but has improved over the past
year. The program assesses this time with several measures,
including Mean Corrective Maintenance Time for Critical
Failures (MCMTCF) and Mean Time To Repair (MTTR)
for all unscheduled maintenance. MCMTCF measures
active maintenance time to correct only the subset of
failures that prevent the F-35 from being able to perform a
specific mission; it indicates how long it takes, on average,
for maintainers to return an aircraft from NMC to Mission
Capable (MC) status. MTTR measures the average active
maintenance time for all unscheduled maintenance actions;
it is a general indicator of the ease and timeliness of repair.

 FY16 DOD PROGRAMS
Both measures include active touch labor time and cure times
for coatings, sealants, paints, etc., but do not include logistics
delay times, such as how long it takes to receive shipment of
a replacement part.
• The tables below compare measured MCMTCF and MTTR
values for the 3-month period ending in July 2016 to the
ORD threshold and the percentage of the value to the
threshold for all three variants. The tables also show the
value from May 2015, the month reported in the FY15
DOT&E Annual Report, for reference. [Note that the
May 2015 values may be different than those in the FY15
DOT&E Annual Report due to the revision of the scoring
rules described at the beginning of the Reliability Growth
section above.] For maintainability, lower repair times are
better. Three of six metrics improved marginally, while three
metrics, F-35B and F-35C MCMTCF, and F-35A MTTR,
increased or worsened. Currently, all mean repair times are
at least or nearly twice as long as their ORD threshold values
for maturity, reflecting a heavy maintenance burden currently
being carried by fielded units.
F-35 MAINTAINABILITY: MCMTCF (HOURS)

(3 Mos. Rolling
Window)

Observed
Value as
Percent of
Threshold

(3 Mos. Rolling
Window)

Values as of
July 31, 2016

Values as of
May 2015

Variant

ORD
Threshold

F-35A

4.0

10.6

265%

11.4

F-35B

4.5

13.2

293%

12.7

F-35C

4.0

10.1

253%

8.4

F-35 MAINTAINABILITY: MTTR (HOURS)

(3 Mos. Rolling
Window)

Observed
Value as
Percent of
Threshold

(3 Mos. Rolling
Window)

2.5

6.3

252%

4.7

F-35B

3.0

7.3

243%

7.7

F-35C

2.5

4.9

196%

5.3

Variant

ORD
Threshold

F-35A

Values as of
July 31, 2016

Values as of
May 2015

- A more in-depth analysis of data from between
August 2015 and July 2016, in order to capture
longer-term 1-year trends, shows that for the MCMTCF
metric, the F-35A and F-35B repair times are decreasing,
while for the F-35C it is relatively flat. For overall
mean repair times, however, the F-35A exhibited a slight
increasing, or worsening trend; the F-35B showed a
slight decreasing, or improving, trend; and the F-35C was
relatively stable. Prior to May 2015, all six metrics were
improving. In contrast, the more recent trend from this
period generally indicates a slowing of improvement in the
maintainability metrics.
- All six maintainability metrics exhibit high month-tomonth variability. Due to this variability, it is difficult
to make projections in trends for maintenance metrics;
however, it will be challenging for the program to meet the

threshold values by maturity with the rate of improvement
slowing and when current values for repair times are at
least twice as high as requirements.
- Several factors negatively influenced the ability to conduct
quick and efficient maintenance. Extensive adhesive cure
times for structural repairs, such as attaching hardware
(e.g., nutplates and installing heat blankets around the
engine bay), as well as long material cure times for low
observable (LO) repairs, remain drivers. The cure time
for some LO materials can be as high as 168 hours, for
example, although units can accelerate this if they have
appropriate tools.
- Other factors that indirectly affect maintainability
metrics have also been raised as concerns by maintainers.
Maintainers must physically connect Portable Maintenance
Aid (PMA) laptops to the aircraft in order to conduct most
maintenance activities. The PMAs enable the maintainers
to get status and configuration information from the
aircraft, as well as control aircraft functions to enable other
maintenance, such as opening the bomb bay doors where
the cooling-air receptacle is located in order to apply
air conditioning while running avionics on the ground.
Maintainers also access the Anomaly Fault Resolution
System (AFRS), which automatically troubleshoots Health
Reporting Codes (HRCs) generated by the on-aircraft
PHM system, and access JTD, which tells maintainers
how to effect repairs identified by AFRS, via the PMA.
Finally, maintainers record their work with the PMAs
as well. However, synching the PMAs to the aircraft to
conduct maintenance has been difficult, time-consuming
and, in many instances, maintainers must attempt to
synch several PMAs with an aircraft before finding one
that will successfully connect. These connections are
called Maintainer Vehicle Interface (MVI) sessions.
Occasionally PMAs disconnect in the middle of an MVI
session, which also hampers efficient maintenance.
Recently, the program introduced improved MVI cable
adapters to prevent accidental physical disconnection,
which has helped. Software-related problems persist as
well, such as PMAs taking anywhere from seconds to
minutes to connect. This occasionally leads maintainers
to disconnect a PMA they incorrectly believe is failing to
connect, which prevents that PMA from connecting to an
aircraft until an Automatic Logistics Information System
(ALIS) administrator resets it, which can be a lengthy
process.
• Maintainers have reported several difficulties with
troubleshooting the aircraft, which is the first step in many
maintenance actions. Normally, the aircraft PHM system
produces HRCs and then maintainers use AFRS to identify
possible root causes for those HRCs as well as determine
the appropriate repair action. Often, AFRS will provide a
“solution set,” which lists several possible root causes for
an HRC, rank ordered by probability of occurrence. While
AFRS coverage is improving, it currently provides effective
solution sets only approximately 70 percent of the time.

F-35 JSF

93

 FY16 DOD PROGRAMS
Particularly, when an aircraft fails a Vehicle Systems (VS)
Built-In Test (BIT), an aircraft self-check conducted pre- and
post-flight, there is no specific HRC produced, making these
relatively frequent occurrences difficult to troubleshoot.
When there is no HRC, such as in a VS BIT failure or
manually reported fault, or AFRS does not produce a solution
set for an HRC, or all the solutions offered by AFRS fail to
resolve a fault, units must use other resources to troubleshoot
the discrepancy. The primary method is to submit Action
Requests (AR) to the joint JPO-Lockheed Martin Lightning
Support Team (LST), whose engineers will further
troubleshoot the aircraft remotely. The AR response times
vary significantly, depending on category and urgency, but
average several days to get a final response. Alternatively,
or in conjunction, maintainers can use experience to
troubleshoot on their own; however, in most cases they lack
any system theory-of-operation or troubleshooting manuals
that tell them how aircraft systems work. The current JTD
are primarily dedicated to instructions only for repair actions
for which AFRS has already identified a solution, and not
for teaching maintainers the details of systems operations.
Recently, the program and Lockheed Martin have started to
provide some troubleshooting manuals to field maintainers
for select mission systems to try to improve the poor fleet
FMC performance. The extent to which these manuals will
help troubleshooting and result in higher FMC rates remains
to be determined.
• F-35 flying squadrons also have a heavy burden of scheduled
maintenance. In particular, maintenance units have reported
that daily servicing and inspection tasks, known as the
Before-Operations Servicing (BOS), Inter-Operations
Servicing (IOS), and Post-Operations Servicing (POS),
are very time-consuming compared to similar inspections
on legacy aircraft. Some of these daily inspections also
require power and cooling air application on the aircraft, so a
unit’s ability to perform them is a function of the amount of
Support Equipment (SE) assigned or available when needed.
As the fleet matures and more data become available, the
Services may be able to increase intervals between certain
scheduled inspection tasks to reduce the man-hours that units
must dedicate to this type of maintenance, if field experience
warrants this. However, it is not clear the scheduled
maintenance burden will reduce in the near future.
Autonomic Logistics Information System (ALIS)
• The program continues to fall behind in ALIS development
and fielding. Although the program planned to test and field
the next iteration of capability, designated ALIS 2.0.2, in
2016 to support the Air Force’s decision to declare Initial
Operational Capability (IOC) in August, the program failed
to do so. Additionally, the program continued to defer
planned content from ALIS 3.0 to post-SDD development.
• ALIS includes hardware and software that connects with
all aspects of F-35 operations, including maintenance
management, aircraft health, supply chain management,
Offboard Mission Support (OMS) mission planning, along

94

F-35 JSF

with tracking and management of pilot and maintainer
training. Units rely on ALIS for planning and executing
deployments by managing the data required to transfer
aircraft, materiel, and personnel from home station to a
deployed or expeditionary environment. Similar to the
manner in which the program develops and fields mission
systems capability in the air vehicle, it fields ALIS in
increments.
- The program fielded ALIS software version 2.0.1.1 in
late 2015. Since that time, the program has released
two updates, 2.0.1.2 and 2.0.1.3, to address previously
identified, usability-related deficiencies. These software
updates include fixes to existing deficiencies and usability
problems, but do not add new capabilities to ALIS. Prior
to the release of the first update with ALIS 2.0.1.2, the
program attempted to field ALIS software versions with
both new capabilities and deficiency corrections, a process
which tended to add new problems while fixing some
existing problems. Instead, the program now plans to
continue fielding updates dedicated only to correcting
deficiencies every three months until the release of ALIS
3.0, the final release scheduled for SDD.
- Although the program had planned to field a new
version of ALIS software, version 2.0.2, in the second
half of 2016, in time to support the U.S. Air Force IOC
declaration, it was unable to do so. ALIS 2.0.2 includes
propulsion integration, a key capability the Air Force
had planned to have for IOC; however, the Air Force
declared IOC with ALIS 2.0.1 in August, forgoing those
capabilities. Because the program continued to experience
technical difficulties integrating propulsion functionality
into ALIS, fielding of 2.0.2 slipped into CY17. As a result,
operational units began 2016 with ALIS 2.0.1.1 and will
finish the year with ALIS 2.0.1.3; receiving only updates to
address deficiencies and without any additional capability
fielded in ALIS.
- Delays in ALIS 2.0.2 have affected the development of the
next, and last, major release of ALIS software within SDD,
ALIS 3.0, because Lockheed Martin shifted personnel
from ALIS 3.0 development to support completing
ALIS 2.0.2 development. Because the program can
no longer complete ALIS 3.0 with all of the additional
capability development planned by the end of SDD, it
has restructured the planned ALIS increments for the
remainder of SDD and for Follow-on Modernization
(FoM). This restructuring reduces the content of ALIS 3.0
from earlier plans, defers content from ALIS 3.0 that the
program has now determined is not required for IOT&E
to post-SDD development, and also adds Service and
partner priorities and emerging requirements for security
updates. The resulting plan from the restructuring was
to field four increments of software at 6-month intervals;
the first, ALIS 3.0, scheduled to field in mid-to-late 2018,
which is required for IOT&E, followed by the remaining
three after SDD. These incremental software releases are
also intended to resolve ALIS deficiencies and usability

 FY16 DOD PROGRAMS
problems. At the mid-point between each of these major
releases, the program plans to deliver software updates to
continue addressing usability problems and deficiencies.
Because no fielding or Logistics Test and Evaluation
(LT&E) events of additional ALIS capability have occurred
for over a year, the program’s plan to develop, test, and
field these ALIS 3.0 and later versions appears overly
ambitious with a low likelihood of actually being realized.
Regardless of whether ALIS 3.0 or a later version has
been fielded, or which capabilities are included, IOT&E
will evaluate the suitability of the F-35 and ALIS in
operationally realistic conditions.
• Until 2016, formal testing of ALIS software only took
place at the Edwards AFB, California, flight test center on
non-operationally representative ALIS hardware, which
relied on reach-back capability to the Lockheed Martin
facilities at Fort Worth, Texas. Although some formal testing
will continue to occur in this manner, the program developed
and fielded a dedicated end-to-end developmental testing
venue for ALIS located in part at Edwards AFB and in part at
Lockheed Martin in Fort Worth in 2016. This venue, referred
to as the Operationally Representative Environment (ORE),
reflects the end-to-end Autonomic Logistics infrastructure
used to support fielded operations, including one Autonomic
Logistics Operating Unit (ALOU), which represents the
main hub at Lockheed Martin Fort Worth, two Central
Points of Entry (CPEs), representing the country-unique
portal from the main hub, and two Standard Operating Units
(SOUs), representing squadron-level ALIS components, all
networked together in a closed environment. Although the
ORE provides for more realistic developmental testing of
ALIS hardware and software for early problem discovery
and fixing deficiencies, the current closed environment does
not adequately represent the variety of ways in which the
Services operate ALIS in different environments. ALIS
testing at the flight test center is limited in several ways.
First, the inability of ALIS to support their engines and lift
fans, which differ from production models, so LT&E of
propulsion functionality in ALIS cannot take place there.
Also, the flight test center does not use ALIS capabilities
routinely, such as Squadron Health Management (SHM),
AFRS, or the Computerized Maintenance Management
System (CMMS), as operational units do. Finally, the flight
test center does not use PHM capabilities, as they are used by
operational units, since the flight test aircraft have additional
sensors and onboard instrumentation that provide the flight
test center with more information than is available through
PHM.
ALIS Software Testing and Fielding in 2016
• Although the program planned to test and field new capability
with ALIS 2.0.2 software release in 2016, it failed to do so.
The plans for added capability in ALIS 2.0.2 include:
- Life Limited Parts Management (LLPM), which includes:
▪ Propulsion integration. Currently propulsion data are
downloaded from aircraft portable memory devices and

provided to Pratt & Whitney Field Service Engineers for
processing and generation of maintenance work orders.
Propulsion integration will allow ALIS to process
propulsion data in the same manner as aircraft data.
▪ Production Aircraft Inspection Requirements (PAIRs).
ALIS 2.0.2 will include the first phase of the PAIRs
system. The program added PAIRs as part of the
PHM after eliminating most of the originally planned
prognostic algorithms. The program plans to include
8 prognostic algorithms in ALIS 2.0.2 and 8 in ALIS 3.0
out of the originally planned 128 SDD algorithms.
- Sub-squadron reporting. This will allow the air vehicle
to report its status back to the home squadron SOU
even when it is deployed away from the majority of a
squadron’s assets.
- SOU-to-SOU communication. Currently, information on
one U.S. SOU is transferred to another by routing files
from the originating SOU through the CPE at Eglin AFB,
Florida, to the ALOU at Fort Worth, Texas, back through
the CPE and to the receiving SOU. This new capability
will permit targeted routing of files between SOUs under
specific circumstances and is geared primarily toward
making aircraft deployments more efficient.
- Deployability improvements. This includes improved
deployment planning and the bulk transfer of all deploying
assets at once. The current release of ALIS makes
deployment planning inefficient as it does not provide a
centralized location in ALIS for this function. During
deployments, squadrons currently transfer aircraft, supply,
and support equipment data files individually.
- Commercial Off-the-Shelf (COTS) hardware replacement.
This allows the program to plan for hardware obsolescence
and substitute newer hardware over time.
- ALIS Readiness Check. Improves the health monitoring
of ALIS processes.
• Testing of ALIS 2.0.2 will occur in multiple stages at
multiple venues. The program plans to conduct an LT&E on
the air vehicle portion of the ALIS 2.0.2 software package in
early 2017, including initial testing of the propulsion module
of the software in the ORE. Once those tests are complete,
the program plans to do a validation and verification of
the process to upgrade to ALIS 2.0.2, including the data
migration, at an operational unit – possibly Luke AFB,
Arizona – before fielding ALIS across the rest of the F-35
operating locations.
• Releasing ALIS 2.0.2 to field units will require significant
manual intervention and data verification efforts to transition
each site, which will likely affect flight operations. The
data migration effort for ALIS 2.0.2 will be more complex
and will take longer than previous ALIS releases because of
propulsion integration and changes in data structures. For
example, the Program Office noted that one ALIS domain
alone, Customer Relationship Management, will require
40 man-hours for data migration and verification. Currently,
the program estimates that each site will require 8 days

F-35 JSF

95

 FY16 DOD PROGRAMS
to complete the transition of all assets. Lockheed Martin
will conduct the migration and has plans to complete the
transition at each site by using the Friday through Monday
time period of two consecutive weeks. Whether or not the
affected squadron can continue flying operations between
the two transition periods is unknown. As of September
2016, the program must transition 56 sites—either SOUs or
CPEs—through this process. As of the time of this report,
the program had not released a comprehensive transition
plan.
Assessment of ALIS Support to Deployment Demonstrations
with Operational Units
• Because of delays in ALIS release 2.0.2, fielded units have
operated with ALIS 2.0.1 since October 2015. As planned,
the Marine Corps used this release for a deployment
demonstration to the Marine Corps Air Ground Combat
Center (MCAGCC) Twentynine Palms, California, in
December 2015, which DOT&E reported on in the FY15
DOT&E Annual Report. Similarly, the Air Force conducted
a deployment demonstration to Mountain Home AFB, Idaho,
in February 2016. The operational test squadrons from
Edwards AFB participated in each of these demonstrations;
however, the ALIS hardware came from operational units
(a Marine Corps squadron from MCAS Yuma for the
MCAGCC demonstration and an Air Force squadron from
Hill AFB, Utah, for the Mountain Home demonstration).
• The Air Force completed its first F-35A deployment away
from Edwards AFB, California, with six aircraft from the
31st Test and Evaluation Squadron (31TES) to Mountain
Home AFB, which has no organic F-35 capability, from
February 8 to March 2, 2016. All aircraft that participated
in the deployment were in Block 2B configuration
with software version 2BR5.2. This deployment was a
Service-led assessment.
- This deployment was the first time the Air Force deployed
with a modularized, more transportable version of the
ALIS hardware, referred to as SOU v2. ALIS software
version 2.0.1 was used for this deployment, as well as
for the Marine Corps’ deployment to Twentynine Palms;
the previous “cross ramp” deployment at Edwards
AFB in May 2015 used the bulky SOU v1.1 Deployed
personnel had no difficulty setting up and configuring the
ALIS network at Mountain Home AFB; however, they
had a great deal of difficulty using ALIS on the local
base network. After several days of troubleshooting,
Information Technology (IT) personnel and ALIS
administrators determined that they had to change several
settings on the base network at Mountain Home and in
the web interface application (i.e., Internet Explorer) to
permit users to log on to ALIS. One of these changes

1.

The 31st TES previously conducted a “cross ramp deployment” at its home
base, Edwards AFB, from April 27 to May 8, 2015, to support deployment
concept of operations development. DOT&E reported on this activity in the
FY15 Annual Report.

96

F-35 JSF

involved lowering the security setting on the base network,
an action that may not be compatible with required
cybersecurity and network protection standards in place.
- Data file transfers took place more quickly than in the
previous F-35 deployment demonstrations, (i.e., the
F-35A cross ramp deployment and the Marine Corps’
deployment demonstration to MCAGCC Twentynine
Palms). However, Lockheed Martin provided the five
ALIS administrators normally assigned to the 31TES and
three additional, highly experienced ALIS administrators
from other locations to provide deployment support, more
than for any previous deployment. Whether the Service’s
concept of operations for deploying ALIS will call for this
level of ALIS administrative support, to ensure timely
and accurate transfer of aircraft data at the deployed
location, is still not known. Although the process was
time-consuming and labor-intensive, they completed
the transfer of all data to the deployed SOU v2 before
deployed flight operations were scheduled to begin. To
account for the expected extended time for data transfers,
the 31TES allocated the ferry date and two additional
days to complete the transfers; flight operations began
on the third day of the deployment, as planned. Service
deployment concepts of operations may need to account
for time to transfer aircraft data files and ensure accuracy
before beginning – or at least sustaining – operations at
deployed locations.
- Because of ambiguity in the ordnance loading technical
data, one aircraft experienced major damage to a weapons
bay door and horizontal tail early in the deployment when
a bomb, which was incorrectly loaded, struck the aircraft
following release. Aircraft repairs were extensive enough
to require most of the remainder of the deployment to
complete. The Marine Corps had previously discovered
this ambiguity in the technical data, but the program did
not disseminate this information across the F-35 enterprise.
- Preparations to redeploy back to Edwards AFB began on
March 1, 2016, with aircraft departing on March 2 and
aircraft data file transfer from the deployed SOU beginning
as soon as the aircraft took off from Mountain Home AFB.
Though ALIS administrators transferred all data off the
deployed SOU at Mountain Home AFB, administrators
at Edwards AFB did not finish inducting aircraft files
back onto the Edwards AFB SOU until March 4. The
redeployed aircraft were ready for flight at Edwards on
March 5, a 4-day transition period.
• Since the Services have not yet completed ALIS Concept of
Operations (CONOPs) development, they will likely need
to take into account the results of these deployments when
determining the procedures and timing of F-35 deployments.
Although the aircraft may be flown for short periods of
time without ALIS, operational planners may need to allow
for additional time between aircraft deployment and the
beginning of deployed flight operations, compared to legacy

 FY16 DOD PROGRAMS
platforms. Deployed operations, including the set-up and
support from ALIS, will be evaluated during IOT&E.
• The challenges facing the Services and program in making
ALIS deployable now involves software. Previously, the
program identified the need to move from the bulkier, heavy
SOU version 1 (v1) racks, which weighed approximately
1,600 pounds each, to the more customizable, modularized,
two-man portable components in the SOU v2, so that
the ALIS “footprint” could meet F-35 deployability
requirements. Although the SOU v2 has improved
the deployability of the ALIS hardware, these recent
deployments show that lack of flexibility exhibited in
integrating ALIS into new or existing networks, along
with deficiencies in ALIS functionality and usability,
contribute more to deployability problems than just the
previously-identified hardware limitations.
ALIS Software and Hardware Development Planning from
2016 through the End of SDD
• In CY16, the program continued to struggle with providing
the planned increments of capability to support the scheduled
releases of ALIS software 2.0.2 to such an extent that
the program now cannot accomplish the original plan for
ALIS 3.0 development. As the objective date for Air Force
IOC neared, the program considered releasing ALIS 2.0.2
in two increments: the first with all capabilities aside from
propulsion integration in time to support an August 2016
Air Force IOC declaration; the second with propulsion
integration, when the program overcame technical problems
and completed formal testing. When the Air Force declared
IOC without ALIS 2.0.2, using the already-fielded version
of ALIS 2.0.1.3 instead, the need for a two-phase release
no longer existed. As a result, the program now plans to
conduct the LT&E of ALIS 2.0.2 in two parts in early 2017;
the first with all functionality except propulsion integration at
the flight test center, then propulsion integration in the ORE.
ALIS 2.0.2 has been delayed for over a year from the release
schedule approved in CY15.
• The Program Office planned for the release of ALIS 3.0
in June 2017, in time to support its planned start date for
IOT&E, but now plans to release it in mid-to-late 2018.
However, the ongoing delays with ALIS 2.0.2 and the
resulting restructuring of ALIS 3.0 and beyond, have caused
the program to defer capability that had been planned to be
delivered with ALIS 3.0. The following list includes major
capabilities the program planned for ALIS 3.0 inclusion, and
identifies which ones are now being deferred – in full or in
part – out of SDD:
- Decentralized maintenance. This will enable execution
of the sortie generation cycle with a deployable PMA for
independent maintenance workflow while maintainers
work in the shadow of the aircraft. Decentralized
maintenance is now divided into two parts, both deferred
to post-SDD software versions.
- Resource sharing. This capability will allow the sharing
of tools, support equipment, pilots, and training records

-

-

-

-

across squadrons without requiring the transfer of data
between SOUs. Deferred to post-SDD software release.
Security enhancements. This includes additional ALIS
readiness checks to validate and monitor user accounts and
additional penetration testing.
Offboard Prognostic Health Management (PHM).
Additional algorithms to assess materiel condition
independently of ALIS releases and to implement a
correlation function between the Integrated Caution,
Advisory and Warning (ICAW) system and HRCs.
Partially deferred to post-SDD software release; only 16
of 128 planned prognostic algorithms are now included
within SDD.
Life Limited Parts Management (phase 2). Adds an
Identify Locate (IDLO) viewer for product life-cycle
management, support for lightning protection and
On-Board Inert Gas Generation System (OBIGGS),
Illustrated Parts Breakdown product, Complex PAIRs to
manage remaining life of aircraft components, support
for quick engine changes, the HMDS, and back-shop
visibility for supply chain management. Full Life Limited
Parts Management in ALIS was a capability the program
originally planned for ALIS 2.0.0 to support Marine
Corps IOC; however, the re-baselining of this technically
difficult-to-implement capability has resulted in it not
being fielded for at least 2 years after IOC declaration.
COTS hardware replacement.
Corrosion Management System. Will improve the ability
of ALIS to track and report the corrosion conditions of
aircraft using two sensors located in designated positions
within the aircraft and includes corrosion HRCs in ALIS.
Deferred to post-SDD software release.
Low Observable Health Assessment System (LOHAS)
enhancements. Partially deferred to post-SDD release.

Prognostic Health Management (PHM) within ALIS
• The PHM system is designed to collect performance data to
determine the operational status of the air vehicle and, upon
reaching maturity, will use data collected across the F-35
enterprise and stored within PHM to predict maintenance
requirements based on trends. The PHM system is
designed to provide the capability to diagnose and isolate
failures, track and trend the health and life of components,
and enable autonomic logistics using air vehicle HRCs
collected during flight and saved on aircraft PMDs. The
F-35 PHM system has three major components: fault and
failure management (diagnostic capability), life and usage
management (prognostic capability), and data management.
PHM diagnostic and data management capabilities remain
immature. The program has yet to integrate any prognostic
capabilities; the first set of algorithms is planned for ALIS
2.0.2.
• Diagnostic capability should detect true faults within the
air vehicle and accurately isolate those faults to a line
replaceable component. However, to date, F-35 diagnostic
capabilities continue to demonstrate poor accuracy, low

F-35 JSF

97

 FY16 DOD PROGRAMS

•

•

•

•

detection rates, and also have high false alarm rates.
Although coverage of the fault detection has grown with
the fielding of each Block of F-35 capability, all metrics of
performance remain below threshold requirements. The
table below compares specific diagnostic measures from the
ORD with current values of performance through April 2016.
PHM monitors nearly every on- and offboard system on the
F-35. It must be highly integrated to function as intended
and requires continuous improvements for the system to
mature.
Poor diagnostic performance increases maintenance
downtime. Maintainers often conduct BITs to see if the
fault codes detected by the diagnostics are true faults. False
failures (diagnostics detecting a failure when one does not
exist) require Service personnel to conduct unnecessary
maintenance actions and often rely on contractor support
to diagnose system faults more accurately. These actions
increase maintenance man-hours per flight hour, which
in turn can reduce aircraft availability rates and sortie
generation rates. Poor accuracy of diagnostic tools can also
lead to desensitizing maintenance personnel to actual faults.
The number of false alarms recorded within ALIS can be
artificially lowered, as qualified maintenance supervisors can
defer or cancel an HRC without generating a work order for
maintenance actions, if they know that the HRC corresponds
to a false alarm not yet added to the nuisance filter list. The
deferred or canceled HRC will not result in the generation
of a work order, and it will not count as a false alarm in the
metrics in the table below. The program does not score an
HRC as a false alarm unless a maintainer signs off a work
order indicating that the problem described by the HRC did
not occur. Because PHM is immature and this course of
action saves time for the maintainers, it occurs regularly at
field locations; however, this means the number of recorded
false alarms is not always an accurate reflection of the HRC
false alarm rate.
Comparing the values in the table below with those in the
FY15 DOT&E Annual Report shows improvement in Fault

Detection Coverage, Fault Detection Rate, Fault Isolation
Rate for non-electronic faults to one Line Replaceable
Component (LRC), and – most significantly – Mean Flight
Hours Between Safety Critical False Alarms. Mean Flight
Hours Between False Alarms and Fault Isolation Rate
for non-electronic faults to three or fewer LRCs show
no significant improvement, and Fault Isolation Rate for
electronic faults to one LRC has gotten worse since last
year’s report. At this time, Mean Flight Hours Between
Flight Safety Critical False Alarm and Fault Isolation Rate
for non-electronic faults to one LRC are the only diagnostic
metrics which appear to be improving adequately toward
meeting their threshold requirements. The program planned
for accurate diagnostics to support a planned level of
sustainment; poor diagnostics contribute to poor reliability
and maintainability metrics, reducing aircraft availability and
increasing aircraft downtime.
• Following are the systems most likely to result in missed
fault detections, incorrect fault isolations, and false alarms as
of April 2016.
- Missed detections: Integrated Core Processor (ICP),
Communications, Navigation, and Identification (CNI)
rack modules, Panoramic Cockpit Display, Power and
Thermal Management System (PTMS), and vehicle system
processing.
- Incorrect isolation: ICP, PTMS, EW, electric power, and
hydraulic power system.
- False alarms: Propulsion, CNI system, EW, ICP, and
displays and indicators in general.
• The Program Office initiated a PHM maturation plan in
2015 to improve the performance of each of the three major
components of PHM:
- Improving BIT functionality, PHM software handling of
BIT results, and off-aircraft filter lists and fault isolation
instructions; also focusing on identified high-fault drivers
to prioritize developing AFRS solutions with the greatest
impact on fault detection and isolation, false alarm

METRICS OF DIAGNOSTIC CAPABILITY
(6-month rolling window as of April 2016. Data provided by Program Office considered “preliminary” as they have not completed formal adjudication process by the data review board.)

Threshold
Requirement

Diagnostic Measure

Demonstrated Performance
Block 1

Block 2

Block 3

Developmental Test and Production Aircraft
Fault Detection Coverage (percent mission critical failures detectable by PHM)

N/A

88

88

93

Fault Detection Rate (percent correct detections for detectable failures)

98

88

88

93

Fault Isolation Rate (percentage): Electronic Fault to One Line Replaceable Component (LRC)

90

65

64

42

Fault Isolation Rate (percentage): Non-Electronic Fault to One LRC

70

71

73

86

90

87

87

100

Mean Flight Hours Between False Alarms

50

0.09

0.41

0.50

Mean Flight Hours Between Flight Safety Critical False Alarms

450

61

537

437

Fault Isolate Rate (percentage): Non-Electronic Fault to Three or Fewer LRCs
Production Aircraft Only

Accumulated Flight Hours for Measures

N/A

61

6,440

6,111

Ratio of False Alarms to Valid Maintenance Events

N/A

135:1

19:1

19:1

98

F-35 JSF

 FY16 DOD PROGRAMS
performance, unnecessary maintenance, high maintenance
man-hours, aircraft availability, and excess cost
- Improving the functionality of PAIRS and algorithms
which assess materiel condition based on usage and repair
feedback, potentially adding new life tracking items based
on fleet experience
- Improving or adding data collection from the air vehicle,
improving data downloading and processing from the
aircraft to ALIS, and improving distribution and storage of
data to better support user needs
• Structural PHM (SPHM) is a key element of overall
airframe life-cycle management. It includes conditional
event detection and analysis, including over-g, hard landing,
overspeed, and overload conditions, and is planned to
provide a corrosion monitoring and predictive modeling
capability. The air vehicle currently includes two corrosion
sensors—one on the forward face of the radome bulkhead
and the other on the wall of the bay housing the fuel/heat
exchanger. ALIS 2.0.0 included a logging function for these
corrosion sensors. A Program Office study completed in
November 2015 determined that 27 percent of the corrosion
sensors in the fleet had failed, so the program is in the
process of developing a new sensor manufactured with more
precise sealing applications to be used during production
instead of upon installation.
Air-Ship Integration and Ship Suitability Testing
F-35B
• The integrated test team from Patuxent River, Maryland,
conducted the third and final planned set of F-35B ship
trials, referred to as Developmental Test III (DT-III), from
October 28 through November 17, 2016, on USS America.
The objectives for this 3-week developmental test event
included:
- Expanding the vertical landing flight envelope for both day
and night operations (higher wind-over-deck conditions
and operations at higher sea states than earlier ship trials,
as well as operating from additional landing spots farther
forward on the flight deck)
- Evaluating the Gen III HMDS for nighttime landings, with
or without landing aids on the ship
- Assessing Joint Precision Approach Landing System
(JPALS) functionality
- Conducting vertical landings and short take-offs with
symmetric and asymmetric external loads carriage
- Expanding vertical take-off capability
- Evaluating environmental effects from flight operations,
such as the thermal tolerance and response of the flight
deck to vertical landings and noise surveys from various
ship locations
- Conducting maintenance demonstrations – including
engine and lift fan removal and replacement actions, and a
power module maintenance demonstration – and loading
and unloading of external stores

- Evaluating the operational capability of the first
deployment of an ALIS SOU v2 on the ship
• Besides the two developmental test aircraft from the Patuxent
River test force (BF-1 and BF-5), the Marine Corps also
supported the test activities by providing an additional three
instrumented operational test aircraft assigned to VMX-1,
the operational test unit at Edwards AFB, California, and two
fleet aircraft from VMFA-211, one of the two operational
units at MCAS Yuma, Arizona. Although primarily a
developmental test event, the Marine Corps embarked fleet
and operational test squadron personnel for training, and
to inform the JSF Ship Integration Team in preparation for
the first operational F-35B deployment onboard USS Wasp,
planned for late 2017. From November 17 – 21, the Marine
Corps also conducted a “Lightning Carrier” proof of concept
demonstration, with an additional five F-35B fleet aircraft
plus two MV-22 and two H-1 Air Combat Element (ACE)
assets deployed to the ship to assess interoperability and
the suitability of F-35B “Heavy” ACE configurations on
LHA-class ships. Observations from this testing included:
- The specialized secure space set aside for F-35-specific
mission planning and the required Offboard Mission
Support (OMS) workstations is likely too small and
therefore unsuitable for regular ACE operations with the
standard complement of six F-35B aircraft – let alone
F-35B Heavy ACE configurations with more aircraft. Due
to the classification of certain F-35 capabilities, pilots
must conduct mission planning in a secure space. The
ALIS SOU v2, which has several classified components,
was also located in this space. However, pilots, the ALIS
administrator, and security personnel commented that the
compartment designated for the secure workspace onboard
USS America was too small to accommodate enough
OMS workstations and a sufficient briefing and debriefing
area. Marine Corps and ship personnel are investigating
using this compartment for ALIS only, and designating an
alternate compartment for mission planning.
- The power module maintenance demonstration was
intended to show that a deployed unit could conduct
modular engine maintenance at-sea. The F135 engine
is modular, with a fan and compressor section; a power
section with the combustion chamber and turbine stages;
an afterburner section, which on the F-35B consists
of a Three-Bearing Swivel Module (3BSM) that can
rotate downward to more than 90 degrees for vertical
flight; and a nozzle section. The general maintenance
concept for a failed engine is to replace only the defective
module on any given engine to return the overall engine
to service more quickly, and send the defective module
to depot-level repair. The demonstration consisted of
splitting open an F135 engine mounted on two aligned
Maintenance and Transportation Trailers (MTTs) into its
modularized sections, removing a “bad” power module,
taking a “good” spare power module out of its shipping
and storage container, placing the good module into the

F-35 JSF

99

 FY16 DOD PROGRAMS
engine, and containerizing the bad module, all with the
use of an overhead bridge crane in the aft high bay of the
hangar bay. The demonstration showed that maintainers
could swap a module at sea; however, the evolution took
up a large amount of space in the hangar bay and occurred
without a full ACE onboard. The Navy and Marine
Corps should conduct some further analyses, such as an
operational logistics footprint study which simulates flight
deck and hangar bay spotting with a full ACE onboard,
using data from this evolution to determine what the
impact of this maintenance would be on integrated ship
and ACE operations with a full ACE onboard.
- The detachment planned to stage an F135 engine removal
and installation (R&I) demonstration, but early in the
deployment maintainers discovered, during a PostOperations Servicing, that one of the OT aircraft (BF-20)
had a thrust pin that had unseated. There are several thrust
attachments between the engine and the airframe that
transfer the propulsive forces produced by the engine to
the airframe, and this was the first time in program history
that maintainers discovered a thrust pin had backed out
of full engagement, a serious safety of flight concern. As
a result, the unit submitted an AR to request disposition.
The AR response directed that the engine be removed from
the aircraft, and the thrust pin attachment points on both
the engine and airframe be thoroughly inspected. This
provided a natural opportunity to evaluate an actual engine
R&I as opposed to a staged demonstration. The unit
provided photos and dimensional data to the Lightning
Support Team (LST), initiating a long investigation
process to determine the root cause, but there were no
immediately obvious signs of wear or damage. The LST
eventually directed the squadron to replace the engine, as
there was a full spare engine onboard, and the lift fan drive
shaft. The squadron completed this maintenance in the
hangar bay and, on November 16, conducted a High-Speed
Low-Thrust (HSLT) engine operation on the flight deck
to confirm that the new engine was installed correctly and
fully functional. The unusual circumstances of this event
primarily drove the 2-week long R&I process, as opposed
to specific shipboard conditions and, by the time of this
report, the program had not yet determined a root cause.
However, the engine R&I was practically aided by the fact
that, for this detachment, a full spare engine was available
for immediate installation. Currently, the program’s
planned Afloat Spares Package of spare parts that will
be loaded onboard the USS Wasp for the first F-35B
deployment in 2017 will not have a full spare engine, only
spare propulsion modules. See the F-35C ship suitability
section for further details on F135 engine R&I concerns at
sea.
- The squadron also conducted a staged lift-fan R&I
demonstration on BF-20 while it was in an NMC status in
the hangar bay for the engine R&I. Maintainers positioned
the aircraft along the ship’s centerline and directly beneath
the bridge crane in the forward of two high bays. Organic

100

F-35 JSF

Marine squadron personnel first used a collapsible, portable
floor crane and an assembled support frame to cradle the
upper lift fan door and remove it from the aircraft, and then
place it on the deck. After maintainers attached another
assembled frame to the top and sides of the lift fan, ship
personnel used the overhead bridge crane to raise the lift
fan out of the aircraft cavity and, via attached tether ropes
to each of the four top corners of the frame to guide the
lift fan, lowered it to a support cradle on the deck. Service
personnel then reversed this process to reinstall the lift fan.
After the upper lift fan door was reinstalled and maintainers
were disassembling the support frame that attaches the
door to the crane, a portion of this assembly fell onto the
lift fan, damaging a stator strut at the top of the lift fan.
Repairs to this strut took another couple of days to complete.
Maintenance personnel noted several improvements that
should be incorporated into this process; most importantly,
the tether points for the lift fan support assembly need to
be moved to the bottom four corners for better control, as
the tethers provided very little control near the hook point
of the crane; also the program should provide a protective
maintenance cover for the lift fan to prevent damage during
future lift fan R&I’s or upper lift fan door maintenance.
- On November 15 and 16, a single fleet aircraft from
VMFA-211 departed from USS America to drop live
ordnance on targets on an inland range, hot-pitted for fuel
from MCAS Yuma, Arizona, and returned to the ship each
day. Both sorties dropped one GBU-12 laser-guided bomb
and one GBU-32 JDAM. The Marine Corps originally
intended to fly two loaded aircraft each day, but the lack of
available mission-capable aircraft drove the detachment to
launch only a single aircraft each day.
- While the set of sea trials were not focused on operational
realism, several aspects were more operationally
representative than the 2015 F-35B deployment
demonstration onboard USS Wasp. The aircraft had a full
suite of Block 2B electronic mission systems installed,
unlike onboard USS Wasp; however, like the USS Wasp
demonstration, these aircraft mission systems were not
maintained to a full combat-mission-capable state of
readiness. Unlike in 2015, the OT and fleet aircraft were
cleared to carry live ordnance on the flight deck, with
some workarounds. With this clearance, the test team
intended to employ live ordnance on missions. Productionrepresentative support equipment (SE) was onboard ship
for the first time as well for use on the non-DT aircraft.
Similar to the 2015 demonstration, the operational logistics
support system, known as the Autonomic Logistics Global
Sustainment system, was still not available. As a result,
spares provisioning and supply support were not necessarily
the same as would be expected on a combat deployment.
F-35C
• The third and final phase of F-35C ship suitability testing,
designated Developmental Test III (DT-III), was conducted
by VX-23, the developmental test team from Patuxent River,

 FY16 DOD PROGRAMS
from August 10 – 26, 2016, aboard USS George Washington.
The primary objective of DT-III was to complete
characterization of the flying qualities of the F-35C aircraft
for catapult launches and arrested recoveries, building on the
results from two previous at-sea developmental test periods.
The test team explored aircraft flight operations around the
carrier in high crosswind conditions and, for the first time,
with external ordnance, including asymmetric load-outs.
Both day and night operations were conducted, allowing for
assessments of the Gen III HMDS for night approaches and
landings under varying light conditions. These investigations
will help develop aircraft launch and recovery bulletins to an
expanded envelope to support fleet operations. Also, while
the ship was underway, VFA-101, the Navy’s F-35C training
squadron at Eglin AFB, Florida, participated in the event for
other test objectives, including a Commander of Naval Air
Forces (CNAF)-directed proof-of-concept demonstration
of an F-35C engine R&I in the ship’s hangar bay as well
as initial day carrier qualifications for 12 pilots that would
assess overall suitability of catapult launches and the Delta
Flight Path capability for carrier approaches and landings.
- Initially, only developmental test aircraft CF-3 and CF-5
(transient aircraft needed for logistical support) and search
and rescue helicopters deployed to the carrier. No air wing
was present. Five VFA-101 aircraft deployed onboard
the ship from August 14 – 18. The major contractor and
test team were responsible for maintenance of CF-3 and
CF-5, although fleet maintenance personnel supported
the VFA-101 carrier qualifications and the engine R&I
demonstration. ALIS was not installed on the carrier; it
was accessed via satellite link to a location ashore.
- The developmental test team conducted night operations
with modifications to the Helmet Display Unit for the
Gen III HMDS that permitted lower illumination settings,
intended to reduce the amount of “green glow” in the
helmet display that makes seeing the lights on the carrier
difficult during night operations. The test pilots reported
that the refined brightness control somewhat improved the
night carrier approaches; however, “green glow” was still
a significant problem and is the subject of two Category 1
deficiency reports.
- From the carrier qualifications, the VFA-101 pilots found
the F-35C catapult shot not operationally suitable due
to excessive vertical (Nz) oscillations during launch.
Although numerous deficiencies have been written against
the F-35C catapult shot oscillations – starting with the
initial set of F-35C ship trials (DT-I) in November 2014 –
the deficiencies were considered acceptable for continued
developmental testing. The fleet pilots reported that the
oscillations were so severe that they could not read flight
critical data, an unacceptable and unsafe situation during
a critical phase of flight. Most of the pilots locked their
harness during the catapult shot, which made emergency
switches hard to reach, again creating an unacceptable and
unsafe situation.

- The VFA-101 pilots reported that the Delta Flight Path
mode of operation made carrier approaches easier on
pilot workload and touchdown points more consistent.
During the qualifications, pilots made 154 approaches and
landings with 100 percent boarding rate and no bolters.
- The engine R&I proof-of-concept demonstration took
55 hours to complete and used about one-third to one-half
of one of the three hangar bay partitions; this is much
more space than that needed for an F/A-18 engine change.
Because it was the first F-35C engine R&I demo at-sea,
maintainers moved through all required steps at a slow
pace to ensure safety first, which may have extended
the timeline relative to what an experienced crew could
achieve during routine maintenance operations. On the
other hand, the maintainers had practically free use of
most of the hangar bay space, which may have facilitated
speedier maintenance relative to conducting an engine R&I
with a full air wing onboard. As a result, actual engine
R&I’s during deployments may not differ drastically in
time from this demonstration.
- While the proof-of-concept demonstration showed that
an engine could physically be swapped at sea, it also
revealed that such a major maintenance evolution would
be very difficult, time consuming, take up a large amount
of space, and be a drastic change from the engine R&I on
legacy aircraft. The F-35C engine change is also more
labor- and space-intensive than the F-35B engine R&I,
such as conducted onboard the USS America. The F-35B
engine R&I is aided by the aircraft’s 3BSM doors, which
open during regular operation to enable the exhaust nozzle
to rotate downward to more than 90 degrees for vertical
flight. Opening these doors for engine maintenance avoids
the need to remove fixed panels, such as on the F-35A
and F-35C. For the F-35C, many more skin panels and
a large piece of structure known as the tail hook trestle,
although not the tail hook itself, must be removed for an
engine R&I. Storing these items, and the associated tubes
and wire harnesses, so they will not be damaged while off
the aircraft, also takes up additional space. The fact that
the demonstration was conducted without a full air wing
on the ship additionally limited the test team’s ability to
assess the likely impact of an F-35C engine change on
integrated carrier-air wing operations. Such an assessment
will be needed for IOT&E. Because of the complexity
and time required to conduct an engine change, the Navy
and JPO should investigate alternatives for determining
the impact of an R&I while conducting carrier-air wing
operations as well as improving the maintainability of the
F-35 system at sea.
• Both the F-35B engine R&I onboard USS America and the
F-35C engine R&I onboard USS George Washington were
hampered by the lack of suitable strut locks approved for
at-sea use, considering the rolling and pitching motion that
may be experienced while underway. Since the engine is
a significant part of the aircraft weight, without strut locks

F-35 JSF

101

 FY16 DOD PROGRAMS
the airframe would raise up on the pressurized landing
gear struts as soon as the engine was detached. This could
potentially damage either the engine or airframe due to
tight tolerances, or injure maintainers with hands in the
area. In both cases, maintainers put the aircraft up on jacks
to de-service the struts before the engine change, and then
raised the aircraft back up on jacks to re-service the struts
after the change, adding significant time to the process.
Further, ship maneuvering is restricted when raising and
lowering aircraft on jacks; engine R&I times could be
decreased if the program develops, and the Navy approves,
appropriate strut locks for at-sea use.
• Maintainers conducted a less extensive power
module maintenance demonstration onboard
USS George Washington than the one performed on
USS America, consisting of removing a power module
from its container in the hangar bay, moving it to the
engine repair shop aft of the hangar bay, and returning it
to its container. To open the container, maintainers used a
motorized, wheeled, mobile crane that is part of the ship’s
SE complement to raise the container lid, which is composed
of the roof and four side walls, over the encapsulated power
module, and set it to the side in the hangar bay. A specialized
Electric Pallet Jack (EPJ) was then used to move the power
module, still attached to the container bottom, to the engine
repair shop, where it could be transferred to an MTT via an
overhead bridge crane. Maintainers expressed dissatisfaction
with the container design, which required a large amount
of space and a large piece of SE to remove, and stated that,
while suspended on a possibly pitching and rolling ship, such
a heavy item could present a safety hazard. They stated a
preference for the type of container used for the T56 engine,
installed on the E-2 Hawkeye and C-2 Greyhound aircraft.
This type of container has a door on one side that opens
outward, with the engine mounted on rails inside. An MTT
can be wheeled up to the container and the engine slid onto
it by hand. This configuration takes up less space to remove
an engine, doesn’t require any SE, is quicker, and presents
fewer hazards. The current container is designed to a very
high standard of structural integrity in order to withstand
a fall if ever resupplied by moving it across a wire strung
between a resupply ship and a carrier, a standard form of
resupply at sea. However, only the planned heavy E-Stream
wire system was capable of moving the heavy power module
container, but this program is now canceled. The Navy
now plans to resupply un-containerized power modules
via internal carriage on a CV-22 aircraft, and containerize
any spare modules onboard ship if needed for storage. The
program and the Navy should investigate if the heavy power
module container should be redesigned for better usability at
sea.
• Current program plans do not provide a full spare engine
for the envisioned Afloat Spares Package of parts that will
go onboard Navy CVN and L-class ships to support F-35C
and F-35B squadrons, respectively. This will significantly
increase the amount of time required to conduct an actual

102

F-35 JSF

engine change. The 55-hour timeline measured during the
proof-of-concept demonstration provided above assumed
a full spare engine ready for immediate install once the
down engine is removed from the aircraft. Without a spare,
the time required to troubleshoot the down engine to a bad
module, disassemble the engine to swap that module, and
then reassemble the engine to reinstall it into the aircraft
must be added to the overall process; this can easily add
several more days of downtime to the affected aircraft.
Further, the probability of Foreign Object Damage (FOD)
to engines is higher at sea than ashore, which may drive
more frequent engine R&Is at sea. This is due to the close
proximity of aircraft maintenance to the ship landing areas
allowing foreign objects to migrate, and the more stressing
arrested or vertical landings at sea, which can increase the
probability of items like fasteners falling off an aircraft into
the landing area.
• Access to ALIS offboard the ship via the ship’s satellite
communications was intermittent and troublesome, making
transmitting large file sizes difficult. For example, a 200 MB
file required 2 days to successfully transfer due to bandwidth
limitations and inconsistent connectivity. These issues
drove VFA-101 to operate in an ALIS offline mode for the
majority of the detachment. While the root cause appeared
to be due to limitations with the shipboard communications
equipment vice ALIS directly, and deployed units will have
an SOU onboard ship, the SOU will occasionally have to
transmit large files to the CPE due to how data-intensive
ALIS is. This requirement to communicate large amounts of
information will likely be exacerbated after a ship emerges
from a restricted Emissions Control (EMCON) period where
transmissions from the ship are severely limited or cut-off
completely. The program and the Navy should investigate
potential options to improve ship-based communications
bandwidth dedicated to ALIS connectivity off-ship, such as
increasing the priority of ALIS transmissions, or reserving
low-use times of the day for transmitting large volumes of
ALIS message traffic.
• VFA-101 brought a suite of production-representative SE to
the aircraft carrier, including electrically powered hydraulic,
air conditioning, and polyalphaolefin (PAO) carts for use
in the hangar bay. Personnel use the PAO cart to service
the aircraft with this special fluid that cools the radar and
some other avionics. The Navy prefers that SE for use in
hangar bays be electrical vice diesel powered because of
the enclosed environment. They also brought an engine
R&I trailer and an engine maintenance trailer, needed for
the engine maintenance demo. Collectively, these items
of SE were larger than legacy items and took up a large
amount of deck space. Hangar bay personnel commented
that the size of the SE would also make them more difficult
to move around a crowded hangar bay with a full air wing
onboard. The Navy should investigate any efficient, multiuse opportunities for F-35 SE, such as using legacy SE on
the F-35 or F-35 SE on legacy aircraft, to try to limit the
impact on the overall SE footprint for an air wing with F-35

 FY16 DOD PROGRAMS
included. Additionally, the JOTT will evaluate SE operation
and movement around the flight deck and hangar bay during
IOT&E.
• Since the hangar-bay SE items are electrical, they rely
on 440V power from outlets in the walls of the ship.
Maintenance on a single F-35C can sometimes require
external power, provided by a small transformer power cart
that converts the 440V wall power to the 270V and 28V
DC power used by the aircraft, along with air conditioning
and hydraulic power, each requiring separate carts. Such
maintenance activities would require the use of three wall
outlets. However, most hangar bay partitions had four
outlets, which would make simultaneous maintenance on
more than one F-35C in a partition a coordination challenge.
The Navy should investigate options for increasing the
number of wall power outlets in hangar bays to help facilitate
simultaneous maintenance on multiple F-35Cs, or the ability
to interconnect multiple pieces of support equipment from a
single outlet to permit simultaneous operations.
• The Navy is working on the following air-ship integration
issues, primarily for carrier operations. Some of the
following issues also apply to F-35B operations on L-class
ships:
- Flight deck Jet Blast Deflectors (JBDs) will require
additional side panel cooling in order to withstand regular,
cyclic limited afterburner use, during F-35C catapult
launches. JBDs are retractable panels that redirect hot
engine exhaust up and away from the rest of the flight
deck when an aircraft is at high thrust for take-off.
During IOT&E, an F-35C detachment will deploy to a
CVN to evaluate sortie generation rate capability within
an air wing context. The CVN used for IOT&E must
have additional side panel cooling installed in the JBDs
to enable the most operationally representative test to
evaluate this Key Performance Parameter of the F-35C.
- The Navy continues to procure a replacement mobile
Material Handling Equipment crane for several purposes
onboard carriers, including lifting the power module
container lid as described above. This crane will only be
used on CVNs, for F-35 maintenance only, as they lack
the hangar-bay overhead cranes that L-class ships come
equipped with. Since the FY15 DOT&E Annual Report,
the crane acquisition has proceeded at a pace such that
sufficient articles should be in the fleet in order to support
a first F-35C deployment in the 2020 timeframe.
- Two methods of shipboard aircraft firefighting for the F-35
with ordnance in the weapons bays are being developed,
one for doors open and one for doors closed. Each method
will use an adapter that can fit to the nozzle of a standard
hose. The open door adapter will also attach to a 24-foot
aircraft tow bar so firefighters can slide it underneath
the aircraft and spray cooling water up into the bay.
Development of this open door adapter is proceeding well
and it was deployed to the USS America to support live
ordnance carry by the OT and fleet F-35B aircraft during
DT-III. However, the closed bay adapter, which intends
to use water pressure to drive a saw to cut into the aircraft

and lock a hose in place to douse a loaded weapons bay
during a flight deck fire, was not yet ready for deployment.
As a workaround, F-35B aircraft on USS America with
live ordnance taxied with their weapons bay doors open,
closing them only right before take-off, to mitigate the
risk, but this will not be a standard practice for combat
deployments.
Cybersecurity Operational Testing
• The JOTT continued to accomplish testing based on the
cybersecurity strategy approved by DOT&E in February 2015,
with some modifications due to test limitations, discussed
below. In accordance with this strategy, in FY16 the JOTT
conducted adversarial assessments (AA) of the ALIS 2.0.1
Squadron Kit and Central Point of Entry (CPE), completing
testing that began in Fall 2015, and conducted cooperative
vulnerability and penetration assessments (CVPA) of the
mission systems Autonomic Logistics Operating Unit (ALOU)
used to support developmental testing (referred to as the
DT-ALOU), and the operational ALOU. The JOTT also
completed a limited cybersecurity assessment of the F-35
air vehicle. These tests were not conducted concurrently
as originally planned; therefore, end-to-end testing of
ALIS, from the ALOU to the air vehicle, has not yet been
accomplished. The JOTT initially tested the DT-ALOU in
lieu of the operational ALOU because the JPO did not approve
an Interim Authority to Test for the ALOU due to concerns
that cybersecurity testing would adversely affect the ALOU’s
operations; however, a limited test of the operational ALOU
was completed in October 2016 and an AA was scheduled for
December 5 – 9, 2016.
- The U.S. Navy’s Commander, Operational Test and
Evaluation Force (COTF) conducted a CVPA and limited
AA against the DT-ALOU, from April 1 – 15, 2016, at
Lockheed Martin’s Fort Worth facility. The COTF testing
verified that the DT-ALOU, configured with ALIS 2.0.1.3,
had mitigated several key vulnerabilities discovered on
ALIS 2.0.1.1 systems during fall 2015 testing. However,
this testing of the DT-ALOU was not operationally
representative because several key systems and external
interfaces, from which cyber-attacks might originate,
were not present. The testing was further constrained
because the Program Office and Lockheed Martin only
permitted testing to occur during overnight hours while the
DT-ALOU was disconnected from external networks to
minimize interference with operations. The COTF testing
still discovered several minor security problems with the
DT-ALOU. The operational ALOU is still configured with
ALIS 2.0.1.1.
- The U.S. Marine Corps Information Assurance Red Team
(MCIART) conducted an AA of the Marine Fighter Attack
Squadron 211 (VMFA-211) ALIS 2.0.1.3 Squadron Kit
at Marine Corps Air Station Yuma, Arizona, April 25
through May 6, 2016. The unit’s Squadron Kit was in
the process of being stood up, so it was not in a fully
operational configuration during the test. The operational

F-35 JSF

103

 FY16 DOD PROGRAMS
VMA-121 Squadron Kit was declared off-limits by Marine
Corps personnel. MCIART verified that several key
vulnerabilities discovered during the 2015 Squadron Kit
testing had been mitigated; however, MCIART discovered
several new vulnerabilities from insider and outsider threat
postures.
- The U.S. Air Force 177th Information Aggressor Squadron
(IAS) conducted an AA against the ALIS 2.0.1.3 Central
Point of Entry (CPE) at Eglin AFB, Florida, from
June 2 – 10, 2016. The 177 IAS assessed the system
as an outsider and near-sider threat, and discovered
vulnerabilities with various components of the CPE,
despite the fact that Lockheed Martin administrators and
ALIS users had implemented new operating procedures
during the test to improve the CPE security posture.2 The
CPE classified servers were not adequately assessed due
to time constraints and a lack of approval for connecting
177 IAS equipment to the classified CPE network.
- The JOTT, with support from the Air Force
Research Laboratory (AFRL), conducted a limited
CVPA of the F-35A Block 2B air vehicle, from
September 26 – 27, 2016, at Edwards AFB, California.
The CVPA tested the process by which the air vehicle
validates the digital signature of files within the operational
flight program when it is loaded onto the aircraft via the
aircraft media device. This test was one of the test cases
proposed by cybersecurity subject matter experts, and was
the first cybersecurity assessment of an operational F-35
air vehicle. The successful accomplishment of this initial
test should encourage the Program Office to examine
other planned test cases in future air vehicle cybersecurity
assessments. Analyses of the test results are ongoing.
- The COTF and the JOTT conducted a CVPA of the
operational ALOU October 17 – 28, 2016, at Lockheed
Martin’s Fort Worth facility. The test team was augmented
by Lockheed Martin Red Team members so that the
ALOU could be examined for vulnerabilities from the
Lockheed Martin Intranet (LMI). COTF and the JOTT
were not permitted to conduct any test activities on the
ALOU unless it was disconnected from the LMI, limiting
the operational realism of the test and precluding certain
vulnerabilities from being assessed. Detailed analyses of
the data collected are ongoing.
• In response to DOT&E’s recommendation that active
intrusion discovery and forensics, referred to as a Blue
Hunt, be conducted on the Squadron Kit and CPE, the JOTT
has scheduled the 855th Cyber Protection Team (CPT) to
conduct two events for the end of CY16. Current plans are to
perform mostly vulnerability assessment and traditional Red
Team activities against these systems —not active intrusion
discovery and forensics—and so it is still unclear whether
these events will fulfill DOT&E’s request. Additionally, the
2

Outsider threats have neither physical access nor account privileges to a
network; near-sider threats have physical access to a system, but no account or
log-in privileges to a network.

104

F-35 JSF

JOTT will need to conduct a Blue Hunt on the ALOU once
ALIS 2.0.2.4 is loaded and then additional Blue Hunts on all
ALIS levels (ALOU, CPE, and Squadron Kit) each time a full
increment of ALIS software is released.
• While progress towards fulfilling missed test opportunities in
2015 was considerable in 2016, full end-to-end cybersecurity
testing of the ALIS architecture, from the operational ALOU
to the air vehicle, remains to be completed. The JOTT is
planning concurrent assessments of the ALIS 2.0.2 Squadron
Kit, CPE, and ALOU in 2017. The JOTT is also exploring
testing opportunities on the F-35 training systems, and has
begun exploring options for testing systems at the U.S.
Reprogramming Laboratory, which generates mission data
files for the F-35.
• The JPO continued to develop its Operationally Representative
Environment (ORE); it plans to perform verification,
validation, and accreditation (VV&A) testing in order to
conduct future operational testing on ALIS components within
the ORE. Regardless of whether the ORE completes VV&A,
the JOTT is working with the JPO and Lockheed Martin
to plan cybersecurity testing of ALIS components within
the ORE for purposes of risk reduction ahead of continued
cybersecurity testing of the operational ALIS systems.
DOT&E Response to Senator McCain’s Questions Regarding
the Completion of SDD
In a letter to the SECDEF on November 3, 2016, Senator McCain
asked the Department to respond to questions regarding the
completion of SDD. The letter was prompted by, and cited,
recent revelations that the program would be experiencing yet
another delay in completing SDD and cost overruns that may be
upwards of $1 Billion.
Although USD(AT&L) responded to the Senator on behalf of the
Department in a letter dated December 19, 2016, the following
are DOT&E’s responses to each of the questions.
Question #1: When will the Department complete the SDD
phase of the F-35?
• DOT&E Answer: SDD will close out in multiple phases.
Developmental flight testing is projected to end no
earlier than mid-2018, based on independent estimates
on completing mission systems flight testing – the testing
that will likely take the longest to complete. These
estimates—from the Director of Cost Assessment and
Program Evaluation (CAPE) of March 2018, the Director
of Developmental Test and Evaluation of March to
June 2018, Deputy Assistant Secretary of Defense for
Systems Engineering of July 2018, and my office of July
2018—are all later than the program’s estimate, based
on the amount of planned mission systems test points
remaining. (These estimates are optimistic because they
do not fully account for the corrections and verification
testing needed for the more than 270 high-priority
deficiencies in Block 3F performance identified by a recent
review.) Then, incremental deliveries of the Block 3F

 FY16 DOD PROGRAMS
capabilities (i.e., flight envelope, weapons, and avionics)
for each variant will likely not be completed until late
2018 due to continued delays and discoveries with F-35B
and C flight sciences testing, along with weapons testing.
Finally, contract close out actions, including specification
compliance and verification and validation, will complete
no earlier than late 2019. Completion of all required
contracting action for the SDD phase will likely continue
for a number of years.
Question #2: How many additional funds, in each
upcoming fiscal year budget, will be required to complete
F-35 SDD?
• DOT&E Answer: Although DOT&E does not conduct
independent cost estimates, CAPE estimated that the
program would need an additional $550 Million in FY18 to
finish the necessary and planned developmental test points
and produce additional software versions to fix and verify
the important known and documented deficiencies, then an
additional $425 Million in FY19 and $150 Million in FY20
to complete SDD. These estimates add up to an additional
$1.125 Billion required to complete SDD. The Program
Office estimate is about one-half of the CAPE estimate.
Question #3: What other Service priorities will not receive
funding in fiscal year 2018 due to the SDD delay and cost
overrun?
• DOT&E Answer: Although the program recently claimed
that their estimated SDD overrun can be covered by
reallocating existing JSF program funding (other than
$100 Million in flight test risk), the SDD cost increase will
be much larger than the current program estimate for the
reasons described in this report. Therefore, the overrun
will not be completely covered with only program funds
and the Services will likely need to address the SDD cost
increase from within their budgets, or funding currently
designated for Follow-on Modernization (FoM) will need to
be reallocated to complete SDD.
Question #4: Is Secretary James’ Block 3F full combat
capability certification, as required by the Fiscal Year 2016
NDAA, still valid?
• DOT&E Answer: For many reasons, it is clear that the
Lot 10 aircraft that will begin delivery in early 2018 will
not initially have full Block 3F capability. These reasons
include, but are not limited to, the following:
- Envelope limitations will likely restrict the full planned
Block 3F carriage and employment envelopes of the
AIM-120 missile and bombs well into 2018, if not later.
- The full set of geographically specific area of
responsibility mission data loads (MDLs) will not be
complete, i.e., developed, tested and verified, until 2019,
at the soonest, due to the program’s failure to provide
the necessary equipment and software tools for the U.S.
Reprogramming Laboratory (USRL).
- Even after the MDLs are delivered, they will not be
tested and optimized to deal with the full set of threats

-

-

-

-

-

-

-

present in IOT&E, let alone in actual combat, which is
part of full combat capability.
The program currently has more than 270 Block 3F
unresolved high-priority (Priority 1 and Priority 2, out of
a 4-priority categorization) performance deficiencies, the
majority of which cannot be addressed and verified prior
to the Lot 10 aircraft deliveries.
The program currently has 17 known and acknowledged
failures to meet the contract specification requirements,
all of which the program is reportedly planning to get
relief from the SDD contract due to lack of time and
funding.
Dozens of contract specification requirements are
projected to be open into FY18; these shortfalls in
meeting the contract specifications will translate into
limitations or reductions to full Block 3F capability.
Estimates to complete Block 3F mission systems extend
into the summer of 2018, not just from DOT&E, but
other independent Department agencies, making delivery
of full capability in January 2018 nearly impossible to
achieve, unless testing is prematurely terminated, which
increases the likelihood the full Block 3F capabilities
will not be adequately tested and priority deficiencies
fixed.
Deficiencies continue to be discovered at a rate of about
20 per month, and many more will undoubtedly be
discovered during IOT&E.
ALIS version 3.0, which is necessary to provide full
combat capability, will not be fielded until mid-2018;
also, a number of capabilities that had previously been
designated as required for ALIS 3.0 are now being
deferred to later versions of ALIS (i.e., after summer of
2018).
The Department has chosen to not fund the CAPE
estimate for the completion of Block 3F mission systems
testing lasting until mid-2018, an estimate which is
at least double the Program Office’s latest unrealistic
estimate to complete SDD. This guarantees the program
will attempt a premature resource- and schedule-driven
shutdown of mission systems testing, which will increase
the risk of mission failures during IOT&E and, more
importantly, if the F-35 is used in combat.
Finally, rigorous operational testing, which provides the
sole means to evaluate actual combat performance, will
not complete until at best the end of 2019—and more
likely later.

Question #5: How will this delay and cost overrun affect
the current overall schedule for Joint Strike Fighter
deliveries to the Services?
• DOT&E Answer: The Program Office currently has no
plans to delay the production and delivery schedule of
aircraft to the Services. However, since Lot 10 aircraft
will not initially be delivered with full combat capability,
including operational MDLs for Block 3F, the Services
will need to plan for accepting aircraft with less capability,

F-35 JSF

105

 FY16 DOD PROGRAMS
possibly with Block 3i capability, until full Block 3F
capability can be delivered.
Question #6: When will you complete the operational test
and evaluation phase?
• DOT&E Answer: The IOT&E is planned to cover a
span of approximately 12 months, and will start after the
program is able to meet the TEMP entrance criteria and
the Department certifies that the program is ready for test.
These entrance criteria are common-sense and carefully
defined requirements that were well-coordinated with the
Services and JPO as the TEMP was being staffed. Meeting
these criteria to enter IOT&E is necessary to ensure the
test is conducted efficiently and effectively within the time
span planned and to minimize the risk of failing IOT&E,
or causing a “pause test” and having to reaccomplish costly
test trials, which would only further delay the completion of
IOT&E and increase program costs. Since the program will
not be ready to start IOT&E until late 2018, at the earliest,
and more likely 2019, completion of IOT&E will not occur
until late 2019 or early 2020.
Question #7: When will you make the
Milestone C/Full-Rate Production decision?
• DOT&E Answer: Since the Milestone C/Full-Rate
Production decision cannot be made until after IOT&E is
completed and DOT&E has issued its report, it cannot occur
by the threshold date of October 2019 and will likely not
occur until early 2020, at the soonest.
Question #8: Will you defer any planned F-35 capabilities
from SDD into the F-35 Follow-on Modernization (FoM)
program?
• DOT&E Answer: Multiple F-35 capabilities will be
deferred from SDD or not function properly in Block 3F
unless the program continues testing and fixing deficiencies.
The program currently has hundreds of unresolved
deficiencies and immature capabilities, including 17
documented failures to meet specification requirements
for which the program acknowledges and intends to seek
contract specification changes in order to close out SDD.
Question #9: How will the SDD delay affect the Follow-on
Modernization (FoM) program?
• DOT&E Answer: Delays to the completion of SDD will
impact both the FoM program schedule and content. While
FoM is critical for the capabilities needed with the F-35 and
the program is attempting to minimize delays, the program
does not appear to be ready to complete all prerequisites to
start full development in FY18, as planned. Also, IOT&E
will not be complete until late 2019 or early 2020, which
overlaps with the planned test periods for Block 4.1.
Finally, the program’s current plans for FoM are not
executable, for many reasons, which include the following:
- Too much technical content for the production-scheduledriven developmental timeline

106

F-35 JSF

- Overlapping capability increments without enough time
for deficiencies from OT to be fixed prior to releasing the
next increment
- High risk due to excessive technical debt and
deficiencies from the balance of SDD and IOT&E being
carried forward into FoM because the program does not
have a plan or funding to resolve key deficiencies from
SDD prior to attempting to add the planned Block 4.1
capabilities
- Inadequate test infrastructure (aircraft, laboratories,
personnel) in the current FoM plan to meet the testing
demands of the capabilities planned and the multiple
configurations (i.e., TR2, TR3, and Foreign Military
Sales)
- Insufficient time for conducting adequate DT and OT for
each increment
Question #10: When will you provide your final response
either to validate the current requirement for the F-35 Joint
Strike Fighter total program of record quantity or identify
a new requirement for the total number of F-35 aircraft
that the Department would ultimately procure?
• DOT&E Answer: DOT&E is not aware of when the
Department will complete these actions.
Recommendations
• Status of Previous Recommendations. The program
adequately addressed 5 of the 14 previous recommendations.
As discussed in the appropriate sections of this report, the
program did not, and still should:
1. Acknowledge schedule pressures that make the start of
IOT&E in August 2017 unrealistic and adjust the program
schedule to reflect the start of IOT&E no earlier than late
CY18.
2. The Department should carefully consider whether
committing to a “block buy” is prudent given the state of
maturity of the program, as well as whether the block buy is
consistent with a “fly before you buy” approach to defense
acquisition and the requirements of title 10 U.S. Code.
3. Plan and program for additional Block 3F software builds
and follow-on testing to address deficiencies currently
documented from Blocks 2B and 3i, deficiencies discovered
during Block 3F developmental testing, and during IOT&E,
prior to the first Block 4 software release planned for 2020.
4. Ensure the testing of Block 3F weapons prior to the start
of IOT&E leads to a full characterization of fire-control
performance using the fully integrated mission systems
capability to engage and kill targets.
5. Provide the funding and accelerate contract actions to
procure and install the full set of upgrades recommended
by DOT&E in 2012, correct stimulation problems, and fix
all of the tools so the USRL can operate efficiently before
Block 3F mission data load development begins.
6. Complete the planned testing detailed in the
DOT&E-approved USRL mission data optimization
operational test plan and amendment. Although some

 FY16 DOD PROGRAMS
testing was completed, the program should ensure all
operational Block 3i MDLs are tested per the approved test
plan.
7. Along with the Navy and Marine Corps, conduct an actual
operational test of the F-35B onboard an L-class ship
before conducting a combat deployment with the F-35B.
This test should have the full Air Combat Element (ACE)
onboard, include ordnance employment and the full use
of mission systems, and should be equipped with the
production-representative support equipment.
8. Develop a solution to address the modification and retrofit
schedule delays for production-representative operational
test aircraft for IOT&E. These aircraft must be similar to, if
not from, the Lot 9 production line.
9. Develop an end-to-end ALIS test venue that is production
representative of all ALIS components. Although the
program has developed the ORE, only limited testing has
occurred.
• FY16 Recommendations.
1. The program should complete all necessary Block 3F
baseline test points. If the program uses test data from
previous testing or added complex test points to sign off
some of these test points, the program must ensure the data
are applicable and provide sufficient statistical confidence
prior to deleting any underlying build-up test points.
2. In light of the fact that the program is unable to correct
all open deficiencies prior to IOT&E, the program should
assess and mitigate the cumulative effects of the many
remaining SDD deficiencies on F-35 effectiveness and
suitability, especially those deficiencies that, in combination
or alone, may cause operational mission failures during
IOT&E or in combat, prior to finalizing and fielding Block
3F. The program will need to add test points to troubleshoot
and address deficiencies that are currently not resolved.
3. The program should consider developing another full
version of Block 3F software to deliver to flight test in order
to address more known deficiencies.
4. The program should ensure adequate resources remain
available (personnel, labs, flight test aircraft) through
the completion of IOT&E to develop, test, and verify
corrections to deficiencies identified during flight testing.
5. The program should address the deficiency of excessive
F-35C vertical oscillations during catapult launches within
SDD to ensure catapult operations can be conducted safely
during IOT&E and during operational carrier deployments.
6. The Program Office must immediately fund and expedite
the contracting actions for the necessary hardware and
software modifications to provide the necessary and
adequate Block 3F mission data development capabilities
for the USRL, including an adequate number of additional
radio frequency signal generator channels and the other
required hardware and software tools.
7. The program should address the JOTT-identified shortfalls
in the USRL that prevent the lab from reacting to new
threats and reprogramming mission data files consistent
with the standards routinely achieved on legacy aircraft.

8. The program should correct deficiencies that are preventing
completion of all of the TEMP-required Block 3F Weapons
Delivery Accuracy (WDA) events and ensure the events are
completed prior to finishing SDD.
9. The program should ensure Block 3F is delivered with
capability to engage moving targets, such as that provided
by the GBU-49, or other bombs that do not require
lead-laser guidance.
10.The program should complete additional testing and
analysis needed to determine the risk of pilots being harmed
by the Transparency Removal System (which shatters the
canopy first, allowing the seat and pilot to leave the aircraft)
during ejections in other than ideal, stable conditions (such
as after battle damage or during out-of-control situations).
The program should complete these tests as soon as
possible, with the new equipment, including the Gen III
Lite helmet in a variety of off-nominal conditions, so that
the Services can better assess risk associated with ejections
under these “off-nominal” conditions.
11. The program needs to conduct an assessment to determine
the extent to which the results of further durability
testing with BH-1, the F-35B durability test article, are
representative of production aircraft and, if necessary,
procure another test article for the third life testing.
12.The Navy and the Program Office should investigate
alternatives for determining the operational impact of an
engine removal and install while conducting carrier air wing
operations at sea.
13.The Navy and Marine Corps should conduct an analysis,
such as an operational logistics footprint study, which
simulates flight deck and hangar bay spotting (aircraft
placement) with a full ACE onboard, using data from the
DT-III ship trials to determine what the impact of an engine
removal and installation would be on integrated ship and
ACE operations with a full ACE onboard.
14.The program and the Navy should investigate if the heavy
power module container should be redesigned for better
usability at sea.
15.The program and the Navy should investigate potential
options to improve ship-based communications bandwidth
dedicated to ALIS connectivity off-ship, such as increasing
the priority of ALIS transmissions, or reserving low-use
times of the day for handling large volumes of ALIS
message traffic.
16.The Navy should investigate any efficient, multi-use
opportunities for F-35 support equipment (SE) such as
using legacy SE on the F-35 or F-35 SE on legacy aircraft.
17.The Navy should investigate options for increasing the
number of wall power outlets in CVN hangar bays to help
facilitate simultaneous maintenance on multiple F-35Cs,
or the ability to interconnect multiple pieces of support
equipment from a single outlet to permit simultaneous
operations.

F-35 JSF

107

 FY16 DOD PROGRAMS

108