Comments on:

President’s Council of Advisors on Science and Technology

REPORT TO THE PRESIDENT

Forensic Science in Federal Criminal Courts: Ensuring Scientific

Validity of Pattern Comparison Methods

The FBI agrees with the authors of the President’s Council of Advisors on Science and Technology
(PCAST) report that forensic science plays a critical role in the criminal justice system, and therefore
needs to be held to high standards. Further, the FBI agrees with the PCAST report as well as the 2009
National Research Council report (2009 NAS report) that significant funding is needed to develop
stronger ties between the academic research community and the forensic science community. It is
inherent within science that over time, our knowledge of a subject evolves. It is critical that continued
research be pursued in order to ensure that forensic science meets the high standards necessary to be used
in a court of law.
However, the FBI disagrees with many of the scientific assertions and conclusions of the report. The
report makes broad, unsupported assertions regarding science and forensic science practice. For example,
the report states that “the only way” to establish “validity as applied” is through proficiency testing, and
requires a measurement of how often the examiner gets the correct answer, which is fundamentally at
odds with a report of the National Academy of Sciences.1
The report also creates its own criteria for scientific validity and then proceeds to apply these tests to
seven forensic science disciplines, failing to provide scientific support that these criteria are well accepted
within the scientific community. In fact, PCAST defines their internally developed criteria as “scientific
criteria” by which forensic feature-comparison methods must be supported by. However, PCAST does
not apply its own criteria consistently or transparently. The PCAST criteria define “black box” studies as
the benchmark to demonstrate foundational validity, but provide no clarification on how many studies are
needed or why some studies that have been conducted do not meet their criteria. These criteria seem to be
subjectively derived and are therefore inconsistent and unreliable.
The report does not mention numerous published research studies which seem to meet PCAST’s criteria
for appropriately designed studies providing support for foundational validity. That omission discredits
the PCAST report as a thorough evaluation of scientific validity.
The report proposes federal government criminal justice related-databases should be made available to
researchers for independent studies while consistently overlooking the legal authorization and limitations
set out in statutes and regulations regarding the use of such databases.
Finally, the report ignores important differences between forensic science disciplines, conflating
fundamental differences between class-level and identification-level evidence, leading to troubling
generalized conclusions about all forensic science disciplines.
1

National Academy of Sciences, STRENGTHENING 
  FORENSIC SCIENCE IN 
  THE UNITED 
  STATES, 9 (2009); see also id.
At 207 (“[b]lind proficiency testing is recommended . . . not as a way to determine error rates, but as a more 
  
precise test of a worker’s accuracy.”)

September 20, 2016
1