EDWARD H. HAERTEL, Ph.D.
Jacks Family Professor of Education, Emeritus

May 23, 2019

The Honorable Connie M. Leyva
Chair, Senate Education Committee
State Capitol, Room 2083
Sacramento, CA 95814
Dear Senator Leyva:

I am writing to share my serious concerns regarding Assembly Bill 751, the “Pathways to College
Act.” For over 35 years, I served on the faculty of the Stanford University Graduate School of
Education. I am recently retired, now the Jacks Family Professor of Education, Emeritus. I have a
Ph.D. from the University of Chicago, Department of Education, with a concentration in
measurement, evaluation, and statistical analysis. I am a recognized expert in educational
research, and in particular in educational testing and measurement, as shown on my
Curriculum Vitae (available at https://people.stanford.edu/haertel/curriculum-vitae).
My history with testing issues in the State of California reaches back to membership on
Technical Advisory Committee for the California Assessment Program in the 1980s, and I served
on the Advisory Committee for the Public Schools Accountability Act of 1999 from the time that
committee was first formed until it ceased to function in 2015. In 2007, I received the Lifetime
Achievement Award from the California Educational Research Association. I currently serve on
the Technical Design Group working with the CDE’s Analysis, Measurement, and Accountability
Reporting Division, and I also serve on the Technical Advisory Committee for the Smarter
Balanced Assessment Consortium. In this letter, I am expressing my own views. I am not
representing the views of Stanford University.
I am deeply concerned about the implications of AB 751 for California’s school accountability
system and ultimately, for the quality of education offered to public-school students in our
great state of California. There is a natural desire to reduce testing burden, and on the face of
it, using the same test for multiple purposes seems like a sensible solution. Since many high
school students are already taking the ACT or the SAT, for example, why should they also have
to take the 11th-grade Smarter Balanced assessments? Moreover, encouraging more students
to take college entrance exams seems like a good thing—As suggested by the name of the
“Pathways to College Act,” mandating one of these tests might encourage college application
and college enrollment. In addition, high school students might be more motivated to try hard
on a college entrance exam than on Smarter Balanced assessments for which they perceived
fewer personal consequences.
As a psychometrician who has devoted my entire professional career to understanding and
improving achievement tests and accountability testing programs, I certainly share the
concerns of policy makers and the public over testing burden, access to higher education, and
student motivation, but despite these concerns, I believe that AB 751 is very unwise. The
authors of the bill were clearly thoughtful and well-intentioned, having included numerous
sensible requirements concerning rigor, coverage, inclusiveness, score reporting, and other
matters, but as a practical matter, these paper requirements would be quite easy to satisfy. In
485 Lasuen Mall, Stanford, CA 94305-3009 T 650.725.1251 F 650.725.7412

http://people.stanford.edu/haertel/

 Haertel letter re AB 751 – Page 2
the end, what the bill comes down to is a proposal to take two (or more) very different
examinations, developed according to different specifications to serve different purposes,
administered under different conditions (including different motivational contexts), and then to
treat the resulting scores as interchangeable. We may wish such a thing were possible, but it
simply cannot be done. Methods for expressing scores from different tests on a common scale,
or for using and interpreting them as if interchangeable, fall far short of providing the degree of
comparability California’s state-of-the-art school accountability system would require.
I cannot help but think back to the short-lived “Pupil Testing Incentive Program” (PTIP) enacted
in California over 20 years ago, after the demise of the California Learning Assessment System
(CLAS) and before the enactment of the Standardized Testing and Reporting (STAR) program.
PTIP was supposed to, in fact was mandated to, produce comparable scores across a menu of
approved off-the-shelf achievement tests that districts could choose from. I was among the
testing experts who were concerned at that time that the legislation’s aspirations had gotten
out ahead of what the testing field would be able to deliver, and that in fact proved to be the
case. The effort was abandoned within two years. I am afraid that despite its good intentions,
AB 751 likewise reaches well beyond the range of technical feasibility.
The comparability requirements included in AB 751 might sound rigorous, but in practice, it
would be very easy to satisfy “alignment,” depth-and-breadth, and similar technical criteria
despite material differences in item formats, content coverage and weighting, and ultimately in
the constructs assessed by the Smarter Balanced tests versus some college entrance exam. (In
the PTIP program, a seemingly rigorous test review process was in practice easily
circumvented.) Scores on different tests have different meanings, even if the names of the tests
are the same or similar, even if the tests are nominally aligned to the same content
frameworks. Even if it were possible to scale the scores from such different tests so as to
provide adequate comparability at the level of individual students (which is far from
guaranteed), when scores were aggregated to the level of schools, or demographic subgroups
within schools, small biases and differences in score meaning, as well as differences in the tests’
precision and error structures, would take on much greater importance, resulting in deeply
flawed comparisons across schools participating in different testing regimes.
In addition, many colleges and universities in California have already agreed to accept specified
performance levels on the Smarter Balanced tests as evidence that students can bypass
remedial courses or waive co-requisites. As Smarter Balanced scores are increasingly accepted
as evidence of readiness by California’s community colleges and CSU institutions, 11th-grade
Smarter Balanced scores will become even more valuable and meaningful for our high school
students. AB 751 would deprive students in opt-out districts not only of valuable information
concerning their own college readiness, but also of tangible evidence they could otherwise use
to jumpstart their college careers.
Finally, I am seriously concerned with “wash-back” or “up-stream” effects of abandoning the
Smarter Balanced assessments for high school students. While we can never rely solely on
incentives to “teach to the test” as a tool for curriculum reform, we can at least choose tests

 Haertel letter re AB 751 – Page 3
with the virtue that sound test preparation is also sound instruction. In 2001, University of
California President Richard C. Atkinson proposed that the UC system stop using the SAT-I for
college admissions and instead use a mix of achievement tests, based in large part on evidence
that widespread SAT-I test preparation was of little educational value. Preparation for
achievement tests (the SAT-II exams at that time) was believed to have greater pedagogical
value. There are striking parallels in the use of the Smarter Balanced assessments versus college
entrance tests today. The Smarter Balanced tests have been designed, from the ground up,
with the goal of incentivizing sound classroom instruction. This is “in the DNA” for Smarter
Balanced. To my knowledge, the same simply cannot be said of any of the college entrance
exams envisioned in Section 1111(b)(2)(H) of ESSA, or other similar examinations that AB 751
might make available.
I hope that these remarks are helpful to you in your deliberations.
With best regards,

Edward H. Haertel, Ph.D.
Jacks Professor of Education, Emeritus
Graduate School of Education
Stanford University

cc:

Lynn Lorber
Staff Director,Senate Education Committee
Brandon Darnell
Principal Consultant