SCHOOL
Douglas G. Kubach
President
5601 Green Valley Drive
Bloomington, MN 55437 USA
T: 952 681 3100
E: douglas.kubach@pearson.com

March 30, 2015
Tom Torlakson
State Superintendent of Public Instruction
California Department of Education
1430 N Street
Sacramento, CA 95814
RE:

Dr. Michael Kirst
State Board President
California State Board of Education
1430 N Street, Suite 5111
Sacramento, CA 95814

Selection of California Assessment of Student Performance and Progress (“CAASPP”) contractor
following CAASPP Request for Submissions CN150012 (“RFS”)

Dear Mr. Torlakson and Dr. Kirst:
Open and fair competition for government contracts is in the best interests of citizens and
taxpayers. Competition spurs innovation and ensures that California receives the best value. In a climate
where there are doubts about the fairness and openness of a procurement process, contractors who are not
incumbents may not be willing to invest to compete and win contracts, and taxpayers will suffer as a
result, losing the benefits of innovation and best value that come from competition.
I am writing this letter to express concerns about the process used to select Educational Testing
Service (“ETS”) as the new CAASPP contractor. Allowing ETS to modify their proposal after
submission violates the rules of the procurement. The fact that the State Board of Education (“SBE”)
preferred NCS Pearson, Inc.’s (“Pearson”) approach to scoring, yet our scoring proposal was rated the
lowest, points to serious flaws in the proposal evaluation process. The fact that public records of the
proposal evaluation were destroyed is illegal and renders the procurement invalid. Finally, there is no
evidence that the actual costs were ever considered in the evaluation, to the detriment of California’s
taxpayers.
Pursuant to the RFS and Education Code section 60643(b)(2), the California Department of
Education’s (“CDE”) was obligated to use a competitive and open process utilizing standardized scoring
criteria through which to select a CAASPP contractor. Although the contract in question was made
exempt from the California Public Contract Code, the RFS and Education Code section 60643(b)(2)
require that a competitive process be used and the process chosen, as well as the decisions made during
the procurement, were made subject to California Code of Civil Procedure sections 860 to 870 under an
arbitrary and capricious standard. Pearson believes that the CDE and SBE have failed to follow the
competitive and open process set forth in the RFS in selecting ETS as the CAASPP contractor, and
selection of ETS as the CAASPP contractor was arbitrary and capricious.

1 of 5

 1.
Allowing ETS to materially modify its proposal by discarding a significant portion
of that proposal and replacing it with a significant portion of another bidder’s proposal at
costs not to exceed the cost proposed for that task in the other proposal violated the
standard of review.
The CDE recommended that the SBE approve ETS as the CAASPP contractor expressly
conditioned on ETS meeting specified conditions, including “If directed by the SBE or the CDE, the
designated contractor and/or their subcontractors for a specific task will agree to provide the same
approach/work described in another bidder’s submission for that task (the desired approach) at costs not
to exceed the cost proposed for that task in the other submission.” Such an approach is unprecedented in
public procurements at virtually any level — Federal or State.
In reference to this condition, Board Member Burr specifically asked the CDE, “Can we, I’ll just
be real specific, we want ETS to do what Pearson said they would do on scoring?…Is that what that
means? You can substitute a task….” In response, Keric Ashley, Interim Deputy Superintendent of
Public Instruction stated, “The statement is there exactly for that reason — the Board hearing something
in one proposal that they did not hear in another proposal and want to negotiate that in the other proposal.
That’s exactly why the statement is there.”
As stated by the CDE, it included the condition that ETS “agree to provide the same
approach/work described in another bidder’s submission for that task (the desired approach) at costs not
to exceed the cost proposed for that task in the other submission” to involve “California teachers to the
maximum extent possible with a large component focused on professional development” for the purpose
of asking ETS to do what another bidder proposed. Such a condition calls into question the validity of
the RFS process and runs afoul of generally recognized principles of competitive bidding in California
and elsewhere.
First, the aforementioned condition essentially permits ETS to materially change its bid proposal
after submission and bid opening, which provides it an unfair advantage over the other bidders and
contravenes California law on public contracts.1 See, e.g., Valley Crest Landscape, Inc. v. City Council,
41 Cal. App. 4th 1432, 1435 (1992) (trial court erred in not granting runner-up's petition to set aside
contract awarded to the successful bidder where changes after bidding closed gave the successful bidder
an “unfair advantage”); National Coach Corp. v. State Bd. of Control, 137 Cal. App. 3d 750, 756 n.3
(1982) (normally bids may not be modified after bid opening). This principle was eventually adopted by
the CDE in the RFS. Paragraph 5.5(m) of the RFS addressed the submission and provided that the bidder
could modify its proposal, but only up until the bid submission deadline. In this case, the CDE ignores
the time limit and permits ETS to modify its offer months after the submission of offers. Under any
standard of review, this approach fails.
Second, the aforementioned condition violates the RFS process and the process for developing
the final Scope of Work. The RFS required bidders to submit proposals for certain tasks and the CDE
indicated that the successful bidder’s proposal would be used to negotiate the final Scope of Work. The
aforementioned condition essentially makes the proposal process irrelevant because it permits the CDE
and SBE to require ETS to abandon its own proposal and adopt the approach proposed in another bidder’s
submission. In other words, it does not matter what ETS actually proposed in its submission. In addition,
the condition permits the final Scope of Work to be negotiated based not on what ETS proposed, but
rather what was proposed by another bidder, such as Pearson.
Third, the aforementioned condition permits ETS to negotiate a contract based on a matter that
was not set forth in the RFS. No other bidder, including Pearson, was given the opportunity to “provide
the same approach/work described in another bidder’s submission for that task (the desired approach) at
1
While the specific procurement was exempted from the California Public Contract Code, cases interpreting California
procurements and Federal procurements are evidence of what type of conduct is considered arbitrary and capricious in a
procurement process.

2 of 5

 costs not to exceed the cost proposed for that task in the other submission.” In effect, the CDE and SBE
have changed the rules of the procurement after the proposals have been submitted. Under California
law, a contract awarded following a change in specifications after the bidding is closed is subject to be set
aside. See Eel River Disposal & Resource Recovery, Inc. v. Humboldt, 221 Cal. App. 4th 209, 237 (2013)
(public agency improperly deviated from applicable bidding requirements when awarding contract based
on criteria not contained in the listed evaluative criteria). Here, ETS was the only bidder evaluated on its
willingness to meet the technical and cost factors of the other two bidders.
Fourth, the condition adopted by the SBE essentially also changed the type of contract being
negotiated. Section 5.3(H) of the RFS states that “the resulting contract will be a cost reimbursement
contract based on actual documented expenses.” The SBE, however, is requiring ETS to provide the
technical solutions offered “at costs not to exceed the cost proposed for that task in the other submission.”
In effect, ETS is being asked to place a cap, in the nature of a fixed price contract, which no other bidder
was offered the opportunity to provide. This alteration to the RFS, along with the others described above,
unfortunately invalidates the entire process.
2.
If ETS is required to discard the Scoring and Analysis portion of its technical
proposal and replace it with the scoring and analysis approach proposed by Pearson,
despite the fact that Pearson received a significantly lower score on that section of the
proposal than ETS, the scoring of the technical proposals by CDE was necessarily flawed,
and fell below the legally acceptable standard.
ETS received the highest score for its technical proposal, significantly higher than the score
received by Pearson. However, SBE directed ETS to discard its proposed scoring approach and instead
adopt the approach proposed by Pearson, despite the fact that Pearson was given a significantly lower
score. This is unavoidably an admission by SBE that the scoring of the technical proposals was
significantly flawed. It also represents actions that are clearly arbitrary and capricious. In fact, Pearson’s
Submission Consensus Score for RFS Section 3.8 Scoring and Analysis was 43 compared to ETS’
Submission Consensus Score of 55 — a significant difference. Pearson receiving a significantly lower
score, when the SBE’s actions indicate that Pearson had a better technical approach for this section,
indicates the evaluation scoring process was flawed. If, as the evidence suggests, the scoring process for
Section 3.8 Scoring and Analysis was flawed, this suggests the possibility that the scoring processes for
other sections were flawed, as well.
3.
There is no evidence that the CDE used standardized scoring to evaluate the cost of
the assessment system proposed by the bidders.
Pursuant to Education Code section 60643(b)(2), the CDE and SBE are required to use a
competitive and open process utilizing standardized scoring criteria and are required to consider “[t]he
cost of the assessment system” in selecting a CAASPP contractor. However, there is no indication that
the CDE used standardized scoring criteria in evaluating the cost of the assessment system proposed by
bidders. Also, there is no indication that CDE even properly considered the actual costs of the assessment
systems proposed.
The RFS sets forth the requirements for the cost submission, including for example, providing the
costs for labor, travel, overhead, and subcontractors associated with each task under the Scope of Work.
(See RFS, section 5.3; Attachments 10A, 10B, 10C, and 10D.) Based on the CDE’s memorandum dated
March 6, 2015 to the SBE summarizing the RFS and recommending the CAASPP contractor, a bidder
could be awarded a total of 200 points for its cost submission. However, the RFS is silent on the criteria
used to evaluate the cost of the assessment system and silent as to how the 200 points were awarded to
each bidder’s proposal. This non-disclosed process can be contrasted against the specific process and
procedures set forth on technical issues where the RFS set forth what was “sufficient” and what was
“insufficient” for each technical issue. The CDE’s failure to articulate how it evaluated the cost of the
assessment system proposed by each bidder or to demonstrate that it used standardized scoring for

3 of 5

 evaluating this criteria is especially troubling to Pearson, who proposed the lowest cost assessment
system, but did not receive the highest score for this criteria.
The failure to properly define the cost criteria adversely impacted the review process. At the
March 11, 2015 presentation, Board Member Burr asked the CDE whether each bidder was given specific
instructions on how to characterize their costs. In response, Mr. Kilmer failed to address the specific
question but did admit that not every contractor placed its costs in the same task area. Rather, he said
they were just spread out and “everyone did it differently.” The CDE’s description demonstrates that
there was not a defined process for the evaluators to apply consistently over the cost proposal, that the
bidders were confused and that, therefore, the evaluation of costs is highly suspect.
The objectionable nature of the cost evaluation is supported by the scoring. Although Pearson
was the lowest proposed cost, Pearson received the second highest points for costs. CTB/McGrawHill
received 148 points for a cost of $223,769,974.58. Pearson received 145 points for a cost of
$205,840,739. Essentially a proposal that was $18 million higher than Pearson received a higher number
of points for costs. No explanation for the process of scoring costs has been provided, nor has there been
any logic provided to explain how the lowest cost proposal did not receive the highest score for costs.
Moreover, it appears that the CDE did not in fact consider the actual cost of the assessment
system proposed by the bidders. Under the Cost Submission findings, CDE makes no mention of the
actual costs of the assessment systems proposed by the bidders. Rather, the CDE focuses on the format of
the costs in summarizing the weaknesses of each bidder indicating that it considered the format in which
the costs were presented instead of the actual costs. Under federal law, courts have found an agency’s
decision to be reversible when, for example, the agency “entirely failed to consider an important aspect of
the problem . . . .” Ala. Aircraft Indus., Inc.-Birmingham v. United States, 586 F.3d 1372, 1375 (Fed. Cir.
2009). Similarly here, the CDE failed to consider the cost of the assessment system as required by the
RFS and Education Code section 60643(b)(2). Thus, its evaluation of the cost submission was arbitrary
and capricious.
4.
Since critical public records have been destroyed, the process needs to be rebid to
remove any taint to the procurement process.
Government Code section 6200 prohibits CDE from destroying any record placed in its custody.
That section provides:
Every officer having the custody of any record, map, or book, or of any paper or proceeding of any court,
filed or deposited in any public office, or placed in his or her hands for any purpose, is punishable by
imprisonment pursuant to subdivision (h) of Section 1170 of the Penal Code for two, three, or four years if,
as to the whole or any part of the record, map, book, paper, or proceeding, the officer wilfully does or
permits any other person to do any of the following:
(a) Steal, remove, or secrete.
(b) Destroy, mutilate, or deface.
(c) Alter or falsify.

At the March 11, 2015 hearing, the CDE representative stated that the CDE had “shredded”
documents in its possession relating to the procurement. Specifically, the CDE representative stated:
So after the whole work was done by the panel and everyone else, the individual scores, the individual
notes, any personal notes that people took, those are deleted/shredded because it is not about an individual
process. What’s kept are the consensus scoring of those panels and the consensus comments of the panels.

Recently, when asked about this process of destroying public records, CDE representatives stated
that they were following a common practice at CDE. The CDE appears oblivious to its violation of
California law.
The documents that have been destroyed would have likely shed light on many of the serious
issues raised by Pearson in this letter. Since such critical documentation has been destroyed, the entire

4 of 5

 process is suspect. Frankly, one would have expected a process more focused on transparency rather than
a process designed to frustrate any dissention.
As set forth above, Pearson believes that the CDE and SBE have failed to use a competitive and
open process in selecting ETS as the CAASPP contractor, and the selection of ETS as the CAASPP
contractor was arbitrary and capricious. For the foregoing reasons, we request that the CDE and SBE
rebid the CAASPP procurement.
We look forward to the CDE and SBE’s response to this letter. If you would like to discuss this
matter directly, please feel free to contact me.
Sincerely,

Douglas Kubach

cc:

Ilene Straus, Vice President, SBE
Sue Burr, SBE
Bruce Holaday, SBE
Aida Molina, SBE
Patricia Ann Rucker, SBE
Nicolasa Sandoval, SBE
Trish Boyd Williams, SBE
Kenton Shimozaki, SBE
Karen Stapf Walters, Executive Director, SBE
Judy M. Cias, Chief Counsel, SBE
Brooks Allen, Deputy Policy Director & Assistant Legal Counsel, SBE

5 of 5