Office of

Inspector General
Chicago Board of Education
Nicholas Schuler, Inspector General

SIGNIFICANT ACTIVITY REPORT

W E D N E S DAY , F E B R U A R Y 2 6 , 2 0 2 0

ADMINISTRATION OF CPS HIGH-STAKES TEST, GRADES 3–8

OVERVIEW

Over the past year, the OIG’s Performance Analysis Unit conducted a review of CPS’s
high-stakes test for third through eighth graders, focusing on the Spring 2018 test.
Last fall, the OIG submitted a written report on the results of the review, including
numerous recommendations, to the Board of Education and CPS leadership. Since
then, the OIG and CPS administration have discussed CPS’s response to our report,
and CPS has agreed to take action on all OIG recommendations.
In short, the OIG found a concerning level of unusually long test durations, high
counts of test pauses and other irregularities during CPS’s Spring 2018
administration of this untimed, adaptive test. This occurred in a minority of cases,
but enough to be worrisome and to warrant action.
Specifically, we found that tens of thousands of CPS students are taking at least twice
the national average duration to complete their tests, and some are taking three,
four and five times. The test vendor has warned that excessive durations can make it
difficult to accurately compare CPS results to national norms. As it stands now, CPS’s
average durations have been above national norms since at least 2016 and increased
even more in each grade and subject in 2017, 2018 and 2019.
The OIG also is concerned that a small number of tests — about 4% — had at least
five pauses each. Some were paused 10, 20 and even more times. Some of these
pauses could reflect attempts to game the test, which could compromise the results.
Excessive durations and pauses can occur for benign reasons. Importantly, even if
benign, such irregularities risk making CPS results less meaningful. This is a major
concern, given the many ways CPS uses these results.

567 West Lake Street  Suite 1120  Chicago, Illinois  60661-1405
Phone (773) 534-9400  Fax (773) 534-9401  inspectorgeneral@cpsoig.org

 However, in untimed tests that carry stakes for students, teachers, principals and
schools, high durations and pauses also could be indicative of improper attempts to
win higher scores or gains. OIG interviews with a small sample of 20 students and
10 teachers suggested this might be the case in certain instances. To be sure, data
alone cannot say whether high durations and pause counts are due to improper
motives. The OIG also has not sustained here any individual cheating cases. The OIG
does not have — and perhaps no office has — the resources to do a deep individual
dive into a significant number of unusual 2018 tests. Accordingly, the OIG has taken
the approach that improving the administration procedures and general security of
the test going forward is the far better solution from a cost-benefit standpoint.
During its performance review, the OIG had extensive exchanges with the test
vendor and in the wake of the OIG’s review, the vendor has published clearer
guidance on some of the duration and pause concerns raised by the OIG. This further
reinforces the need for swift action.
It is worth noting that many of the issues discussed in this report may well have
arisen because this untimed, adaptive test may not have been designed with CPS’s
multiple high-stakes pressures in mind. It’s possible that some unknown level of
gaming or cheating sits in the mix of potential reasons for the high durations and
pause counts. The recommendations the OIG has made — and which CPS is acting
on — are intended to ensure that this test can be used effectively for CPS’s current
purposes. At some point, CPS might want to consider whether this test is the right
test for its multiple high-stakes needs.
RECOMMENDATIONS

Based on the OIG’s performance review, the OIG has recommended that CPS
overhaul its procedures for administering and monitoring this untimed, high-stakes
test by:
o Reducing durations, preferably by establishing test time limits for general
education students.
o Taking concrete steps to shrink pause counts.
o Finding an auditable way to record each test’s proctor, preferably in a test
data field, so test results can be analyzed by proctor.
o Using new proctor data to identify which proctors to audit during testing,
rather than which grades and subjects per school.
o Barring those Math and Reading teachers whose evaluations are tied to the
growth of their students’ test results from being their students’ sole proctors.
Even the test vendor recommends two proctors in high-stakes situations.

Page 2 of 15

 o Bolstering CPS test training and a five-question quiz that must be passed to
proctor the test. Clear examples of improper behavior should be covered. The
OIG should be cited as an office to be contacted about test-irregularity
concerns.
o Inserting penalties for test cheating in the Test Security Agreement all
proctors must sign.
o Hiring a test security expert to help CPS implement these and other reforms,
including improving the criteria for identifying classrooms to be flagged
and/or audited as well as the auditor checklist, and expanding CPS’s test data
information. If the testing company cannot provide needed reforms, the test
security expert should help CPS write an RFP for a new test contract.
CPS RESPONSE

CPS has committed to implementing all OIG recommendations with the support and
guidance of a test security expert.
METHODOLOGY

The OIG analyzed CPS Reading and Math tests, grades 2 to 8, given in the Spring of
2016, 2017 and 2018, focusing mainly on 2018 tests and durations, pauses and
score gains. The OIG then interviewed 20 students and 10 teachers, almost all in
schools with unusual results, and spoke with the test vendor and other testing
experts.
TEST STAKES

The test in question is so integral to so many aspects of CPS that the accuracy of its
results is paramount.
This tests’ results impact, in varying degrees: student promotions in third, sixth and
eighth grades; for seventh graders, admission to selective enrollment high schools
and programs; the teacher evaluations of Math and Reading teachers; principal
evaluations; Independent School Principal status; and the School Quality Rating
Policy levels set annually for most schools in the system. The test results of students
from four “Priority Groups” carry extra SQRP weight.
The tests’ results also help drive curricular decisions on the school and district level
that can involve CPS resources.
DISTINCTIVE TEST FEATURES

The test in question is CPS’s primary assessment for grades 2 through 8. The Math
test contains 52 or 53 questions. The Reading test contains 42 to 43 questions. The
test’s distinctive features include:

Page 3 of 15

 Untimed — Both Diverse Learners and Non-Diverse Learners can take as long as
they need to complete the untimed test. The test vendor expects the test to be
finished in 45 to 75 minutes, according to one company blog. A sample testing
schedule created by CPS generally allots one hour for the test.
To guide school districts, in August of 2018 (after CPS took the tests analyzed in this
report), the test vendor released the average duration of each test by grade and
subject. In that document, the national average duration for Reading and Math tests
in grades 3 to 8 ranged from 57.7 minutes to 70.7 minutes, based on all students
nationally who took the Spring 2017 version of the test.
Computer-adaptive — The test is administered on a computer and adapts to a
student’s ability level. Thus, a correct answer is followed by a harder question and
an incorrect answer is followed by an easier question. Students are expected to get
half the questions at their achievement level right and half wrong.
Pauses — A proctor can pause the test while a question is on the screen if a student
needs a bathroom, water, lunch or wiggle break. Once a proctor resumes the test, a
question appears that is of similar difficulty to the paused one. This feature is
designed to ensure breaks cannot be used to obtain answers to pending questions.
Time-outs — Generally, after 25 minutes without an answer a test will “time out” and
send students back to the login page. The proctor then must resume the test,
resulting in a new question of similar difficulty.
DURATION FINDINGS

The OIG found that in the Spring of 2018, in every tested grade, the average CPS
student took longer than the national duration1 average to complete their tests.
Chart 1 shows these results for Math; Reading results are similar.
CPS’s high durations were not driven by Diverse Learners who needed extended
time. In fact, as indicated in Appendix A, an OIG analysis found that Diverse Learners
were less likely to have long tests than non-Diverse Learners.2
In particular, the gap between national duration norms and CPS average Math
durations significantly increased in grades six, seven, and eight — all of which carry
stakes for students.

Durations reflect the time spent on questions that students actually answered. All time spent
on questions that eventually timed out or were paused is excluded from duration counts.
1

A similar analysis conducted by the OIG after the performance review was completed indicated
that English Learners also were less likely to take longer tests than other students.
2

Page 4 of 15

 There could be many benign reasons for long durations, including the high stakes
some CPS students face. However, even if long durations are benign, some unusually
long tests can be an inefficient use of student and teacher time. In addition, long
durations can be problematic for other reasons.
The test vendor warned that it compares CPS’s results to those of a national norming
sample so “for the inferences from that comparison to be accurate, CPS testing
conditions should be reasonably reflective of testing conditions of other schools”
that take this test. Durations that vary excessively from the norms “may pose a risk
to the accuracy of inferences” made from CPS results, the test vendor said.
An OIG analysis of Spring 2018 CPS duration data found that tens of thousands of
CPS tests well exceeded national duration norms, as shown in Table 1.
One in four CPS tests took at least twice the national norm to complete; some took
three, four and five times the national average.
Plus, tests with long durations tended to concentrate in certain schools. Twenty
percent of tests that were at least three times as long as the national norm were
clustered in three percent of schools, an OIG analysis found.
In addition, about 30 percent of CPS’s Spring 2018 tests took multiple days to
complete, the OIG found. Some students and teachers described tests that took a
week or more.

Page 5 of 15

 Table 1: CPS Test Durations vs. National Norm
2018 Test Duration vs. National Norm

# of Tests

% of Tests

320,561

100%

160,906

498

At least 2 times the national norm

82,824

25.8%

55,630

495

At least 3 times the national norm

24,269

7.6%

17,853

482

At least 4 times the national norm

7,448

2.3%

5,832

401

At least 5 times the national norm

2,388

0.7%

1,966

258

All CPS 3rd – 8th grade tests

Students*

Schools**

*Reflects number of students with the indicated duration ratio on at least one test.
**Reflects number of schools with at least one test with the indicated duration.
Source: OIG Analysis of CPS Data and National Duration Data.

Importantly, CPS’s duration problem is getting worse over time.
The average duration of CPS tests in every grade 3 to 8 was above the national
duration norm in the Spring of 2016, and those durations increased every year in
every grade in both 2017 and 2018.
Table 2 shows the double-digit increases in each grade over those three testing
cycles. In each subject, the biggest duration hike was in seventh grade, where results
impact selective-enrollment high school admissions.
Table 2: Increases in Avg.
CPS Spring Durations
from 2016 to 2018

A more recent OIG analysis indicated that CPS’s
average durations increased yet again in the Spring of
2019 in every grade and subject tested.

CPS
Grade

Math
Increase

Reading
Increase

3

12.4%

17.2%

Thus, if no action is taken, CPS durations may continue
to move farther and farther from national norms,
putting the accuracy of some results at increasing risk.

4

12.5%

21.7%

5

10.7%

22.6%

6

17.4%

18.5%

7

22.2%

24.0%

8

18.0%

18.5%

Source: OIG Analysis of CPS Data.

In addition, some might contend that spending
multiple days on a 53-question test that the average
student nationally completes in roughly an hour is a
questionable use of instructional time. Even the test
vendor warns in a blog that “An efficient measure of
student learning shouldn’t have the student away from
the classroom for several hours at a time.”

The test vendor has suggested that CPS reserve the
right to retest students whose test durations are too long, but was unable to tell the
OIG what it considered an excessive or problematic duration as it had not done
research on this issue.

Page 6 of 15

 Two independent testing experts told the OIG that they do not favor using an
untimed test in a high-stakes situation.
Marc Weinstein, chief of investigations at Caveon Investigative Services,
recommended that high-stakes tests be administered in a single setting or in
discrete, timed parts. “There are too many things that can happen during those
breaks that can affect the validity of the test results,” Weinstein said.
Gregory Cizek, a professor of educational measurement and evaluation at the
University of North Carolina at Chapel Hill and past president of the National Council
on Measurement in Education, told the OIG that he “would definitely set a time limit”
for tests that carry stakes for teachers. With an untimed high-stakes test, this expert
said, “Educators have absolutely no incentive to tell kids to finish. If I know my rating
depends on this, I’m gonna tell them to keep checking their work and taking their
time. I’m not going to pressure that kid to finish.”
To address the duration issue, the OIG recommends that CPS try to establish time
limits for general education students. If the test vendor will not sanction such limits,
CPS may want to consider setting rules for the retest of students whose durations
exceed a certain amount of time or weigh switching to a different, timed test.
DURATION UPDATE

In the wake of the OIG’s performance review, which involved multiple exchanges
with the test vendor, the test vendor published new December 2019 guidance on
maintaining the integrity of its test, including more specific duration information.
The vendor noted that between Spring 2014 and Spring 2018, the test’s average
national duration had increased by 21 minutes in Reading and 17 minutes in Math.
Pressure on teachers to improve scores could be one reason for this, the vendor said.
It added: “We are increasingly concerned that assessments may be taking longer to
complete than is necessary to produce an accurate score.”
In this new guidance, the vendor recommended that districts create procedures
stating that:
o Average test durations of classrooms and grade levels “should not
substantially differ” from its published norms.
o Durations should “remain relatively consistent across terms.”
o Durations should be monitored periodically to ensure consistency across
schools, classrooms and terms.
The updated guidance did not set a clear standard for excessive durations, but did
offer examples of problematic durations. That included one fifth-grade class that
averaged 150 minutes on a Spring test in an unnamed subject. The class’s 2.5-hour

Page 7 of 15

 average was beyond the 99th percentile for fifth-grade tests nationally, and was
“unreasonably long, given that there is no reason for students to need to average 2.5
hours to complete [the test]. Durations this long invalidate comparisons between
[these] students’ test results and [the test’s] norms, because the conditions vary so
much from the typical test durations.”
This latest guidance makes it all the more imperative that CPS find a way to rein in
its durations to protect the validity of comparisons to national norms and therefore
the utility of CPS test results.
PAUSE FINDINGS

CPS rules for administering its high-stakes test allow even general-education
students to take “frequent breaks,’’ according to CPS training materials. Bathroom,
water or wiggle breaks are only supposed to occur while a test is paused.
Also, generally, if a question is on a computer screen for 25 minutes without an
answer, the test will time out and send the student back to the login page. The
proctor must then resume the test.
Both pauses produced by proctors to provide testing breaks and time-outs resulting
from student inactivity were counted as pauses in custom reports requested by the
OIG. Current data does not distinguish between the two.
Current CPS rules allow even general education students to take their tests over
“several days,’’ according to CPS training materials. To continue the test a second day,
it would have to be suspended at the end of the first day — an action the test vendor
counts in custom reports as a pause.
Nearly 48 percent of CPS tests in Spring 2018 involved no pauses at all, and thus
were taken in one sitting without any breaks or pauses. Conversely, about 52
percent of tests had at least one pause.
More importantly, as indicated in Table 3, more than 12,000 tests had at least five
pauses. More than 1,600 had at least 10 pauses and more than 200 tests had 20 or
more pauses. At a handful of schools, some individual tests had more than 40
pauses, according to special counts provided to the OIG by the test vendor.
Some might question whether pauses were mostly taken by Diverse Learners whose
IEPs allowed breaks. This was not the case. A much larger percentage of non-Diverse
Learners’ tests had at least 10 pauses than those of Diverse Learners, as indicated in
Appendix B.
As with longer durations, higher pause counts tended to cluster in certain schools:
20 percent of tests with at least five pauses were concentrated in 1.7 percent of
schools.

Page 8 of 15

 Table 3: CPS Tests by Number of Times Paused or Timed Out
Total

0

1-4

5-9

10-14

15-19

20+

Tests

302,993

145,424

145,388

10,524

1,149

290

218

Students*

152,128

91,221

94,309

8,904

1,053

268

180

463

459

462

401

165

60

24

Schools*

*Reflects students and schools with at least one Reading or Math test in the indicated pause range.
Note: The OIG did not receive pause data for some tests. Those tests are excluded from this analysis.
Source: OIG Analysis of Spring 2018 CPS Data from Grade 3-8 Tests

One OIG analysis made clear that the percent of CPS tests with at least five pauses
increased as grade levels increased.
Table 4: Pct. of Tests w/ 5
or More Pauses by Grade
Grade

% of Tests w/ 5+ Pauses

3

1.7%

4

1.8%

5

2.3%

6

4.6%

7

7.0%

8

7.1%

All

4.0%

Note: The OIG did not receive pause
data for some tests, which are
excluded from this analysis.
Source: OIG Analysis of CPS Spring
2018 Test Data

As shown in Table 4, a significant jump in the percent
of tests by grade with five or more pauses occurred in
seventh and eighth grade. These tests can impact
high school admissions and eighth-grade graduation,
so they have especially high stakes for students.
The OIG interviewed a small sample of students and
teachers from schools with unusual results, including
high pause counts, to probe why high pause counts
might be occurring.
In doing so, the OIG was told that in a few classrooms,
proctors were pausing the test to allow students to
skip difficult questions.
Such tactics are an improper use of the pause
function, the test vendor said. “The validity of the
assessment can be compromised if tests are paused
for the purpose of producing a new question,’’ the
vendor told the OIG.

In several interviews, both students and teachers told the OIG that in some cases
kids were intentionally timing out questions so they could get new questions.
For example, one CPS seventh grader told the OIG that her classmates would rather
let a question time out than guess and be wrong. “We were so worried about high
school,’’ the seventh grader said. “Guessing — we would never do that.”
According to the test vendor, an “assessment’s validity may be compromised” if
students intentionally time out hard questions so they can get new questions.

Page 9 of 15

 “In general, there is no reason for a student or proctor to allow a question to time
out,’’ the vendor told the OIG. Doing so “should be considered a possible ‘gaming’
practice.”
As a result of these findings, the OIG is recommending that proctors be given clear
instructions during training on how the pause function works and how to prevent its
abuse. The test vendor has suggested that proctors be required to document all
time-outs and that students with excessive pauses be required to retake their tests.
However, given the closeness of CPS Spring tests to the end of the school year,
presumably large numbers of retests would be difficult to manage.
UPDATED PAUSE GUIDANCE

Recent guidance from the test vendor, released in December of 2019, publicly
affirmed some of the information the vendor had shared with the OIG concerning
pauses. This new public guidance stated that pausing a test in order to replace a
hard question with another question is an “unsupported” practice that “could affect
the validity of the assessment.”
As an adaptive test, the guidance said, the test was designed “with the expectation
that students would be given challenging items that they were likely to get wrong
about half of the time, and with the expectation that students would attempt those
items. If the pause function is used to skip items that are perceived to be too difficult,
then item responses that are essential to the accuracy of the measure are not
considered in assessing the student’s achievement. This could make the resultant
score invalid.”
This new guidance makes it clear that CPS must rein in improper pause tactics to
protect the validity of its test results.
GROWTH FINDINGS

There are many legitimate reasons why a student’s test may show unusually large
gains. This can include that the student’s previous test was uncharacteristically low;
the student made large strides in language proficiency from the previous year; the
student was diagnosed with a learning disability and given needed accommodations
for the first time; or a student clicked with a particularly effective teacher, a new
curriculum or a new learning strategy.
However, an OIG analysis found that a higher percentage of high-growth tests
occurred among very long-duration tests, as indicated in Chart 2.
The OIG analyzed student growth by comparing each test’s score gain from Spring
2017 to Spring 2018 to that of other CPS tests in the same subject and grade with
the same Spring 2017 score.

Page 10 of 15

 The OIG defined “unusually high growth” as two or more standard deviations above
the average growth of comparable tests — also known as a “z-score” of at least 2.
Only 1.9 percent of CPS 2018 tests in grades 3 to 8 met this standard.
An OIG analysis found that a larger percentage of high-growth scores occurred
among tests that were much longer than the typical nationwide test. The OIG’s
analysis looked at the frequency of high-gaining tests broken down by duration
ranges and found that as durations increased, so did the occurrence rate of highgaining tests.
By the time a student was taking 5 or 6 hours (or 301 to 360 minutes) to complete
the test — most likely spilling over into at least a second day of testing — 7.6
percent, on average, were posting unusually large gains. That’s four times the
system-wide average of 1.9.
Students who took more than 6 hours to complete their tests were producing more
than 6 times as many high-gaining tests as the average 1.9 percent systemwide.
Although Diverse Learners had a higher rate of achieving an unusually high-growth
score in each duration band, both Diverse and non-Diverse Learners who spent

Page 11 of 15

 longer on their tests produced a higher rate of high-gaining scores than those whose
durations were typical nationally.
CPS has emphasized that the OIG’s analysis does not show a systemwide correlation
between duration and academic growth. However, this was not the purpose of the
OIG’s analysis. Instead, the OIG sought to determine whether those CPS tests that
were very long had a greater frequency of unusual growth. According to Cizek, an
educational measurement expert, a correlation analysis “masks the relationship
between these variables” while the OIG’s charts “describe the relationship in the
most accurate way.” Cizek said the OIG’s charts depict a “strong relationship”
between these variables.
As with durations, the OIG analyzed the distribution of unusually high-gaining tests
among those with high pause counts.
CPS tests with 5 to 9 pauses had unusually high growth nearly 2.5 times more often
than CPS tests overall, Chart 3 shows. Tests with 10 or more pauses had unusually
large gains 4 to 5 times more often than the CPS average.

Page 12 of 15

 The occurrence rate of unusually large gains among both Diverse Learners and nonDiverse Learners was higher for those with more pauses. This is curious, because as
one testing expert told the OIG, other than for students with special needs who
require accommodations involving breaks, there’s “no educational explanation for
why pauses would improve scores.”
When the OIG talked to students and teachers in schools with unusual results to
probe possible reasons for high pause counts and long durations, the OIG was told of
some tactics that could have boosted pause counts or durations and could have
improved scores.
These interviews occurred with a very small subset of the more than 150,000
students who took the test, and were not intended to reflect a representative sample
of the district. However, in several cases the OIG heard similar concerning anecdotes
involving multiple schools or multiple classrooms within the same school.
Some pauses clearly could be benign, but the way that high pause counts tended to
cluster in certain schools and the comments of students at some of those schools
indicated that the pause function was being abused in some cases.
All of the improper techniques described to the OIG occurred at the instigation of, or
with the acquiescence of, the test proctor. Therefore, the OIG strongly recommends
that CPS find an auditable way of recording the proctor of each test.
The test vendor does not currently provide such data, but two other major testing
companies have told the OIG that they do. The OIG believes that collection of this
data alone would serve as a deterrent to improper practices. The OIG has
recommended that such information be used to identify classroom audit sites, rather
than tying one year’s audits to how a certain grade in a school performed in a certain
subject the previous year.
CPS AUDIT RECOMMENDATIONS

In April of 2018, the CPS Office of Internal Audit and Compliance released an audit
concerning the testing protocols and detection methods used by CPS during the
administration of the test. This Audit had been done at the request of the CPS
Departments of Student Assessment and School Quality Measurement and Research.
This audit found that CPS’s “detective controls” surrounding the administration of
the high-stakes test as well as its “preventive controls” needed strengthening. Audit
proposed a long list of recommendations to address these issues.
As a result, among other things, CPS boosted its preventive training efforts, created a
new method of detecting unusual test results that warranted audits in a joint effort
among several departments, and audited a much larger number of classrooms in
2018 and 2019 than in 2017.

Page 13 of 15

 While the CPS Audit clearly resulted in reforms, the OIG also found that some key
Audit suggestions were never executed, were partially executed or were
inadequately executed.
For example, CPS did not implement Audit recommendations or suggestions that:
o CPS try to rotate proctors so teachers would not be proctoring their own
students. In guidance on giving high-stakes versions of its test dated January
of 2017, the test vendor recommended that teachers of record be
accompanied by a second proctor with no direct stake in the test.
o the Test Security Agreement that all proctors were required to sign include a
warning of the penalties for violating test administration rules.
o CPS use pause3 data as part of its process for identifying unusual results.
This was one of about seven test data points the audit suggested. CPS
ultimately used only 3 of them, and two of the three were redundant,
according to one test security expert.
Although CPS improved its training methods as a result of the CPS audit, the OIG
found that the new training and the five-question proctor quiz that had to be passed
to proctor the high-stakes test did not adequately cover the test’s unique features,
proper and improper testing behaviors or how to prevent certain test irregularities.
CPS proctors need clearer rules and new guidance on how to prevent time-outs and
unusually long durations.
In addition, as a test security guidebook published by the Council of Chief State
School Officers recommends, proctors need “clear examples of what behavior is
unacceptable” because “One source of cheating by staff is lack of understanding
about what are acceptable and unacceptable behaviors.”
Student interviews also indicated that some irregularities could be occurring during
small-group testing, including of general-education students, which is not allowed
under current CPS rules. Therefore, CPS’s audit checklist should include monitoring
of whether small-group testing is being used correctly.
INSUFFICIENT DATA

The OIG also found that the comprehensive data file currently provided by the
vendor is missing certain information that would be helpful in flagging unusual test
results.
This includes, as mentioned, who proctored each test; who paused a test; whether a
pause was a time-out due to student inactivity or a pause instigated by a proctor; the
3

CPS would likely have had to pay the vendor extra for a custom report with pause data.

Page 14 of 15

 number of pauses on each test; how many days each test took; the length of each test
session; and which dates the test was worked on.
CONCLUSION

The test discussed here carries so many stakes for so many CPS parties that its
accuracy is absolutely critical. Concerning patterns in the data and comments from a
small sample of students and teachers in schools with unusual results indicate that
swift action must be taken to tighten up the administration of this test. The area is so
complex and technical that the help of an independent test security expert is
warranted.
This expert can advise CPS on improving its test administration rules, training,
proctor quiz, criteria for flagging unusual results, audit checklist and the test-related
data CPS collects, as well as shrinking CPS durations and pause counts. This expert
also should help CPS write new requirements into its testing contract.
If the current vendor cannot provide the recommended changes, the expert should
assist CPS in writing a proposal for a new test contract after the current $2.2 million
contract expires at the end of June 2020.

Page 15 of 15

 Appendix A: Spring 2018 Tests by Duration and Diverse Learner Status
Duration
(Minutes)
Total Tests
Tests by DLs

Total

0 to
75*

76 to
120

320,561 105,682

121 to
180

181 to
240

241 to
300

301 to
360

Greater
than 360

111,308

68,386

23,010

7,839

2,764

1,572

45,243

21,043

13,663

7,214

2,189

723

257

154

275,318

84,639

97,645

61,172

20,821

7,116

2,507

1,418

% of DL Tests

46.51%

30.20%

15.95%

4.84%

1.60%

0.57%

0.34%

% of non-DL
Tests

30.74%

35.47%

22.22%

7.56%

2.58%

0.91%

0.52%

Tests by non-DLs

*An article on the testing company’s website says that, in general, it expects students to complete a test in about
45 to 75 minutes.
Note: The OIG identified Diverse Learners using 2017-18 data from CPS’s Special Education indicator, which is
based on whether the student has an Individualized Education Program.
Source: OIG Analysis of CPS Data from Grade 3-8 Tests

Appendix B: Spring 2018 Tests by Pauses* and Diverse Learner Status
Pauses

Total

0

1-4

5-9

10-14

15-19

20+

Total Tests

302,993

145,424

145,388

10,524

1,149

290

218

Tests by DLs

42,760

19,014

21,944

1,649

112

26

15

260,233

126,410

123,444

8,875

1,037

264

203

% of DL Tests

44.47%

51.32%

3.86%

0.26%

0.06%

0.04%

% of non-DL Tests

48.58%

47.44%

3.41%

0.40%

0.10%

0.08%

Tests by non-DLs

*The test company’s data does not distinguish between pauses and time-outs.
The OIG did not receive pause data for some tests, which are excluded from this analysis.
Note: The OIG identified Diverse Learners using 2017-18 data from CPS’s Special Education indicator, which is
based on whether the student has an Individualized Education Program.
Source: OIG Analysis of CPS and Vendor Data from Grade 3-8 Tests