July 26, 2019
VIA HAND DELIVERY and E-MAIL
J. Mitchell Armbruster, Esq.
Smith Anderson, L.L.P.
150 Fayetteville Street, Suite 2300
Raleigh, NC 27601
marmbruster@smithlaw.com
RE: Final Decision in Response to Amplify’s Protest
Dear Mr. Armbruster:
Istation is the best diagnostic tool for the state of North Carolina. Istation received the
contract award through a fair and objective process conducted by an evaluation committee that
adhered to all laws, rules, and policies. The evaluation committee recommended Istation to me and
to the State Board of Education (the “State Board”). I approved that recommendation, and the State
Board unanimously approved that recommendation.
Pursuant to 09 NCAC 06B .1102(c)(2), this letter and its attachments constitute my decision
on behalf of the North Carolina Department of Public Instruction (the “Department”) to the protest
by your client, Amplify Education, Inc. (“Amplify”), of the Department’s June 7, 2019 Read to
Achieve contract award to Imagination Station, Inc. (for this letter, “Istation” refers to the company
or Imagination Station Inc.’s reading diagnostic tool, Istation, as the context requires).
I have reviewed the claims in your protest letter, the claims expressed during the in-person
protest meeting, and the claims in your supplemental letter. After careful review, I stand by the
Department’s original contract award and deny Amplify’s protest.
This letter will give you and the public the clarity necessary to understand the entirety of this
procurement process. Amplify’s claims about Imagination Station, Inc.’s reading diagnostic tool,
Istation, and its claims about the procurement process, are incorrect and many are based on
misstatements of fact.
Please consider these four pages a summary. Details regarding each section can be found in
the provided attachments.
I.

Read to Achieve Diagnostic Tool Procurement
North Carolina law requires one reading diagnostic tool for every kindergarten, first, second,
and third grade class. This same law directed the Department to conduct a Request for Proposal
(“RFP”), consider certain factors, and select one tool to be implemented beginning with the 20192020 school year. I was responsible for assembling an evaluation committee of Department
employees who would recommend a tool for my approval. (Please see Attachment A).
OFFICE OF THE NORTH CAROLINA SUPERINTENDENT
Mark Johnson, Superintendent of Public Instruction   mark.johnson@dpi.nc.gov
6301 Mail Service Center, Raleigh, North Carolina 27699-6301   (919) 807-3430   Fax (919) 807-3445
AN EQUAL OPPORTUNITY/AFFIRMATIVE ACTION EMPLOYER

 The State Board, the Department, and I awarded this contract to Istation because it is the
best diagnostic tool for North Carolina.
•
•

•
•
•
•

Istation provides immediate results for parents to view, track, and better help their
students’ literacy progress; even allowing families to log in at home.
Istation provides immediate data and results for teachers and administrators to view,
analyze, and personalize their teaching to better meet the needs of North Carolina
students.
Istation minimizes the amount of instructional time impacted by assessments. Put another
way, teachers can spend more time teaching, rather than administering tests.
Istation is a highly integrated tool that provides assessment results with instructional
support for teachers and students.
Istation provides the Department and schools across North Carolina with the best value.
Istation provides more options for less money than does your client.

This list goes on.
Unfortunately, this positive story for North Carolina teachers and students has been
overlooked due to misstatements of fact from your client, unintentional or not, and inappropriate
actions by employees no longer with the Department.
At the start of the process in 2017, evaluation committee members promised to prioritize
fairness and objectivity over improper biases; going so far as to sign non-disclosure agreements
and statements affirming that they had disclosed all possible conflicts of interest. Unfortunately,
a small number of career Department employees did not honor these commitments.
•

•
•

•

•

•

One did not disclose that she had previously been paid by Amplify, her most recent
employer before the Department, even though she signed the no conflict of interest
document.
One violated the non-disclosure agreement to leak updates to outsiders about confidential
issues.
A whistleblower provided evidence of a text message discussion detailing how
committee members had voted and characterizing my priorities and efforts for educators
as attempts to “appease lazy ass teachers.” (Their words, not mine.)
Members of the selection committee employed biased procedures that benefited the
incumbent vendor (Amplify) over other vendors by not clarifying questions on
specifications. If the questions had been clarified, as required by the procurement rules
and procedures, it would have benefitted other vendors.
Members of the selection committee employed biased procedures that benefited the
incumbent vendor (Amplify) over other vendors by allowing untrue statements or nonfactual opinions to be presented as fact during their ranking process. If factually correct
statements had been presented, it would have benefitted other vendors.
Members of the selection committee violated procurement policies outlined in the
Department of Information Technology procurement manual by improperly adding
weight to select criteria when ranking the vendors.
2

 These employees are no longer with the Department. And, to be clear, the majority of
career Department employees are honorable civil servants who place the needs of educators and
students as their top priority. Please do not let the bad actions of a small minority portray all
Department employees in a negative light.
Regrettably, so as to conduct a fair and unbiased procurement process, the Department
had to cancel two RFPs. (Please see Attachment B). Pursuant to state law and procurement rules
and polices, and in consultation with the Department of Information Technology and the State
Board, the Department entered negotiations with Amplify and Istation. (Please see Attachment
C).
The evaluation committee for the negotiations process presented to me its unanimous
recommendation to select Istation, and I approved the recommendation to select Istation.
Department staff then presented the recommendation to the State Board, and the State Board
unanimously approved the recommendation to select Istation. For clarity, there was only one
recommendation in the entire procurement process, and it was Istation. (Please see Attachment
B).
II.

Amplify’s Incorrect Claims about Istation

Throughout this protest for Amplify, you have made incorrect claims about Istation to
argue in favor of Amplify’s request. Istation has already been used successfully in other states
and more than satisfies the requirements to be the reading diagnostic tool for North Carolina.
(Please see Attachment D). Contrary to your incorrect claims, and without limitation:
•
•
•
•
•
•
•
•
•
•
•
•
III.

Istation satisfied the requirements of the procurement.
Istation meets the requirements of law.
Istation predicts students who are at risk of a reading difficulty.
Istation screens for dyslexia.
Istation is developmentally appropriate.
Istation assesses required measures.
Istation is effective at measuring a student’s progress.
Istation allows for students to interact with and read aloud to teachers.
Istation provides sufficient data to allow determinations of short-term student progress.
Istation accurately and reliably determines which students will be reading on grade level
by the end of the school year.
Istation is already on track for successful implementation in North Carolina.
Istation provides the Department and schools across North Carolina with the best value.
Issues with Amplify and mCLASS in North Carolina

In your protest letter, you state that the Department “must also consider factors including
whether the vendor complies with industry standards, the vendor’s past performance with the
state, and the probability of the vendor providing the required service on time.” Amplify has

3

 presented problems meeting the needs and requirements for the Department and North Carolina
school systems. (Please see Attachment E).
IV.

Final Decision

As described and detailed in the attachments, the claims in your protest letter that the
award to Istation does not meet the requirements of the procurement process or North Carolina
law are incorrect.
The Department and the State Board followed all laws, policies, and rules related to this
procurement process and contract award. Because of the reasons I have incorporated herein, and
without waiving the procedural defects of Amplify’s untimely protest letter, I stand by the
Department’s original contract award and deny your client’s protest.
Istation is the best reading diagnostic tool for North Carolina educators, students, and
parents.

Sincerely,

Mark R. Johnson
North Carolina Superintendent of
Public Instruction

4

 ATTACHMENTS
Attachment A

Overview

Attachment B

Read to Achieve Diagnostic Tool RFPs

Attachment C

Negotiations Process

Attachment D

Istation

Attachment E

Amplify

EXHIBITS
Exhibit A

Evaluation Committee Member’s Confidentiality Agreement

Exhibit B

Evaluation Committee Member’s Statement Regarding Conflict of Interest

Exhibit C

January 8, 2019 Text Message Chain – Breach of Confidentiality

Exhibit D

November 19, 2018 Evaluation Consensus Meeting Notes (pp. 1-2)

Exhibit E

Read to Achieve 2018 Weighted Criteria Chart

Exhibit F

June 7, 2019 Contract Award Recommendation

Exhibit G

Positive Responses from North Carolina Educators After Istation Training

Exhibit H

North Carolina Reading Scores

Exhibit I

40-IT00107-16 / BAFO 3 (pp. 20-21)

5

 Attachment A
Overview
State law requires North Carolina to have one reading diagnostic tool for every K-3 class
and required the Department to evaluate diagnostic tools and select one to be implemented
beginning with the 2019-2020 school year.
“The State Superintendent shall form and supervise an Evaluation Panel to review the
proposals received pursuant to the RFP issued in accordance with subsection (b) of this section.”
Section 7.27(c) of S.L. 2017-57, as amended by Section 2.6 of S.L. 2017-197. This law also
required that “[t]he Evaluation Panel shall be composed of persons employed within the
Department of Public Instruction.” Id.
“In determining which vendor to select, the Evaluation Panel shall consider, at a
minimum, all of the following factors: (1) the time required to conduct formative and diagnostic
assessments with the intention of minimizing the impact on instructional time; (2) the level of
integration of assessment results with instructional support for teachers and students; (3) the
timeliness in reporting assessment results to teachers and administrators; and (4) the ability to
provide timely assessment results to parents and guardians.” Id.
As directed by the General Assembly in Section 7.27(c) of S.L. 2017-57, as amended by
Section 2.6 of S.L. 2017-197, “the Evaluation Panel, with the approval of the State
Superintendent, shall select one vendor to provide the assessment instrument or instruments for
the 2019-20 school year.” For clarity, any recommendation from the evaluation committee would
also require my approval to become the selected vendor, which underscored my obligation, and
enhanced my ability, as State Superintendent to ensure a fair and objective process.
The Department, the State Board, and I fulfilled our statutory duties. In doing so, we
consulted with the Department of Information Technology (“DIT”), the agency ultimately in
charge of approving information technology (“IT”) procurement awards. The Department
regularly consults with DIT to ensure the Department adheres to North Carolina’s procurement
laws, rules, and policies.

6

 Attachment B
Read to Achieve Diagnostic Tool RFPs
1. Introduction
The procurement process for the Read to Achieve diagnostic tool faced unexpected
challenges from within the Department. The process had to be cancelled and restarted twice to
ensure fairness and objectivity, while also ensuring the Department and the State Board adhered
to all state procurement laws, rules, and policies.
At the start of the process in 2017, evaluation committee members promised to prioritize
fairness and objectivity over improper biases; going so far as to sign non-disclosure agreements
and statements affirming that they had disclosed all possible conflicts of interest. Unfortunately,
a small number of career Department employees did not honor these commitments.
Each evaluation committee member signed a Confidentiality Agreement. (See Exhibit A).
Each member also signed a statement indicating that he or she had no conflict of interest. (See
Exhibit B).
The first RFP (“RFP-1”) was issued on December 6, 2017 and publicly opened on
January 11, 2018. RFP-1 was cancelled on March 14, 2018 due to a lack of competition and
errors in specifications. Even if RFP-1 had not been cancelled for this reason, the Department
would have had to cancel RFP-1 due to an undisclosed conflict of interest.
During the evaluation phase of RFP-1, the Department was made aware that a voting
member of the evaluation committee, who was also one of the two business owners responsible
for managing the process, did not disclose that she had previously been employed by your client,
Amplify. She was paid by Amplify immediately preceding her employment with the Department.
The Department was made aware of this conflict by a whistleblower who is a career school
district employee.
The second RFP (“RFP-2”) was issued on September 6, 2018 and publicly opened on
October 2, 2018. RFP-2 was cancelled on March 21, 2019. RFP-2 had many legal and fairness
issues (detailed below) that tainted the process, and the evaluation committee did not reach the
consensus required by procurement rules to recommend one of the four diagnostic tools to me.
The Department discovered improper and unfair procedures were used during the evaluation and
that a voting member of the evaluation committee breached the Confidentiality Agreement by
disclosing confidential details about the procurement process.
2. RFP-2 Cancellation
An agency can cancel an RFP when there are “circumstances that prevent determination
of . . . the best value offer; or [there is] any determination that rejection would be in the best
interest of the State.” 09 NCAC 06B .0401. The RFP-2 evaluation process was plagued by
multiple issues, resulting in the Department recommending to DIT that the RFP be cancelled.

7

 Among these issues:
a.
b.
c.
d.
e.

breach of confidentiality agreement;
unfair and improper ranking procedures;
misstatements of fact leading to improper rankings;
improper weights leading to improper rankings; and
lack of consensus and unclear basis for comparison
a. Breach of Confidentiality Agreement During RFP-2

An anonymous whistleblower provided the Department with a screenshot of a text
message chain between two former Department employees showing that a voting member of the
evaluation committee breached the Confidentiality Agreement.
Pursuant to 09 NCAC 06B .0103, the Confidentiality Agreement provides that “. . . all
information and documentation (verbal and written) relative to development of a contractual
document is deemed ‘confidential’ and shall remain confidential until successful completion of
the procurement process. Therefore, Evaluation Committee Members are required to keep all
comments, discussions, and documentation confidential . . . .” (See Exhibit A). “If it is
discovered that there has been a breach of confidentiality by a member of this Committee . . .
[t]he solicitation may be cancelled . . . .” (See Exhibit A).
During the text exchange, dated January 8, 2019, which is the same day as a meeting of
the evaluation committee, one of the texting participants stated: “Well, just got off another
marathon call with [employee’s initials redacted]. 1 hour 45 minutes all about RFP what a
mess!” (See Exhibit C).
The text messages describe who was on the evaluation committee and how each member
voted during the meeting. The texting participant editorialized that some evaluation committee
members “voted for children” while others “voted for helping teachers” so as to “appease lazy
ass teachers.” (See Exhibit C).
The Department still has an on-going investigation into this matter and the extent of the
breach.
b. Unfair and Improper Procedures Corrupted the Evaluation Committee’s
Process in RFP-2
The text exchange discussed above added to the concerns the Department and I already
had about improper bias jeopardizing the legality and impartiality of RFP-2.
The ranking process for RFP-2 was deficient. During the November 2018 meeting, the
evaluation committee members were directed to vote “yes,” “no,” or “maybe” on each
specification. Members were directed to vote “maybe” if clarification was needed. (See Exhibit
D).
However, the evaluation committee decided to rank vendors during the draft specification
assessment without clarifying any specifications that received “maybe” votes. Instead, “maybe”

8

 votes were treated like “no” votes, and only “yes” votes were awarded points for ranking
purposes.
A proper clarification process would have yielded more “yes” votes for vendors. The
Department discovered many specifications that received “maybe” votes (and counted as “no”
votes) should have received “yes” votes.
A clarification process should have occurred prior to any ranking by the evaluation
committee. Vendors should have been receiving credit in this ranking process but were not,
which corrupted the outcome. Further, misstatements of fact resulted in improper “no” votes and
“maybe” votes (which counted as “no” votes) for Istation.
c. Misstatements of Fact
A PowerPoint presented to me in December 2018 contained multiple misstatements of
fact and omissions of important vendor information. The presentation also had rankings of
vendors based on these misstatements and omissions.
As documented in the released public records, multiple misstatements of fact were put
forth by members of the evaluation committee during the RFP-2 meetings. Many of these
misstatements, such as those regarding dyslexia screening, required subsequent clarification and
correction.
The numerous misstatements of fact and biased information found in the December 2018
PowerPoint presentation were never corrected on that document because the presentation was
never delivered publicly.
The voting members of the evaluation committee for RFP-2 followed guidance from
whomever they considered to be a given topic’s subject matter expert on the evaluation
committee. Unfortunately, in multiple cases, the opinion voiced by a subject matter expert was
not grounded in fact. Members of the evaluation committee, whether intentionally or not, were
putting forth untrue statements about vendor products, but these falsehoods were treated as facts
by the rest of the evaluation committee. Consequently, RFP-2 voting and rankings were based on
misstatements of fact and incorrect data.
The Department will likely never know the full extent of the number of points improperly
denied to Istation during the RFP-2 process due to these misstatements of fact. At best, this was
an unprofessional and unfair process. At worst, this allowed for improper bias on the part of an
evaluation committee member to skew the results. This jeopardized the objectivity and fairness
of the entire RFP-2 process.
Even if the Department ignored these issues, the evaluation committee for RFP-2
attempted but failed to reach a consensus, as per procurement guidelines, on a recommendation.
Then the Department learned that a committee member had broken the Confidentiality
Agreement. The related text exchange, unfortunately, featured participants accusing that certain
voting members would “vote for children” while others were voting to “appease lazy ass

9

 teachers.” This is disappointing evidence that misstatements of fact could have been intentional
forms of improper bias.
The following points, a non-exhaustive list, are key examples of misstatements of fact
that led to improper votes.
Misstatements of Fact = Improper Votes = Flawed Results
Misstatement (evaluation committee opinion (summarized in December 2018 PowerPoint)):
Istation is not a valid diagnostic assessment (based on 10 maybe votes, 1 no vote).
Fact:
Istation is a valid and reliable diagnostic literacy assessment and is used in all 50 states at
the campus, district, or state level. It has been implemented statewide in eight states and is an
approved K-3 screener in five additional states. The ISIP ER technical manual provides research
underlying the assessment and includes results of research conducted on ISIP ER over multiple
years showing strong concurrent and construct validity. ISIP ER reliability results for overall
reading range from .927 to .970, which well exceeds required reliability at .80 or validity at .60.
Results of predictability studies show ISIP ER can predict with confidence students’
performance on End of Grade exams and is a stronger predictor than DIBELS ORF for Texas
Achievement of Knowledge and Skills (TAKS).
Misstatement (evaluation committee opinion (summarized)): Istation is not a valid dyslexia
screener (based on 2 maybe votes, 9 no votes).
Fact:
Istation is a valid dyslexia screener and ISIP ER is already used as a screener of risk for
dyslexia in two states per legislation requiring dyslexia screening. ISIP ER provides reliable,
continuous assessment in each of the critical areas of reading, including development of
phonemic awareness and alphabetic knowledge and skills. Therefore, ISIP ER can be a powerful
aid in identifying students with dyslexia early in their development. In a telephonic interview of
Dr. Joseph Torgesen, one of the top experts in the field of dyslexia, conducted on July 12, 2019,
he stated: “I consider this to be as good a screener as any I know of.”
Misstatement (evaluation committee opinion (summarized)): Istation does not meet the
requirements for progress monitoring (based on 5 maybe votes, 5 no votes, 1 yes vote).
Fact:
Istation is a valid progress monitoring tool and is currently being used for progress
monitoring in all 50 states at the campus, district, or state level. Istation meets the requirements
as a strong progress monitoring tool because it: (a) is brief – the assessment may be given to
students in a one-on-one setting or with an entire classroom in approximately 30 minutes, (b) is
repeatable – numerous items are available for each subtest, making the subtests repeatable
throughout the school year, (c) is sensitive to improvement over time – ISIP ER has been
designated to automatically provide continuous measurement of student progress throughout the
year in all critical areas of reading, (d) documents improvement over time with instantaneous
10

 reports to show student progress, and (e) allows for off grade progress monitoring as the
assessment will automatically adapt to student ability even if it is below the students current
grade.
Misstatement (evaluation committee opinion (summarized in December PowerPoint)): Istation
parent reports require the teacher to manually fill in the report for each child (based on 6
maybe votes, 2 no votes, 3 yes votes).
Fact:
Istation automatically generates reports for teachers and parents that are available in
English and Spanish, including the Student Summary Report which tracks each student’s
progress in critical areas being assessed with complete graphic and contextual analysis. Several
user-friendly printable and customizable templates are also available that teachers can choose to
use in addition to the automatically provided reports. Also, parents in North Carolina will have
access to the online parent portal where they can access student level reports that show specific
skills in which each student struggles, and it links directly to the appropriate lessons and
resources.
Misstatement (evaluation committee opinion (summarized)): Screen-based assessments are not
developmentally appropriate for Kindergarteners and struggling learners (based on 4 maybe
votes, 7 no votes).
Fact:
ISIP ER computer-adaptive assessment is developmentally appropriate due to the very
nature of the way items are administered that are individually selected to be at the right level of
difficulty for each child. The measures are presented in an engaging format, thus reducing many
of the problems, such as distraction and lack of interest, that are sometimes encountered when
young children, especially struggling learners, are assessed. The computer-adaptive format
engages young students as sights and sounds come to life with animated game-like interaction
for young readers. The use of technology in the classroom has increased and will continue to
increase in the future with all students. In a recent study of kindergarten students, Putnam (2016)
found the level of teacher literacy support with the use of Istation had a statistically significant
effect on early literacy achievement, explaining 17.7% of the variance in group differences. As
the use of technology becomes more prevalent, the relationship between technology and literacy
is at the forefront of educational decisions. This study contributes to the growing evidence that
“the teacher’s role in the classroom is strengthened by the introduction of new technologies,” and
“technology supplements, not supplants the teacher in the classroom.”

d. Improper Weights Given to Criteria
Pursuant to the North Carolina Administrative Code and the DIT manual, an IT
procurement process conducted by the Department must use both the source selection method
and the evaluation method designated in the solicitation. RFP-2 directed the use of the “tradeoff”
source selection method and the use of the “ranking” evaluation method (see RFP-2 pp. 13-14).
11

 Under the tradeoff source selection method, “the criteria are stated in relative order of
importance in the bid” but the evaluation must consider the criteria “without weighting them.”
Statewide IT Procurement Office Manual 2017, Section 12.7.2. The evaluation committee
improperly added weight to certain criteria in the RFP-2 process even though the solicitation
document required use of the tradeoff source selection method. (See Exhibit E).
Each evaluation criterion should be given the same weight, and each bid receives a
relative overall ranking. In the next phase, the ranking evaluation method, the evaluation
committee is then allowed to adjust those rankings “up or down when considered with, or tradedoff against, other non-price categories.” 09 NCAC 06B .0302(2)(c).
The State emphasizes the importance of equal weight as the first part of the process
followed by potential adjustments for non-price categories because a goal of State IT
procurements is to drive best value principles when buying services on behalf of the citizens and
taxpayers of North Carolina. A best value procurement is “a procurement process with the
objective of reducing the total cost of ownership.” 09 NCAC 06A .0102(2). The “particular
procurement methods used [in a best value procurement] are selected so as to result in the best
value for the State in terms of the function to be performed or delivered.” Id.
Procurement laws and policies, when followed, drive the best value by having equal
weight for cost at first while allowing an evaluation committee to later adjust for other factors if
necessary. If multiple vendors can provide products, which meet the requirements of North
Carolina law, an agency must consider cost in proper stewardship of the taxpayer dollars
entrusted to it. This process allows agencies to procure products and services that are the best
value to the State and its people.
The RFP-2 evaluation committee improperly added weights to the scoring that prevented
a best value procurement process. Because of this violation combined with the unfair and
improper procedures described above, even a worthy product that was offered at no cost could
have scored lower than Amplify.
e. A Flawed Ranking, Not a Recommendation
Even though, as discussed, there was improper bias, flawed procedures, lack of fairness,
misstatements of fact, and other issues from RFP-2 that jeopardized the results, the evaluation
committee did bring both Amplify and Istation to my attention as the top-ranked vendors.
However, to reach an agreement to recommend one tool to me, the evaluation committee had to
reach a consensus. The only vote in RFP-2 occurred in January 2019 – it was a vote on which
vendor(s) to continue negotiating with – and did not result in consensus, so there was no
recommendation.
f. Cancellation of RFP-2
With guidance from DIT, RFP-2 was properly cancelled.

12

 Attachment C
Negotiations Process
1. Introduction
The Department, after consulting with the State Board and DIT and properly cancelling
RFP-2, entered the negotiations process pursuant to 09 NCAC 06B .0316. An important aspect
of the negotiations process is that “[n]egotiations shall not materially alter the intent or scope of
the original solicitation document.” 09 NCAC 06B .0316(e). The Department did not materially
alter the intent or scope of the original solicitation document.
2. Intent of the Original Solicitation
The intent of the original solicitation document was to effectuate the goal of the Read to
Achieve legislation. RFP-2 stated that “[t]he purpose of this RFP and any resulting contract
award is to solicit offers for Read to Achieve Diagnostic Software . . . to meet NCDPI’s
obligations under state laws, which can be found at N.C.G.S. 115C-83.1 et. seq.” (see RFP-2 p.
6). The solicitation was intended to “ensure that difficulty with reading development is identified
as early as possible; students receive appropriate instructional and support services to address
difficulty with reading development and to remediate reading deficiencies; and each student and
his or her parent or guardian be continuously informed of the student’s academic needs and
progress.” N.C. Gen. Stat. § 115C-83.2 (2018). The intent of the negotiations process was to
effectuate the goal of the Read to Achieve legislation. The intent did not materially change.
3. Scope of the Original Solicitation
In RFP-2, the Department listed six criteria for evaluation. In the negotiations process,
with guidance from DIT, the Department reworded certain criteria for brevity and clarity and
removed two criteria that had already been fully addressed. The scope did not materially change.
The negotiations process, however, did correct a flaw in RFP-2 by removing the weights from the
scoring of the criteria.
The chart below shows how the changes were not material alterations.
RFP-2 Criteria
•

Negotiations Criteria
•

•

Substantial Conformity to Solicitation
Specifications
RFP Desired Specification

•

Proof of Concept/Demonstration

•

•

Vendor Cost Proposal

•

Vendor Cost Proposal

•

Vendor Relevant Experience and Reference
Checks

•

•

Vendor Financial Stability

•

Removed since this part of the process was
properly completed by the evaluation
committee during RFP-2 and did not need
repeating
Vendor Financial Stability

•

Reworded as “Formative & Diagnostic
Assessment” for clarity*
Reworded as “Personalized Learning” for
clarity**
Removed since this part of the process was
properly completed by the evaluation
committee during RFP-2 and did not need
repeating

13

 *As you state in the supplemental letter, “the whole purpose of the RFP is to procure ‘valid, reliable, formative and
diagnostic reading assessments.’” (See RFP-2 p. 6).
**In RFP-2, the Department had included personalized learning as a substantive desired specification. Personalized
Learning is a strategic direction of the State Board and clarification was added through this rewording in the
negotiations process.

4. Amplify’s Claims in the Supplemental Letter
The theories purported in your protest are incorrect. The scope of the negotiations did not
materially change from those of RFP-2. The Department did not change the rules of the
procurement. Your claim that the Department “threw out” criteria is wrong. With guidance from
DIT, certain criteria were removed because they were already satisfied for the evaluation
committee and others were reworded for clarity.
Properly following IT procurement policies, as discussed above, the criteria were not
weighted during the negotiations process and were equal as a result. References and product
demonstrations for Amplify and Istation were both properly completed and satisfactory. Being
unweighted and equal, the vendors were equal for these criteria.
With the guidance of DIT, the Department properly moved the cost proposal and vendor
financial stability, both criteria from RFP-2, higher in priority without materially changing the
scope. This did not have an impact on the outcome, though, because no adjustments were made
by the evaluation committee based on the order of priorities.
All the criteria were unweighted and equal. Amplify and Istation were both satisfactory
for vendor financial stability. Being unweighted and equal, the vendors were equal for this
criterion. Istation’s cost proposal provided during the negotiations had more strengths and fewer
weaknesses than Amplify’s cost proposals provided during the negotiations.
When given all the facts and the strengths and weaknesses for all criteria, the evaluation
committee for the negotiations process unanimously agreed to continue further negotiations with
Istation and recommended awarding to Istation. The evaluation committee had no need to adjust
for other factors.
5. Cost and Value
As discussed above, the State emphasizes the importance of equal weight first. This can
potentially be followed by adjustments for non-price categories because a goal of State IT
procurements is to drive best value principles when buying services on behalf of the citizens and
taxpayers of North Carolina. A best value procurement is “a procurement process with the
objective of reducing the total cost of ownership.” 09 NCAC 06A .0102(2). The “particular
procurement methods used [in a best value procurement] are selected so as to result in the best
value for the State in terms of the function to be performed or delivered.” Id.
Procurement laws and policies, when followed, drive the best value by having equal
weight for cost at first while allowing an evaluation committee to later adjust for other factors if
necessary.
Cost was a significant factor in RFP-2. If multiple vendors can provide reading diagnostic
tools that each meets the requirements of North Carolina law, the Department must consider cost
14

 in proper stewardship of the taxpayer dollars entrusted to it. Per procurement policies, this
maximizes value for the citizens of North Carolina. The Department made clear to Amplify and
Istation that cost would continue to be a significant factor in the negotiations process. To
emphasize the importance of cost in the process, and with the guidance of DIT, cost was given a
high priority in the Request for Negotiation communication sent to both vendors.
To maximize fairness during the negotiations process, the Department gave each vendor
the opportunity to improve their price offered to the State. Amplify offered a reduced bid of $3.8
million annually (plus professional development costs) compared to its bid of $6.1 million
annually (plus professional development costs) in RFP-2. This bid was still objectively higher
than Istation’s bid of $2.8 million annually (plus professional development costs).
Though cost was given a high priority, the evaluation committee for the negotiations
process did not use such priority in its determination. Every criterion was unweighted and equal.
The evaluation committee made its recommendation based on evaluations of the unweighted
criteria and had no need to adjust for other factors.
Your claims in the protest are wrong. Also, in your protest letter dated June 24, 2019, you
take issue with a newspaper article that reported Istation’s contract amount for the contract award
and Amplify’s current contract amount. The Department answered the questions asked by the
reporter. While we cannot control what a reporter decides to compare, you use the opportunity to
point out that Amplify’s bid was $3,755,560 a year, which you say represents more than a 40%
reduction from prior years.
In the next paragraph, however, you claim that Amplify’s tool offers more than Istation,
specifically, the TRC. You do not properly qualify this argument with the fact that the additional
offerings would cost more than the amount of Amplify’s bid. Amplify’s bid referenced in the
protest letter was $3.8 million (plus over $500,000 for professional development) and did not
include the TRC. Amplify also proposed an alternate bid, which included the TRC and added
Amplify Reading National Edition, along with additional professional development. This bid
was for $10.1 million in the first year (plus $1.2 million for professional development).
A top priority of any IT procurement is to find the best value for the State in terms of the
functions to be performed or delivered. After a fair and unbiased process, Istation proved to be
that best value to the State.
6. Recommendation to and Approval by the State Superintendent and the State Board
of Education
The evaluation committee for the negotiations process presented to me its unanimous
recommendation to select Istation, and I approved the recommendation to select Istation. (See
Exhibit F). Department staff and I then presented the recommendation to the State Board, and the
State Board unanimously approved the recommendation to select Istation.

15

 Attachment D
Istation
1. Introduction
Throughout this protest for Amplify, you make incorrect claims about Istation to argue in
favor of Amplify’s request. Istation has already been used successfully in other states and more
than satisfies the requirements to be the reading diagnostic tool for North Carolina. Contrary to
your incorrect claims:
•
•
•
•
•
•
•
•
•
•
•

Istation satisfied the requirements of the procurement,
Istation meets the requirements of law,
Istation predicts students who are at risk of a reading difficulty,
Istation screens for dyslexia,
Istation is developmentally appropriate,
Istation assesses required measures,
Istation is effective at measuring a student’s progress,
Istation allows for students to interact with and read aloud to teachers,
Istation provides sufficient data to allow determinations of short-term student progress,
Istation accurately and reliably determines which students will be reading on grade level
by the end of the school year, and
Istation is already on track for successful implementation in North Carolina.

2. Istation’s Use in Other States
Istation has been adopted for statewide implementation in eight states: Texas, Florida,
Colorado, Kansas, New Mexico, Idaho, Arkansas, and North Carolina. In addition to the eight
statewide implementations, Istation has been approved by state departments as a K-3 literacy
screener in the following states: Massachusetts, Mississippi, Michigan, Ohio, and Oklahoma.
Idaho
The State Department of Education in Idaho recently released preliminary spring reading
scores on July 1. Scores went up across the board from fall to spring.
“The preliminary results from this first year of the new, comprehensive Idaho Reading
Indicator are promising,” state superintendent Sherri Ybarra said in a news release. “The
improvements were considerable. In each grade, the number of students reading at grade level
improved by at least 12 percentage points. In first grade, the improvement approached 24
percentage points. Put another way, more than 14,500 students who scored below grade level on
the fall IRI advanced to grade level by spring.”
Kevin Richert, What the New Reading Scores Say – And What They Don’t Say, IDAHO ED.
NEWS (July 11, 2019), https://www.idahoednews.org/news/what-the-new-reading-scores-sayand-what-they-dont-say/.
16

 Florida
Based on a recent report from Florida, results are promising for third grade students who
have used Istation, as districts partnering with Istation indicate higher growth for third grade
standardized testing in May. In the districts of Alachua, Monroe, Pinellas, and Polk, third grade
scores “averaged a growth rate of 5.58% over the course of the FSA 2015-2019 testing results.
This is over 2% more than average growth rate of districts that have not partnered with Istation.
This average growth rate comes out to 3.4% over the same span of years.”
“Istation helps raise the potential for growth in each student by allowing both teachers
and students to monitor student growth, and by providing individualized education plans for each
student based off of needs. These plans are scripted so a new teacher can easily use them, or if
there is a seasoned educator using the technology they can smoothly add their own material to
the personalized lesson so that each student gets the education they need rather than the
traditional “one-size-fits-all” approach. “I love that it (Istation) tiers my students and provides
individualized interventions that I can use to help my struggling students,” said Jessica
Lockwood, a third-grade teacher at Poinciana Elementary School on Key West.”
Online Staff, Higher Growth Reported for Third Grade Standardized Testing In Florida Districts
Partnering With Istation, THE BOCA RATON TRIB. (June 18, 2019),
http://www.bocaratontribune.com/bocaratonnews/2019/06/higher-growth-reported-third-gradestandardized-testing-florida-districts-partnering-istation/.
Arkansas
A recent press release out of Arkansas speaks to reliability and validity of using Istation
as a K-3 assessment. “Up to 40% of Arkansas's public kindergarten, first and second graders
need additional support to achieve at a desired level in reading. That’s based on the 2019 results
from the three different nationally standardized tests from which school districts can choose to
give to pupils to gauge their achievements in reading and math. The state-authorized tests -Istation, Renaissance, and NWEA -- produced similar results, which indicates the tests are
reliable measures,” said the state’s director of assessment, Hope Worsham.
Cynthia Howell, Education Notebook, ARKANSAS DEMOCRAT GAZETTE (July 15, 2019),
https://www.arkansasonline.com/news/2019/jul/15/english-reading-scores-stir-queries-tes-1/.
New Mexico
New Mexico awarded Istation with a statewide contract in 2016 and faced a four-week
timeline for implementation. Cloudcroft Municipal Schools K-8 Principal and DTC shared the
following comments from her faculty and staff:
“This experience has changed our minds about Istation. We weren't a fan [sic] because of
the quick turnaround we had to do. But the people with Istation have been very helpful and so
nice so that was not as scary as we thought it would be. My teachers loved how quickly they got
their data and access to intervention for students” (response on file with the Cloudcroft
Municipal Schools).
17

 ". . . I have an autistic student in third grade who is very negative, and it is hard to test
because he refuses. He finished the entire test without redirecting from us. And even asked after
the test when he could do it again. At one point he was belly laughing! That just doesn't happen
with him on a test or a computer.” Id.
“You personally relieved my personal concerns for utilizing the program with five-year
old children! I'm on board now to support and encourage our teachers to give this a try . . . .” Id.
“I am so relieved to say I really think we will love this new program! The Istation staff
are all so very passionate and most of them are former teachers who used this in their own
classrooms!” Id.
“I feel very much at ease with it, and the other administrators I talked with were also very
relieved and optimistic.” Id.
“I have really been impressed with your company's responsiveness. I have dealt with
support twice, and they were incredible.” Id.
Kansas
Kansas approved the entire program for Kansas Reading Success in 2015. A fifth grade
teacher at Chanute Elementary School in Kansas shared how Istation has impacted her students’
learning, her teaching, and her classroom. Though Kansas differs from North Carolina because
all schools use the curriculum, her comments about reports and simplicity for gathering data
provide an important example of how teachers quickly gain insight into using data to inform
instruction.
“I love the reports! They give so much information on exactly what the students are
getting and what they aren’t. For example, I had one Tier 3 student who was doing great on
Istation. With each passing week, her scores were improving. When the monthly assessment was
pushed in, however, she tanked it. I was able to drill in and see exactly which questions she had
missed. After the analysis and a conversation with her, it was apparent she bombed it on purpose;
she was nervous about moving up to a harder level. After much encouragement she is back on
track and moving forward!”
“I love that it is pushed in each calendar month. It helps keep the kids motivated and is so
easy for the teacher. If a student is absent the day of the assessment, it’s just there the next time
they log on . . . so easy on the teacher! Also, the reports generated from the assessments are
amazingly data rich. I feel that it is meaningful data that drives both instruction and intervention.
My Professional Learning Community (PLC) is considering using the Istation assessments as our
common assessment next year to track student growth.”
Lindsey Joyce, Year One With Istation: A Grade 5 Teacher Reflects, ISTATION BLOG (May 4,
2016), https://blog.istation.com/year-one-with-istation-a-grade-5-teacher-reflects.

18

 3. Istation Feedback from North Carolina
Eric Davis, Chairman of the State Board, told me that he discussed the State Board’s
decision with education experts he trusts, and they told him they consider Istation one of the top
diagnostic tools available (second only to another vendor that applied but did not meet all the
criteria of the Read to Achieve law (not Amplify’s tool)).
Multiple meetings of the State Board have occurred since the contract award to discuss
Istation and its implementation in the 2019-2020 school year. The State Board has been
supportive of the contract award and has not budged from its decision. When discussing the
training schedule for Istation in a board meeting on June 28, 2019, State Board member Amy
White said: “This is not a discussion about whether we’re going back to mCLASS.”
T. Keung Hui, After outcry from educators, NC will delay use of computer-based reading test,
THE NEWS & OBSERVER (June 29, 2019, updated June 29, 2019),
https://www.newsobserver.com/news/politics-government/article232045842.html.
At the same meeting, Amy White said: “I think having something new is always hard.
Moving from one platform to another is always hard. But I think once they see it, touch it, feel it,
they will realize that [Istation’s] going to be something very positive.”
Board member Dr. Olivia Oxendine said: “This is more of an instructional feature
advantage that I see in Istation. The other thing is that I go back to the year 2013, definitely
2014-15, and there was an incredible angst around the state among classroom teachers and
curriculum specialists and superintendents about the amount of time taken away from sheer, pure
instruction so that so much could be done with assessment, the assessment of students. So, we
received during that period of time, probably about 18 months to two years, a barrage of emails
from superintendents and teachers about not enough, not sufficient time given to the direct
instruction of reading. So, I believe that we are going to have the opportunity to allow teachers to
do what is called teaching reading and acknowledging and teaching to the science of reading,
which is music to my ears.”
Jack Hoke, the Executive Director of the NC School Superintendents Association,
emailed me and told me that he attended a session in Asheville with Istation. He said it appeared
to be an excellent tool.
The number of positive responses from North Carolina educators continues to grow as
more and more attend Istation trainings. Please see the current compilation of responses collected
after educators completed face-to-face training with Istation. (See Exhibit G).

19

 4. Amplify’s Claims are Incorrect
a. Istation Meets the Requirements of Law
Before this process, Istation was approved by the State Board as an assessment for
proficiency in third grade reading under the Read to Achieve law and was used by others in
North Carolina at their own expense.
Istation meets all the criteria of the Read to Achieve law (G.S. § 115C-83.6):
i.
ii.

iii.

iv.

Istation serves as a valid, reliable, formative, and diagnostic reading assessment for
students in kindergarten, first, second, and third grades.
Istation provides immediate data that can be used with the Education Value-Added
Assessment System (EVAAS) to analyze student data to identify root causes for
difficulty with reading development and to determine actions to address them.
Istation provides formative and diagnostic assessments and resultant instructional
supports and services to address oral language, phonological and phonemic awareness
phonics, vocabulary, fluency, and comprehension using developmentally appropriate
practices.
Istation is administered by computer or some other electronic device as allowed by law.
b. Istation Adequately Predicts Students Who Are at Risk of a Reading Difficulty

The authors, Dr. Joseph Torgesen and Dr. Patricia Mathes took great care in constructing
the ISIP ER item pool, basing the types and content on contemporary findings in early reading
research. Istation’s ISIP ER measures are designed to be used for Universal Screening
(benchmarks three times a year) and Progress Monitoring (monthly). The benchmarks and
progress monitoring assessments are designed to test the same skills in the same format and with
the same consistent and comparable results. While the questions will change for the individual
student, the subtest, the format, and overall score composition will remain consistent between
progress monitoring and benchmarks to ensure consistency in instructional design and
instructional decisions between benchmark months.
The advanced technology used in Istation’s adaptive format provides data to help
teachers determine which students are at risk for reading difficulties and those who have missed
critical foundational skills that are typically not assessed at later grade levels. For each student,
the tasks adapt to lower or higher-level skills – meeting the student where he or she is on the
student’s reading journey – and produces a reliable estimate of their ability.
Reliability and validity are two important qualities of measurement data. Reliability can
be thought of as consistency and validity can be thought of as accuracy. The ISIP Early Reading
Technical Report (c 2011; 2016) describes a study conducted using ISIP Early Reading data
collected from kindergarten through grade 3 in five elementary schools in north Texas.
Regarding measures of reliability, data suggest consistently high levels of internal consistency in
the subtest ability scores as well as the overall reading ability scores. ISIP ER also produces
stable scores over time. Evidence of concurrent validity is found in the numerous strong, positive
20

 relationships to external measures of reading constructs. Data suggest mostly large to very large
criterion validity with scores from well-known measures of reading constructs, such as CTOPP,
GORT-4, PPVT-III, TOWRE, WJ-II ACH, WLPB-R, and WIAT-II, as well as with TPRI and
ITBS. In addition, classification accuracy and area under the curve estimates for third grade
accountability measures meet generally accepted standards.
Furthermore, predictive validity results show that ISIP Overall Reading is a stronger
predictor than DIBELS ORF for state mandated Texas Assessment of Knowledge and Skills
(TAKS) Reading, using scores from 1 to 5 months prior. Taken together, the evidence supports
that ISIP ER produces reliable and valid data for measuring key areas of reading development
such as phonemic awareness, alphabetic knowledge, vocabulary, and reading comprehension, as
well as overall reading ability.
Additionally, Campbell, L. O., Lambie, G. W., and Sutter, C. conducted a study to
examine third grade ISIP-ER scores and Florida Standards Assessment English Language Arts
(FSA-ELA) scale scores. Data were collected during the 2016-2017 school year from five public
school districts in Florida. Researchers used a simple linear regression analysis to examine the
correlation of Overall Reading Ability and Reading Comprehension to the FSA-ELA scores.
Results of the study indicated ISIP ER scaled scores had a strong correlate to FSA-ELA scores
and predictability bands were computed to identify cut scores to predict all achievement levels.
This study provides evidence that Istation’s reading assessment can predict the FSA-ELA
statewide examination scores for all achievement levels among third grade students.
Campbell et. al., (2018). Measuring the Predictability of Istation Indicators of Progress Early
Reading (ISIP-ER) Scores on Florida Standards Assessment Scores. University of Central
Florida, www.ucf.edu/mirc.
Recently, Campbell, Sutter, Lambie, and Tinstman (2019) conducted a study to measure
the predictability of Istation’s Indicators of Progress (ISIP) Early Reading Scores on Renaissance
STAR Reading. Researchers examined kindergarten, first, and second grade ISIP-ER scores
(Overall Reading Ability and Reading Comprehension) in one county in the state of Florida.
Data were collected during the 2017 – 2018 school year and a simple linear regression analysis
was conducted to determine the correlation of the STAR scaled scores and the ISIP-ER overall
reading scores. Results indicated ISIP-ER scaled scores had strong correlate to the STAR scores
and predictability bands were computed to identify cut scores for accurate prediction for all
achievement levels. This study provides evidence that ISIP Reading cut scores can predict the
STAR statewide examination scores for all achievement levels among first and second grade
students using computer-adaptive testing.
Campbell et. al, (2019). Measuring the predictability of Istation Indicators of Progress Early
Reading (ISIP-ER) scores on Renaissance STAR Reading scores. University of Central Florida,
www.ucf.edu/mirc.

21

 c. Istation Screens for Dyslexia and Meets the Statutory Requirements for
Screening Dyslexia
Istation is used in other states as a dyslexia screener for children. One of the authors of
ISIP Early Reading (ISIP ER) was Dr. Joseph Torgesen, an Emeritus Professor of Psychology
and Education at Florida State University. He is also the author, with Dr. Richard Wagner and
Dr. Carol Rashotte, of two of the most widely used diagnostic tests for dyslexia, The
Comprehensive Test of Phonological Processes (CTOPP) and the Test of Word Reading
Efficiency (TOWRE). Dr. Torgesen was asked to establish the Florida Center for Reading
Research (FCRR), which has been the premiere site for disseminating information about
research-based practices related to literacy instruction and assessment for children since 2002. In
the development of ISIP ER, several subtests were included that are often used to help in an
initial screening for dyslexia. ISIP ER measures have high correlations with other measures that
are used for dyslexia screening; therefore, ISIP ER measures can confidently be used to screen
for dyslexia, particularly if they are given on a monthly basis. Additionally, Dr. Torgesen stated,
“I consider this to be as good a screener as any that I know of.”
Data related to student performance in the areas that can be predictive red flags of
dyslexia and difficulties in other areas of reading (e.g., phonemic awareness, letter sound-symbol
recognition, letter knowledge) are provided by ISIP ER to identify student skill weaknesses,
evaluate student intervention plans, discuss student performance with administrators, and plan
for parent-teacher conferences. From research, we know that early diagnosis of dyslexia is more
likely when the problems occur in students who have strong abilities in other areas of language
such as vocabulary. Istation measures listening comprehension of students in kindergarten as
well as vocabulary K-3, thus allowing educators to gather assessment data for vocabulary, which
has been an optional measure with mCLASS.
Characteristics associated with reading difficulties are connected with spoken language.
Difficulties in young children can be assessed through screenings of phonemic awareness and
other phonological skills (Sousa, 2005). Additionally, Eden (2015) points out that “when
appropriate intervention is applied early, it is not only more effective in younger children, but
also increases the chances of sparing a child from the negative secondary consequences
associated with reading failure. ISIP ER is a tool that can be used to screen for risk of dyslexia
early in students’ academic careers.”
Beginning with phonological processing, measures of phonological awareness tend to be
the most predictive of success at early reading. As stated by Dr. Joseph Torgesen, “In young
children, the most reliable indicators of risk for dyslexia are difficulties in the development of
phonemic awareness and alphabetic knowledge and skills. The latter category includes lettersound knowledge and alphabetic reading skills (ability to “sound out” unknown words). If
difficulties in these areas are not addressed by powerful instruction early in a child’s
development (kindergarten and first grade), the student typically does not progress in the ability
to read text fluently, and this is an extremely reliable indicator of dyslexia by the end of first
grade. Since the ISIP-ER provides reliable, continuous assessment in each of these critical areas,
it can be a powerful aid in identifying students with dyslexia early in their development.”
22

 It is important to note that ISIP ER does include measures that can be used to screen
students for dyslexia, and Dr. Joseph Torgesen has confirmed that requiring any additional
measures is not necessary to accurately identify students who are at risk for dyslexia. However,
Istation has worked with North Carolina to identify a process that has been used in Arkansas to
further help and flag students for further evaluation. Using the model that has been successful in
Arkansas, educators in North Carolina could use results from an ISIP screening that would be
gathered at a time other than a benchmark, including measures that are not typically administered
at that time/grade level, in order to identify students who are in need of additional advanced
screening. The initial screening can take effect as follows:
•

•

•

•

Kindergarten: Kindergarten students can be automatically screened upon login, at which
point they can receive the additional subtest of alphabetic decoding. This subtest can be
given later in the year in addition to the normal ISIP subtests for kindergarten:
listening/language comprehension, phonological and phonemic awareness, letter
knowledge, and vocabulary.
1st Grade: 1st graders will automatically receive the relevant subtests based on the initial
screening. These subtests include phonological and phonemic awareness, letter
knowledge, vocabulary, alphabetic decoding, reading comprehension, and spelling.
2nd Grade: 2nd grade students can be automatically screened upon login with the
additional subtests of alphabetic decoding, letter knowledge, and phonological and
phonemic awareness. These subtests would be in addition to the normal ISIP subtests for
2nd graders: vocabulary, reading comprehension, spelling, and text fluency.
3rd Grade: 3rd grade students can be automatically screened upon login with the
additional subtests of alphabetic decoding, letter knowledge, and phonological and
phonemic awareness. These subtests would be in addition to the normal ISIP subtests for
3rd graders: vocabulary, reading comprehension, spelling, and text fluency.

For clarity, while the Department and I desire these functions for our Read to Achieve
diagnostic tool, and Istation provides these functions, your claim that Session Law 2017-127
(HB 149) requires the Read to Achieve diagnostic tool to screen for dyslexia is not accurate.
Session Law 2017-127 states that “it is the intent of the General Assembly that all students with
specific learning disabilities, including dyslexia and dyscalculia, receive the necessary and
appropriate screenings, assessments, and special education services to provide interventions for
learning difficulties with language, reading, writing, and mathematics.” S.L. 2017-127, §1.
The law goes on to state that “prior to the start of the 2017-2018 school year, local boards
of education shall review the diagnostic tools and screening instruments used for dyslexia,
dyscalculia, or other specific learning disabilities to ensure that they are age-appropriate and
effective and shall determine if additional diagnostic and screening tools are needed.” Id., at §4.
The law also requires the State Board to craft definitions related to dyslexia identification and to
ensure the availability professional development opportunities related to identification of
dyslexia. The law does not actually require the Read to Achieve tool to serve as a dyslexia
screener for the state. Id., at §2-3.

23

 Again, though, I agree that the Read to Achieve diagnostic tool serves as a good
opportunity to provide local school districts with dyslexia screening and am pleased that Istation
does so.
d. Istation is Developmentally Appropriate
One important piece of Istation is the modeling opportunity provided by the teacher for
students before students interact with the assessment. The computer-adaptive nature of Istation
ensures that each student, regardless of grade level, will be assessed in a way that identifies his
or her individual proficiencies as well as challenges in reading. Data are immediately available
through benchmarks and progress monitoring to differentiate instruction in order to address
individual needs. Istation reports also provide for grouping of students by skill performance for
small group instruction. Recommendations for individual and small group instruction, as well as
interventions are provided through Istation to support the teacher in data-based decision making
and instructional planning. Teachers have access to a repository of resources, including lesson
plans and instructional activities specific to each student’s needs.
Dr. Joseph Torgesen states, “A computer-adaptive assessment like ISIP ER is
developmentally appropriate because it administers items that are individually selected to be at
the right level of difficulty for each child. In addition, because of its lively and engaging format,
it reduces many of the problems, such as distraction and lack of interest, that are sometimes
encountered when young children are assessed.”
With Istation, teachers will assess their own students and obtain valid, reliable, and
authentic data using technology tools and skills. For mCLASS: Reading 3D, there is variance in
the level of teacher training that often yields unreliable data, especially as it relates to the
components with more difficult scoring rules, such as NWF and PSF, and TRC. There are
considerable concerns related to the subjectivity associated with administration of mCLASS
Reading 3D, especially TRC measures. This is evident in the fact that teachers do not assess
their own students at MOY and EOY. Instead, teachers across the state have assessed students in
another classroom on their grade level to increase the objectivity in administration using
mCLASS Reading 3D and obtain scores for students that are nonbiased and accurate. Even
though students interact via technology with Istation, students are expected to read text just as
students read text on Flex or any other mCLASS measure, but ISIP measures will adjust based
on the student’s performance.
With Istation, students absolutely can produce the sounds and read the words out loud or
silently. The teacher will have the flexibility to monitor students based on individual need as they
apply their knowledge to complete tasks to demonstrate proficiency on the critical skills in
reading. In addition, K-12 students have Information and Technology Essential Standards
required by the state which include the following: Kindergarten: Use technology tools and skills
to reinforce classroom concepts and activities.
As documented in the ISIP Early Reading Technical Report: following a Computer
Adaptive Testing (“CAT”) model, each item within the testing battery is assessed to determine
how well it discriminates ability among students and how difficult it actually is through a process
24

 called Item Response Theory (IRT) work. Once item parameters have been determined, the CAT
algorithm can be programmed. Using this sophisticated algorithm, the computer selects items
based on each student’s performance, selecting easier items if previous items are missed and
harder items if the student answers correctly. Through this process of selecting items based on
student performance, the computer generates “probes” that have higher reliability than those
typically associated with alternate formats. This results in a better reflection of each student’s
true ability. The process described above provides evidence that the true ability of students in K3 will be established using the computer-based assessment model of ISIP ER Assessment.
As it relates to the developmental appropriateness of using a computer-adaptive
assessment like ISIP ER for young children, Dr. Joseph Torgesen stated, “There is no research
that I am aware of to indicate that the small increases in time spent in front of a computer screen
using the ISIP-ER constitute a risk for young children. In fact, monthly assessments using the
ISIP-ER very likely involve an extremely small proportion of the total time young children spend
in front of computer, or TV, or IPAD screens in our modern world.”
Lastly, while Istation does meet the criteria as described above, in an attempt to support
Amplify’s approach, your protest letter added inappropriate context for “developmentally
appropriate” attached to the description of this RFP specification around observation-based
practices. The actual specification is: “Describe how the proposed solution directly assesses
reading and pre-reading behaviors to support students’ learning development at the various grade
levels to inform instruction, including any observation-based practices if applicable.”
e. Istation Assesses Required Measures
As noted in the ISIP Early Reading Technical Report (c 2011; 2016), ISIP Early Reading
follows a continuum of learning that, research indicates, is predictive of later reading success.
Skills build upon skills, and the sequence of subtests builds upon prior subtests. As skills of
lower-level difficulty are achieved, they are eliminated from the test battery and more advanced
skills are added. Importantly, there is no other continuous progress monitoring assessment tool
that measures vocabulary and comprehension accomplished through short tests, or “probes,”
administered at least monthly, that samples critical areas that predict later performance. Id.
ISIP Early Reading provides growth information in the five critical domains of early
reading: phonemic awareness, alphabetic knowledge and skills, fluency, vocabulary, and
comprehension. It is designed to (a) identify children at risk for reading difficulties, (b) provide
automatic and continuous progress monitoring of skills that are predictors of later reading
success, and (c) provide immediate and automatic linkage of assessment data to student learning
needs, which facilitates differentiated instruction. Below is a grade level breakdown of how
Istation ISIP ER measures assess all required measures plus listening comprehension,
vocabulary, and spelling that are not assessed with mCLASS: Reading 3D. Id.
Kindergarten (Subtests - Listening Comprehension, Phonemic Awareness, Letter Knowledge,
Vocabulary):
Kindergarten students require assessment of their growth in phonemic awareness,
alphabetic knowledge and skills, and vocabulary. Fluency in letter names and sounds facilitates
25

 spelling, but these skills are usually not developed sufficiently to assess spelling ability. Their
reading skills are also rarely sufficiently developed to usefully assess reading fluency and
reading comprehension. In general, research has shown that phonological awareness and letter
sound knowledge in Kindergarten are predictive of Grade 1 outcomes. For children at risk of
reading difficulty due to poverty or language background, vocabulary is critical to reading
success (Foorman, Anthony, Seals, & Maouzaki, (2016); Snow et al., 1998; Dickinson & Tabors,
2001). Vocabulary assessments for Kindergarten students are mostly “tier 1” words and items to
assess children’s knowledge of prepositions and verbs of varying tense, since these classes of
words are particularly difficult for young children. Id.
Grade 1 (Subtests - Phonemic Awareness, Letter Knowledge, Vocabulary, Alphabetic Decoding,
Comprehension, Spelling, Connected Text Fluency, Oral Reading Fluency):
It is important to continue to monitor students’ development of phonemic awareness and
alphabetic knowledge and skill, because struggling students may continue to have difficulty in
these areas. The development of accurate and fluent decoding skills should be monitored, since
these foundational skills for reading accuracy undergo major development. Word recognition at
the beginning of Grade 1 has been found to be predictive of Grade 1 outcomes. Spelling has also
been found to be a predictor of oral reading fluency. Vocabulary growth is important in the
development of reading comprehension. As soon as students can demonstrate the ability to read
connected text with reasonable accuracy and understanding, reading fluency (timed reading with
meaning) should be monitored. Continued growth in vocabulary should be assessed, as well as
reading comprehension. Id.
Grade 2 (Vocabulary, Comprehension, Spelling, Connected Text Fluency, Oral Reading
Fluency):
Word reading continues to be a strong predictor of Grade 2 outcomes, with reading
fluency and comprehension becoming increasing important predictors. Second graders need
continued monitoring of their decoding abilities, because struggling students may still have
difficulty in this area. Reading fluency is critical through second grade, since students must make
strong growth in this skill to maintain grade level reading proficiency. The development of
reading comprehension is dependent on fluency and vocabulary. Sight vocabulary must grow
rapidly in second grade to keep pace with expected reading outcomes. Thus, continued growth in
Spelling, Vocabulary and Reading Comprehension should be measured. Id.
Grade 3 (Vocabulary, Comprehension, Spelling, Connected Text Fluency, Oral Reading
Fluency):
Reading fluency and comprehension are strong predictors of Grade 3 outcomes. The
primary dimensions of reading growth that should be measured in third grade are reading
fluency, reading comprehension, spelling and vocabulary. Id.
f. Istation is Effective at Measuring a Student’s Progress
ISIP ER measures are designed to be used for Universal Screening (benchmarks three
times a year) and Progress Monitoring (monthly). The benchmarks and progress monitoring
26

 assessments are designed to test the same skills in the same format and with the same consistent
and comparable results. While the questions will change for the individual student, the subtest,
the format, and overall score composition will remain consistent between progress monitoring
and benchmarks to ensure consistency in instructional design and instructional decisions between
benchmark months.
As stated in the ISIP ER Technical Manual (c 2011; 2016), “With recent advances in
Computer Adaptive Testing (“CAT”) and computer technology, it is now possible to create
assessments that adjust to the actual ability of each child. Thus, CAT replaces the need to create
parallel forms. Assessments built on CAT are sometimes referred to as “tailored tests” because
the computer selects items for students based on their performance, thus tailoring the assessment
to match the performance abilities of the students. This also means that students who are
achieving significantly above or below grade expectations can be assessed to more accurately
reflect their true abilities.”
Furthermore, the Technical Manual (c 2011; 2016) indicates, “There are many
advantages of using a CAT model rather than a more traditional parallel forms model, as is used
in DIBELS. First, it is virtually impossible to create alternate forms of any assessment that are
truly parallel. Thus, reliability from form to form will always be somewhat compromised.
However, when using a CAT model, it is not necessary that each assessment be of identical
difficulty to the previous and future assessments. Following a CAT model, each item within the
testing battery is assessed to determine how well it discriminates ability among students and how
difficult it actually is through a process called Item Response Theory (IRT) work. Once item
parameters have been determined, the CAT algorithm can be programmed. The use of CAT
algorithms also creates efficiencies in test administration. The adaptive item algorithm allows the
computer to adjust item difficulty while the child is taking the test, quickly zeroing in on ability
level. Thus, the use of CAT algorithms reduces the amount of time necessary to accurately
determine student ability.”
The entire assessment battery for any assessment period requires 40 minutes or less. It is
feasible to administer ISIP Early Reading assessments to an entire classroom, an entire school,
and even an entire district in a single day, given adequate computer resources. Classroom and
individual student results are immediately available to teachers, illustrating each student’s past
and present performance and skill growth. Teachers are alerted when a particular student is not
making adequate progress so that the instructional program can be modified before a pattern of
failure becomes established.
With the advanced computer technology, data are provided in real time to help teachers
determine which students are at risk for reading difficulties and those who have missed critical
foundational skills that are typically not assessed at later grade levels. For each student, the
computer adapts to lower or to higher level skills and produces a reliable estimate of their ability.

27

 g. Istation Provides Opportunities for Students to Read Aloud to and Interact with
their Teachers
Students can read to and interact with teachers with Istation in place. The teacher can and
very much should be highly involved in the reading and learning of a child. Teachers are
encouraged to sit with the students as they read aloud and participate in learning activities. For
example, in Istation’s Oral Reading Fluency subtest, the teacher or reading specialist is asked to
listen to the child read aloud and identify key reading behaviors and skills.
With Istation, students absolutely can produce the sounds and read the words out loud to
teachers. The teacher will have the flexibility to monitor students based on individual needs as
they apply their knowledge to complete tasks to demonstrate proficiency on the critical skills in
reading.
The purpose of K-3 formative assessments is to provide data pertaining to a student’s
performance level, not to substitute for teacher interaction. Teacher interaction is critical during
instructional time, as research consistently demonstrates evidence-based instruction informed by
meaningful data produces changes in student outcomes. Teachers who monitor their student’s
progress and use the data to inform instruction and decision making have higher student
achievement. In fact, results from a study conducted by SERVE (2015) indicated teachers
reported using many different measures to assess students prior to the introduction of mCLASS:
Reading 3D and 96% of teachers indicated that they used formative assessment to inform
instruction before mCLASS: Reading 3D.
Since a CAT like the ISIP-ER is efficient, and items are individually selected each time a
student takes the test, it can be given frequently enough to produce reliable estimates of student
growth from month to month. This is, of course, very important information for the teacher, who
can adjust teaching strategies, or teaching intensity, based on whether or not each individual
student is growing in the critical reading skills assessed by the ISIP-ER. Valid, timely, and
reliable assessment data gathered in an effective and efficient way provides more time for the
teacher to interact with students as he or she engages in effective small group, targeted reading
instruction. Gains in instructional time allow teachers to use small group instructional sessions to
determine where individual children fall on the continuum of reading skills and provide
systematic, explicit instruction to promote reading proficiency.
Flex assessments in mCLASS are also computer-based and gather data through a device.
Students are asked to read and stop to select a correct answer. Teachers have reported an
appreciation for the option to use Flex due to the reduced time spent assessing their whole class.
Istation takes this flexibility to another level in a way that still allows for audible phonemics to
be sounded out by students, while saving more instructional time for both students and teachers.
ISIP-ER can provide an accurate and efficient assessment of student ability because it
uses advanced computer technology to select items that are “just right,” in terms of their level of
difficulty, for each individual student. Thus, students do not waste time responding to test items
that are “way too hard” or “way too easy.” The computer can also estimate when the student has
28

 responded to enough items of the appropriate level of difficulty in order to produce a reliable
estimate of their ability. As the Putnam (2016) study demonstrates, with the relationship between
technology and literacy being at the forefront of many educational decisions, “the teacher’s role
in the classroom is strengthened by the introduction of new technologies,” and “technology
supplements, not supplants the teacher in the classroom.”
h. North Carolina is Already Engaged in a Successful Implementation of Istation
The Department and Istation have developed a statewide implementation schedule to
have initial training for all educators complete before the start of school. With the ease of
administering Istation in combination with the level of support offered in online modules, faceto-face training, webinars, and on demand links, successful implementation of Istation is well
underway. The training and implementation plan include the following steps:
•
•
•
•
•
•

Begin using Istation at the start of the 2019-2020 school year (with delay in metrics
described below)
Gather data during the fall to start becoming familiar with the assessments
Start using data to inform instruction
Delay the use of data to measure growth for EVAAS until MOY benchmark;
Use first official benchmark in the winter (MOY) and end of year benchmark (EOY) for
EVAAS purposes
Train all teachers by start of school with a schedule including:
− In-person workshops (face-to-face regional-based half day train the trainer model for
one campus leader per school and district leader/RtA Contact)
− On-demand webinars (monthly technical and educator live webinar in Q&A
environment recorded and available for on demand viewing)
− Learning modules (monthly timely topics in released online modules/podcasts for
educators to view brief course, complete quiz, get certificate of completion)
− Technical assistance
− Ongoing support from the K-3 Literacy team

29

 Below is the “Getting Started with Istation Roll Out Plan,” which has the support of local
superintendents and the State Board.
Summer 2019 (June, July, August)
• Istation enrollment and deployment activated in schools
• Live webinars hosted
• In-person regional trainings hosted
• On-going implementation support
• Districts can begin using Istation as early as July, but it will not count in metrics
Fall 2019 (September, October, November)
• Students take Istation’s ISIP assessment to begin to learn about the program
• Additional in-person regional trainings hosted
• Additional live webinars hosted
• Fall is a “Getting Started” learning opportunity. Data will not feed into EVAAS
Winter 2019-2020 (December, January, February)
• Ongoing progress monitoring continues
• Ongoing training continues
• January: The first benchmark window for 2019-2020 opens for EVAAS purposes
(MOY)*
Spring 2020 (March, April, May)
• Ongoing progress monitoring continues
• Ongoing training continues

30

 •

May: End-of-year benchmark window for 2019-2020 opens for EVAAS purposes
(EOY)*

(*Traditional School Calendar)

As of July 25, 2019, over 2,735 North Carolina educators and districts leaders have
engaged in webinars and in-person trainings. Educators are already modeling ISIP measures with
students in year-round schools.
• 989 total attendees in tech webinars
• 1,241 total attendees in live educational webinars
• 505 total attendees in person face to face workshops
On a survey following training, participants were asked to, “Rate how helpful you feel
Istation’s ISIP assessment will be for you and your campus on a scale of 1-5 stars, with a 1 being
not very helpful, and a 5 being very helpful.”

To date, results indicate:
32% = 5 stars, 38% = 4 stars, 23% = 3 stars, 6% = 2 stars, 2% = 1 star
Additionally, technical integration of Istation and the Department has been a smooth
process. Integration with PowerSchool has been seamless. Rostering occurs automatically with
nightly updates so that any changes are applied instantaneously. Integration with NCEdCloud
has been successful, and we already have teachers logging in successfully to Istation via
NCEdCloud.
• 114 districts/charters signed up through registration
• 450 campuses rostered
• 105,000 students rostered
• 629 teacher logins
i. Istation Satisfied the Requirements of the Procurement
As discussed in this response, Istation satisfied all the requirements of the procurement.
j. Istation’s ISIP-ORF was not Considered in the Contract Award
This feature described by Amplify was not a factor in the contract award.

31

 Attachment E
Amplify
In your protest letter, you state, “The agency must also consider factors including
whether the vendor complies with industry standards, the vendor’s past performance with the
state, and the probability of the vendor providing the required service on time. 09 NCAC
06B.0302(1)(f)(iii).” In responding to your letter with these discussions of Amplify’s tool,
mCLASS, and Istation, I would like to take this opportunity to share concerns with Amplify’s
product.
a. Teachers’ Concerns with mCLASS
A study was conducted by the SERVE Center at UNC Greensboro who contracted with
the Department to report on initial Read to Achieve implementation. One major finding for the
2014 Year One Report was, “implementation challenges identified by respondents that need
attention, mostly related to administering benchmarking and progress monitoring assessments
and a perception that reading assessments are taking too much time from instruction.”
Given the results of 2014 report, the Department again contracted with the SERVE
Center to conduct the current (2015) study to further examine teacher perceptions and
assessment processes of mCLASS: Reading 3D in classrooms due to the questions from
educators and the State Board as to how long the assessments in mCLASS: Reading 3D were
taking, whether students lost too much instructional time or time spent on other educational
activities, and whether the results of the formative assessments were being used to modify and
tailor individual instruction.
Researchers analyzed data from teacher logs, observations, and interviews of 100
teachers from 37 randomly selected schools across the state and 6 schools that the Department
recommended as exemplars, as well as results from 749 teachers’ responses to a survey in order
to gain an understanding of the assessment process. Results indicated an overwhelming concern
with the time involved with mCLASS: Reading 3D that is taken from providing quality
instruction to students that improves reading skills. Nearly half (48%) of teachers indicated
concern about what students were doing while others were being assessed and reported they felt
their students were not receiving high quality instruction. While teachers reported they like the
data, they did not feel it was worth the time spent assessing and away from instruction.
All K-3 teachers who were surveyed or interviewed indicated concern for time spent
doing benchmark assessment and progress monitoring. More specifically, K-2 teachers who were
interviewed stated they spent 24 hours on average assessing during benchmark periods, while K2 teachers who were surveyed indicated 26 hours. Third grade teachers’ survey results indicated
28 hours. Among the teachers surveyed, 74% indicated progress monitoring takes up too much
instructional time. Teachers were concerned with the frequency required to assess students at
“yellow” level (every 20 days) and “red” level (every 10 days) because the frequency did not
allow sufficient time to teach deficit skills to make improvement between monitoring.
Participants reported they would prefer to spend the time currently devoted to mCLASS:
32

 Reading 3D assessments for guided reading and/or small group instruction (42%) and more or
higher quality instruction (38%).
Overall, 50% of third grade teachers surveyed were “slightly” or “not at all satisfied”
with the overall utility of Amplify in providing data to identify and intervene with students
experiencing reading difficulty. Teachers reported insufficient time to use benchmarking and
progress monitoring data, 52% and 46%, respectively. While 72% agreed or strongly agree that it
is a meaningful use of time to analyze benchmarking data, 89% of teachers surveyed thought that
the Amplify assessment instrument took too much time away from instruction and 64% thought
the information gained through Amplify assessment was not worth the loss in instructional time.
Anderson, J., Hutchins, B., Howse, R., Amwake, L., Dufford-Melendez, K., & Wynn, L., et al
(2015). Read to Achieve Evaluation Year Two: An Evaluation of the Implementation of Read to
Achieve Assessment and Reding Camp. SERVE Center at the University of North Carolina at
Greensboro.
The complete (2014) Year One Report can be accessed at:
http://www.serve.org/uploads/files/RtA%20_report_SERVE_9.5.14.pdf.
b. Amplify’s Accuracy on Universal Screening
You note in your letter that a core component of the Read to Achieve program is
universal screening. Recent research conducted by REL Southeast (2019) examined longitudinal
data from mCLASS: Reading 3D to predict outcomes on grade 3 EOG. In the study, data were
analyzed using classification and regression tree (CART) analysis to identify decision rules for
identifying students in grades K-3 as at risk or not at risk for reading problems at the end of
grade 3. The following administration points of the mCLASS 3D reading were used as predictors
of grade 3 EOG: kindergarten middle of the year (MOY), grade 1 beginning of the year (BOY)
and MOY, grade 2 BOY, and grade 3 BOY.
The study used a longitudinal statewide sample of 91,855 students entering kindergarten
in the 2014-2015 school year and completing grade 3 in 2017-2018. Results revealed a large
number of students were misidentified as likely to be non-proficient in grade 3 in all of the
models, with the exception of a model in grade 3. While prediction models beginning with grade
1 MOY were acceptable for identifying students who are likely to be proficient by the end of
grade 3, more information was needed in order to accurately identify students who are at risk of
not being proficient at the end of grade 3. Less than 80 percent of students were accurately
identified as at risk of poor performance (that is, the sensitivity rate was found to be below a
recommended minimum standard of .80). Grade 3 BOY with North Carolina’s grade 3
beginning-of-grade assessment as an additional predictor was the only model that correctly
classified over 80 percent of students as at risk or not at risk for reading problems at the end of
grade 3 (that is, both the sensitivity and specificity rates, respectively, were found to be greater
than .80). Findings of the study are limited to the cohort and assessments included in the analysis
and may differ with additional cohorts and/or assessments. However, the study findings point to
the need to supplement the mCLASS student scores with informal, curriculum-based
assessments when setting instructional objectives and differentiating classroom instruction. The
REL Southeast is in the process of preparing a full report with the study findings.
33

 c. Amplify is Only One of Many Opportunities for Teacher-Student Engagement
You claim in your letter that Amplify’s “direct engagement with students’ thinking
during the assessment also provides more reliable results.” But teachers should be listening to
their students reading daily, regardless of the benchmark assessment used three times yearly.
While different, both Istation and Amplify offer opportunities for teachers to monitor progress,
determine areas of need, and work closely with students.
Additionally, with implementation of Amplify’s mCLASS, teachers have not been
allowed to assess their own students for MOY or EOY due to fidelity issues. Therefore, the
student is assessed by another teacher with whom the student may not have rapport, which can
certainly impact the reliability of results. On the other hand, with Istation, students can be
assessed by their own teachers in their own classrooms.
d. Decreasing Reading Scores in North Carolina
In your letter, you also state that “mCLASS’s benefits have not only been seen in North
Carolina schools, but nationwide. Amplify has partnered with schools across the country and has
a demonstrated history of successful implementations.” While we do not dispute that Amplify’s
mCLASS has some strengths, reading scores have gone down in North Carolina since Amplify
was fully implemented across the state. (See Exhibit H).
e. Amplify is Open to Inappropriate Actions by Teachers
Despite years of training on the assessment, school districts continue to report issues with
inappropriate actions (such as sharing secure test materials) by teachers using mCLASS,
resulting in testing irregularities with mCLASS. In 2018-19 alone, 23 districts reached out to the
Department for guidance in handling these issues. While formal investigation results are still
coming in, at least 14 testing irregularities have been formally reported to the Department’s
accountability division for 2018-19, with retesting being the usual remedy.
f. Amplify Technical Issues Present Challenges for North Carolina Educators
Educators across the state, tech directors at school districts, the Department’s IT division,
and K-3 Literacy personnel experienced significant challenges in 2018 due to issues with
Amplify incorrectly assigning students to class rosters. Amplify failed to timely address the
situation, making matters worse. There were 250 mCLASS related issues created in our DPI-RtA
Jira ticketing system and October of 2018 during BOY (Note: In September and October, the
issues were imported into Jira from the classing Department spreadsheet in Sharepoint, so a
resolution is not reported, but data are included). Because of the magnitude of mCLASS related
issues in October, the Department had to hire a temporary technical specialist to manage and
remedy the volume of Amplify issues.

34

 Below is a breakdown of the time Amplify took to resolve issues reported each month
from September 2018 to July 2019, with tables for both normal priority and high/critical priority
issues.
On average, it took 21 days for Amplify to resolve normal priority issues, ranging from
0-46 days for resolution. However, a requirement from the existing contract with Amplify
(BAFO 3 Contract) for resolving a normal priority issue states the following: “The Contractor
will provide a preliminary status update and, to the extent practical, preliminary remedy to the
State not later than the next business day after the problem is initially logged with technical
support.” (See Exhibit I, pg. 20).

Furthermore, the average resolution time for Amplify for high priority/critical issues was
9 days, ranging from 0-43 days for resolution. Please note the requirement from the contract with
Amplify in effect in 2018-19 (BAFO 3 Contract) for resolving a high to critical priority issue
states the following: “The Contractor will provide a preliminary update and, to the extent
practical, a preliminary remedy to the State within one (1) hour after the High Priority problem is
initially logged with technical support.” (See Exhibit I, pg. 20).

35

 Amplify’s past performance with the State and untimely provision of services leave much
to be desired. Amplify’s unreliable reading diagnostic tool threatens to delay the identification of
poorly performing students, which ultimately delays providing these students with the additional
help they need.
g. No Viable Reading Camp Solution
An important component of the Read to Achieve legislation is the opportunity for
students to attend summer reading camps to improve their skills. Amplify does not provide a
viable solution for these reading camps because it only has an offering for BOY, MOY, and
EOY periods in a given school year. Amplify’s common practice since the state of North
Carolina began using mCLASS has been to roll the entire state into the next school year and then
invalidate and delete all TRC data following the conclusion of a reading camp and prior to the
start of BOY.
Yes, delete data. This process was challenged this year when it was discovered and
Amplify justified deleting data because they had always done it this way. It became a major issue
for the 2018-2019 school year and beyond due to new requirements to report 1st and 2nd grade
data, which was not required in previous years. In previous years, 3rd grade data were reported,
then invalidated and deleted once PowerSchool rostering data file was reactivated at the
beginning of the year. This was not possible for 1st and 2nd grade, however, because they would
continue to use mCLASS in subsequent years. When pressed to provide an alternative solution
for reading camps that provides an “isolated environment,” it was discovered that Amplify
“shares their environment” with all clients making it difficult for Amplify to create an isolated
instance to meet the needs for collecting and reporting data for reading camps. It should be noted
36

 that Istation offered a solution within 48 hours of the request that will also flag the 3/4 transition
students for continued monitoring and never delete data.
h. Lexile Level Issues
Amplify identifies texts of their Atlas Kits with Lexile levels. Based on the levels
provided by Amplify, TRC level Q was identified as the text level for third grade students to
demonstrate grade level proficiency. Based on the Lexile level of 770, level Q most closely
aligned with NC End of Grade Lexile of 725. Department and local district staff have raised
concerns about inaccuracies in the levels of the books provided by Amplify. This “mis-leveling”
means that students’ reading proficiency was tested using books that were too hard, or too easy,
for them. As a result, teachers and administrators are getting inaccurate information on student
reading abilities and making instructional and promotional decisions for students based on that
information. Examples of concerns include:
•

•

•

Nearly half of Amplify test books do not have a Lexile Level identified on the common
search engine Lexile.com. MetaMetrics (Lexile.com headquarters) is well known among
educators as identifying Lexile Levels of texts.
A book that Amplify identified as Level Q (Lexile 770) is listed as Level M (Lexile 570)
on Lexile.com, potentially allowing students to pass third grade without reaching a high
enough Lexile level to read at grade level.
A book that Amplify identified as Level C was actually Level E. This may have affected
kindergartners who are supposed to reach at least a Level C to be considered proficient.

On Lexile.com, the Lexiles are very different within the same band of letters in the Atlas
TRC Inventory. Many of the texts, within the Atlas Kits, cannot be found using a common
search engine, and the majority of the identifiable books are mis-leveled. As of April 16, 2019,
32 of the 74 books from levels A-Z in the Amplify Atlas Kits had no measured listing at
Lexile.com.

37