dsib-adad-may14item03 Attachment 1 Page 1 of 4 California Assessment of Student Performance and Progress, Contract # CN150012—Concordance Table The following summarizes the changes to the proposed scope of work (SOW) for the California Assessment of Student Performance and Progress (CAASPP) Contract # CN150012 as agreed to by all parties during contract negotiations conducted between staff from Educational Testing Service (ETS), the California Department of Education (CDE), the State Board of Education (SBE), and the Department of Finance (DOF). Per the SBE motion, the proposed budget does not exceed the amount that is estimated to be included in the annual Budget Act for this contract; nor exceed ETS’s published cost submission. ETS did not make any changes to the previously proposed cost submission. TASK 1: Comprehensive Plan & Schedule of Deliverables As a result of contract negotiations, ETS revised the proposed SOW and incorporated the following change:  Substituted American Institutes for Research (AIR) for all work that the SOW identified as being completed by Computerized Assessments and Learning (CA&L). TASK 2: Program Support Services ETS agreed to revise the proposed SOW and incorporate the following changes:  Added 51 additional workshops to be completed during the initial contract period (July 2015 through December 2018.) Regional workshops and trainings were increased from a total of five to no fewer than eight site locations (2 north, 2 central, and 4 south).  Increased to 45 total (increase of 30) focus groups over the initial contract period (July 2015 through December 2018.) TASK 3: Technology Services ETS agreed to revise the proposed SOW and incorporate the following changes:  Supported a single sign-on solution in coordination with Smarter Balanced that allows California users to log onto the CAASPP assessment delivery system and the Smarter Balanced systems, including the Digital Library, with the same logon credentials. 4/24/2015 11:18 AM dsib-adad-may14item03 Attachment 1 Page 2 of 4  In addition to the required annual submission, ETS will also provide results from the Smarter Balanced Implementation Readiness Package to the CDE upon request by the CDE. TASK 4: Test Security ETS agreed to revise the proposed SOW and incorporate the following changes:  In addition to the required 100 security audits ETS will conduct up to an additional 30 test security site visit audits annually, as directed by the CDE. TASK 5: Accessibility and Accommodations ETS agreed to revise the proposed SOW and incorporate the following changes:  Provide the same translations for the new California Next Generation Science Standards (CA NGSS) assessments as provided by the Smarter Balanced summative assessments.  Provide stacked translations or full Spanish versions for the CA NGSS assessments. TASK 6: Assessment Development ETS agreed to revise the proposed SOW and incorporate the following changes:  Add clarifying language that the SBE must approve all test development plans prior to implementation.  ETS will work closely with CDE to develop CA NGSS-aligned assessments that best meet the state’s vision. Other sources of appropriate items may be considered to create the necessary pool for test development and other assessment and/or instructional resources.  Include support of the same languages supported by the Smarter Balanced summative assessments for the CA NGSS and CA NGSS Alternate Assessments. TASK 7: Test Administration ETS agreed to revise the proposed SOW and incorporate the following changes:  Provide additional detail in Table 11a. regarding the secure browser support policy for new operating systems. 4/24/2015 11:18 AM dsib-adad-may14item03 Attachment 1 Page 3 of 4 TASK 8: Scoring and Analysis ETS agreed to revise the proposed SOW and incorporate the following changes:  Expand efforts to maximize the involvement of California educators in hand scoring student responses to CAASPP items and increase professional development opportunities. ETS will involve teachers in five types of large-scale scoring activities: 1) Interim Assessment Scoring Workshops, 2) Summer Scoring Institutes, 3) Range-Finding Meetings, 4) Constructed Response Scoring Modules, and 5) Live Operational Scoring o 1) Provide Interim Assessment Scoring Workshops to train teachers in the effective and consistent use of scoring rubrics and materials so that they may accurately score their own students’ interim assessments. o 2) Conduct Summer Scoring Institutes that will provide training on the scoring of released operational items from both the summative and interim administrations from the previous year. o 3) Conduct Rangefinding Meetings for the new CA NGSS and Primary Language assessments to be developed that will take place after the pilot and field tests will provide input into score ranges for each item, scoring rationales, and identify anchor sets with exemplar responses. o 4) Provide Constructed Response Scoring Modules by working with the Beginning Teacher Support and Assessment (BTSA) Induction to engage preliminary credentialed teachers in a job-embedded formative assessment system of support and professional growth by providing opportunities to include modules on constructed response scoring in these locally implemented programs with the goal of improving classroom practice. o 5) Conduct Live Operational Scoring to provide current California teachers, to the maximum extent possible, the option to engage in operational scoring of CAASPP English Language Arts and Mathematics student responses. In conjunction, ETS will:  Conduct Weekend Scoring Institutes for California teachers who have applied and have been accepted for the operational pool. ETS will hold scoring institutes on alternating weekends in March, April, and May for the purposes of training and certifying California teachers in a face-to-face setting. Teachers that qualify during these weekend sessions will score for the remainder of the 4/24/2015 11:18 AM dsib-adad-may14item03 Attachment 1 Page 4 of 4  weekend and will be certified to score via a distributed model at the end of the institute session. California teachers will be eligible for a higher hourly rate for scoring in the program  ETS will use hand scoring for items that elicit more complex and elaborate student responses.  ETS will not expand the current levels of artificial intelligence (AI) scoring unless they meet specific technical criteria and the expectations of the CDE.  AI scoring will target only those constructed response items that are designed to elicit a specific correct answer from students. TASK 9: Reporting ETS agreed to revise the proposed SOW and incorporate the following changes:  Provide the online student-level test results to LEAs within two to three weeks after the student completes a content area.  Provide teachers with direct access to the online summative test results reporting system.  Provides an opt-in option for LEAs to receive electronic version of the ISR within four weeks after the student completes testing.  Deliver the paper individual student report (ISR) one week earlier (within five weeks) after the student completes testing.  Design and produce an ISR that includes dynamic text on the front and back.  Provide the option for an ISR to be reproduced in Spanish if an LEA marks in TOMS that the student comes from a Spanish-speaking home.  Provide interpretation guides for each ISR format in up to five (5) languages (including Spanish) other than English. 4/24/2015 11:18 AM