State Assessment Update February 6, 2019 Eric Hall, Ed.D. Deputy State Superintendent of Innovation Tammy Howard, Ph.D. Director Accountability Services Initiatives to Reduce Testing INITIATIVE Reduce the length of required tests Reduce the stress at schools around testing time PROCESS Reduce the number of questions and review historical data on actual time students require to take the online tests Revising test administration guides: use of proctors a local decision, allow reviewing of testing strategies immediately prior to the test Reduce the number of locally required tests Working with local leaders to identify assessments that are useful and contribute to educational planning Pushing to eliminate tests not required by Washington, D.C. Continue to review testing requirements with State decisionmakers 2 Initiatives to Reduce Testing INITIATIVE PROCESS • Giving students other ways to show progress if they have a bad test day • Providing guidance to districts on how to consider other data beyond test scores and how to scale • Using the appropriate amount of technology as a tool for students and teachers to personalize learning and eliminate tests • Pursuing innovative methods for testing such as embedding assessments in instruction 3 Background • In 2017, the State Board of Education adopted new content standards for mathematics and English language arts/reading, requiring new assessments – Mathematics: 2018–19 School Year – English language arts/reading: 2019–20 school year 4 Designing New Assessments Prior to Field Testing Convene Test Specifications Panelists (NC teachers) for their recommendations Determine the percentage of items and the item type for each assessed standard After Field Testing Determine the number of test items for the entire test; aligns to the test specifications already set Determine if there needs to be discussion with leadership on any of the design features of the test 5 Designing New Assessments • Must consider technical quality and statutory requirements – Validity: Are the tests measuring the specified grade level content (content validity)? – Reliability: Are the tests constructed with consistency and stability in measuring what it is intended to measure? – Classification Consistency: Are the reported academic achievement levels reliable with minimum standard error? 6 Designing New Assessments U.S. Department of Education Peer Review: Do the assessments meet the requirements of federal law as stated in the Elementary and Secondary Education Act, as reauthorized and specified in the Every Student Succeeds Act? • Aligns to grade level content standards • Measures the breadth and depth of the content standards • Report student achievement by at least three academic achievement levels 7 State Board Policy Considerations • Academic Achievement Levels’ Scale Scores: Recommended for approval after standard setting occurs (August 2019) – Test-033 (End-of-Grade) – Test-036 (End-of-Course) – Test-037 (NCExtend1) 8 Factors in Determining the Number of Items • Adequately cover the assessed content standards • Reliably report student performance by Academic Achievement Levels – Determine number of levels (ESSA requires at least 3 levels) – More levels require more items to ensure reliability of student classifications 9 Proficienc Level Number of Items Classification LGVGIS Consistency Number of HTS Public Schools of North Carolina Designing New Assessments • For the new mathematics assessments, a critical question was whether to have five or four Academic Achievement Levels – The current assessments were originally set on four Academic Achievement Levels – It was not possible to reduce the number of test items and maintain five levels – It was possible to reduce the number of test items and maintain technical quality and statutory requirements 11 Implementation ASSESSMENT All Mathematics (EOGs and EOCs) All English Language Arts/Reading (EOGs and EOCs) All Science (EOGs and EOCs NEW TESTS DATA REPORTED BY FOUR LEVELS 2018–19 Reports to parents after SBE approves standard setting in August 2019 2019–20 Reports to parents after SBE approves standard setting in August 2020 2019–20 Reports to parents after SBE approved revised standards in November 2019 By 2019–20 all EOGs and EOCs will be reported based on four levels 12 Academic Achievement Levels Current Five Levels: SBE Policy TEST-033 LEVEL 5 LEVEL 4 LEVEL 3 LEVEL 2 LEVEL 1 Superior Solid Sufficient Partial Limited • Students at Level 4 and above are On-Track to be Career and College Ready at graduation – Level 5 is used to determine if a student must be offered an advanced math course (NC Math 1) • Students at Level 3 and above are Grade Level Proficient – Used to determine whether a student has met Read to Achieve 13 Academic Achievement Levels Recommended LEVEL 5 LEVEL 4 LEVEL 3 LEVEL 2 Advanced/Ontrack Career and College Ready On-track Career and College Ready Grade Level Proficient Basic Other Options for Names of Levels LEVEL 5 LEVEL 4 LEVEL 3 LEVEL 2 Accelerated CCR Advanced CCR Grade Level Proficient Limited Proficiency Distinguished Exceeds Meets Partially Meets 14 And…most tests will require less administration time NEW TESTING TIME TEST (Two Hours Estimated and an Additional Hour if Needed) Mathematics End-of-Grade (3–8), Science End-of-Grade (5 and 8) and Biology End-of-Course 2018–19 School Year English language arts/reading End-of-Grade (3–8) and possibly English II End-of-Course 2019–20 School Year NC Math 1 and NC Math 3 timing data does not support reducing the number of items at this time 15 Number of Items: Then and Now PREVIOUS NO. OF ITEMS CURRENT NO. OF ITEMS Mathematics End-of-Grade (3–4) 54 46 Mathematics End-of-Grade 5 54 48 Mathematics End-of-Grade (6–8) 60 53 Science End-of-Grade (5 and 8) 75 65 Biology End-of-Course 75 65 TEST 16 Federally-Required Tests • End-of-Grade (3–8) Mathematics and English language arts/reading • End-of-Grade (5 and 8) Science • End-of-Course: – NC Math 1 – NC Math 3 – English II – Biology • Language Proficiency Test for English Learners • Career and Technical Education Assessments (also SBE policy) 17 State-Required Tests State Law • K–2 Diagnostic Math Assessment • K–3 Diagnostic Reading Assessment • ACT and ACT WorkKeys State Board of Education Policy • NC Final Exams: – Grade 4, 6, and 7 Science and Social Studies – Grade 5 and 8 Social Studies – High School: Math, English, Science, and Social Studies where there is not an End-Of-Course test 18 North Carolina Final Exams • Background – Developed to meet Race to the Top requirement to provide a growth measure for each teacher ▪ NCFEs for core content (mathematics, science, English language arts/reading and social studies where there was not an end-of-grade or end-of-course assessment ▪ Analysis of Student Work for content areas not conducive to multiple choice format (music, art, health, etc.) – NCFEs and Analysis of Student Work continued through State Board policy after Race to the Top ended ▪ Analysis of Student Work was eliminated by the General Assembly effective with the 2016–17 school year ▪ NCFEs were not eliminated 19 North Carolina Final Exams • Background – Currently, there are no state or federal statutes that requires NCFEs; however, NCFEs’ growth data may contribute to how districts approach the following: ▪ § 115C-269.35: The State Board of Education’s evaluation of Educator Preparation Programs shall include “Proficiency and growth of students… when available, EVAAS data shall be used to measure student proficiency and growth.” ▪ § 115C-105.27: Requires that school improvement teams use student-level data (specifically EVAAS growth data) to inform their school improvement plans, conduct root cause analyses, and develop appropriate goals for improvement. ▪ Though Standard 6 in the teacher evaluation process was eliminated, NCFE growth data is used as an artifact for all standards in the evaluation process. 20 North Carolina Final Exams • Background – 94,809 teachers employed (March 2018) – 66,954 (70.6%) teachers with EVAAS growth for 2017-18 school year • 51,199 (54.0%) teachers with EVAAS growth through state and federal-mandated tests • 11,620 (12.3%) teachers with EVAAS growth through NCFEs Growth Through Federal or StateMandated Tests Growth Through NC Final Exams Growth Through CTE Exams • 4,135 (4.4%) teachers with EVAAS growth through CTE exams 21 Grade 4 Science and Social Studies Grade 5 Social Studies School Year 2017-18 TEACHERS NCFE Science Grade 4 With EVAAS With EVAAS Only from Elementary NCFEs 229 78 134 7 144 20 5,647 Social Studies Grade 4 Social Studies Grade 5 With EVAAS from Elementary EOGs & NCFEs 5,675 22 Grade 4 Science and Social Studies Grade 5 Social Studies School Year 2017-18 NCFE NUMBER OF STUDENTS TAKING ASSESSMENT Science Grade 4 11,497 Social Studies Grade 4 5,035 Social Studies Grade 5 6,655 23 Local Testing • As with the state assessments, it is useful to review local assessments to ensure the tests administered are not duplicative or not used to improve student achievement in ways intended – Cumberland County Schools recently conducted such a review and eliminated some assessments • Recommend engaging in collaborative conversations with districts and other public school units – Arkansas has had success with this work utilizing Achieve’s tool: The Student Assessment Inventory for School Districts 24 POLICIES PROCEDURES Changes to the Test Administration Guide Public Schools of North Carolina Release from Test Session • Allow locals to determine WHEN to remove students from the testing setting – Previous protocol allowed removing students at the end of the estimated time – Timing data show many students complete the test prior to the estimated time – Every situation is different; this change allows those closest to the test administration to make the decision of when to release students 26 Proctors • Allow locals to determine whether to have proctors for each test session – Previous protocol required proctors for each administration: one-to-one, whole class – Availability of proctors has continued to be very limited in some communities – Test security practices continue to be shared and many schools will utilize roving proctors, so an adherence to security is maintained 27 Testing Strategies • Allow teachers to remind students of test strategies immediately prior to the test – Previously, this was not permitted; testing began as soon as the students arrived for the school day – Having the flexibility to give some reminders may create a more relaxed test environment 28