Consortium 2017 State-by-State Comparisons D. J. McRae, Ph.D. As of 09/27/17 When two consortiums of states were chosen by the US Department of Education in 2010 to develop new statewide assessment systems, one of the purposes was to generate state-by-state comparable achievement data. Roughly 45 states initially signed up for potential use of consortium tests, but by 2015 when the new tests were ready for their first “operational” use, only 18 states administered the Smarter Balanced tests and 11 states (plus the District of Columbia) administered the PARCC tests, representing just under 50 percent of total K-12 enrollments across the country. In 2017, 14 states administered Smarter Balanced tests, and 6 states (plus DC) administered PARCC tests, representing just over 30 percent of K-12 enrollments across the country. In addition, Louisiana and Massachusetts (PARCC) and Michigan (Smarter Balanced) used public domain test questions for their statewide tests but are not using the full consortium protocols and hence are not included in this set of state-by-state comparisons. Plans to use consortium tests in 2018 are noted on the data charts below. The data charts on pages 2 and 5 provide state-by-state results for the Smarter Balanced and PARCC states, respectively, for spring 2017 testing. The results are expressed as “percents meeting target” grade-by-grade for English Language Arts and Mathematics, along with average percents across grades. On pages 3-4 and 6, the average gain scores for each state for 2015 thru 2017 are provided, respectively, for Smarter Balanced and PARCC states (plus DC). Notes describing the data in the charts are provided at the bottom of pages 3-4 and 6, respectively, for SBAC and PARCC states. Doug McRae is a retired educational measurement specialist living in Monterey, California. In his more than 45 years in the K-12 testing field, he has served as an educational testing company executive in charge of the design and development of K-12 tests widely used across the country, as well as an advisor for the design and development of California’s STAR system which was used from 1998 through 2013. He has a Ph.D. in Quantitative Psychology from the L. L. Thurstone Psychometric Laboratory at the University of North Carolina, Chapel Hill. 1 Smarter Balanced 2017 State-by-State Comparisons [Level 3 & Above Percents] Compiled by D. J. McRae, Ph.D. [As of 09/27/17] English/Language Arts Grade 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3 4 5 6 7 8 Ave California Connecticut Delaware Hawaii Idaho Montana Nevada New Hampshire North Dakota Oregon South Dakota Vermont Washington West Virginia 44 52 52 48 47 49 45 54 NA 45 48 49 53 45 45 54 54 48 48 50 46 56 NA 48 49 49 55 47 47 56 60 53 55 54 50 61 NA 53 51 55 59 49 47 54 52 49 51 49 43 57 NA 52 49 52 56 45 49 55 53 50 54 53 47 63 NA 56 53 55 60 48 49 54 52 47 52 48 46 58 NA 55 49 55 59 45 46.8 54.2 53.8 49.2 51.2 49.8 46.2 58.2 NA 51.5 49.8 52.5 57.0 46.5 Averages 49 50 54 50 54 51 51.3 Mathematics 1 2 3 4 5 6 7 8 9 10 11 12 13 14 California Connecticut Delaware Hawaii Idaho Montana Nevada New Hampshire North Dakota Oregon South Dakota Vermont Washington West Virginia 46 53 53 53 50 48 48 55 NA 46 54 52 58 48 40 50 50 48 47 45 41 51 NA 43 50 47 54 43 34 43 44 42 42 40 34 46 NA 39 42 42 49 34 36 44 41 41 40 38 30 47 NA 40 42 39 48 32 37 43 41 37 42 40 29 50 NA 42 45 44 50 31 36 42 38 38 39 36 18 45 NA 41 42 41 48 29 38.2 45.8 44.5 43.2 43.3 41.2 33.3 49.0 NA 41.8 45.8 44.2 51.2 36.2 Averages 51 47 41 40 41 38 42.8 2 Smarter Balanced 2015-17 Gain Scores English/Language Arts 2015 2016 2017 State ELA ELA ELA Gain Gain 1 California 42.3 46.7 46.8 +4.5 +0.1 xx* 55.8 54.2 xx* -1.6 51.7 54.8 53.8 +2.1 -1.0 4 Hawaii 47.7 50.5 49.2 +1.5 -1.3 5 Idaho 49.7 51.8 51.2 +1.5 -0.6 6 Montana xx* 50.0 49.8 xx* -0.2 7 Nevada xx* 48.3 46.2 xx* -2.1 8 New Hampshire 58.7 59.8 58.2 -0.5 -1.6 9 North Dakota xx* 50.2 NA* xx* xx* 10 Oregon 53.8 53.3 51.5 -2.3 -1.8 11 South Dakota 47.5 51.2 49.8 +2.3 -1.4 12 Vermont 53.7 56.5 52.5 -1.2 -4.0 13 Washington 55.5 57.8 57.0 +1.5 -0.8 14 West Virginia 45.5 48.0 46.5 +1.0 -1.5 2 Connecticut 3 Delaware 2015-17 2016-17 Notes for Smarter Balanced Data: All averages and gains are based on Grade 3-8 data only. Less than half Smarter Balanced states uniformly administer Smarter Balanced tests for High Schools, and hence HS results are not included. *Montana, Nevada, and North Dakota participated in Smarter Balanced testing in 2015, but all three states experienced technology difficulties that prevented generation of representative scores for the entire state. These circumstances prevent calculation of selected gain scores. Connecticut indicated they discontinued the Performance Task portion of ELA tests in 2016, so for comparability reasons the 2015 Connecticut average is not recorded. North Dakota’s numerical results for 2017 are not available yet. New Hampshire, North Dakota and West Virginia have indicated plans to discontinue use of Smarter Balanced tests spring 2018. 3 Mathematics 2015 State 2016 2017 2015-17 2016-17 Math Math Math Gain Gain 1 California 34.2 37.3 38.2 +0.9 2 Connecticut 40.3 44.2 45.8 +5.5 +1.6 3 Delaware 40.7 43.7 44.5 +3.8 +0.8 4 Hawaii 42.2 43.0 43.2 +1.0 +0.2 5 Idaho 40.8 43.3 43.3 +2.5 0.0 6 Montana xx* 41.0 41.2 xx* +0.2 7 Nevada xx* 33.8 33.3 xx* -0.5 8 New Hampshire 47.7 50.3 49.0 +1.3 -1.3 9 North Dakota xx* 41.7 NA* xx* xx* 10 Oregon 43.5 42.8 41.8 -1.7 11 South Dakota 41.2 44.5 45.8 +4.6 +1.3 12 Vermont 43.2 46.7 44.2 1.0 -2.5 13 Washington 49.8 51.5 51.2 +1.4 14 West Virginia 30.8 34.8 36.2 +5.4 +1.4 +4.0 -1.0 -0.3 Notes for Smarter Balanced Data: All averages and gains are based on Grade 3-8 data only. Less than half Smarter Balanced states uniformly administer Smarter Balanced tests for High Schools, and hence HS results are not included. *Montana, Nevada, and North Dakota participated in Smarter Balanced testing in 2015, but all three states experienced technology difficulties that prevented generation of representative scores for the entire state. These circumstances prevent calculation of selected gain scores. North Dakota’s numerical results for 2017 are not available yet. New Hampshire, North Dakota and West Virginia have indicated plans to discontinue use of Smarter Balanced tests for spring 2018. 4 PARCC 2017 State-by-State Comparisons [Level 4 & Above Percents] Compiled by D. J. McRae, Ph.D. [As of 09/27/17] English Language Arts 1. 2. 3. 4. 5. 6. 7. Grade 3 4 5 6 7 8 Ave Colorado Dist Columbia Illinois Maryland New Jersey New Mexico Rhode Island 40 29 36 40 50 26 40 44 34 37 42 56 26 41 46 35 37 41 59 29 42 41 33 35 38 53 25 38 44 34 40 43 59 26 40 43 30 37 39 59 28 37 43.0 32.5 37.0 40.5 56.0 26.7 39.7 Averages 37 40 41 38 41 39 39.3 Mathematics 1. 2. 3. 4. 5. 6. 7. Grade 3 4 5 6 7 8 Ave Colorado Dist Columbia Illinois Maryland New Jersey New Mexico Rhode Island 40 38 39 43 52 30 44 34 33 31 37 47 23 35 34 31 35 36 46 23 35 31 26 28 32 44 20 29 26 23 27 xx 40 16 27 xx xx 32 xx xx xx xx 33.0 30.2 31.2 37.0 45.8 22.4 34.0 Averages 41 34 34 30 27 xx 33.4 5 PARCC 2015-17 Gain Scores English/Language Arts 2015 2016 2017 2015-17 2016-17 State Ave Ave Ave Gain Gain Colorado Dist Columbia Illinois Maryland New Jersey New Mexico Rhode Island 40.3 26.2 36.0 38.8 50.0 23.2 36.8 40.5 26.5 36.5 38.7 52.8 24.5 39.8 43.0 32.5 37.0 40.5 56.0 26.7 39.7 +2.7 +6.3 +1.0 +1.7 +6.0 +3.5 +2.9 +2.5 +6.0 +0.5 +1.8 +3.2 +2.2 -0.1 1. 2. 3. 4. 5. 6. 7. Mathematics 1. 2. 3. 4. 5. 6. 7. Colorado Dist Columbia Illinois Maryland New Jersey New Mexico Rhode Island 31.2 25.3 28.8 31.8 41.0 21.4 28.2 32.6 33.0 +1.8 +0.4 29.5 30.2 +4.9 +0.7 31.3 31.2 +2.4 -0.1 37.3 37.0 +5.2 -0.3 45.6 45.8 +4.8 +0.2 23.0 22.4 +1.0 -0.6 34.0 34.0 +5.8 0.0 Notes for PARCC Data: Most PARCC states utilized PARCC End-of-Course Math tests for the High School level rather than PARCC grade level tests. Since course taking patterns differ from state-to-state, HS results are not included. All averages and gains reflect grades 3-8 only, and the xx under a grade means the state does not uniformly administer PARCC grade level tests to all students at that grade level. All xx’s reflect administrations of Algebra and/or Geometry End-of-Course tests to grade 7 and 8 students taking these courses, rather than the regular grade level tests. The pattern of xx’s in the 2017 chart are identical to the patterns for 2015 and 2016. Averages are based on grade levels with comparable results. Colorado and Rhode Island have indicated plans to discontinue use of PARCC tests for spring 2018. 6