Memorandum To: Mike Kirst, President, California State Board of Education From: Michael Hanson, President, CORE Districts Board of Directors Subject: Item 02: Establishing CORE Districts as a Research Pilot for the State Accountability System Date: July 7, 2016 __________________________________________________________________________________________ ________ On behalf of my colleagues and the CORE Districts, I would like to respond to item 02 on your upcoming agenda and to request the board consider officially establishing CORE Districts as a Research Pilot for the State Accountability System under Every Student Succeeds Act (ESSA). During the past six years, the nine school districts that collaborate through CORE and represent over a million students have helped identify specific local measurements we believe can help prepare more students for college and career readiness. Together we are empowering educators within and across districts to build the capacity of schools and support instruction in ways that help all students to learn with a specific emphasis on equity and honestly confronting issues of disproportionality that exist on all metrics. It’s led to frank conversations among educators about where we can do better because the measures we are using are supportive and matter locally. The CORE districts collaborate through a continuous improvement system that helps us gauge progress, determine where improvement is needed, and ensure student outcomes improve. Usable information is being provided back to schools and classrooms to help improve teaching and learning. This innovative system, the School Quality Improvement System (SQIS), is aligned with both LCAP and ESSA requirements and includes many measures that the State Board of Education is now considering and may consider in the future for inclusion in California’s emerging accountability system. All told, the participating CORE districts are utilizing shared data to better support the 1 million students we serve so that they can achieve and succeed in college and career. Given our extensive system and learnings about how our measures can be calculated and used for improvement, we respectfully request that the State Board of Education adopt CORE as a pilot for California’s accountability system under the state’s upcoming ESSA proposal. In this memo, we expand on this proposal, further explaining how CORE can offer the California State Board of Education and California Department of Education critical feedback for the evolving accountability rubrics and inform the efforts of working groups on culture/climate surveys, English Learners and college and career readiness) and why the research pilot would allow us to continue to perform our innovative work. For our districts, holding schools accountable for local data is key - but for the purpose of improvement not sanctions. We are building educational capacity by pairing schools that have shown success with certain student populations with similar schools that are struggling to meet their improvement goals while providing them with support and effective strategies. The system is also being used to form communities of practice – groups of people from different schools and districts collaborating to solve common issues – based on greatest needs and the similarity of problems of practice. This model is helping our districts identify effective assistance and intervention strategies. th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org Sharing our successes and failures and working together, we can promote innovation and bring new strategies to scale to improve student learning and achievement. Local measures are helping us identify weaknesses and address areas of need. For example, in addition to measuring four-year graduation rates, CORE Districts are measuring five- and sixyear cohort graduation rates. In doing so, the CORE Districts are placing high value on continuing to work with youth who may need additional time to complete high school graduation requirements, such as late-entering language learners who learn English along with their academic coursework. Within our districts, we have established model programs to reduce absenteeism. Educators across the CORE districts gather to share and learn from one another about effective strategies. Local data has helped identify and learn from exemplars of success. Many educators, researchers and education policy makers have come to believe that positive social and emotional skills are critical to student success. At CORE Districts, we agree, and are conducting an important pilot as the first in the nation to include the measurement of social and emotional factors in a system of school improvement and accountability. As a result, we have already contributed to the public discourse through new findings around deep correlations between SEL skills and academic outcomes as well as exciting findings around the ability to assess these skills at scale. While our findings are far from complete, we are in the process of significant learnings that can help guide the state and nation. What’s more, our districts represent nearly 300,000 students who aren’t fluent in English. Research shows that if they don’t become fluent within a certain time period, they won’t have access to college prep curriculum, will be two to three years behind in math and English language arts, and are likely to have a GPA that’s lower than 2.0. So we are holding ourselves mutually accountable and working collaboratively on timelier reclassification of English learners. We also spent a significant amount of time together designing an innovative high school readiness indicator which allows us to send a very clear signal to middle schools on ensuring they are matriculating students on track to graduate high school. Imbedded in this measure are shared definitions about 8th-grade English language arts and math courses that we consider core courses. We discovered that we couldn’t just use the state’s CALPADS course codes for this, however, because they are too broad. So our work with course-taking is compiled district by district. We further explore what sets the CORE system apart, and why it makes for an important pilot opportunity for California in the materials that follow. District-led development of CORE’s innovative accountability metrics CORE’s continuous improvement system embodies local control, directly reflecting the measures that our districts believe are important for helping students achieve and succeed and supporting local implementation. In fact, the SQIS originated from an analysis of participating districts’ data dashboards and strategic plans, and district experts and stakeholders led the development and refinement of the included metrics (this process is described in detail in the blog posts in Appendix A). Ultimately, the SQIS indicators are a collection of measures that educators believe make up the key measures for ensuring schools are best preparing their students for college and career readiness. And although CORE’s continuous improvement system was developed to meet the requirements of CORE’s waiver from the NCLB rules, the resulting system is fully aligned to current state and federal accountability rules. th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org The SQIS includes multiple measures of academic progress such as growth in student performance over time, high school readiness of 8th-graders, and multi-year cohort high school graduation rates. It also includes measures of a student’s social and emotional (SEL) skills, as well as of a school’s overall culture-climate. Additional indicators include chronic absenteeism and suspension and expulsion rates, how quickly English learners are reclassified as proficient in English, and other information gathered by conducting school cultureclimate surveys. Many of the metrics that are included in the SQIS rely on data that is not collected in a state data system, so CORE has developed an underlying data system (e.g., data warehouse, data dashboard), and a robust research partnership with PACE to support this continuous improvement system. Thus, we have a fully built model for moving from raw data to metric results and school ratings that facilitates goal setting and enables the identification of schools under the ESSA model. CORE’s metrics are impacting practice in participating school districts The nine collaborating districts received their CORE multi-metric school and district data dashboards, which include performance benchmarked against peers across California, for the first time in the 2015-16 school year. These data and reports are now being used by districts to learn and change practice. For example: Long Beach Unified School District used their SQIS report to drive their LCAP conversation with parents. Specifically, they shared the SQIS Report and LCAP template together with parents, and then asked parents to use the data in their SQIS report to drive their LCAP budget decisions. This new LCAP utilizes the district’s SQIS data to inform their plan to address the state’s priorities. San Francisco Unified School District redesigned their elementary report cards based on CORE’s socialemotional learning metrics and research findings. The CORE SQIS use of social-emotional learning caused the district to review how they graded behavior on their report cards compared to the SEL measures in the SQIS. What they found was that their current report cards were primarily focused on self-management as opposed to a continuum of skills that research says matters. As a result, their newly designed report cards provide information about all four areas of our social-emotional learning: growth mindset, self-efficacy, selfmanagement, and social awareness. Here in Fresno, we have designed a system that ensures all of our site leaders must first review and consider overall achievement and, critically, the gaps highlighted in their CORE School Quality Improvement report before designing their Single Plan for Student Achievement. In this way, we have leveraged the work of CORE and our shared network of mutual accountability to further the equity and access focus in our local district. These are just a couple examples of how CORE’s locally driven, multiple measures model has directly resulted in positive change at the district and classroom level. In addition to these examples of improving district LCAPs and reports, the SQIS data and research results are also being used to support school improvement through communities of practice and a school pairing program. Across the CORE districts, suspension rates have decreased and graduation rates have increased. CORE’s system is leading to changes in district practice, helping districts improve, and supporting teachers. CORE’s system as a pilot for the state under ESSA As described above, CORE’s SQIS was developed from the needs and knowledge of participating districts. It enables local innovation and implementation, is supported by a locally-driven, robust data system not available at the state, and is already leading to district improvement. The ability of the CORE districts to continue this important work will be undermined under the state’s proposed accountability system given the limited data the th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org state collects. Continuous improvement relies on coherence at all levels, and if the state’s system includes different measures and thus reports different outcomes at the school level or identifies the bottom 5% for intervention using different metrics than our system, it will deter the deep work currently underway for one million youth, lead to confusion for both educators and parents, and miss a tremendous opportunity for the state and the nation to learn about the efficacy of our approach that uniquely includes the indicators discussed below. It is worth noting that our internal analyses show that a substantial percentage of schools in the “bottom 5%” would be different using our metrics vs. metrics the state has available to them; 25-40% of schools are differently identified using different assumptions. We also strongly believe that our educator-created metrics offer a more complete and accurate portrayal of what’s happening with an individual student, aggregated up to any given classroom, and to the school level as well. It also stands that these metrics will help guide district decision making. But perhaps most vital, the level of information derived from our metrics not only guides our identification of the necessary intervention, but quite powerfully, it directs us as to at what level in our districts it should be appropriately applied (ie student, classroom, grade level, school, district). We are well poised to serve as a pilot for the state for new measures that could be included in the state’s system, as well as a data-driven continuous improvement approach to accountability. Thus, we propose that the state include in its ESSA plan a waiver request to allow CORE’s pilot. We believe that there are several options through which the state could propose that CORE remain as a California pilot under ESSA, and we are working with DC-based experts to explore this further. In particular, in addition to requiring each state to establish an accountability system and returning authority to each state in the design of its system, ESSA also includes a new and improved waiver authority for states. This provision (ESSA sec. 8013) allows districts to request of the state and then the state to request of the Department a waiver to ESSA provisions (including the accountability provisions) as long as the state provides sufficient information to demonstrate how the waiving of such requirements will advance student academic achievement and provides plans for adequate evaluation to ensure review and continuous improvement. Further, the Department is expressly prohibited from denying a waiver based on conditions outside the scope of the request. This waiver provision seems directly in line with the idea of CORE serving as pilot for the state for new accountability measures to better advance student learning, as well as a data-driven continuous improvement approach to accountability. And ESSA’s waiver authority is in addition to the separate transition authority included in ESSA (ESSA sec. 4), which requires the Department to take steps to ensure orderly transition from NCLB to ESSA, and further reinforces the case for California and the CORE districts to continue our joint efforts. Next, we’d like to explain how this pilot could benefit the state: CORE’s system includes LCAP-related measures at scale, utilizing data that the state cannot collect. CORE’s robust data collection includes data from all schools and at additional grades that the state does not collect annually at scale. We also collect a wider range of data, including social-emotional information and school culture and climate information. All of this information relates directly to state priorities and can be used for the LCAP. Culture and climate Surveys: Common survey instruments are given annually to students, staff, and parents, so the SQIS takes into account a wide range of important voices. We are engaged in extensive research on our surveys and will therefore be able to provide critical information to support the ongoing conversation at the state level around the use of surveys for accountability. Social-Emotional Skills (SEL) surveys: There is tremendous national interest in assessing SEL skills, and no one knows for certain how the inclusion of assessments of these skills in accountability systems will impact th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org students’ academic and non-academic success. The SQIS is already providing a tremendous amount of data to help answer this question. So far, initial results suggest that the CORE SEL survey is reliable and that SEL skills are more highly correlated to academic achievement than the correlation between culture and climate and academic outcomes. What we do not know is how these data will fare in a scenario where stakes are attached. We will only learn this under the auspices of the pilot program we propose. CORE looks at measures that the state could consider now or in the future. The SQIS includes several measures that the CORE districts believe are important but that have not been included in an accountability system at this scale before. These are measures that California could consider including in the state accountability system now or in the future, and CORE’s role could be developing ways to compute the measures and understand how they can be used. Such measures include: High school readiness: CORE’s high school readiness indicator was created based on the fact that our internal student level data showed that if the following four things were true, a student leaving 8th grade had approximately a 95% chance of graduating high school: a 2.5 GPA or higher, no D’s or F’s, no suspensions or expulsions and at least a 96% attendance rate. Although the state does not currently collect the necessary attendance data, they will in the 2016-17 school year. The state has the data to calculate GPA in 8th grade, although there are likely data quality issues as the state hasn't attempted to calculate GPAs to our knowledge. In addition, based upon our research, looking at Ds and Fs in core ELA and math courses is not possible with CALPADS, as CALPADs does not separate “core” ELA and math from “helper” ELA and courses (e.g., math tutorials, reading labs). We collect supplementation information to be able to make that distinction. English learner reclassification that accounts for length of time as an English Learner: Our approach to measuring English Learner reclassification takes the time that a student has been an English Learner into account. Growth Measures: Our growth models estimate school impact on student year over year performance, starting with English Language Arts and math. While the state could adopt our model, we’d like to note that this work has involved over a year’s worth of work to iterate on options, simulations, etc. with analytical partners that have over a decade of experience supporting this kind of work, and in consultation with the leaders of our districts. In addition, we have empaneled a group of technical advisors that have expertise in growth modeling and school accountability to inform our development process. We are poised to release preliminary growth results to our districts in mid-August with final results ready in October. Our expectation, based on numerous conversations, is that our final growth model will differ substantially from where the state is likely to land. As a result, this will be another great opportunity for the state to study different models in action particularly in regards to how they translate into informing instruction. Why Adding Our Measures on to the State System as a Supplement is not the Same Thing as This Pilot Many have asked why the CORE districts cannot simply continue measuring what they want to measure, and use it how they like. How does the state’s system change things? What we know from experience is that state’s accountability system will have a driving influence on what districts and schools focus on, and that while it is feasible for districts to look at other data for decision making and improvement efforts – indeed this was permitted under NCLB and API – having multiple measurement/accountability systems presents communication and implementation challenges for schools, districts, and other stakeholders. What’s more, we would not be able to truly learn about what happens when survey-based measures are included in accountability th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org system that leads to identification for intervention if we were using these only as local indicators disconnected from a state system. CORE’s Efforts Are of Statewide and National Interest The work of the CORE Districts is at the forefront of state and national efforts to emphasize capacity building and continuous improvement as levers for change, to examine school quality across multiple domains, and to make all students visible through the examination of subgroup performance where there are 20 or more students. For example, ASCD released a white paper on multi-metric accountability systems that highlights CORE (see http://www.ascd.org/ASCD/pdf/siteASCD/policy/MultimetricAccountability-WhitePaper.pdf). Further, Policy Analysis for California Education has released a piece on chronic absenteeism (see http://edpolicyinca.org/publications/using-chronic-absence-multi-metric-accountability-system) and n-size (see http://edpolicyinca.org/publications/making-students-visible-comparing-different-student-subgroup-sizesaccountability).We have also contributed to the nascent knowledge base around assessing SEL skills with a report from our partner Transforming Education around developing our measures (see http://www.transformingeducation.org/s/160406_Measuring-MESH_For-Release2.pdf), as well as a report from our independent evaluator Dr. Marin West (see http://www.brookings.edu/research/reports/2016/03/17-noncognitive-skills-school-accountability-california-core-west). Finally, CORE leadership have participated in numerous high profile, national roundtable discussions about the future of school accountability. Summary: A California Pilot of the CORE System Under ESSA Creates a Powerful Opportunity for Our Students Supporting the CORE Districts to become a state pilot for ESSA would provide an exciting opportunity for partnership, collaboration, and leadership, led by the State Board of Education. It would provide critical information for the future refinement of the state accountability system. In addition, given the attention that CORE’s social-emotional measures have gained across the nation, utilizing CORE as a pilot would also allow California to play an even larger role in the national conversation on school and district accountability. The CORE pilot under the auspices of the state ESSA system is: An embodiment of local control; An opportunity to test the use of measures Californian’s and the country care about, but that California as a whole is not ready to utilize state-wide; and An opportunity to minimize disruption and to allow change to happen over time. The CORE Districts care deeply about all of the students in our schools and are committed to addressing historic inequities that have persisted for far too long. We recognize the great value of ongoing collaborative input from experienced educators in the evolution of California’s new accountability system. We remain th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org committed to supporting local innovation and we believe that as a pilot under ESSA, we can help provide critical feedback about local measures that matter most to improving student outcomes. We look forward discussions with you, your office and the other members of the California State Board of Education. th 1107 9 Street, Ste. 500 Sacramento, CA 95814 916 569-2548 COREdistricts.org