2018 State Review Panel Progress Monitoring Recommendation Form School/code & District/code: Adams County 14/0030 State Review Panelists: Tacy Killingworth, Starla Pearson, Amy Weed, Andy Franko Recommendation Meeting Date: October 11, 2018 Current State-directed Action: Management by a private or public entity other than the district. Panel’s 2018 Recommendation: The State Review Panel recommends management by a private or public entity other than the district for Adams 14 School District, in addition to a continued partnership with Beyond Textbooks based on an analysis of compiled data and documentation, as well as a site visit conducted October 4-5, 2018. Evidence and Rationale: The State Review Panel recommends continuing the partnership with Beyond Textbooks, along with the addition of management by a private or public entity to provide clear direction and oversight. There is lack of evidence indicating that district leadership has the capacity to act as a change agent to drive dramatic achievement gains. District leadership continues to have high turnover, and there is limited evidence to indicate that the current leadership team has created a sense of urgency or that they have developed a strategic plan to lead change that will result in improved student outcomes. The current cabinet-level leaders are showing some signs of increased awareness around the need for dramatic change; however, it is clear they need the support and guidance of strong leadership to make the necessary changes to facilitate increased achievement. The district is in year eight on the State accountability clock. Achievement data show that students continue to perform well below State expectations in both achievement and growth. According to the five areas outlined in the Education Accountability Act, the district is rated Developing in all areas except for leadership being adequate, which is rated as Not Effective. The previous State Board-directed action for Adams 14 was management by a private or public entity other than the district. The district chose Beyond Textbooks (BT) as a partner and developed a Revised Pathways Proposal dated June 6, 2017 in which the State Board approved and mandated implementation of the plan. In addition, the State Board directed the district to develop a progress monitoring plan that must be submitted to the Colorado Department of Education no later than August 15, 2017, and to submit local data at least quarterly to allow the Department to assess if the implementation is on track according to the submitted progress monitoring plan. A review of district documents revealed that Adams 14 has a District Progress Monitoring Tool 2018-19; however, there is no evidence to show the district is using the data collected to monitor progress of student performance or improvements to instruction. Although the partnership with BT is serving a purpose by providing a foundation for instruction focused on curriculum, assessment, and intervention, there has not been a formal program evaluation to determine true effectiveness of the partnership. BT does not have a strong plan of accountability to ensure the district is implementing the plan effectively. BT representatives have indicated that they view themselves as a partner with the district, rather than an external manager. During the site visit, all stakeholders reported that BT was not enough to help the district move out of Priority Improvement. Both a document review and a site visit revealed that there is a clear lack of leadership and infrastructure at the district level to support improvement efforts. Also, the district and community do not have a shared vision or belief around what the district needs to move forward. © 2018 SchoolWorks, LLC. All rights reserved. Recommendation Form - 1 Because a review of documents and evidence from the site visit indicates some traction in improving instruction, the State Review Panel (SRP) team is recommending the continuation of the partnership with BT to help provide some consistency in supporting teachers in providing quality instruction. Termination of this partnership might cause anxiety and frustration among staff; it would contribute to the past experiences of constant change and turmoil. In recommending the addition of a management by a private or public entity, it is important to note that the SRP team feels strongly that this entity must provide true management – not a partnership – and that it must provide clear direction and oversight of the district. In addition, it will be essential that the Colorado Department of Education has close communication to help create urgency and to ensure high accountability. Adams 14 needs a management entity that will: • Understand and support the continuation of the partnership with BT and help develop a strong academic program infrastructure; • Identify and document concerns and areas of focus and be clear with the district around expectations for implementation of turnaround strategies; • Establish and develop a district leadership team that can implement and monitor effectiveness of the turnaround strategies; • Develop organizational structures and a clear leadership model at the district and school levels; • Be attentive to the culture and climate and establish strong community engagement; • Establish processes and procedures to hire and retain quality staff; • Establish clear communication structures that will enhance a shared vision toward common goals; and • Focus on developing the capacity of the staff and not just bringing in programs. The State Review Panel does not recommend innovation district status. The district has neither adequate leadership capacity nor the infrastructure to support innovation. District leadership has not shown evidence of being able to lead dramatic change and, during the site visit, leadership reported lack of experience with innovation. Change to innovation status would require consent from the Adams 14 community. As evidenced by a document review and site visit, the district does not have a shared vision, and stakeholders are divided in their beliefs around what is best for the district. In addition, the continued turnover in staff, from teachers to leaders, would make it difficult to develop, implement, and maintain quality innovation status. Although changing to innovation status might increase the district’s ability to recruit teachers and leaders, it is unclear that this benefit is understood by leadership and that leadership has the capacity to manage the transition and leverage the benefit. Adams 14 requires a common vision with stability and consistency. The introduction of initiatives would have a negative impact on staff and community morale and buy-in. The State Review Panel does not recommend conversion of one or more schools to a charter school. The district does not have the capacity to support the conversion process and converting one or two schools to a charter would not address the systemic need of the district to support multiple schools that are on Priority Improvement Status. The district’s lack of leadership capacity would impact its ability to effectively manage the authorization of a charter. District representatives have clearly expressed that they are not interested in a charter status. Although a charter would provide options for students, which parents and community members have expressed they would like, the lack of consistency in leadership would make it challenging to adequately plan, implement, and support a charter with fidelity. The SRP team did consider possibilities that included the idea that a charter entity with a strong history of performance could help improve the achievement for some students. Also, a slow launch of charter schools beginning with the lowest-performing schools might be a way to see if there is a management partner that could support the © 2018 SchoolWorks, LLC. All rights reserved. Recommendation Form - 2 district in this process. The State Review Panel does not recommend closure of one or more district schools. Closing a school will not make an impact on overall district performance. Because seven of the eleven schools are on Improvement or Priority Improvement Status, there are no other viable options or choices for students that will lead to better outcomes. In addition, there is likely not the physical space to support closure of one or more schools. A review by the SRP in 2015 noted schools at that point were already 6% above capacity. The State Review Panel does not recommend district reorganization. The SRP gave serious consideration to this option, but because district reorganization procedures are less clear, and due to the significant lack of leadership capacity and infrastructure, it was determined that a district reorganization would require management by an outside partner, which ultimately led to the team’s recommendation. There is also concern that with district reorganization, there would not be clear direction and oversight – thus, a continuation of lack of defined roles and responsibilities for the Board of Education and district leaders. In addition, the current leadership team has not proven they are able to lead dramatic change. Although there is a clear need for some reorganization, the SRP strongly believes that Adams 14 would need the clear direction and oversight from an external management entity to support this reorganization. The continued turnover in leadership and the challenge of recruiting people to work in Adams 14 could also impact the ability to effectively reorganize the district in a way to facilitate positive outcomes for students. © 2018 SchoolWorks, LLC. All rights reserved. Recommendation Form - 3 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 Purpose: The State Review Panel (SRP, or the Panel) was created by the Accountability Act of 2009 to provide a critical evaluation of the State’s lowestperforming schools’ and districts’ plans for dramatic action and provide recommendations to the Commissioner and the State Board of Education. The Panel’s work is informed by a review of documents (e.g., Unified Improvement Plan) and, in some cases, by a site visit. The site visit component was added in 2013 to strengthen panelists’ understanding of the conditions in the schools and districts that are further along on the accountability clock. For schools and districts that continue to remain on the accountability clock, the SRP will conduct an additional progress monitoring site visit that will be used to assess the actions the school or district was previously directed to take, the fidelity to which the district has implemented directed actions, and the amount of time the district has had to implement the actions to achieve results. The expectation is the site visit will inform their recommendations to the Commissioner and the State Board of Education about potential actions at the end of the accountability clock. Prior to arriving on site, panelists conducted a document review aligned to the six key areas in the Accountability Act. On site at the district, the site visit team used evidence collected through focus groups, interviews, and document review to come to consensus on capacity levels in relation to the six key areas. This report presents the district’s capacity levels in relation to the six key areas and a summary of evidence for each. Reviewer Name(s): Starla Pearson, Amy Weed, Andy Franko, Tacy Killingworth Date: 10/10/18 District Name/Code: Adams County 14/0030 Capacity Level: SRP Progress Monitoring Site Visit Summary 1. The leadership is adequate to implement change to improve results. Not Effective 2. The infrastructure is adequate to support school/district improvement. Developing 3. There is readiness and apparent capacity of personnel to plan effectively and lead the implementation of appropriate action to improve student academic performance. Developing 4. There is readiness and apparent capacity to engage productively with, and benefit from, the assistance provided by an external partner. Developing 5. There is likelihood of positive returns on State investments of assistance and support to improve the performance within the current management structure and staffing. Developing 6. There is necessity that the district remain in operation to serve students. © 2018 SchoolWorks, LLC. All rights reserved. Yes District Progress Monitoring Site Visit Feedback Form - 1 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 SRP Evaluation based on Progress Monitoring Site Visit State Review Panel Criteria 1. The leadership is adequate to implement change to improve results. 1.1: Leadership acts as a change agent to drive dramatic achievement gains. 1.2: Leadership establishes clear, targeted and measurable goals designed to promote student performance. 1.3: Leadership analyzes data to identify and address high priority challenges, and to adjust implementation of the action plan. 1.4: Leadership establishes high expectations for student learning and behavior. • Leadership communicates a relentless commitment to the district turnaround. • Leadership makes data-driven changes to the academic program and organization to promote dramatic achievement gains. • Leadership conveys clear expectations for performance for all stakeholders, including district staff, school leadership, teachers, district accountability committee, parents, and community members. • Leadership communicates clear and focused goals that are understood by all district and school personnel. • District and school staff understand their responsibilities for achieving goals. • Leadership maintains district-wide focus on achieving established goals. • Leadership allocates resources in alignment with goals and critical needs. • Leadership has established systems to measure and report interim results toward goals. • Leadership communicates data trends and issues, ensures timely access to data, and models and facilitates data use. • Leadership openly shares results and holds staff accountable for results and effective use of data. • Leadership first concentrates on a limited number of priorities to achieve early, visible wins. • There is regular progress monitoring of performance and implementation data and, as appropriate, results lead to elimination of tactics that do not work. • Benchmarks are used to assess progress toward goals; goals are adjusted as progress is made. • Data on progress toward goals drives organizational and instructional decision making • • • • The district holds high expectations for academic learning. Educators set high expectations for learning and clearly convey these to students. Educators convey that students are responsible for raising their performance and encourage their participation in learning. The district provides a safe environment to support students’ learning and, in the case of a virtual school, ensures that students’ © 2018 SchoolWorks, LLC. All rights reserved. Claims & Evidence Capacity Level: [ ] Highly Effective [ ] Effective [ ] Developing [ X ] Not Effective Constant changes in district-level and building-level leadership has contributed to a lack of urgency and inability to drive dramatic achievement gains. • District leadership stated that there has been a restructuring at the district level to address specific needs of both elementary and secondary schools. However, through multiple focus group discussions, there was no evidence to support that there is a strategic plan to lead dramatic change that will lead to increased student achievement. • Although leadership developed a partnership with Beyond Textbooks (BT) a little over a year ago, district leadership was unable to provide evidence that revisions or new work has been adopted to strengthen this partnership. • As reported by teachers, schools not involved with the BT partnership were able to choose strategies that worked in their school and were not required to follow the BT requirements. • District and school leaders, teachers, and community members reported that both district-level leadership and school leadership turnover has been high from year-to-year, thus causing a lack of consistency to lead change. • Teachers reported that in buildings in which overall leadership is strong, those leaders are leading instructional change. In addition, teachers indicated that district-level leadership changes have had a negative impact in the district, but when school leadership is strong, the work continues at the building level. District Progress Monitoring Site Visit Feedback Form - 2 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 • interactions between and among themselves and school staff are respectful and supportive. Leadership ensures that the school’s physical environment is clean, orderly, and safe. • Although all focus groups have acknowledged gaps in the turnaround efforts, there is little evidence that indicates there is an urgency to take action to lead dramatic change to address these gaps. In addition, it was often reported that “everything is new,” thus making it hard to start the work to lead change that will result in increased student achievement. District stakeholders do not articulate clear, targeted, and measurable goals to drive improvement and increase student achievement. © 2018 SchoolWorks, LLC. All rights reserved. • When asked, focus groups were unable to cite the State Board’s turnaround directive for the district. Rather, they stated that the district was directed by the State Board to improve in areas such as school culture, test scores, and post-secondary readiness. Additionally, focus groups did not articulate specific and measurable district goals. • In the review of district documents, goals are established in documents such as the Unified Improvement Plan (UIP) and the Adams 14 District Progress Monitoring Tool 2018-19. However, when asked in focus groups, specific measurable goals were not articulated. Most cited responses to what the district goals were including, to “get off of turnaround status” or to “change the School Performance Framework (SPF) rating.” • District leadership was unable to describe a succinct plan of action that was being used to achieve district goals. Most often, leaders referenced the partnership with BT and gave general actions such as monthly walkthroughs at each building and meetings between district leaders and BT representatives. • District leadership stated that there is use of data indicators that drive conversations at the district office level with the instructional team; however, it was unclear what these data indicators included. District Progress Monitoring Site Visit Feedback Form - 3 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 It was stated that the district instructional team then works down to the building level to address concerns with school leaders. • Teacher focus groups reported, and it was confirmed by school and district leaders, that there is a progress monitoring tool provided to each school to track progress in various areas, such as Colorado Measure of Academic Success (CMAS) literacy achievement, student attendance, teacher attendance, and Dynamic Indicators of Basic Early Literacy Skills (DIBELS). • District leaders were unable to describe what specific student assessment outcomes meet desired expectations, nor were they able to define desired expectations. • District leaders reported program evaluation is something that needs improvement. Leadership is beginning to establish high expectations for student learning and behavior. © 2018 SchoolWorks, LLC. All rights reserved. • When asked, there were inconsistent responses about how teachers set high expectations and use data to realize student learning. For instance, some teachers were able to clearly identify academic goals and how they monitor progress toward those goals, yet not all teachers indicated this level of goal-setting and monitoring. • Parents reported that they believe there are some high expectations being set for students, but that it depends on the school leadership. • Some school leaders reported that the district has expectations for them that included conducting data team meetings twice a month and consistent Professional Learning Communities (PLCs) at the secondary level. District Progress Monitoring Site Visit Feedback Form - 4 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 © 2018 SchoolWorks, LLC. All rights reserved. • Some community members reported low expectations for students’ achievement and for attendance, and that school staff are not holding students accountable to high expectations for behavior, learning, or attendance. • Some district leaders reported an effort to shift the mindset from focusing only on growth to increasing student achievement and proficiency. • District leaders and school leaders reported that implementation of BT has begun to increase the rigor of instruction in some classrooms. For example, both district and school leaders reported seeing more teachers using higher-level questioning with students. • When asked, parents stated they believe certain schools are safe, but the level of the school (i.e., elementary, middle, high) and leadership of the school characterize the level of cleanliness, order, and safety. • A review of the Adams City Youth Initiative (ACYI) Survey showed that 62% of high school student respondents say they disagree that students tell teachers when other students are being bullied. In addition, 43% of middle school student respondents say they disagree that students tell teachers when other students are being bullied. • Although the district document for the 2018-19 Professional Development Plan indicated that a 2-day Positive Behavior Intervention and Supports (PBIS) training occurred in July 2018 and a 5-year plan for Kagan program implementation and Responsive Classrooms existed, focus groups did not identify how specific behavior expectations for students in schools are developed and enforced. District Progress Monitoring Site Visit Feedback Form - 5 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 SRP Evaluation based on Progress Monitoring Site Visit State Review Panel Criteria 2. The infrastructure is adequate to support school/district improvement. 2.1: The district leads intentional, strategic efforts to ensure the effectiveness of the academic program and the sustainability of the organization. 2.2: District leadership has a strong focus on recruiting and retaining talent; creates and implements systems to select, develop, and retain effective leaders, teachers, and staff who can drive dramatic student gains; evaluates all staff; and dismisses those who do not meet professional standards and expectations. 2.3: District leadership ensures that the district has sound financial and • Leadership ensures ongoing development for emerging and current school leaders with a focus on building leadership capacity to lead turnaround efforts and sustain improvement. • Leadership provides adequate oversight in schools’ work to deliver the curriculum, monitors instruction on a regular basis, and provides adequate support and feedback to principals to improve instruction. • The district provides adequate systems by which to capture and store data, report it to schools, and make it accessible for instructional staff to utilize. • Leadership has created and/or implemented an organizational and staffing structure that will drive dramatic student gains. • Leadership recruits and hires leaders, teachers, and staff members with commitment to, and competence in, the district’s philosophy, design, and instructional framework (e.g., trained and experienced with curriculum, certified/licensed to teach, qualified to teach subject area). • Leadership ensures the evaluation of all staff, and dismisses those who do not meet professional standards and expectations. • Leadership provides leaders, teachers, and staff members with active, intense, and sustained professional development (PD), including guidance on data analysis and instructional practice, aligned to school improvement efforts. o PD is informed by ongoing analysis of student performance, instructional data, and educators’ learning needs. o PD requires leaders, teachers, and staff members to demonstrate their learned competency in a tangible and assessable way. o PD engages leaders, teachers, and staff members in active learning (e.g., leading instruction, discussing with colleagues, observing others, developing assessments), and provides follow-up sessions/ongoing support for teachers’ continued learning. o The quality of professional development delivery is regularly monitored, evaluated, and improved. • District leadership ensures that the organizational structure supports essential district and school functions, and that roles and responsibilities of all individuals at the school are clear. • District leadership has established effective means of communicating with district and school staff. © 2018 SchoolWorks, LLC. All rights reserved. Claims & Evidence Capacity Level: [ ] Highly Effective [ ] Effective [ X ] Developing [ ] Not Effective The district is beginning to lead intentional, strategic efforts to ensure the effectiveness of the academic program and the sustainability of the organization. • While the district has outlined targets in the UIP, has identified progress monitoring tools for students’ academic performance, and has outlined some strategies that can be used to improve student performance, district and school leaders were not able to consistently articulate the documented targets, tools, and strategies to guide the direction of the academic program. • District focus groups stated that the academic program focuses on the implementation of BT and Schools Cubed and that schools, (except the three original schools designated in the BT partnership) are able to opt into BT. In addition, some teachers reported that departments are using the BT framework to create their own frameworks to support students. • District leaders stated that the curriculum and instruction department is newly-formed and reorganized this year and is just beginning to create some systems and structures to support teaching and learning. For example, principal and assistant principal meetings are now being held monthly, progress monitoring indicators are being identified, and increased district walkthroughs are occurring more consistently. • Both teachers and district leadership reported that classroom walkthroughs occurred once a month at BT schools and that a common classroom walkthrough form is used to gather data on District Progress Monitoring Site Visit Feedback Form - 6 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 operational systems and processes. 2.4: Leaders provide effective instructional leadership. • District leadership ensures that all compliance requirements and deadlines set by the State are met, including the submission of school improvement plans, financial statements, school audit, calendar, and student attendance. • District leadership effectively manages the budget and cash flow, and there is a plan for long-term financial sustainability. • District leadership effectively manages operations (e.g., food services, transportation, school facilities). • District leaders ensure that schools implement a coherent, comprehensive, and aligned curriculum. o District leaders ensure that schools’ curriculum, instruction, and assessments are aligned with State standards, with each other, and coordinated both within & across grade levels. o District leaders ensure that instructional materials are selected and/or developed in accordance with a district-wide instructional framework and aligned with established curriculum standards. o District leaders ensure the curriculum is periodically reviewed and revisions are made accordingly. • The district ensures that school leaders provide meaningful feedback on teachers’ instructional planning and practice. o The district ensures that leaders regularly provide meaningful feedback on instructional planning. o The district ensures that leaders regularly observe instruction and provide meaningful, timely feedback that helps teachers improve their practice. • District leaders provide conditions that support school-wide data cultures. o Staff, school leaders, and teachers have easy access to varied, current, and accurate student and instructional data. o Staff, school leaders, and teachers are provided time to collect, enter, query, analyze, and represent student data and use tools that help them act on results. o District leaders ensure that all staff, school leaders, and teachers receive professional development in data use (e.g., how to access, read, and interpret a range of data reports; frame questions for inquiry; analyze data, assessment literacy; use data tools and resources). © 2018 SchoolWorks, LLC. All rights reserved. implementation of the expectations. Additionally, each walkthrough visit is followed by a debrief with school leaders regarding strengths and recommendations and, also, a debrief with district personnel is conducted. • District leadership conveyed that the schools are able to use Illuminate as the platform to house common assessments and that it has the capability of disaggregating the data. However, it was reported due to the late set up of Illuminate, they have not yet used it effectively in this manner. The district struggles to recruit and retain talent. The district is beginning to implement systems to select, develop, and retain effective leaders and teachers. • Across focus groups, district and school leaders reported that staff (district, principal, and teacher) turnover is one of the district’s biggest challenges. • School staff reported that teacher turnover often occurs because of leadership turnover (e.g., bringing initiatives into the building that they do not like, constant change). Some school staff reported having recently experienced a 95% staff turnover. • Community and parent members also described how the turnover has impacted a school’s performance and the stability of the school environment. • Community group members questioned the strategy for selecting school leaders who are skilled in turnaround efforts. They indicated that they felt principals were selected in spite of what community and staff preferred. • District leadership stated that hiring school leaders is a challenging process. It was stated that principal candidates are informed that increased achievement and growth for students results within oneDistrict Progress Monitoring Site Visit Feedback Form - 7 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 to-two-years are expected. This creates some hesitation for future principal candidates to accept positions. © 2018 SchoolWorks, LLC. All rights reserved. • District leadership shared that emerging leaders are developed through the RELAY fellowship. This was confirmed by teacher leaders. However, school leaders reported that the RELAY development is around procedures and technical leadership and not focused on building relationships. • In addition, district leaders stated that in an effort to retain school leaders, the district is holding monthly school leader meetings to help with communication and provide support, as well as the hiring of a coach to support school leaders. In focus groups, school leaders reported problem solving, reviewing data, and conducting classroom walkthroughs with the school leader coach. • District leaders reported that consistent hiring processes were being developed and that the district has developed a teacher and principal induction program but could not articulate the specifics of the programs. • A district focus group stated that the human resource department offers a bonus for ”hard-to-staff” positions (i.e., math, science, special education teachers). • Teachers indicated that BT pacing guides calendars were provided three days before students arrived for the 2018-19 school year and that there were inconsistencies in how teachers were trained to implement the BT strategies. School leaders confirmed that BT training on implementation was inconsistent across the district. • Teachers indicated that more professional development (PD) has recently been provided to train staff on conducting “real” PLCs. However, teachers were unable to specifically address how PLCs or data discussions were being used to drive instruction. District Progress Monitoring Site Visit Feedback Form - 8 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 • District leaders reported the district is partnering with Regis University to support teachers to earn the certificate for Culturally Linguistic teachers. It was reported that the cost is shared and the district pays one-third, the partner pays one-third, and the teachers pays one third. The district leaders are beginning to provide instructional leadership. © 2018 SchoolWorks, LLC. All rights reserved. • School leaders and teachers reported a variety of curricular resources used to support student learning. For example, it was stated that schools engaged with BT reported using BT resources in math and English language arts (ELA) as a curriculum and that nonBT schools were using Benchmark Literacy, which is a comprehensive Common Core Program K-6 to support literacy. However, teachers, district leaders, and BT representatives reported curriculum gaps to support phonics instruction across the district and that Schools Cubed was supporting efforts to address these gaps. • School leaders and teachers reported that at the elementary level, grade-level teams conduct weekly data meetings, and at the secondary level, they conduct PLCs 3-to-4 times a week. However, the level of data analysis is reported to depend on school leadership guiding the process. • District leaders stated that the district is using Illuminate to house common assessments; however, they were unable to articulate data trends for achievement and growth. • Additionally, district leaders reported that the use of Illuminate is so new that they are not yet able to see data trends. Rather, they reported that they are able to view common formative assessments and interim assessments in the platform but have not collected results from these assessments. District Progress Monitoring Site Visit Feedback Form - 9 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 © 2018 SchoolWorks, LLC. All rights reserved. • Teachers stated that data are collected from internal formative assessments, normed assessments (STAR), and BT assessments. Some teachers stated that they use the data to help group students. However, they were unable to describe how these data are used in a collaborative manner with other teachers to drive instruction. • District leaders and school leaders reported that district leaders visit all schools weekly for classroom walkthroughs, using a districtdeveloped look-for document and that feedback on practices observed is shared with the school leader. • Teachers, however, reported that district leaders come into their classrooms or into their schools and that they believe the district leaders are looking for specific elements. Also, teachers reported that they do not receive feedback on their instructional practice but wish that they would. • When asked how school leaders are supported in being developed as instructional leaders, both district leaders and school leaders reported that a principal coach frequently visits their buildings to mentor, conduct classroom walkthroughs, look at data, and help problem solve potential issues. • School leaders reported that they are provided verbal feedback during walkthrough visits and during the afternoon school leader meetings. In addition, school and district leaders stated that school leaders did not consistently receive formal evaluations and feedback the past two years. District Progress Monitoring Site Visit Feedback Form - 10 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 SRP Evaluation based on Progress Monitoring Site Visit State Review Panel Criteria 3. There is readiness and apparent capacity of personnel to plan effectively and lead the implementation of appropriate action to improve student academic performance. 3.1: Educators’ mindsets and beliefs reflect shared commitments to students’ learning. 3.2: The district has established conditions that support educators’ learning culture. 3.3: District personnel collaborate regularly to learn about effective instruction and students’ progress. • District personnel convey shared vision and values about teaching and learning and reference these to guide their instructional decision making. • District personnel convey a shared commitment to the learning of all students in the district. • District personnel convey that students’ learning is a collective responsibility, regardless of their personal/home situations. • District personnel convey that it is important not to give up on any students, even if it appears that they do not want to learn. District personnel convey commitment to, and hold each other accountable for, improvement goals and tasks. • Communications among all stakeholder groups are constructive, supportive, and respectful. • Communications between leadership and district/school staff are fluid, frequent, and open. • District leaders model and convey well-defined beliefs about teaching and learning, and convey value for innovation, learning from mistakes, and risk taking. • District leaders participate in formal and informal professional learning, including their own leadership development about how to improve curriculum and instruction in a leadership context (i.e., high- or low-poverty; urban or rural district). • District personnel meet frequently during regularly scheduled uninterrupted times to collaborate, establish improvement goals, and make data-informed instructional decisions. • District personnel’s collaborative meetings have a clear and persistent focus on improving student learning and achievement. • District personnel describe sharing knowledge and expertise among colleagues as an essential collaborative activity for success. • District staff and school leaders are willing to talk about their own practice, to actively pursue and accept feedback from colleagues, and to try new leadership strategies. The district has created a performance-driven culture in which district staff, school leaders, and teachers effectively use data to make decisions about daily instruction/organization of students. © 2018 SchoolWorks, LLC. All rights reserved. Claims & Evidence Capacity Level: [ ] Highly Effective [ ] Effective [ X ] Developing [ ] Not Effective Educators’ mindsets and beliefs do not yet reflect shared commitments to students’ learning. • District leaders reported that some adults in the district have not demonstrated a commitment to students, have not made students their top priority, and some come to the district with an agenda that is not focused on putting students first. They reported that in the recent past, the culture at the district level was a competitive environment in which you could not have transparent conversations; however, there is indication that the adult culture is showing improvement. In addition, community members stated that the biggest problem is the amount of politics within the district that has contributed to the competitive environment. It was stated that some people in leadership positions come in with their own personal agendas that do not always include what is best for kids. • Teachers indicated there are inconsistencies in the instructional decision-making process. Some stated that many decisions are made at the building level, while others indicated that the district directs instructional decisions. • District personnel stated that student learning is a top priority; Board members confirmed the same. • Parents stated that the district discontinued parent/teacher conferences during the 2017-18 school year. When asked, parents stated the reason for cutting the conferences was to force building District Progress Monitoring Site Visit Feedback Form - 11 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 3.4: The district engages the community and families in support of students’ learning school improvement efforts. • The district includes parents/guardians in cultivating a culture of high expectations for students’ learning and their consistent support of students’ efforts. • The district invites family participation in district activities (e.g., volunteering in on committees; attendance at organizational meetings) and regularly solicits their input. • The district offers workshops and other opportunities for parents/guardians to learn about home practices that support student learning. • District personnel communicate with parents/guardians about instructional programs and overall student progress within the district. level employees (i.e., teachers and administrators) to call home more often and communicate more regularly. Parents confirmed that conferences were reinstated for the 2018-19 school year. • School leaders reported that teachers are committed to students. District leaders agreed that teachers care about students, but do not always know how to teach them. It was stated that teachers “…love our kids so much they have gone to the ‘poor kids’ mindset and sometimes forget to care about student learning.” The district is working to establish conditions that support educators’ learning culture. © 2018 SchoolWorks, LLC. All rights reserved. • When asked about the adult culture, district leaders reported that the culture and climate has been poor, and that there has been “adult drama,” a lack of trust, and a toxic environment. District leaders attribute this to the high amount of turnover and the inconsistency in district leadership. However, district leaders reported that the adult culture is improving and trust is increasing. • School leaders reported that the adult climate in the district is improving. It was reported that the climate at the building level is leader dependent. Parents and community members confirmed that the building culture is dependent on the school leader. • Some staff indicated that the political climate has been a barrier to getting things done at times. • Both district and school leadership reported that principals attend monthly meetings and that these meetings are facilitated by the superintendent and district leadership. • District leaders, school leaders, and teachers reported that communications go from the district to principals and then to teachers. Teachers report that they typically receive District Progress Monitoring Site Visit Feedback Form - 12 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 communication from their principals and that there is limited communication that comes directly from the district office. • Parents reported that communication from elementary schools tends to be better than communication at the middle and high school level. In addition, they reported that it depends on the leader of the building. • Community members also reported a lack of communication from the district office around district and school events, as well as district and school progress and performance. • The 2018 Teaching and Learning Conditions in Colorado (TLCC) district survey results reported that only 40% of the staff feel there is an atmosphere of trust and mutual respect between district and school administrators. The district leaders are beginning to collaborate regularly to learn about effective instruction and students’ progress. © 2018 SchoolWorks, LLC. All rights reserved. • The BT representative group stated that they convene following monthly BT school visits to debrief observation results and identify recommended next steps both at the school level and district level. • District leaders reported comparing results from weekly classroom walkthroughs. However, they did not indicate how those data points are used to change programming or make adjustments to district actions in support of instruction. • School leaders reported that they are beginning to collaborate more in principal meetings; however, this was not a focus last year. • District leaders reported that they are paying for teachers to have collaborative PLC time to analyze data. Some school leaders added that they are being intentional to try to provide this collaboration time during the school day. District Progress Monitoring Site Visit Feedback Form - 13 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 • Teacher focus groups reported that collaboration at schools is leader-dependent. Some teachers reported more intentional focus on data analysis to drive instructional practices and intervention groups. Other teachers reported that they do not have consistent collaboration time with their teams. The district does not purposefully engage the community and families in support of students’ learning. © 2018 SchoolWorks, LLC. All rights reserved. • The community reports that there is a district advisory council and school advisory councils and that there are family events at the schools, such as Donuts with Dads and Muffins with Moms. However, they stated that the district did not take into account parent availability when scheduling these meetings and events. They wonder if they occur as authentic ways to engage parents or as a requirement for the UIP process. • The Board reported that the superintendent holds four forums a year to solicit input and to promote communication. These forums are held quarterly and meet both in the mornings and evenings to help accommodate schedules so that more people are able to attend. However, community members and parents reported that these forums have low attendance and, due to lack of attendance, are sometimes canceled. • Parents stated that the district used to offer more workshops for parents, but then the workshops all started to become the same and parents stopped attending. • In addition, parents reported that that when they are engaged, it is often because they are intentional about being present at district events and activities, not because the district has invited them. District Progress Monitoring Site Visit Feedback Form - 14 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 © 2018 SchoolWorks, LLC. All rights reserved. • The Board spoke about requiring district and school leaders to attend events in the community, stating, “If they see you there, they will know you care.” • Some parents stated that some schools communicate the performance level of the school through banners inside and outside of the school and through social media; however, they reported this was leader-dependent. • When asked, most parents were not aware of the partnership with BT. However, a couple of parents were aware of the partnership and stated it was a waste of time and money, because when you look at the schools that are doing well they are not BT schools. • District leaders reported that there is a Parent University that is a 4-week class to support parents in how to support student learning. In addition, it was stated that there is a Support Team of Motivated Parents (S.T.O.M.P.) that work with students who are struggling academically in one-on-one or small group settings. • A review of district documents reveals that the district sends flyers out in English and Spanish for events, such as the science fair, math night, and events at the Denver Art Museum. • The community reported that one of the biggest issues across the district is that the lack of communication creates a culture of fear that leaves stakeholders uninformed and uncertain about whom to go to for information. District Progress Monitoring Site Visit Feedback Form - 15 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 SRP Evaluation based on Site Visit State Review Panel Criteria Claims & Evidence Capacity Level: Effective 4. There is readiness and apparent capacity to engage productively with, and benefit from, the assistance provided by an external partner. 4.1: The district collaborates effectively with existing external partners. 4.2: The district leverages existing partnerships to support of student learning. 4.3: Leadership is responsive to feedback. • The district seeks expertise from external partners, as appropriate (i.e., for professional development, direct support for students). • The district ensures that roles and responsibilities of existing partners are clear. • There are designated district personnel to coordinate and manage partnerships. • The district maximizes existing partners’ efforts in support of improvement efforts. • All externally provided professional development is aligned to improvement efforts. The district collaborates with its existing management partner. • District and school leaders and teachers reported engaging with BT as the management partner. • The BT contract with the district outlines specific roles and responsibilities of BT and the district; however, during focus groups with school and district leaders, these roles and responsibilities were not understood consistently. For example, the contract specifically states, “The BT administration team will oversee implementation of the instructional framework as the authority for determining quality implementation.” Yet some leaders reported that the recommendations were not mandatory to implement, and the BT representatives indicated they wanted to give schools autonomy. • Both district and school leaders reported that the district has a BT liaison who meets with building level BT coordinators to support implementation of BT. Additionally, leaders noted there are two BT representatives who work with the district BT liaison to also support implementation. • District leaders stated that the district started the partnership with three BT schools (Adams City High School, Rose Hill Elementary, and Central Elementary) and have had four schools join for the 2018-19 school year. • District and school leaders, as well as teachers, reported that BT visits schools monthly and conducts classroom walkthroughs. After the site visits, there are two debrief sessions around strengths and • District leadership seeks feedback on improvement plans. • District leadership seeks feedback from key stakeholders. • District leadership integrates feedback into future improvement efforts. © 2018 SchoolWorks, LLC. All rights reserved. [ ] Highly Effective [ ] Effective [ X ] Developing [ ] Not District Progress Monitoring Site Visit Feedback Form - 16 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 recommendations for growth – one with the school leader and one with district leaders. • Teachers reported that they receive PD for BT to include BT 101 and 201, and that new teachers receive onboarding training for BT. In addition, teachers and building leaders reported that secondary teachers receive PLC training from a PLC consultant to support implementation of BT. • Some teachers reported that BT training for the second year of implementation was the same as the first year, and that they have received less support this year than in the first year of implementation and less clarity on what full implementation will look like. • District and school leaders, as well as teachers, reported that BT provides various resources to include frameworks for curriculum, assessment, and intervention. Examples of these resources include common formative assessments, lesson plan templates, and pacing guides. The district is working with its external management partner; however, the impact of the partnership is unclear. © 2018 SchoolWorks, LLC. All rights reserved. • Both district and school leaders reported that BT has fulfilled a certain component that was needed; however, many of them acknowledge that it does not address all of their needs. Some reported that the district needs to additionally address climate and culture, develop a strategic five-year plan, and build and implement consistent structures and processes. • District leaders reported that BT has provided an instructional framework and that teachers are now having conversations around unpacking the standards. Additionally, BT has unpacked the standards and provided pacing guides for the teachers and that they only have to decide how to deliver the instruction. Leaders stated that this support District Progress Monitoring Site Visit Feedback Form - 17 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 guides teachers in the instructional process, including conducting formative assessments; as a result, district leaders report they are seeing higher-level questioning being used by teachers. © 2018 SchoolWorks, LLC. All rights reserved. • When asked specifically about the impact of the partnership, stakeholders in focus groups could not articulate measurable results. Some reported, “We want to see it in action in the classrooms;” or “We do sessions with BT where they present results;” or “We are hearing teachers have conversations about standards.” However, district leaders could not articulate specific data points that show the impact of the partnership. • District leaders stated that they need more support at the district level and at the high school in order to implement BT effectively. For example, the district has limited resources to provide adequate PD for staff. Also, the high school scheduling is challenging and not all departments have a common plan time in order to effectively collaborate to implement BT strategies with fidelity. • All focus groups reported that BT is supporting efforts toward instructional consistency, but it is not enough to get the district off the accountability clock. • District and school leaders reported that BT makes recommendations that the district and school can decide whether to implement; however, they were not able to articulate how BT holds the district accountable for implementation. • District and BT leaders reported that although additional schools have been added to the BT framework this school year, no additional district or BT positions have been added to support this scaling. District Progress Monitoring Site Visit Feedback Form - 18 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 SRP Evaluation based on Site Visit State Review Panel Criteria 5. There is likelihood of positive returns on State investments of assistance and support to improve the performance within the current management structure and staffing. 5.1: Leadership monitors the return on investment of specific improvement initiatives and uses that data to inform decision-making. 5.2: Leadership has demonstrated an ability to produce positive returns on state investment and uses resources effectively. 5.3: Students demonstrate academic progress over time. • Leadership identifies turnaround strategies and implements programs/initiatives designed to improve student performance. • Leadership assesses the cost and impact (effect on student achievement and number of students served) of each program/initiative to determine its academic return on investment. • Leadership makes decisions regarding continuation or discontinuation of programs/initiatives based on this analysis. • Leadership establishes systems and structures to support regular and ongoing monitoring. • Programs and initiatives are designed to support turnaround efforts and have demonstrated results. • Leadership seeks resources aligned to its improvement efforts and programs/initiatives with high academic return on investment. • Any additional resources received (i.e., specialized grant funding) are aligned, strategic, and showing evidence of results. • Leadership treats resources flexibly, and implements focused improvement efforts with a focus on early wins. • Students demonstrate progress on internal measures linked with the district’s promotion or exit standards. • The performance of student subgroups on State assessments demonstrates that the district is making progress toward eliminating achievement gaps. • Students meet proficiency and grade-level targets across subjects and grade levels on norm-referenced benchmark assessments and State assessments. • Matched cohorts of students who score proficient or advanced (or equivalent) on State assessments maintain or improve performance levels across continuous enrollment years. • The percentage of all students performing at proficient or advanced (or equivalent) on State assessments increases over time. • Students demonstrate academic growth as measured by valueadded or State growth percentile measures. • Students demonstrate progress toward attaining expected knowledge and skills as measured by interim assessments. © 2018 SchoolWorks, LLC. All rights reserved. Claims & Evidence Capacity Level: [ ] Highly Effective [ ] Effective [ X ] Developing [ ] Not Effective Leadership has demonstrated an ability to produce positive returns on some State investments. • District focus groups reported that Title I funds are used at the building level to provide teacher stipends for after-school tutoring, PD, and data team meetings. Funds are also going to be used to pay a coordinator a stipend to organize tutoring for the 2018-19 year, because the prior year’s practice indicated that tutoring time was being lost due to the time being used to set up the tutoring. • According to district focus groups, as a part of Title I funding and programming, the district is using pre- and post-assessments to determine if tutoring is effective, as well as using these data to adjust the programs being used and the time being spent on tutoring in order to meet student needs. • In addition, district leaders stated that Title I programming included collecting data on early literacy and that these data showed results on the SPF as some elementary schools moved up performance levels. Both district and school leadership report that the investment and partnership with Schools Cubed to support early literacy has produced positive results. • According to district leaders, Title I funding and programming included collecting data on after-school data meetings among teachers to monitor participation and also administering surveys to collect information on the benefit of these data teams. Data teams were a practice highlighted for return on investment; district District Progress Monitoring Site Visit Feedback Form - 19 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 leaders stated they see improved practices and participation and that some surveys also show there is a benefit to data team meetings. © 2018 SchoolWorks, LLC. All rights reserved. • District leaders stated that they provide support to school leaders in putting together a plan to spend their Title I funds. Past practices indicated they were not spending the money or were not maximizing expenditures according to Title I guidelines. • District leaders also noted that Title IV funding was used to support Team Tipton, which is focused on bringing the Board and district senior leadership together for a retreat, on developing a process to establish mission and vision in the district, and on developing a strategic turnaround plan. • Additionally, district leaders reported the Empowering Action for School Improvement (EASI) competitive grant was being used to support school leaders in attending RELAY training, support school turnaround through increased community engagement, and provide consultants to support math instruction that included sideby-side coaching. • District leaders stated that the consultants hired at the high school level to help with systems and structures and leadership development in areas such as communication, climate and culture, staff and student attendance, and behavior systems did not have a positive impact. There have not been measureable improvements in attendance or decreases in behavior incidents. • District leaders reported that they are beginning to monitor initiatives for effectiveness in areas such as teacher retention, recruitment spending, and turnover rates; however, since these monitoring processes are new, trends have not been identified. District Progress Monitoring Site Visit Feedback Form - 20 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 • According to district leadership, BT is all funded through the general fund and its impact is not being strategically monitored as a whole. Both district and school leaders noted that data are being collected in monthly walkthroughs at BT schools, but they could not articulate whether the results of those walkthroughs were positive. While some schools have experienced increases in achievement, students, overall, are not demonstrating sufficient academic growth and achievement. © 2018 SchoolWorks, LLC. All rights reserved. • The district’s achievement and growth data have been inconsistent and below the State expectations over the past several years. The 2018 Preliminary District Performance Framework (DPF) reports that the district will enter Year 8 of Priority Improvement or Turnaround status. • The DPF showed slight decreases in the total points earned from 37.6% in 2017 to 36.8% in 2018. The summary of ratings by level indicated that the middle school level showed significant decreases in academic growth from the 73rd percentile in 2017 to the 54th percentile in 2018. • District leadership noted, and the Frameworks Release 2018 PowerPoint presentation confirmed, that although there were some improvements with individual schools in the district (one elementary school moved two levels from Priority Improvement in 2017 to Performance in 2018, and three schools moved up one level rating), the percentages of students performing at or above expectations continues to be below the State expectations. • District leaders stated that interim assessments were used (DIBELS/IDEL, STAR); however, they were unable to articulate the expected data points and whether students were progressing toward meeting expectations. District Progress Monitoring Site Visit Feedback Form - 21 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 SRP Evaluation based on Site Visit State Review Panel Criteria Claims & Evidence 6. There is necessity that the district remain in operation to serve students. 6.1: The district is mission-driven and its mission and vision meet a unique need. 6.2 There are no other viable options for enrolled students that will likely lead to better outcomes. • All stakeholders share an understanding of, and commitment to, the mission and vision. • District programs reflect the mission and vision. • The mission and vision guide decisions about teaching and learning. • The mission and vision meet the needs of an identified student population. • The district serves an isolated and/or remote community. • Closure or consolidation of district would have a significant negative impact on the community. • Comparison districts do not promote better student outcomes. [X] Yes [ ] No The district’s mission and vision are unclear to stakeholders. • When asked, district leaders indicated that mission and vision statements were created by former leaders and have not been revised recently; however, there is a plan to do so in the future. • Community members and parents in focus groups were not able to share the mission and vision of the district; however, some parents reported that, recently, they were a part of developing the vision at one of their children’s schools. • Teachers reported that the district’s vision is included in emails from district personnel but were unable to share their understanding of the mission or vision. • When asked what the district’s mission and vision were, stakeholder focus groups – including district leaders, school leaders, parents and community members – gave various responses: “The only goal I know of is to get out of turnaround status;” “Educate, Inspire, Empower;” “Ensure students are ready and prepared for college;” “We honestly don’t know;” and “Ensure all students succeed and get off the turnaround clock.” There are no other viable options to absorb the district’s approximately 7,500 students. • © 2018 SchoolWorks, LLC. All rights reserved. During focus groups, community members stated they want options for students (e.g., Science, Technology, Engineering, Mathematics [STEM}; innovation; Career Technical Education [CTE] paths; partnerships with universities). Parents also cited this desire and referred to other neighboring districts that have these options. District Progress Monitoring Site Visit Feedback Form - 22 State Review Panel District Progress Monitoring Site Visit Feedback Form 2018-19 • © 2018 SchoolWorks, LLC. All rights reserved. District leadership reported that the current management partner has provided some curricular materials for schools and a strong foundation for a teaching and learning cycle; however, they recognize it is not enough. District leaders, as well as Board members, expressed that a “true” management partner, or a couple of additional external partners, is what the district needs in order to establish and implement effective systems and structures. Some of the systems and structures that district leaders noted are needed include improved communication systems, consistent human resource processes, staffing at levels to support the turnaround work, and improved climate and culture. District Progress Monitoring Site Visit Feedback Form - 23 State Review Panel Document Review Feedback Form 2018-19 Purpose: To critically evaluate the district/school’s plan (i.e., Unified Improvement Plan) and performance. This report will be used as one element of a body of evidence to inform actions that may be undertaken by the Commissioner of Education and the State Board of Education. Reviewer Names: Starla Pearson, Amy Weed, Andy Franko, Tacy Killingworth Date: 9/25/2018 District Name/Code: Adams County 14/0030 School Name/Code: N/A SRP Summary Capacity Level: 1. The leadership is adequate to implement change to improve results. Developing 2. The infrastructure is adequate to support school improvement. Developing 3. There is readiness and apparent capacity of personnel to plan effectively and lead the implementation of appropriate action to improve student academic performance. 4. There is readiness and apparent capacity to engage productively with, and benefit from, the assistance provided by an external partner. 5. There is likelihood of positive returns on State investments of assistance and support to improve the performance within the current management structure and staffing. 6. There is necessity that the school/district remain in operation to serve students. Not Effective Developing Not Effective Yes Based on your professional judgment, will the plan result in dramatic enough change to pull the school/district off the accountability clock if it is implemented as written? [ ] Yes [ ] No [ X ] Not sure, more information is needed. Specify the additional information required. As noted below in the question area, there are several pieces of information needed to make an informed decision. Based on your professional judgment, what is your overall level of concern regarding this school/district’s ability to significantly improve results? Level of Concern: [ X ] High © 2018 SchoolWorks, LLC. All rights reserved. [ ] Moderate [ ] Low [ ] Cannot determine. Specify the additional information required. Document Review Feedback Form - 1 State Review Panel Document Review Feedback Form 2018-19 Overall Comments Adams County 14 School District serves a 33,000-resident community just North of downtown Denver and has been identified as one of the lowest performing districts in the State of Colorado. Adams County 14 includes two preschools, seven elementary schools, two middle schools, and two high schools, one of which is an alternative school. The district has 905 total staff members that includes 469 teachers, 391 support staff, and 45 administrators. The Colorado Department of Education (CDE) SchoolView dashboard indicates that the district serves a minority population of 89.3%, 87.3% of students qualify for free and reduced-price meals, 10.6% are students with disabilities, 54.9% are English Language Learner(ELL) students, and 4.4% of the students are identified as gifted and talented. Student enrollment numbers have increased over the years, from 6,991 in 2013-14 to 6,817 in 2017-18. The 2017 district Unified Improvement Plan (UIP) lists the current student enrollment at 7,467 students. Adams County 14 School District submitted their revised Pathways Proposal - External Management Partnership (PPEMP) plan to the State Board of Education on June 6, 2017. The 2017 District Performance Framework (DPF) shows that the district is rated Accredited with Priority Improvement and will enter Year 7 of Priority Improvement or Turnaround. According to the Pathways Proposal - External Management Partnership (PPEMP), nine of the eleven traditional schools in the district are on the accountability clock. In review of Adams County 14 District documents, the district is rated developing in three criteria and not effective in two criteria. Although the district has entered into a partnership with Beyond Textbooks (BT) and has developed an External Management Partnership Plan, there is limited data that shows this plan has been implemented at a level that has begun to improve the district’s effectiveness. As noted in each of the criteria below, the district is conveying a sense of urgency through establishing structures that address the priority performance challenges; however, there is limited data showing that these structures are moving the district forward. The areas of most concern are the district’s capacity to implement the plan, considering past trends of high turnover rates in leadership and teachers and the consistent low student achievement and growth across multiple years. Areas that should be explored more deeply through an on-site visit: • What does the 2018 DPF rating and the 2018 Colorado Measures of Academic Success (CMAS) data for the district and Adams High School show? • What are the specifics of the Theory of Action plan? • Considering past trends in leadership turnover, what was the retention rate for district leadership and school leadership to begin the 2018-19 school year? • How is the district responding to survey results from the Teaching and Learning Conditions in Colorado (TLCC)? • How is student achievement monitored throughout the school year, and what is the district’s role in monitoring? • What, to date, are the results of the implementation of Beyond Textbooks (BT)? • What specific outcomes were achieved at the focus schools within the BT implementation (ACHS, Central Rose Hill)? • What are the results of the turnaround strategies that all schools are going to implement? • What has been the impact of the ongoing meetings (every three weeks) between the superintendent and the BT schools? • What resulted in the district leadership’s Cultural Proficiency training? How many of those leaders are currently employed by the district? • What has been the result of the recruitment and retention efforts? • What is the status of the Chief Academic and Innovation Officer’s (CAIO) work with standards maps, common assessments, data-driven decisions, extended learning opportunities for students, and professional development? • How are progress monitoring targets developed and when do staff assess for adequate progress toward the end goals? • How are professional development opportunities for staff communicated and implemented? • Does the district intend to update the professional development video channel posted on the district website? • What successes/challenges do both building leaders and district leaders see in relation to the implementation of the PPEMP? • What role does the district play in supporting PLC work at the building level? © 2018 SchoolWorks, LLC. All rights reserved. Document Review Feedback Form - 2 State Review Panel Document Review Feedback Form 2018-19 SRP Evaluation Based on Unified Improvement Plan and Other Available Documents 1. The leadership is adequate to implement change to improve results. Capacity Level: [ ] Highly Effective [ ] Effective [X ] Developing [ ] Not Effective [ ] Unable to Assess State Review Panel Criteria 1.1: Leadership • Leadership communicates a relentless commitment to acts as a change the school/district turnaround. agent to drive • Leadership makes data-driven changes to the academic dramatic program and organization to promote dramatic achievement achievement gains. gains. • Leadership conveys clear expectations for performance for all stakeholders, including leadership, teachers, students, and partners. • Leaders distribute leadership responsibilities to appropriate individuals or groups. Look-Fors • Evidence of leadership driving impactful change. (Major Imp. Strategies & Action Plan) • The degree to which leadership has been continuous over time or change(s) in leadership utilized to activate change. (See HR data.) • • • • • • © 2018 SchoolWorks, LLC. All rights reserved. Evidence According to the Unified Improvement Plan (UIP), in 2016, Adams County 14 saw a change in 27 leadership positions; in 2017, there were 16 changes in district leadership. In addition, the district, in 2017, had 13 out of 25 (52%) school administrators new to their building. The Adams County School District 14 Pathways Proposal-External Management Partner (PPEMP) states that the current superintendent was hired in the summer of 2016 and is the third superintendent in five years. In addition to the hire of a new superintendent, the PPEMP states that in order to address the district’s leadership gaps and as a commitment to turnaround efforts, a Chief Academic and Innovation Officer (CAIO), a director of human resources, a manager of teacher effectiveness, a director of federal programs, and a manager of family and community engagement were all hired. According to the PPEMP, the district leadership team has committed to participating in two annual Beyond Textbooks (BT) events that promote effective district leadership practices. One training focuses on learning how to best implement and monitor the partnership plan and one focuses on leadership capacity in action. The district is driving impactful change through its commitment to partner with Beyond Textbooks (BT) and abide by all of the expectations encompassed in the PPEMP. The UIP shows the change is being driven around the identification of the following major improvement strategies: ensure that a consistent teaching and learning cycle is used across all schools; ensure that evidence-based practices are being used in classrooms; and establish and support the practice of data routines and reporting accountability. Document Review Feedback Form - 3 State Review Panel Document Review Feedback Form 2018-19 • • • 1.2: Leadership establishes clear, targeted and measurable goals designed to promote student performance. • Leadership communicates clear and focused goals that are understood by all staff. • Educators understand their responsibilities for achieving goals. • Leadership maintains school-/district-wide focus on achieving established goals. • Leadership allocates resources in alignment with goals and critical needs. • Leadership has established systems to measure and report interim results toward goals. © 2018 SchoolWorks, LLC. All rights reserved. • High, but realistic, goals are set. • Benchmarks are identified throughout the year. • Plan and narrative convey a sense of urgency. • Clear roles and ownership of action steps are identified. • The UIP also outlines key actions for each major improvement strategy, as well as designating who is responsible for each action. For example, for the major improvement strategy of ensuring a consistent teaching and learning cycle is used, some actions include teachers using a literacy curriculum aligned to the Colorado Academic Standards, and the District Assessment and Accountability Coordinator will support implementation of common district assessments. The PPEMP outlines specific roles and responsibilities for the district superintendent, district Beyond Textbooks (BT) liaison, the BT principals and coaches, and the BT CEO/Director. These responsibilities include: meeting with the Colorado Department of Education for discussion around the external management plan; meeting with both the local and State Board of Education to report progress; utilizing a Professional Learning Community (PLC) to build assessment and data literacy among teachers; compiling and reporting data based on the plan; and conducting classroom walkthroughs three times per semester. According to the PPEMP, the district chose to utilize a tiered level of support; the plan clearly outlines the expectations for both schools working with BT and all other schools in the district. The schools working with BT are required to implement the BT instructional framework, while all other schools are required to implement the outlined district turnaround strategies. According to the UIP, the Board of Education identified three strategic imperatives: dramatically improve academic performance and move the district from Priority Improvement to Accredited with Distinction by the year 2020; significantly empower families and community members to become engaged partners in their child’s education by creating welcoming environments district-wide that reflect and support a culturally diverse population; and intentionally increase and improve communication district-wide, both internally and externally, as well as enhance the resources offered to current clients and community members. Document Review Feedback Form - 4 State Review Panel Document Review Feedback Form 2018-19 • • • • 1.3: Leadership analyzes data to identify and address high priority challenges, and to adjust implementation of the action plan. • Leadership communicates data trends and issues, ensures timely access to data, and models and facilitates data use. • Leadership openly shares results and holds staff accountable for results and effective use of data. • Leadership first concentrates on a limited number of priorities to achieve early, visible wins. • There is regular progress monitoring of performance and implementation data and, as appropriate, results lead to elimination of tactics that do not work. © 2018 SchoolWorks, LLC. All rights reserved. • Evidence that goals are based on data re: past performance. (See data analysis/narrative) • Focus on a limited number of changes. (See Priority Perf. Challenges.) • • As mentioned above, the UIP identified three major improvement strategies; the plan is inclusive of benchmarks for each improvement strategy. For example, for the major improvement strategy of ensuring a consistent teaching and learning cycle is used across all schools, some benchmarks include providing professional development at specific times throughout the year, conducting monthly classroom observations, and having quarterly site visits by Beyond Textbooks who then provides a summary of next steps. The Progress Monitoring section of the UIP does establish specific annual performance goals for 2017-18; however, there are no specific targets established for 2018-19. The PPEMP identifies overall district goals and expected outcomes, as well as specific end-of-year outcomes and targets for the schools included in the partnership (Central Elementary, Rose Hill Elementary and Adams City High School). These outcomes are based on the Colorado Measure of Academic Success (CMAS) achievement and growth data, the Preliminary Scholastic Aptitude Test (PSAT), dropout rates for 9th and 10th grade students, Standardized Test for the Assessment of Reading (STAR) interim assessments in math, reading, and early literacy, gifted and talented and special education referrals, and attendance and truancy rates. Although clear outcomes are established in the PPEMP, based on past achievement, it is unclear if the goals are realistic. For example, when looking at CMAS English Language Arts percentile ranks, Central Elementary was at 4% in 2016, 5% in 2017, and the goal for 2018 is 15% and the goal for 2019 is 50%. Both the UIP and the PPEMP Appendix A show a strong use of data to determine current performance and to set specific goals. As mentioned above, there are annual performance targets established for 2017-18 in the UIP, but not for 2018-19; however, there are clear outcomes established for the participating schools in the PPEMP. In addition, the PPEMP has set clear annual accreditation targets for each school in the district based on prior years’ trend performance. The UIP indicates that the district is addressing three priority performance challenges to include: not meeting State expectations Document Review Feedback Form - 5 State Review Panel Document Review Feedback Form 2018-19 • Benchmarks are used to assess progress toward goals; goals are adjusted as progress is made. • Data on progress toward goals drives organizational and instructional decision making. • Resources are allocated for new programs or identified action steps. • Structures for sharing and using data are present. (See Action Plan.) • • • 1.4: Leadership establishes high expectations for student learning and behavior. • The school/district holds high expectations for academic learning. • Educators set high expectations for learning and clearly convey these to students. • Educators convey that students are responsible for raising their performance and encourage their participation in learning. © 2018 SchoolWorks, LLC. All rights reserved. • Elements of climate and culture are identified and addressed in the plan(s). (See Data Narrative, Root Cause Analysis, • in academic achievement and growth; not meeting State expectations in postsecondary and workforce indicators; and lack of student engagement as evidenced by low student attendance, graduation rates, and academic achievement. Although benchmarks are written in the UIP, most are written around specific actions such as: providing professional development; conducting meetings with staff and/or parents; or creating systems to collect data rather than to assess progress toward specific goals. In addition, there are clear expectations outlined in the PPEMP around conducting frequent progress monitoring within the instructional framework. However, there are no identified progress monitoring data points identified within the school year that would show progress toward goals and guide instructional practices. According to the PPEMP, resources are being provided based on the newly-identified actions and/or programs. For example, the district purchased a research-based reading program for K-5 aligned to the Colorado Academic Standards; also, a teacher and principal induction program are being developed to support retention of staff. The PPEMP outlines specific requirements for the superintendent and building leaders and coaches to review relevant student data; however, there are no specific details on the protocol or process used to analyze and use the data. Timeline requirements for sharing the data are: the superintendent will meet with the local Board of Education monthly; the superintendent will meet with the BT liaison weekly; the district BT liaison will meet with school principals and coaches weekly; the superintendent will meet with the BT liaison and the BT CEO/Director every three weeks; and the superintendent will meet with BT principals every other week. According to the PPEMP, there is some evidence that the district is focusing on both academic achievement and social well-being of all students. For example, in 2015-16 the district implemented a research-based biliteracy program in grades K-2, a Spanish language arts program in 6-8, and graduating high school students with the Adams 14 Seal of Biliteracy. Document Review Feedback Form - 6 State Review Panel Document Review Feedback Form 2018-19 • The school/district provides a safe environment to support students’ learning and, in the case of a virtual school, ensures that students’ interactions between and among themselves and school staff are respectful and supportive. • Leadership ensures that the school’s physical environment is clean, orderly, and safe. Action Plan, TELL data.) • • • © 2018 SchoolWorks, LLC. All rights reserved. In addition, the PPEMP shows evidence that the district holds high expectations for academic learning by ensuring every teacher is highly qualified to meet the needs of its culturally and linguistically diverse student populations by requiring every newly-hired teacher meet Colorado Department of Education’s (CDE) minimum Culturally and Linguistically Diverse (CLD) certified requirement. This is being offered through the University of Colorado Boulder BUENO Center for Multicultural Educations. Also, the PPEMP states that the district is committed to creating a culture of high expectations for all students by ensuring all employees are trained on effective cultural pedagogy. According to the 2018 Teaching and Learning Conditions Colorado (TLCC) Survey, 64% of staff believe that students in Adams County 14 know how they are expected to act in school, compared to 81% for the State results. In addition, 75% of staff believe Adams County 14 is a safe place for students to learn, compared to the State average of 91%. Document Review Feedback Form - 7 State Review Panel Document Review Feedback Form 2018-19 2. The infrastructure is adequate to support school improvement. Capacity Level: [ ] Highly Effective [ ] Effective [ X ] Developing [ ] Not Effective [ ] Unable to Assess State Review Panel Criteria 2.1: The district leads • The district/superintendent ensures ongoing intentional, strategic leadership development for emerging and current efforts to ensure the school leaders with a focus on building leadership effectiveness of the capacity to lead turnaround efforts and sustain academic program improvement. and the sustainability • The district/superintendent provides adequate of the organization. oversight in schools’ work to deliver the curriculum, monitors instruction on a regular basis, and provides adequate support and feedback to principals to improve instruction. • The district provides adequate systems by which to capture and store data, report it to schools, and make it accessible for instructional staff to utilize. Look-Fors • Evidence of district involvement. (See Data Narrative, Action Plan.) • • • • • © 2018 SchoolWorks, LLC. All rights reserved. Evidence The PPEMP states that the district piloted a new data dashboard system on a small scale during the 2016-17 school year to assist with monitoring behavior, academics, attendance, and truancy data, and planned for full implementation at schools in the 2017-18 school year. Additionally, the PPEMP states that academic and growth targets in math, literacy, language acquisition, and science were identified based on percentile cut scores identified by the Colorado School Performance Framework rubrics. The PPEMP indicates STAR math, reading, and early literacy is used as an interim assessment with end-of-year targets indicated for each of the three schools targeted for support with BT. Additionally, goals for the three schools include an increase in gifted and talented referrals, a decrease in students on READ plans, and a reduction of truancy rates; however, the report does not indicate how progress is measured throughout the school year. Appendix K of the PPEMP includes a draft of the 2017-18 School Year Instructional MAP Teaching and Assessment Cycles that describes daily/formative assessments, common assessments on a three-week cycle, and interim assessments; however, structures to support data analysis are not identified. Additionally, the report indicates that STAR serves as the interim assessment platform for the three identified focus schools yet end-of-year rather than benchmark goals are noted, and it remains unclear what the non-identified focus schools use as an interim assessment platform. Appendix K of the PPEMP provides a district-wide daily instructional sequence that includes 30 minutes of breakfast and character education through a morning meeting structure, 180 minutes of balanced literacy, 60 minutes of math with a breakdown for whole group and independent or small group instruction, 30 minutes each of integrated science and social studies, 30 minutes of extended learning, and 45 minutes of electives courses in art, music, and physical education. Document Review Feedback Form - 8 State Review Panel Document Review Feedback Form 2018-19 • • • 2.2: Leadership has a strong focus on recruiting and retaining talent; creates and implements systems to select, develop, and retain effective teachers and staff who can drive dramatic student • Leadership has created and/or implemented an organizational and staffing structure that will drive dramatic student gains. • Leadership recruits and hires teachers with commitment to, and competence in, the school’s philosophy, design, and instructional framework (e.g., trained and experienced with curriculum, certified/ licensed to teach, qualified in subject area). • Trained mentors provide beginning teachers with sustained, job-embedded induction. © 2018 SchoolWorks, LLC. All rights reserved. • Indication of strategic staff changes, particularly at the supervisory level, to support dramatic improvement efforts. (See HR data.) • Evidence of professional • • The PPEMP indicates that specific data cycles for academics, attendance, behavior, and health will be determined by the district and rolled out to the schools. For the schools who are using BT, these cycles will be jointly determined. The plan states the district will also provide protocols, supports, and professional development on how to interpret the data. In addition, the district will provide the action steps to implement the additional support with a focus on prevention. The PPEMP states that the superintendent and BT executives supervise a BT District Liaison who: reports to the superintendent and CAIO; serves as the point person between BT executives and the district regarding scheduled trainings, feedback and general communication; negotiates and maintains all associated terms of service agreements; maintains the relationship between BT schools and all district departments for school leadership to maintain focus on instruction; completes annual evaluations pertaining to BT implementation; monitors the implementation of BT through qualitative and quantitative reporting; manages policy and procedures for BT schools; and supports school-based coaches, teacher leaders, and individuals responsible for data monitoring to provide guidance and ensure fidelity to implementation, including professional development and early release days. A meeting structure table in the PPEMP indicates that the superintendent has a conference call every three weeks with the BT District Liaison and BT CEO/Director, and meets in person every three weeks with principals in schools managed by BT. The PPEMP states that one of the barriers to achievement and growth is a high turnover at the building/central leadership and teacher level. As noted above, to address the district’s leadership gaps, the following positions were hired: superintendent, CAIO, director of human resources, manager of teacher effectiveness, director of federal programs, and a manager of family and community involvement. Appendix C: Restructure of Adams 14 ESS Staff indicates a plan to retain the assistant director of special education to ensure the district has a CDE Lead Educational Authority (LEA) on site at all times, convert three current Teacher on Special Assignment (TOSA) positions to Document Review Feedback Form - 9 State Review Panel Document Review Feedback Form 2018-19 gains; evaluates all staff; and dismisses those who do not meet professional standards and expectations. • Leadership ensures the evaluation of all staff and dismisses those who do not meet standards and expectations. • Leadership provides teachers with active, intense, and sustained professional development (PD), including guidance on data analysis and instructional practice, aligned to school improvement efforts. o PD is informed by ongoing analysis of student performance, instructional data, and educators’ learning needs. o PD requires teachers to demonstrate their learned competency in a tangible and assessable way. o PD engages teachers in active learning (e.g., leading instruction, discussing with colleagues, observing others, developing assessments), and provides follow-up sessions and ongoing support for teachers’ continued learning. • The quality of professional development delivery is regularly monitored, evaluated, and improved. development activities aligned to priorities. (See Data Narrative, Action Plan, TELL data.) • • • • • • © 2018 SchoolWorks, LLC. All rights reserved. Special Education Manager positions (Preschool/Child Find, Autism and 3-4 schools, Significant Needs and 3-4 schools), and using a 5% increase in Title 6 Special Education Funds to create a behaviorfocused manager to support Affective Needs. According to the PPEMP, the BT District Liaison is responsible for ensuring accurate accounting and reporting of staff attendance for mandatory professional development sessions hosted and required by BT and reporting absences to the superintendent, CAIO, and building principal, as applicable, who then take steps according to appropriate Board policy relative to employees who fail to complete assigned duties. The PPEMP states that BT schools will follow the same evaluation system and process for the non-renewals of teachers and school leaders as outlined in local school policy and defined by State statute. Appendix L: Student Services Special Education Strategic Plan for 2016-2020 indicates a conversion of the Teacher on Special Assignment (TOSA) coordinators into peer-administrators, providing special education teams access to a highly-qualified special education coach, evaluator, and appropriate legal support. The PPEMP states that the CAIO is leading district work and implementation of standards-based instructional maps, common interim assessments, data-driven decisions, extended learning opportunities for students including tutoring and summer school, and professional development. Additionally, the PPEMP states that staff from human resources and teacher effectiveness are studying and developing a plan to recruit, develop, retain, and sustain talent. They have restructured the principal/teacher induction program and the mentoring/ mentee program. Professional development is targeted at defining expectations and job-embedded learning, including coaching, mentoring, and observation, as well as including peer observation. District representatives, as stated in the PPEMP, are attending job fairs across Colorado and New Mexico and recruiting at conferences such as CO-ALAS (Colorado Latino Administrators and Superintendents), La Cosecha (Bilingual and Biliteracy Conference Document Review Feedback Form - 10 State Review Panel Document Review Feedback Form 2018-19 • • • • • © 2018 SchoolWorks, LLC. All rights reserved. New Mexico), CABE (Colorado Association of Bilingual Educators) and various Colorado Universities. Additionally, the district has partnered with CU BUENO, a program that graduates Culturally and Linguistically Diverse Education (CLDE) certified and bilingual teachers, in recruiting efforts. Also, the PPEMP states the district has partnered with Metro State University and the University of Colorado Boulder in hosting preservice teachers for field experience and student teaching experience with the expectation that the district will be able to identify talented pre-service teachers and hire them. Further, in the PPEMP, the district identified actions to retain highly effective teachers that include providing tighter feedback loops using short cycle “look-fors” addressing aspects of the State teaching standards, providing teacher leadership opportunities for career advancement, and offering stipends for additional district leadership work. The PPEMP states that the district is committed to ensuring that every teacher is highly qualified to meet the needs of its culturally and linguistically diverse student population by requiring that every newlyhired teacher meet CDE’s minimum CLD Certified requirement. The district is assisting teachers with one-third of the tuition cost of becoming CDL-certified through its partnership with The University of Colorado Boulder Center. Further, the district conducted Cultural Proficiency training with district leadership, provided by the Colorado Department of Education (CDE). The district, according to the PPEMP, will provide teachers with professional development that ensures best first instruction in K-3 Reading Foundations, K-5 Systematic English Language Development, 6-12 Constructing Meaning, K-12 Content Language Objectives, and Culturally Linguistic Diversity, with all grades 6-12 teachers being certified in Constructing Meaning by the end of the 2017-18 school year. The district calendar indicates a total of six full days for professional development, with three of those occurring at the beginning of the Document Review Feedback Form - 11 State Review Panel Document Review Feedback Form 2018-19 • • • 2.3: Leadership ensures that the school/district has sound financial and operational systems and processes • Leadership ensures that the organizational structure supports essential functions; roles and responsibilities of all staff are clear. • Leadership has established effective means of communicating with staff. • Leadership ensures that the school/district meets all compliance requirements and deadlines set by the State, including the submission of school improvement plans, financial statements, school audit, calendar, and student attendance. • Leadership effectively manages the budget and cash flow, and there is a plan for long-term financial sustainability. • Leadership effectively manages operations (e.g., food services, transportation, school facilities). © 2018 SchoolWorks, LLC. All rights reserved. • Evidence of school communication with staff. (See Data Narrative, Target Setting, Action Plan, TELL data.) • Record of compliance. • • • school year, along with three early release days throughout the school year. A review of documents did not indicate how professional development is evaluated for its effectiveness or for expected implementation. Further, the 2018 TLCC District Survey data shows that only 28% of the responses were favorable to the item, “The effectiveness of the professional development is assessed regularly,” compared to 68% at the State. The PPEMP states the implementation of a teacher and leader induction model that: develops a train-the-trainer model with new professional development for induction facilitators; ensures every initial teacher, principal, and assistant principal receives professional induction support, professional development on formative assessments and authentic feedback, and perfecting Content Learning Objectives (CLOs). CLO’s will be one form of formative assessment and it will support lesson planning. Results from the 2018 TLCC survey indicate an agreement rate of 42% (compared to 68% State) in the area of Professional Development. Appendix I of the PPEMP includes the annual fee cost of management by BT for the 2017-18 school year as $12,500; Appendix J includes the annual fee cost for the 2018-19 school year as $39,000. The district’s website includes a link for Financial Transparency that leads to a Preliminary Budget report. The Preliminary Budget report states that the nutritional services program operates on a financially self-funded basis, and that with a free and reduced-price rate over 80%, the Board of Education believes that every student will be offered a free breakfast and lunch regardless of their free and reduced-price status. Additionally, the Preliminary Budget report indicates a 2018-19 Grant Fund Budget of $14.34 million that supports English Language Proficiency Act, READ Act, School Counselor Corps Grant Program, and School to Work Alliance Program, along with federal grant programs (Carl D. Perkins Career and Technical Education Act, Individuals with Disabilities Education Act, and Medicaid). Document Review Feedback Form - 12 State Review Panel Document Review Feedback Form 2018-19 • • • 2.4: Leadership provides effective instructional leadership. • Leaders ensure that the school implements a coherent, comprehensive, and aligned curriculum. o Leaders ensure that curriculum, instruction, and assessments are aligned with state standards, each other, and coordinated within/across grade levels. o Leaders ensure that instructional materials are selected and/or developed in accordance with a school-wide instructional framework and aligned with established curriculum standards. o Leaders ensure the curriculum is periodically reviewed and revisions are made accordingly. • Leaders provide meaningful feedback on teachers’ instructional planning and practice. o Leaders regularly provide meaningful feedback on instructional planning. o Leaders regularly observe instruction and provide meaningful, timely feedback that helps teachers improve their practice. • Leaders provide conditions that support a schoolwide data culture. o Teachers have easy access to varied, current, and accurate student and instructional data. o Teachers are provided time to collect, enter, analyze, represent student data & use tools to act on results. © 2018 SchoolWorks, LLC. All rights reserved. • Instructional needs and associated curricula and assessments are identified as a mechanism to address performance needs. (See Action Plan.) • Organizational routines are established that include ongoing data analysis to improve student learning. (Evidence of interim measures and how they will be used to monitor results.) (See Data Narrative, Target Setting, Action Plan, TELL data.) • • • The Preliminary Budget report designates Title funds as follows: Title I, Part A: Improving the Academic Achievement of the Disadvantaged; Title I, Part C: Education of Migratory Children the Migrant Education Program; Title II, Part A: Improving Teacher Quality; and Title III, Part A: Language Instruction for Limited English Proficient and Immigrant Student. According to the Preliminary Budget report, interscholastic athletics are supported through athletic fees of $35 per sport charged to high school students, as well as income generated from gate receipts at athletic events. Results from the 2018 TLCC Survey indicate an agreement rate of 65% (compared to 81% State) in the area of Facilities and Resources. The PPEMP indicates that BT provides support for effective instructional leadership with a robust 40,000+ item resource bank that includes lessons, resources, and documents that support instructional delivery. Resources in the bank are vetted carefully by a group of trained educational experts. These educators ensure standards alignment, higher order and critical thinking and questions as defined by Depth of Knowledge (DOK) and Bloom’s Taxonomy of Learning Domains and formatting for multiple programs teachers may utilize, such as PowerPoint. According to the PPEMP, district staff members conduct weekly school site visits, actively monitor BT implementation and UIP goals, and progress monitor using quarterly data in BT and non-BT schools; however, the plan did not indicate what data points are being monitored quarterly. Results from the 2018 TLCC Survey indicate an agreement rate of 69% (compared to 85% State) in the area of Instructional Practices and Support, and an agreement rate of 50% (compared to 81% State) in the area of District Supports. Document Review Feedback Form - 13 State Review Panel Document Review Feedback Form 2018-19 • 2.5: The school/ district provides highquality instruction. Leaders ensure that all teachers receive PD in data use (e.g., how to access, read, and interpret a range of data reports; frame questions for inquiry; analyze data, assessment literacy; use data tools and resources). n/a © 2018 SchoolWorks, LLC. All rights reserved. • Not possible to assess from Document Review alone. Document Review Feedback Form - 14 State Review Panel Document Review Feedback Form 2018-19 3. There is readiness and apparent capacity of personnel to plan effectively and lead the implementation of appropriate action to improve student academic performance. Capacity Level: [ ] Highly Effective [ ] Effective [ ] Developing [ X ] Not Effective [ ] Unable to Assess State Review Panel Criteria 3.1: Educators’ n/a mindsets and beliefs reflect shared commitments to students’ learning. 3.2: The school/ • Communications among all stakeholder groups district has established are constructive, supportive, and respectful. conditions that • Communications between leadership and staff support educators’ are fluid, frequent, and open. learning culture. • Leaders model and convey well-defined beliefs about teaching and learning, and convey value for innovation, learning from mistakes, and risktaking. • Leaders ensure that staff and team meeting discussions are structured and facilitated to support the staff’s reflective dialogue around data and instruction (e.g., attend to explicit group norms, use protocols). • Leaders provide guidance to teacher teams (e.g., help to establish meeting routines; model and promote use of discussion protocols; ensure systematic monitoring of student progress; create focus on linking results to instruction) and ensures that teachers utilize tools and time well. • Leaders participate in formal and informal professional learning, including their own leadership development about how to improve curriculum and instruction in a leadership context (i.e., elementary or secondary; high- or low-poverty; large/small schools). Look-Fors • Not possible to assess from Document Review alone. © 2018 SchoolWorks, LLC. All rights reserved. • Evidence of development for leaders. (See Action Plan.) • Structures for collaborative activities are present. (See Action Plan, TELL data.) • Roles are dedicated to supporting teams of teachers. (See Action Plan, TELL data.) • Communication structures are referenced. (See Data Narrative, Action Plan, TELL data.) Evidence • • • The UIP notes that an action step for major improvement strategy #3 (MIS) states that the District Assessment and Accountability Coordinator will complete quarterly data meetings to ensure building leaders are informed and supported with next steps for improvement planning efforts. Another action step states that the District Instructional Leadership Team will conduct monthly classroom visits with principals looking for instructional practices based on a districtdeveloped classroom visit protocol and that the results will be shared with school principals for continuous improvement and making district decisions on supports needed. The classroom visit protocol is not provided in the UIP or in the PPEMP. Another action step in the UIP under MIS #3 mentions that principals will meet bi-monthly to discuss data trends district-wide and use Illuminate and State data. The District Data and Assessment Coordinator and the Chief Academic Officer are listed as the key personnel. Further details of the data meetings are not provided in the plan. The UIP states that data are collected at the schools, but specific routines are not identified or in place to improve the quality of instruction and student learning, as a root cause. Another root cause mentions that a cycle of data inquiry has not been established within the district and that district and school leaders have not established shared goals or a process for monitoring goals. The district responded in Appendix K of the PPEMP that includes a draft of an Instructional Map of the Teaching and Assessment Cycles, which includes specific assessment cycles with common assessments and dates. Document Review Feedback Form - 15 State Review Panel Document Review Feedback Form 2018-19 • • • • 3.3: Educators collaborate regularly to learn about effective instruction and students’ progress. • Educators meet frequently during regularly scheduled, uninterrupted times (e.g., staff, department, grade level meeting times) to collaborate, establish improvement goals, and make data-informed instructional decisions. • Educators‘ collaborative meetings have a clear and persistent focus on improving student learning and achievement. • Educators describe sharing knowledge and expertise among colleagues as essential collaborative activity for job success. • Teachers are willing to talk about their own instructional practice, to actively pursue and accept feedback from colleagues, and to try new teaching strategies. • The school has created a performance-driven classroom culture in which teachers effectively use data to make decisions about daily instruction and the organization of students. © 2018 SchoolWorks, LLC. All rights reserved. • Collaborative meeting times and their purposes are referenced. (See Data Narrative, Action Plan, TELL data.) • • • The UIP mentions the district instructional leadership team as the group involved in the root cause analysis, but the plan does not indicate the specific team members. The UIP lists three Board of Education Strategic imperatives. Number three has a focus on communication structures and reads, “…to intentionally increase and improve communication district-wide, both internally and externally.” Specific details on this communication are not provided in the plan. An action step under MIS #3 in the UIP involves coaching the school’s instructional coaches in the areas of best instructional practices, having critical conversations and effective communication, and using the book, The Art of Coaching by Elena Aguilar. The 2018 TLCC District Survey results indicate: only 33% of the respondents feel that district leadership takes steps to solve problems; only 40% of the staff feel there is an atmosphere of trust and mutual respect between district and school administrators; and only 44% of the responses feel that the district clearly describes expectations for schools. The PPEMP mentions the three schools involved with the BT partnership will have teachers meet in Professional Learning Communities (PLCs) to review formative data from each instructional cycle in order to plan for targeted instruction. Further details about the frequency of the PLCs is not provided. The PPEMP states that district leadership, building leadership and BT executives will meet regularly in order to ensure BT implementation and review student progress. District staff will also make weekly school site visits and have quarterly data and progress monitoring discussions with the schools. In addition, school leadership will conduct daily classroom walkthroughs, monitor the goals of the UIP, and implement the BT framework with fidelity. Appendix K of the PPEMP states that teachers will administer interim assessments during a 3-week learning cycle, in order to monitor students’ mastery of the standards. Document Review Feedback Form - 16 State Review Panel Document Review Feedback Form 2018-19 3.4: The school/ district engages the community and families in support of students’ learning school improvement efforts. • The school includes parents/guardians in cultivating a culture of high expectations for students’ learning and their consistent support of students’ efforts. • The school/district invites family participation in school activities (e.g., volunteering in classrooms or on committees; attendance at performances, sports events, and organizational meetings) and regularly solicits their input. • The school/district offers workshops and other opportunities for parents/guardians to learn about home practices that support student learning. • Educators communicate with parents/guardians about instructional programs and students’ progress. • Strategies for community and family involvement are incorporated throughout the plan. • Parent Involvement Plan is present (for Title I Schools only) and details strategies for involving families to advance student learning. • The 2018 TLCC District Survey indicates that 57% of the respondents feel that schools provide opportunities to learn from other teachers. • The UIP states that the District Accountability and Advisory Committee (DAAC) is made up of three parents and is involved in the development of the UIP. DAAC members provide recommendations to the Superintendent and School Board members on the UIP, budget, safety, student achievement, parent engagement, and the School Performance Framework. The PPEMP mentions that the DAAC and the local Board of Education were engaged during the time feedback was collected about the management selection of Beyond Textbooks. It also states that the three schools selected to implement the BT program will host parent meetings to explicitly share details about how BT will support the school. A Superintendent’s Parent Forum held in October 2016 had a total of 45 parents and community members in attendance. The PPEMP states the superintendent launched six SuperintendentParent Forums and the district implemented its first parent volunteer to paraprofessional program, where 17 parents provide small group reading intervention to more than 70 students in grades K-3. The district’s website has information about Parent Resources that include School Health, 10 Tips for Parents, Helping Your Child, Adult Education, and the DAAC. The superintendent’s Welcome Back letter on the website is not current and dated August 2017. The UIP mentions that the Office of Family and Community Engagement (OFCE) measures the number of parents and families that are engaged in school-based family engagement activities, using a tracking system. The findings indicate that there are approximately 645 parents and community members that were a part of a family engagement tracker; the OFCE hosted over 20 district/school-based events during the 2016-17 school year; 417 parents/community members participated in 1-3 district/school events and 13 parents/ community members participated in 8 or more district/school events. The UIP states that the OFCE wants to increase parent/community engagement by building the capacity of parent leaders from the • • • • • © 2018 SchoolWorks, LLC. All rights reserved. Document Review Feedback Form - 17 State Review Panel Document Review Feedback Form 2018-19 • • • © 2018 SchoolWorks, LLC. All rights reserved. school Parent Teacher Organization (PTO), the Building Accountability and Advisory Committee (BAAC) and the DAAC. MIS #3 includes an action step that the OFCE department will increase parent engagement opportunities in the schools and district through the implementation of quarterly parent forums, and DAAC’s and BAAC’s that meet monthly. The action step is “In Progress.” The UIP states that a Parent Engagement and Satisfaction Survey was administered in April 2017, with nearly 6,000 surveys sent home to families in 8 different languages. The survey measures how well schools are doing in reaching out to parents and the quality of family/school relationships. The plan mentions that the survey is aligned to the national Parent Teacher Association (PTA) family and school partnership standards. Details about the results from the survey are not provided in the UIP because the response rate was low (28%) and the district did not feel that there were enough responses to accurately reflect parent satisfaction. The UIP section, Academic Achievement of Students with a Disability, lists an indicator to measure parent involvement in the individualized education plan (IEP) process, but further information is not provided as to why this is an indicator. The statement, “Adams 14 has increased the number of parent training opportunities that include social events and targeted training based on students’ needs, especially around significant needs, attention deficit and autism,” is mentioned, but additional details are not provided about the specific parent trainings. According to the 2018 TLCC District Survey, only 56% of responses are favorable regarding Community Support and Involvement in comparison to the Colorado favorable rate of 84%. In addition, only 40% of the responses are favorable to the statement, “The school’s efforts to engage families are effective.” However, 66% of the responses are favorable to the statement, “The community is supportive of the schools.” Document Review Feedback Form - 18 State Review Panel Document Review Feedback Form 2018-19 4. There is readiness and apparent capacity to engage productively with, and benefit from, the assistance provided by an external partner. State Review Panel Criteria • The school/district seeks expertise from external partners, as appropriate (i.e., for professional development, direct support for students). • The school/district ensures that roles and responsibilities of existing partners are clear. • There are designated personnel to coordinate and manage partnerships. 4.1: The school/ district collaborates effectively with existing external partners. Capacity Level: [ ] Highly Effective [ ] Effective [ X ] Developing [ ] Not Effective [ ] Unable to Assess Look-Fors • Articulation of roles/responsibilities with external entities (e.g., district level staff, BOCES staff, vendors, CDE) is evident. (See Action Plan.) • • • • • © 2018 SchoolWorks, LLC. All rights reserved. Evidence According to the PPEMP, Adams County 14 received feedback from the State Review Panel (SRP) in 2015 that suggested the district would likely need external support to meet State expectations. According to the PPEMP, Adams County 14 considered an innovation zone strategy, but elected to elicit the support of an external partner, Beyond Textbooks (BT) instead. The partnership process was proposed to the State Board of Education in April 2017 after consulting with advisors from the Colorado Department of Education. The PPEMP identifies four target operations of BT: 1) District professional development calendar; 2) District curriculum, assessments, and instructional support; 3) School day schedule/ reteach and enrichment block; and 4) District and school accountability - implementation, monitoring, and progress. The PPEMP identifies a reallocation of roles and responsibilities within Adams 14 to meet the objectives of the four target operations. The model of leadership includes a BT District Liaison who reports to the BT executives and district superintendent. The primary roles of the BT District Liaison are: 1) report to the Superintendent and Chief Academic Officer; 2) serve as the district point person between BT executives and Adams 14 regarding scheduled trainings, feedback and general communication; 3) negotiate and maintain all associated contracts; 4) maintain relationships between BT and the district; 5) complete an annual evaluation of BT and the implementation within Adams 14; 6) monitor implementation through qualitative and quantitative measures; 7) manage policy and procedures for BT schools; and 8) support school-based coaches, teacher leaders, and individuals responsible for monitoring data. According to the 2018 TLCC Survey, 50% of Adams 14 school leaders responded favorably with regard to their impression of district- Document Review Feedback Form - 19 State Review Panel Document Review Feedback Form 2018-19 provided supports to schools. This compares to a favorable response rate of 81% at the State level. 4.2: The school/ district leverages existing partnerships to support student learning. 4.3: Leadership is responsive to feedback. • The school/district maximizes existing partners’ efforts in support of improvement efforts. • All externally provided professional development is aligned to improvement efforts. • Leadership seeks feedback on improvement plans. • Leadership seeks feedback from key stakeholders • Leadership integrates feedback into future improvement efforts. • Activities of external entities align with major improvement strategies and performance needs of the school/district (not just a list of services the entity provides). (See Action Plan.) • Feedback from CDE on UIP is integrated into subsequent UIPs (i.e., feedback is not repeated for multiple years). (See CDE feedback, previous UIPs, updated UIPs, TELL data.) • • • • • • © 2018 SchoolWorks, LLC. All rights reserved. As noted above, the District UIP identifies three Major Improvement Strategies, and the PPEMP describes the specific actions to be taken to provide training and support to meet the objectives outlined in the MIS of the UIP. According to the PPEMP, BT will work to plan, support, and account for aligned improvement efforts. According to the 2018 TLCC Survey, 62% of respondents reported favorably to the notion that the district makes principal professional development a priority. The PPEMP indicates feedback provided by the State Review Panel in 2015 was used to identify how resources of people and funds will be used to strategically develop and implement the partnership with BT. According to the UIP, Adams County 14 consulted with CDE to develop a Theory of Action. This work was completed by prioritizing the action steps needed to realize the expected outcomes. According to the UIP, Adams County 14 invokes input from stakeholders through the District Accountability and Advisory Committee (DAAC) process. The UIP states the committee includes three parents of enrolled students, one teacher, one administrator, and one business owner who resides in the district. The meetings are open to the public, but the committee is responsible for providing recommendations to the Superintendent on the topics of school safety, budget, student achievement, parent engagement, and school performance frameworks. According to the 2018 TLCC Survey, 46% of respondents reported favorably that the district provides constructive feedback to school leadership to improve performance. Document Review Feedback Form - 20 State Review Panel Document Review Feedback Form 2018-19 5. There is likelihood of positive returns on State investments of assistance and support to improve the performance within the current management structure and staffing. Capacity Level: [ ] Highly Effective [ ] Effective [ ] Developing [ X ] Not Effective [ ] Unable to Assess State Review Panel Criteria 5.1: Leadership • Leadership identifies turnaround strategies monitors the return on and implements programs/initiatives investment of specific designed to improve student performance. improvement initiatives • Leadership assesses the cost and impact and uses that data to (effect on student achievement and number inform decisionof students served) of each program/initiative making. to determine its academic return on investment. • Leadership makes decisions regarding continuation or discontinuation of programs/initiatives based on this analysis. • Leadership establishes systems and structures to support regular and ongoing monitoring. Look-Fors • Additional resources provided through specialized grant funding are aligned, strategic and show evidence of positive results. (for districts/ schools that have received additional funds.) (See Action Plan.) • Programs and initiatives are designed to support turnaround efforts and have demonstrated results. • Leadership seeks resources aligned to its improvement efforts and programs/initiatives with high academic return on investment. • Any additional resources received (i.e., specialized grant funding) are aligned, strategic, and showing evidence of results. • Leadership treats resources flexibly and implements focused improvement efforts with a focus on early wins. • Evidence of the results of previous initiatives. (See Data Narrative, Target Setting, data dashboard, TELL data.) 5.2: Leadership has demonstrated an ability to produce positive returns on State investment and uses resources effectively. © 2018 SchoolWorks, LLC. All rights reserved. • • • • • Evidence The PPEMP states that district personnel used a SWOT (Strengths, Weaknesses, Opportunities, Threat) method to analyze schools and departments to determine their external partner and to determine which schools to support. According to the PPEMP, the district has identified specific turnaround strategies in the areas of: improving and sustaining student growth and achievement; using target supports to meet student needs through Multi-Tiered System of Supports (MTSS); developing, retaining, and sustaining highly-qualified school leaders and teachers; engaging all stakeholders (families, community and partners); and building a strong community intensely focused on student learning. According to the PPEMP, schools not supported by BT are receiving CDE Grants such as the Connect to Success Grant, the Turnaround Network Grant, and the Diagnostic Review Grant to support implementation of specific turnaround strategies. The PPEMP indicates that data used to select BT as an external partner resulted in their work in Vail, AZ where, in a four-year implementation period, the Vail School District climbed from less than 20% of students proficient in math to over 80% of students proficient in math (as measured by the annual Arizona summative assessment). Additionally, the document reports that the significant increases in student performance were not only sustained, but continued to increase and that Vail consistently outperforms the state by 20-to30 percentage points and remains in the top five highly rated districts in AZ. According to the 2017 District Performance Framework, Adams County 14 moved from Turnaround to Priority Improvement. As mentioned in a video on the Adams County 14 website, the superintendent reported that three schools (Alsup Elementary, Monaco Elementary, and Adams City Middle School) moved up one Document Review Feedback Form - 21 State Review Panel Document Review Feedback Form 2018-19 • • • 5.3: Students demonstrate academic progress over time. • Students demonstrate progress on internal measures linked with the school/district’s promotion or exit standards. • The performance of student subgroups on State assessments demonstrates that the school/district is making progress toward eliminating achievement gaps. • Students meet proficiency and grade-level targets across subjects and grade levels on © 2018 SchoolWorks, LLC. All rights reserved. • Achievement and growth data trend up. (See Data Narrative, Target Setting, data dashboard.) • Results of interim assessments show progress. (See Data Narrative, Target Setting.) • • level in the accreditation standings and one of the middle schools (Kearny) moved to Performance. According to the PPEMP, Pathway Grant funds were used to explicitly seek stakeholder input during Fall 2016 through Superintendent Forums with ongoing community and stakeholder communication regarding pathway implementation and progress monitoring, along with a plan to continue to flow through the semi-monthly Superintendent Forum meetings. Various meetings in Spring 2017 sought to provide context and further communication about the external management selection of BT. The DAAC and the local Board of Education were engaged during this time to communicate intentions and collect feedback. Additional meetings were held to ensure feedback loops were conducted with staff, as well as with the teachers’ union representatives. The PPEMP indicates that the Board of Education allocated nearly $500,000 in the 2015-16 school year to special technology projects for doubling the bandwidth and to add a wireless access point to every classroom in the entire district. The Board again allocated another $500,000 to purchase Chromebooks for a 1:1 initiative during the 2016-17 school year to support all district schools. The PPEMP states that, in response to the need for a common resource aligned to the Colorado Academic Standards and State assessments, the School Board allocated $1.2 million from reserves to purchase Benchmark Reading resources for grades K-5 in 2016-17 and allocated one-time funding of $1 million tax refund to Curriculum and Instruction to support the external management partnership with BT. The district’s achievement and growth data have been inconsistent and below State expectations over the past several years. The 2017 DPF shows that the district is on Year 7 of Priority Improvement or Turnaround. The ratings show that the district is Accredited with Turnaround status in 2010, 2011, and 2012; Accredited with Priority Improvement in 2013 and 2014; Accredited with Turnaround in 2016; and Accredited with Priority Improvement in 2017. The 2017 DPF summary shows that the district does not meet expectations in the academic achievement area at every level and in Document Review Feedback Form - 22 State Review Panel Document Review Feedback Form 2018-19 • • • • norm-referenced benchmark assessments and State assessments. Matched cohorts of students who score proficient or advanced (or equivalent) on State assessments maintain or improve performance levels across continuous enrollment years. The percentage of all students performing at proficient or advanced (or equivalent) on State assessments increases over time. Students demonstrate academic growth as measured by value-added or State growth percentile measures. Students demonstrate progress toward attaining expected knowledge and skills as measured by interim assessments. • • • © 2018 SchoolWorks, LLC. All rights reserved. every single student group category, meeting 0 out of 44 indicators. The DPF also shows that the district does not meet expectations in the postsecondary and workforce readiness area, where they met State expectations in 0 of 9 indicators. The district is approaching in the academic growth area at the elementary and high school levels, and meeting at the middle school level. The CDE District Dashboard shows the dropout rates of the district to be significantly higher than the State, with the district at 7.9% in 201617, compared to the State at 2.3%. The Dashboard also shows the district is well below the State expectation for 4-year graduation rates, with the district graduation rate at 65.6% in 2017-18, compared to the State at 79%. The CDE District Dashboard states that the district’s mobility rate has increased over the years, from 16.7% in 2013-14 to 19.2% in 2016-17; the overall attendance rates have been somewhat consistent, with a 90.8% in 2017-18, slightly below the State expectation of 92.5%. The UIP Prior Year Targets were not met in any of the three areas: 1. Every student will meet State and local expectations in achievement and growth in every content area across the district; 2. Every student will graduate within 7 years and will be prepared to succeed in postsecondary and/or workforce pathways; and 3. Every student will be at grade level across the district with no students having a significant reading deficiency. Document Review Feedback Form - 23 State Review Panel Document Review Feedback Form 2018-19 [ X ] Yes [ ] No [ ] Unable to Assess 6. There is necessity that the school/district remain in operation to serve students. State Review Panel Criteria 6.1: The school/ • All stakeholders share an understanding of, and district is missioncommitment to, the mission and vision. driven and its • School/district programs reflect the mission and mission and vision vision. meet a unique need. • The mission and vision guide decisions about teaching and learning. • The mission and vision meet the needs of an identified student population. Look-Fors • Population of students served is clearly identified. (See Data Narrative.) • Mission/vision are evident in plan or publicly available information. (See website, Data Narrative.) • • • • 6.2: There are no other viable options for enrolled students that will likely lead to better outcomes. • There are limited other school/district options available (e.g., online, charter, district). • The school serves an isolated and/or remote community. • Closure would have a significant negative impact on the community. • Comparison schools do not promote better student outcomes. © 2018 SchoolWorks, LLC. All rights reserved. • Number of other available district, online, or charter options and their performance. (See data dashboard, Websites.) • • Evidence According to the PPEMP, Adams County 14 is located North of downtown Denver in Commerce City and is the 26th largest school district. The district serves 7,500 students who speak more than 25 different languages. The student population is 88% students of color, 84% students qualifying for free and reduced-price lunch (FRL), and 49% who are English language learner (ELL) students. The PPEMP states the district engaged parents, community members, and school leadership in a Superintendent Parent Forum for a vision-building session. Both a mission of Inspire, Education, Empower and a vision were created. The vision states, “Adams 14 students, families, staff, community members, and partners will unite to ensure that our students will graduate with the Adams 14 Seal of Biliteracy, two years of college or an associate degree, an apprenticeship certificate or an alternative pathway while demonstrating critical thinking, communication, collaboration, creativity and cultural global competency skills in an ever-changing world.” The PPEMP states that the district believes that in order to achieve its vision, families and community need to be educated, engaged, and empowered. At the Superintendent Parent forums, the district received input from parents regarding what kind of schools they would like to see in the district. The district Turnaround Plan, which includes the mission and vision, is located on the district website. Closure of the district would have a negative impact on the community. Other school/district options are not available in the community. According to the State Review Panel Recommendation in June 2015, closure is not an option; schools across the district are already 6% above capacity. Document Review Feedback Form - 24 State Review Panel Document Review Feedback Form 2018-19 • Performance of neighboring districts. (See data dashboard, Websites.) • Performance of comparison schools. (See data dashboard, Websites.) © 2018 SchoolWorks, LLC. All rights reserved. Document Review Feedback Form - 25