September 20, 2017 Connecticut English Language Learner Pilot Program Final EVALUATION Report Shai Fuxman, EdD; Caroline E. Parker, EdD; Maria-Paz Avery, PhD Education Development Center, Inc. Education Development Center, Inc. edc.org Suggested citation: Fuxman, S., Parker, C. E., & Avery, M. P. (2017). Connecticut English Language Learner Pilot Program: Final Evaluation Report. Waltham, MA: Education Development Center, Inc. Copyright © 2017 by Education Development Center, Inc. EDC designs, implements, and evaluates programs to improve education, health, and economic opportunity worldwide. For more information, visit edc.org. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC i Executive Summary Section 294 of the 2015 Supplement to the Connecticut General Statutes provided funding to the State Department of Education (CSDE) to fund a CT English Learner (EL) Pilot Program to improve outcomes for EL students in four districts. The four districts, as determined in the legislation, were the three with the highest total number of EL students (Bridgeport, Hartford, and New Haven) and the district with the highest percentage of EL students (Windham). The districts were mandated to develop and implement language acquisition pilot programs for EL students that met three requirements: (1) be research-based; (2) be developed in consultation with the CSDE, public institutions of higher learning, or persons with expertise in language acquisition; and (3) take into consideration the various characteristics specific to the region, district, and population being served. The legislation provided funding for two academic years: 2015–2016 and 2016–2017. Furthermore, the legislation mandated that each district evaluate the impact of its pilot, and CSDE contracted with Education Development Center (EDC) to serve as independent evaluator for the pilot programs. Findings EDC found that the four districts, and the pilot overall, met and exceeded the legislation’s three requirements. Each district designed and implemented a unique pilot program aligned with the identified needs in their district. They selected interventions from outside vendors that had a research base, and then adapted those programs in innovative ways to meet the needs of students and teachers. Three of the four districts chose to focus their pilot on the needs of newcomer students (though across different grade spans), and all of the districts engaged general education and EL teachers in professional development opportunities. Despite limitations in the design of the pilot program and evaluation, teachers and pilot leads felt that the pilot improved instruction for students and enhanced the expertise of teachers. Overview of Pilot Programs Bridgeport Bridgeport worked with trainers from California-based Project GLAD (Guided Language Acquisition Design). In addition, Bridgeport developed a new districtwide differentiated EL curriculum, embedded Project GLAD strategies into the curriculum, and aligned the curriculum to Connecticut English Language Proficiency (CELP) standards. Lastly, the district developed and used a plan to monitor the implementation of Project GLAD strategies in the classroom that included student surveys and walkthrough protocols. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC ii Hartford Hartford focused on addressing the unique needs of new arrival students in the district’s only standalone middle school (grades 6–8) through a newcomer center. The district implemented Middlebury Interactive Languages (MIL), and provided training to provide teachers with instructional strategies that support newcomer EL students in participating actively in general studies classrooms. The district also sought to address newcomers’ socio-emotional needs related to the transition into U.S. school culture. New Haven New Haven implemented the pilot in one K-8 school, moving from a transitional bilingual model to a multidisciplinary, thematic-based dual-language model for grades K–3. In addition, the district implemented a new blended learning model using Imagine Learning software, as well as the Schoology platform for grades 4–6. This process included establishing a technology infrastructure, training teachers on the blended learning program and dual language/bi-literacy model, and creating a new multidisciplinary and thematic bi-literacy curriculum. Windham Windham worked to support the English proficiency and socio-emotional needs— particularly related to acculturation—of its grades 6–12 newcomer population. The district purchased a blended learning program called ELLoquence, which offers students and educators a wide range of levels of English. The district also provided multiple professional development opportunities to EL and general education teachers, as well as a summer school program for new arrivals and an event for parents of graduating seniors. The findings from the pilot led to the following recommendations for future programs similar to the EL pilot program: »» Maintain and increase flexibility in choosing the innovation. »» Provide greater flexibility in choosing external support partners. »» Provide adequate planning time and technical assistance to clarify the specific intervention and the expected outcomes from the intervention. »» Provide formal structures for pilot district participants to be part of a professional learning community in order to share challenges and successes throughout the life of the pilot. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC iii »» Design pilot programs to include follow up that builds on the results of the pilot, including sustainability plans. »» Provide greater clarity around evaluation and monitoring requirements for individual districts and for the program as a whole. »» Require greater involvement by superintendents. In addition, the pilot findings led to recommendations for improving EL education: »» Provide pilot districts with formal venues to share their innovations with other districts. »» Provide EL-related professional development to general education and subject teachers and to administrators. »» Continue to incorporate logic model or similar planning tools as part of district planning. »» Continue to invest resources in encouraging innovations to improve teaching for EL students, while also maintaining support for innovations that demonstrate success. Implications for Policy and Practice The CT EL pilot program provided four CT districts with resources to innovatively address the English language acquisition needs of their EL students. Each district identified key needs of a subgroup of EL students in their district, identified programs and strategies based on sound educational research, and implemented those strategies with the support of external partners. Each district was able to fulfill the requirements of the legislation, while also learning lessons from the pilot that will contribute to ongoing practices in the district. If innovative programs such as these are allowed to have more time—both at the beginning, to do careful needs assessment and development of logic models, plans, and evaluation designs, and at the end, to analyze evaluation results—the results will be even higher quality interventions with greater evidence of impact. Even more importantly, additional time will allow district leaders, teachers, parents, and even students to better understand the programs and incorporate them into ongoing practice long after the funding cycle has ended. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC iv Acknowledgments Education Development Center, Inc. (EDC), research staff would like to thank the Bilingual office at the Connecticut State Department of Education (CSDE) for providing funding and support for this study. The research team would also like to thank those who took time out of their very busy schedules to share their experiences. This includes teachers participating in the EL pilot program, pilot leads in each district, and representatives from each of the Regional Education Service Centers (RESCs) and English Language Learning Research Lab Central Connecticut State University (BELL-RL at CCSU). The research team is grateful for your time, honesty, and dedication to improving your programs. Acronyms ACES Area Cooperative Education Services BELL-RL at CCSU English Language Learning Research Lab at Central Connecticut State University CELP Connecticut English Language Proficiency Standards CES Cooperative Education Services CREC Capitol Region Education Council CSDE Connecticut State Department of Education EL English Learner ELD English Language Development ESL English as a Second Language GLAD Guided Language Acquisition Design LAS Links Language Assessment Systems MIL Middlebury Interactive Languages PD Professional Development RESC Regional Education Service Center TESOL Teacher of English to Speakers of Other Languages Connecticut English Language Learner PIlot Program Final Evaluation Report EDC v Contents Executive Summary........................................................................................................................................ i Acknowledgments....................................................................................................................................iv Acronyms....................................................................................................................................................iv Background........................................................................................................................................................ 1 Evaluation Methods....................................................................................................................................... 2 Sample.........................................................................................................................................................2 District leaders, RESC and BELL-RL at CCSU leaders................................................................. 3 Teachers.............................................................................................................................................. 3 Data Collection Methods..........................................................................................................................5 Interviews............................................................................................................................................5 Surveys.................................................................................................................................................5 Logic model workshop..................................................................................................................... 6 Data Analysis...............................................................................................................................................7 Findings............................................................................................................................................................. 8 Bridgeport: Effective Practices for High School Teachers.................................................................. 8 Hartford: Meeting the Needs of Middle School Newcomer Students........................................... 13 New Haven: Implementing a Bi-Literacy Thematic Approach to Dual Language Instruction K–8..................................................................................................................... 18 Windham: Supporting Newcomer Students in Grades 6–12.......................................................... 23 Discussion: Implications from the Four Districts.................................................................................. 27 Recommendations......................................................................................................................................... 31 Recommendations for Future Pilot Programs.................................................................................... 31 Recommendations for Next Steps to Continue to Improve EL Education in Connecticut........ 32 Conclusion...................................................................................................................................................... 34 References...................................................................................................................................................... 36 Appendix A: Baseline interview Protocols............................................................................................. 38 Appendix B: Final interview Protocols.................................................................................................... 44 Appendix C: Baseline Educator Survey.................................................................................................. 50 Appendix D: Final Educator Survey......................................................................................................... 59 Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 1 Background The 2015 Connecticut State bilingual legislation provided funding to the State Department of Education (CSDE) to fund pilot programs to improve outcomes for English learner (EL) students in four districts. The four districts, as determined in the legislation, were the three with the highest total number of EL students— Bridgeport, Hartford and New Haven—and the district with the highest percentage of EL students—Windham. The districts were mandated to develop and implement language acquisition pilot programs for EL students that met three requirements: (1) be research-based; (2) be developed in consultation with the CSDE, public institutions of higher learning or persons with expertise in language acquisition; and (3) take into consideration the various characteristics specific to the region, district and population being served. The legislation provided funding for two academic years: 2015–16 and 2016–17, and also called for an external evaluation of the program. The state implemented the CT EL pilot program beginning in 2015. The four districts were able to access their funds by late January 2016, halfway through the first academic year. As a result, most of the pilot programming did not begin until that time. Similarly, the external evaluator, Education Development Center, Inc. (EDC), also began its work after the pilot had begun. The evaluation began in August 2016, six months after the districts had begun the work. The evaluation included two components: first, EDC evaluated how well each district, and the pilot overall, met the three requirements laid out in the legislation. Second, EDC worked with the districts to help each of them identify how their chosen intervention would lead to desired outcomes, and how they planned to measure those outcomes through the development of a logic model for their pilot program. While each district designed its own pilot, they all followed a similar process. Each district identified an intervention that they determined was research-based, each worked with the Regional Education Service Centers (RESC) Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 2 serving their district and with the Connecticut Bilingualism and English Language Learning Research Lab Central Connecticut State University (BELL-RL at CCSU), each participated in two logic model workshops to help define their intervention, short and long-term outcomes, and plans for measuring success, each contributed to the EDC evaluation through interviews and surveys, and each implemented an intervention whose overall objective was to improve learning for EL students. Evaluation Methods This report summarizes the findings of the EDC evaluation. Specifically, the report describes the methods used to collect and analyze evaluation data, summarizes the main findings through case studies of each district, and proposes two sets of recommendations: one for the implementation of similar pilots in the future, and a second for improving instruction for EL students. As the external evaluator, EDC first met with the EL specialists at the CSDE to identify the priorities of the evaluation of the CT EL pilot program and the overall questions of interest. The following priorities were identified: (a) evaluate the degree to which each district, and the pilot overall, met the three requirements set out in the CT legislation; and (b) provide support to the districts through logic model workshops to help each district identify their pilot goals, how the pilot fit into the district context, and measurable outcomes that would be reasonable within the timeframe of the two-year pilot. To complete the first priority, EDC designed an evaluation that included collecting data from participating teachers and from district and RESC representatives working on the pilot. The second priority was completed by facilitating two workshops and working with each district on their own logic models. Sample Data were collected from the groups most actively involved in the pilot: pilot leaders and participating teachers, as well as leaders from each of the technical assistance centers. District leaders, RESC and BELL-RL at CCSU leaders Each district named an EL leader to be in charge of implementing the pilot, and each RESC also named a representative to work with the districts throughout the pilot. Data were collected from the pilot leads from the four districts (Bridgeport, Hartford, New Haven, and Windham) 1, representatives of the four RESCs assigned 1 The pilot leader from New Haven was unable to participate in the fall 2016 interview due to scheduling conflicts. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 3 to support each district—EASTCONN (Windham), Area Cooperative Education Services (ACES) (New Haven), Cooperative Education Services (CES) (Bridgeport), and Capitol Region Education Council (CREC) (Hartford), and one representative from BELL-RL at CCSU (Table 1). Table 1. Educators participating in fall 2016 and spring 2017 interviews District Organization Bridgeport Cooperative Education Services (CES) Hartford Capitol Region Education Council (CREC) New Haven Area Cooperative Education Services (ACES) Windham EASTCONN Connecticut Bilingualism and English Language Learning Research Lab Central Connecticut State University (BELL-RL at CCSU) Teachers Teachers were administered surveys in October 2016 and in May/June 2017. Surveys were administered anonymously and were not matched from fall to spring. Response rates varied from district to district, although they were fairly consistent from fall to spring (Table 2). In Bridgeport, Hartford and New Haven, the pilot targeted a specific program or gradespan and involved a discrete group of teachers, while in Windham the pilot targeted all teachers. Table 2. Number of educators responding to fall 2016 and spring 2017 surveys by district District Fall 2016 Survey Spring 2017 Survey Bridgeport (n=32) 12 8 Hartford (n=~24)a 14 11 10 16 45 59 New Haven (n=~20) Windham (n=~130) a a Total 81 a Estimated based on target school/program 94 The survey asked educators to provide information about their teaching areas and experience. The majority of educators who participated in both surveys were either general education teachers (39.5 percent and 44.7 percent) or ESL/bilingual teachers (28.4 percent and 33.0 percent) (Table 3). Of note is the large percentage of general education teachers who completed both surveys; while the pilot focused on English acquisition for students, each of the districts targeted general education teachers who work with those students, not only EL teachers. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 4 Table 3. Percentage of teacher respondents by characteristics Role Fall 2016 (n=82) Spring 2017 (n=94) General Studies/Subject Teacher 40 45 ESOL/Bilingual/Dual Language Teacher 28 33 Special Education Teacher 9 9 School Administrator 7 1 Tutor or Paraprofessional 5 2 Other 11 11 49 54 TESOL or Bilingual Certified (or working towards certification) Years of experience working with EL students prior to pilot launcha Number of EL-specific professional development instances prior to pilot launcha a No experience 3 na Less than 1 year 12 na 1–3 years of experience 20 na 4–7 years of experience 17 na More than 7 years of experience 49 na None 18 na One professional development instance 21 na Two to five professional development instances 18 na More than five professional development instances 44 na Only asked in fall 2016 While a similar percentage of the non-ESL or bilingual teachers had more than seven years’ experience working with ELs (46 percent vs. 50 percent), a larger percentage of them had less than one year or no experience (19 percent vs. 9 percent) (Figure 1). Similarly, only nine percent of ESL or bilingual teachers had never participated in an EL-focused in-service professional development compared to 24 percent of non-ESL or bilingual teachers. Thus, on multiple measures, many of the pilot educators had extensive experience working with EL students. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 5 Years experiences working with EL students prior to pilot Amount of EL-focused in-service professional development before pilot 60 60 50 50 40 40 30 30 20 20 10 10 0 0 None <1 1-3 4-7 >7 None One 2-5 % Non-ESL teachers (n=37) % Non-ESL teachers (n=37) % ESL/bilingual teachers (n=22) % ESL/bilingual teachers (n=23) >5 Figure 1. Years of experience working with EL students and previous professional development of ESL/bilingual and non-ESL teachers Data Collection Methods Data were collected from district, RESC and BELL-RL at CCSU representatives via interviews in fall 2016 and spring 2017 and from teachers via online surveys in fall 2016 and spring 2017. CSDE staff reviewed and provided input to the content of both the educator surveys and the interviews. Interviews Each interview was conducted by phone and lasted between 30 and 60 minutes. The fall 2016 interviews provided an opportunity for the district pilot leads to describe their pilot program, focusing on how the program planned to meet the three program requirements, and expected challenges and successes. The RESC and BELL-RL at CCSU contacts described how they were providing technical assistance to each district (see Appendix A for the fall 2016 interview protocol). The spring 2017 interviews focused on describing Year 2 implementation, including pilot successes and challenges, how each district met the three requirements, and future plans and recommendations (see Appendix B for the spring 2017 interview protocol). Surveys For each survey, EDC sent a link to the online survey to all four district leads, who were asked to send it to all educators in the district involved with the EL pilot program. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 6 The survey was administered to collect data on teachers’ views regarding their districts’ EL pilot programs, including (a) the extent to which teachers view their district pilot programs as being beneficial to them and to their students, and likely to succeed in meeting its goals; (b) the extent to which the pilot supports teachers who work with EL students through professional development and other support services; and (c) the extent to which the pilot programs increase teachers’ confidence in their ability to meet the needs of EL students. Because the first survey was administered in fall 2016 and teachers had been participating in the pilot program for some months, there was no baseline survey to record teacher views before participating in the pilot. Instead, both surveys were designed to “take the temperature” of participating teachers at two time points during the pilot. (See Appendices C and D for fall and spring surveys.) Logic model workshop In addition to implementing EL interventions with the pilot funding, state bilingual administrators wanted the districts to align their pilot interventions with other district initiatives and goals and identify ways to measure the success of the initiatives. To accomplish this, the state requested that EDC provide workshops on developing and using a logic model, a tool that provides a framework to connect a specific intervention to a district’s context and goals, identify measurable outcomes, and articulate a theory of change. EDC facilitated two logic model workshops for representatives from the four districts, RESCs and BELL-RL at CCSU. The first took place face-to-face on September 19th and was attended by three representatives from each district, representatives from three RESCs2 and the BELLRL at CCSU consultant. The goals of the workshop were to: introduce logic models as an effective tool for program and policy design, implementation, and evaluation; practice the elements of a logic model; and provide guidance on building a logic model for each district’s EL pilot program. The district teams began drafting their logic models at the workshop, completed them after the workshop, and submitted drafts to EDC two weeks later. EDC provided feedback on the models (e.g., helping the districts to define measurable outcomes) and sent them back to the districts to be finalized. The second (virtual) workshop was held two months later, and focused on developing a feasible evaluation plan using the logic model. The specific goals of this workshop were to: practice using logic models to develop evaluation questions and indicators of success, and to provide guidance on developing an evaluation 2 One RESC was not represented at the workshop. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 7 plan for each district’s EL pilot program. Each district was asked to submit an evaluation plan to EDC; EDC reviewed those evaluation plans, provided feedback, and sent back to the districts. The logic model workshops were not part of the original pilot legislation, but were added by the state bilingual leadership to help districts integrate the pilot into the rest of the district work. Oftentimes single interventions are implemented with limited funding and are not connected to other educational initiatives, are not evaluated, and disappear as soon as the funding ends. By providing the districts with logic model workshops and feedback on the logic models and research designs, the EL pilot program and EDC sought to help districts think about how the discrete pilot program formed part of a broader EL program. As noted above, the legislation designed the EL pilot to be implemented across two academic years, but districts began implementation four months into Year 1 and the EDC-led external evaluation began at the very end of Year 1. Logic model workshops are ideally implemented at the conceptualization stage of an intervention, when changes can be made to align planned activities with expected outputs and outcomes, but that was not possible in this case. Because the workshops took place after the districts had already chosen their intervention and begun implementation, the logic model development process described the components of the existing program and aligned them with expected outputs and outcomes, rather than describing components of a proposed program. It also meant that the evaluation plans developed after the second workshop had to be designed without including baseline data collection from either students or teachers. In essence, the districts were “building the plane as they were flying.” Despite this limitation, workshop leaders emphasized the iterative nature of logic models and recommended that each district identify ways in which the logic model development could contribute to the rest of the pilot and ways in which thinking with an evaluative lens can provide ways to extend the learnings from the pilot itself. Each district completed its logic model and information from each are included in the mini-case studies for each district. Thus, the logic models served as both tools for the district to complete their work and as data sources for the evaluation. Data Analysis Data from each district were analyzed and case summaries of each district were written. The case summaries describe the district context and the intervention, using data from the logic model to show how the pilot project was designed to improve instruction for EL students. Data from both the interviews and the survey Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 8 were used to describe the extent to which each district met the three requirements set out by the legislature and the perceptions of the participants regarding the success of the pilot in improving instruction for EL students. A cross-case analysis identified key issues common to all the districts, and the program-wide successes, challenges, and recommendations are addressed in the discussion section. Findings The following section includes four case studies that describe each district’s pilot program, which together make up the full CT EL pilot program. Districts are described in alphabetical order. meeting pilot requirements Each district was required to meet three conditions for the pilot: they had to choose an intervention that was based on research, they had to consult with external experts, and they had to adapt the pilot to meet the specific needs of students in their district. Bridgeport: Effective Practices for High School Teachers District description. Bridgeport is a mid-sized city with 21,191 students in its public school system as of the 2015–16 school year. Of these students, 48.8 percent are Hispanic/Latino, 35.2 percent are Black/African American, 12 percent are White, and 2.8 percent are Asian. Students of American Indian/Alaska Native or Pacific Islander descent, and students representing two or more races, account for the rest of the student population. EL students make up 14.1 percent of the total student body (CSDE, 2016a). Identified need. In considering the development of a pilot program, Bridgeport administrators and teachers identified the need to “accelerate English linguistic and academic skills in all subject areas in order to meet CELP and grade level standards” (Bridgeport logic model, 2016). The need was particularly acute for high school-age students, many of whom enter the system with interrupted schooling. This decision was based, in part, on the results of a needs assessment conducted with teachers prior to the pilot, and observations of an influx of newcomer students in the district. The district decided to focus on high school-age newcomers, as younger students have more time to acclimate to U.S. school culture and because lower grades have more EL resources available. The pilot focused on providing high school transitional bilingual education and general education teachers with professional development on effective instructional strategies to increase English language acquisition and literacy skills. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 9 Intervention. The district chose to implement Project GLAD (Guided Language Acquisition Design), a K–12 instructional model originally developed in California and now used in 13 states. Project GLAD consists of a core of 35 strategies that can be integrated into content area curricula taught in a general education classroom. Strategies are designed to develop academic English and to build the vocabulary and linguistic skills needed to participate in content-based discourse, foundational skills that are especially important for EL students. Each strategy also includes ways to differentiate instruction to meet the needs of a heterogeneous classroom that includes EL students at different levels of English proficiency, as well as native English speakers. Intervention goals and implementation: As stated in its logic model, the district identified the following three major goals for the pilot: (1) participating teachers will implement a minimum of one Project GLAD strategy per class, per week, starting in September 2016; (2) teachers will create a districtwide differentiated curriculum for beginning, intermediate, and advanced levels of ESL aligned with CELP standards, second language acquisition theories, and cross-cultural skills; and (3) the district will develop and implement a plan to monitor the implementation of Project GLAD and its impact on student achievement. GLAD trainers provided professional development to 32 high school transitional bilingual and EL teachers on Project GLAD strategies. In addition, four teachers received additional training to become certified as Project GLAD trainers. The professional development included both out-of-class professional development workshops and in-class role modeling with students. GLAD trainers worked with participating teachers in their classrooms and provided feedback on their use of the strategies. In addition, several GLAD trained teachers worked with the local RESC, Cooperative Education Services (CES) to develop a new districtwide differentiated ESL curriculum that embedded Project GLAD strategies and aligned with the CELP standards. CES also supported the development of the monitoring plan and participated in monitoring activities. Intervention evidence base: Project GLAD literature states that the strategies included in the instructional model are based on research related to second language acquisition, brain development, and effective literacy practices for second language learners. In a brief search of research on Project GLAD, we were able to identify only one independent study of the effectiveness of the program conducted by Education Northwest. After one year of implementation, study findings indicated that 5th grade EL students in Project GLAD science classrooms Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 10 performed better in vocabulary, reading comprehension, and in two aspects of their essay writing (ideas and organization) than EL students in control classrooms; no differences were found in science. The study findings also suggested that while Project GLAD benefits EL students, it does not hold back their non-EL peers in learning the content (Deussen, 2014). Inclusion of external expertise: The CES and BELL-RL at CCSU provided support to each of the four districts. In Bridgeport, CES staff took an active role in several aspects of the pilot. The CES representative reported regular communications with the district lead and participating teachers to support the implementation of Project GLAD. CES staff attended Project GLAD trainings, visited classrooms, conducted classroom observations, and met with teachers once every two months after school to discuss their progress in implementing Project GLAD strategies. CES also supported teachers developing an ESL/ELD curriculum with embedded Project GLAD strategies. In addition, CES supported the monitoring of Project GLAD implementation by collecting and interpreting data including data from student portfolios, designing district-specific study surveys, and classroom observations. CES reported that throughout the pilot, teachers were very excited to work on the Project GLAD strategies and appreciated the support from CES in helping them implement these strategies. BELL-RL at CCSU provided Bridgeport with professional development workshops related to two topics—working with diverse populations (e.g., refugees, cultural sensitivity), and using the universal backward design model for curriculum writing. The latter helped the district in their efforts to develop the new ESL curriculum. Of the Bridgeport teachers who completed the survey, almost all described receiving support from both the district and CES (Table 4), though a smaller percentage (63 percent) described receiving support from BELL-RL at CCSU in the spring 2017 survey. Table 4. Percentage of Bridgeport teachers somewhat or strongly agreeing with statement Fall 2016 (n=11) Spring 2017 (n=8) I feel that I have adequate support from the school administration and the district to do my part of the pilot program 100 100 I have received support, including but not exclusively professional development training, from our local RESC 100 88 I have received support, including but not exclusively professional development training, from CCSU 100 63 Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 11 Adaptation to district context: As described above, the Bridgeport team administered a needs assessment and considered the changing characteristics of its EL student population in its decision to focus on high school newcomer students. The majority of teachers in both the fall and the spring felt that the pilot goals were realistic, relevant and achievable (Table 5). Table 5. Percentage of Bridgeport teachers somewhat or strongly agreeing with statement Fall 2016 (n=11) Spring 2017 (n=8) I am aware of the goals of the pilot program 100 88 The goals of the pilot program are realistic 91 86 The goals of the pilot program are relevant to our students’ needs 91 71 The goals of the pilot program are achievable, given the timeframe and resources available to us 91 63 The pilot program addresses an important need in the district 91 75 Successes of the Bridgeport pilot. Bridgeport met its three goals for the pilot as outlined in its logic model. Implementation of Project GLAD strategies: The Bridgeport project lead reported in the spring 2017 interview that outside consultants (not with Project GLAD) observed the participating high school teachers and found that 80 percent used at least five different Project GLAD strategies, exceeding the district’s first goal that teachers implement a minimum of one strategy per class per week. The outside observers also reported that teachers seemed to enjoy and benefit from the hands-on/in-classroom approach to professional development that Project GLAD offered. The RESC representative noted that all teachers they observed demonstrated much enthusiasm for the new strategies they learned. The teacher surveys confirmed this perception (Table 6); teachers in the fall and the spring said that both they and their students benefited. Table 6. Percentage of Bridgeport teachers somewhat or strongly agreeing with statement Fall 2016 (n=11) Spring 2017 (n=8) I feel that I have benefited professionally from my involvement in the pilot program 100 86 I feel that my EL students are benefiting from the pilot program 82 80 Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 12 Development of a district-wide high school differentiated ESL/ELD curriculum: The district also met its second goal, the development of a districtwide differentiated curriculum for EL students. The curriculum was completed and it is available for teachers to use. Development of a monitoring plan: The district developed a plan to monitor the implementation of the Project GLAD instructional model and its impact on student achievement. The monitoring plan included a walkthrough of protocols to observe implementation of the Project GLAD strategies in the classroom and student surveys to assess change in student interest in academic content as a result of Project GLAD strategies. As another way of examining pilot success, teachers were asked to identify the success criteria they felt were most important, and then to evaluate whether those criteria were being met by the pilot. Bridgeport teachers identified three criteria related to students and one related to teachers (Table 7). Table 7. Percentage of Bridgeport teachers identifying success criteria. Fall 2016 (n=11) Spring 2017 (n=8) EL students access academic content in English 64 88 Teachers working with EL students are better trained to meet the needs of their EL students 91 75 EL students acquire basic English language skills 73 75 EL students’ social and emotional needs are met so that they are able to learn 64 75 One hundred percent of teachers in the fall, and 88 percent of those in the spring, felt that the pilot was somewhat or very likely to meet those criteria. The percentage of teachers who felt very confident that they could meet the academic needs of their EL students increased from 64 percent in the fall to 88 percent in the spring. Finally, while not a requirement of the pilot, Bridgeport incorporated an important sustainability strategy into its work to support the implementation of Project GLAD districtwide. The four district teachers who were certified as Project GLAD trainers created an in-district resource to train additional teachers on the use of Project GLAD strategies after the pilot. However, the district lead expressed concern that without some level of continued funding, the district would not be able to implement Project GLAD with the same level of fidelity. Funding would be needed for continued professional development not only through workshops but also for in-classroom support, which had been found to be very valuable, and to support external monitoring, in particular the observations by independent consultants. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 13 Hartford: Meeting the Needs of Middle School Newcomer Students District description. Hartford is a mid-sized city with 21,463 students enrolled in its public schools as of the 2015–16 school year. Of these students, 52.5 percent are Hispanic/Latino, 30.6 percent are Black/African American, 11.1 percent are White, and 3.4 percent are Asian/Pacific Islanders. Students of American Indian/Alaska Native and other ethnicities account for the rest. Of these students, EL students represent 17.9 percent (CSDE 2016b). Identified need. Hartford administrators and teachers identified as their priority the unique needs of new arrival students in the district’s only standalone middle school (grades 6–8). Prior to the pilot, Hartford had noted an increase in the number of newcomer students. Hartford had previously offered a newcomer program but had discontinued it because of a drop in the number of newcomer students. The pilot provided an opportunity to address the recent influx of newcomers by creating a newcomer center at the middle school to meet the language, academic, and social-emotional needs of students as they transitioned into U.S. school culture. In particular, the district identified acculturation as an important ingredient for academic success. Interventions. To address the multi-faceted needs of new arrivals, Hartford decided to implement several initiatives—a blended learning program for EL students developed by Middlebury Interactive Languages (MIL), co-teaching in social studies and science, and strategies to create a welcoming environment. The blended learning program is based on MIL’s digital curriculum for EL students and was customized for Hartford’s focus on newcomer students. The instructional modules address the fundamentals of academic English and literacy development through projects in different content areas. Hartford focused on science and social studies. The modules are also aligned with Common Core Grade level standards and correlate with ELPA 21 levels, featuring age appropriate themes. Through this program, each student develops their own success plan with specific action steps and goals for the year. Intervention goals and implementation: The district identified several goals as part of their logic model. These goals included: (1) creation of a newcomer center designed to make newcomer students feel welcome and lead them to academic success; (2) development of individualized student success plans focused on their English language proficiency; (3) development and implementation of a coteaching model for science and social studies as a way of supporting students’ ability to transfer content and literacy skills from their first language to English; (4) implementation of a blended learning curriculum to support the development of English proficiency and literacy skills in specific content areas such as science Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 14 and social studies; (5) provision of wrap-around services for families, including ESL classes for parents, and matching families with services to support their unique needs; and (6) customized experiences where newcomers could utilize their background and culture as an asset to transition to the new educational environment. Hartford trained 24 teachers at the middle school on instructional strategies that support the active participation of newcomer students in social studies and science classrooms as well as on the co-teaching model between the general education teacher and the ESOL teacher. In addition, the district offered professional development opportunities on creating a welcoming environment for new arrivals and offering high- interest extra-curricular activities familiar to students. Lastly, MIL provided professional development on the use of the blended learning curriculum. Intervention evidence base: Hartford used two programs in its pilot, each of which has an evidnece base. Digital curriculum for English learners: Middlebury Interactive Languages offers digital language learning programs for K–12 students. The curriculum is based on research in second language acquisition and cognitive psychology, in particular findings that support authentic, comprehensible learning strategies and participation in frequent opportunities to interact purposefully in the target language. The developers provide a detailed description of the research base for the different instructional practices featured in the modules (Germain-Rutherford & Martinez-Lage, 2012). Co-teaching: Co-teaching as an instructional delivery strategy for EL students in mainstream classrooms borrows from successes in mainstreaming students with disabilities. While there is still a call for more research in the realm of special education to substantiate its effectiveness (Murawski & Swanson, 2001), leading experts in both special education and the education of EL students have begun to explore the transferability of this model and related techniques to mainstream classrooms with EL students (Honigsfeld, 2008; Neely, 2016). Inclusion of external expertise: CREC staff supported the development of the pilot by collecting data on the needs of the EL population, including newcomers, and on teachers’ instructional strategies for EL students and the needs they identified to improve their instruction. CREC continued data collection throughout the pilot to monitor progress, identify areas for improvement, and to support teachers’ professional development needs. CREC staff also coached teachers on the Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 15 implementation of instructional strategies and worked with the district’s curriculum department on the development of the new blended curriculum. BELL-RL at CCSU provided some of the professional development workshops on instructional strategies that supported newcomer students. The majority of Hartford teachers who completed the survey describe feeling that they had not received extensive support from their district, from CREC or from BELL-RL at CCSU (Table 8). The evaluation was not able to explore in greater depth to understand the context for these results, but one possible explanation for this is that as respondents became more immersed in the pilot, they realized how much more support they needed to fully implement the interventions. In addition, in the case of CREC, teachers likely indicated that they did not receive support from CREC because their role focused mostly on data collection rather than directly supporting teachers. Table 8. Percentage of Hartford teachers somewhat or strongly agreeing with statement Fall 2016 (n=13) Spring 2017 (n=9) I feel that I have adequate support from the school administration and the district to do my part of the pilot program 15 11 I have received support, including but not exclusively professional development training, from our local RESC 31 22 I have received support, including but not exclusively professional development training, from CCSU 38 22 Adaptation to district context: As described above, Hartford identified the needs of middle school newcomer students and developed a pilot to respond to that need. While the majority of teachers who completed the survey in fall 2016 did not agree that the pilot programs were realistic, relevant or achievable (Table 9), by spring 2017, a majority did feel that the goals were realistic, relevant and achievable, and 100 percent felt that it addressed an important need in the district. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 16 Table 9. Percentage of Hartford teachers somewhat or strongly agreeing with statement Fall 2016 (n=13) Spring 2017 (n=9) I am aware of the goals of the pilot program 31 78 The goals of the pilot program are realistic 38 67 The goals of the pilot program are relevant to our students’ needs 46 67 The goals of the pilot program are achievable, given the timeframe and resources available to us 38 56 The pilot program addresses an important need in the district 54 100 Successes of the Hartford pilot. From the pilot lead’s perspective, the pilot was successful in achieving most of the goals set out in the logic model. The pilot lead noted the importance of the school principal’s leadership and commitment to the pilot in achieving these goals (although this was not corroborated by the results of the teacher survey in Table 8), while the CREC representative spoke of the productive communication with CREC and the district. Setting up newcomer center. The newcomer center was established at the middle school during the pilot program’s second year. The district used some pilot funding to hire three educators with experience working with newcomer students. In the spring 2017 survey, teachers commented that the additional staff members were a great asset to the district and helped meet student needs. Professional development on co-teaching and meeting newcomer needs. Professional development was offered to both general education and ESOL teachers on the use of a co-teaching model of instruction and on strategies to best meet the academic and socio-emotional needs of newcomer students. The pilot lead observed that many of the general education teachers who participated in the professional development opportunities gained a new mindset regarding the experiences of newcomer students, as well as what newcomer students need to succeed in this new environment. The co-teaching model will continue at the middle school and the district hopes to expand the model to other schools. Such an expansion, however, will require additional professional development and it is unclear how many professional development opportunities the district will be able to offer in coming years. As shown in Table 10, a majority of teachers who responded to the survey said that they feel their EL students benefited from the pilot, but only 38 percent in the fall and 22 percent in the spring felt that they benefited professionally. This aligns with other survey results that show that the Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 17 teachers felt the pilot was addressing the appropriate needs of students, but did not necessarily provide teachers they support they felt was needed. Table 10. Percentage of Hartford teachers somewhat or strongly agreeing with statement Fall 2016 (n=13) Spring 2017 (n=9) I feel that I have benefited professionally from my involvement in the pilot program 38 22 I feel that my EL students are benefiting from the pilot program 54 89 Parent and family engagement. The district was successful in reaching out to parents through a parent survey, administered by CREC, as well as through parent events. Parent engagement efforts succeeded in opening lines of communication between parents and the school. As another way of examining pilot success, teachers were asked in the survey to identify the success criteria they felt were most important, and then to evaluate whether those criteria were being met by the pilot. In both the fall and the spring, the most commonly chosen success criteria was that parents of EL students would be more engaged (Table 11). This aligns with the third goal of the pilot logic model, and is particularly encouraging given that parent engagement can often be an afterthought in interventions. Also of note is that while EL student outcomes were identified in both the fall and spring, teacher outcomes were identified by a higher percentage in the spring than in the fall. This may reflect a changed understanding of the importance of teacher growth as a step to student achievement. Table 11. Percentage of Hartford teachers identifying success criteria Fall 2016 (n=13) Spring 2017 (n=9) Parents of EL students are more engaged with the school and their children’s academic work 85 89 EL students acquire basic English language skills 85 78 EL students access academic content in English 77 78 EL students’ social and emotional needs are met so that they are able to learn 85 78 Teachers working with EL students are better trained to meet the needs of their EL students 62 78 Teachers working with EL students are implementing effective strategies for working with EL students 69 78 Teachers working with EL students feel more confident in their abilities to meet the unique academic needs of EL students 69 78 Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 18 Sixty-nine percent of the teachers in the fall, and 100 percent of those in the spring, felt that the pilot was somewhat or very likely to meet those criteria. Interestingly, the percentage of teachers who felt very confident that they could meet the academic needs of their EL students decreased from 31 percent in the fall to 22 percent in the spring. Again, this may reflect a growing understanding of the skills needed to work successfully with EL students. New Haven: Implementing a Bi-Literacy Thematic Approach to Dual Language Instruction K–8 District description. New Haven is a mid-sized city with 21,752 students in its public school system as of the 2015–16 school year. Of these students, 41.8 percent are Hispanic/Latino, 40.4 percent are Black/African American, 14.3 percent are White, and 2.4 percent are Asian/Pacific Islanders. American Indian/Alaska Native and other ethnicities account for the rest. Of these students, EL students represent 14.6 percent of the student population (CSDE, 2016c). Identified need. In New Haven, the district identified the need to focus support on the K–8 school with the largest concentration of EL students, many of whom are also newcomers. The district had noted that the instructional model that was currently in place at that school, a transitional bilingual educational model, was not meeting the expected annual growth targets for its EL students. Interventions. The district chose to transition into a dual-language model in grades K–3, implement a blended learning model using Imagine Learning and Khan Academy in grades 4–8, and use the Schoology platform—a learning management system for creating engaging content, designing lessons, and assessing student understanding. Intervention goals and implementation: The district identified two program goals for the pilot in their logic model. These were to implement a dual language, biliteracy model, and a blended learning instructional program. It also identified the following outcome goals: (1) students will show growth in English language proficiency levels as measured by LAS Links, the English proficiency assessment used in Connecticut; (2) teachers participating in professional development related to the pilot will rate their experiences positively on a survey; and (3) participating teachers will report increased capacity to serve the needs of their EL students. To meet the goals, the school, with the support of the district and ACES, provided targeted professional development in the dual language, bi-literacy model and the use of the blended learning program, including the integration of supplementary Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 19 instructional strategies for classroom-based and self-directed learning. To support the move to a dual language, bi-literacy model, the district worked with K–3 teachers in creating a new interdisciplinary, thematic bi-literacy curriculum aligned with CELP standards. To support the implementation of the blended learning program, the school invested in setting up the necessary technology infrastructure and provided professional development on the effective use of the supplementary digital resources. In addition, all K–8 teachers were offered professional development on the needs of EL students who were also identified as special education students. These professional development opportunities highlighted the differentiated needs of EL students. Intervention evidence base: New Haven used multiple programs in its pilot, each of which has an evidence base. Dual language programs: The research on dual language programs supports its effectiveness in the development of language proficiency in the two target languages, increased cognitive development, and achievement in content areas when programs are strongly structured and well-implemented (Jacoby & Lesaux, 2017; Kuo, Ramirez, de Marin, Kim & Unal-Gezer, 2017). Imagine Learning: Imagine Learning draws from current research on effective instructional practices to build its language and literacy curriculum. In addition, it uses a strategic, differentiated approach to create an individualized learning path for each student that can be continually adjusted to meet a student’s changing needs. It also provides enhanced first-language support in fifteen languages. Khan Academy: Khan Academy is a free online supplemental resource offering practice exercises, instructional videos, and a personalized learning dashboard, in areas such as math, science, history, and other subject areas. Researchers have examined the use of Khan Academy resources in K–12 settings Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 20 to supplement classroom instruction. The focus of this research, however, has been on the different ways in which teachers have used the resources and not on learning outcomes. However, the research does show some positive associations between use of the resources and student self-reported improvement in nonachievement outcomes such as math anxiety, math self-concept, and academic self-efficacy (Murphy, Gallagher, Krumm, Mislevy & Hafter, 2014). Schoology: Schoology is a learning and assessment management system designed to coordinate standards-based assessments with the standards used to develop curriculum. Inclusion of external expertise: At the beginning of the pilot, New Haven brought in BELL-RL at CCSU to consult on the needs of the district and to plan customized professional development sessions to support specific aspects of the pilot. In particular, BELL-RL at CCSU provided professional development opportunities on dual language programming and instructional strategies, and on understanding the needs of diverse EL students, including students with disabilities. New Haven also took advantage of resources provided by its RESC, ACES. ACES staff reported maintaining open communication with CCSDE to understand what the important aspects of CT policy were in relation to the pilot to support the district. ACES staff also reported that, while communication with district staff was challenging at first, once they agreed on how best to work together, communication was open and productive. ACES helped lead the development of the bi-literacy model, including the mapping and creation of interdisciplinary curricular units. ACES also provided professional development, including workshops on CELP standards and blended learning. As shown in Table 12, a greater percentage of teachers who completed the spring 2017 survey confirmed that they had received support from their RESC and/ or BELL-RL at CCSU, showing that support was provided during the academic year. Table 12. Percentage of New Haven teachers somewhat or strongly agreeing with statement Fall 2016 (n=10) Spring 2017 (n=16) I feel that I have adequate support from the school 70 administration and the district to do my part of the pilot program 69 I have received support, including but not exclusively professional development training, from our local RESC 30 75 I have received support, including but not exclusively professional development training, from CCSU 20 50 Adaptation to district context: As described above, New Haven chose to focus on just one K–8 school, and tailored the interventions to meet differentiated needs Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 21 by grades (K–3 and 4–8). In both the fall and the spring, the majority of surveyed teachers felt that the pilot program addressed an important need in the district, and that the goals were relevant, realistic, and achievable (Table 13). Table 13. Percentage of New Haven teachers somewhat or strongly agreeing with statement Fall 2016 (n=10) Spring 2017 (n=16) I am aware of the goals of the pilot program 70 69 The goals of the pilot program are realistic 70 75 The goals of the pilot program are relevant to our students’ needs 70 81 The goals of the pilot program are achievable, given the timeframe and resources available to us 70 75 The pilot program addresses an important need in the district 80 81 Successes of the New Haven pilot. The district reported achieving the two goals outlined in their logic model: (1) A bi-literacy, interdisciplinary K–3 curriculum was mapped out and implemented, and framework was developed to continue building and improving the model in the future; and (2) the technology infrastructure necessary for the implementation of the blended learning program was put in place, and the program was implemented. From the perspective of the district lead, the three outcome measures were also met. First, students’ proficiency levels were measured using the LAS Links English proficiency assessment. According to the district representative, 60 percent of students in the pilot school met their annual benchmarks, the highest percent of all schools in the district with a comparable number of EL students. At the time of the second interview, the district lead was unable to report on the results of the district-administered survey conducted to determine how teachers rated the professional development activities or on the results of the self-reported perceptions of their increased capacities to meet the academic needs of their students. While results on these last two measures were not ready, the district lead reported that preliminary data indicated success with these two outcomes as well. In the spring 2017 survey, 75 percent of teachers felt that the EL students were benefiting from the pilot and 81 percent felt that they had benefited (Table 14). Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 22 Table 14. Percentage of New Haven teachers somewhat or strongly agreeing with statement Fall 2016 (n=10) Spring 2017 (n=16) I feel that I have benefited professionally from my involvement in the pilot program 70 81 I feel that my EL students are benefiting from the pilot program 70 75 New Haven plans to continue both the bi-literacy and the blended learning programs at the pilot school and is planning to develop thematic, interdisciplinary curricular units for grades 4–8 as was done for K–3. The district is also looking to expand aspects of the pilot in schools across the district. In particular, it plans to offer the bi-literacy K–3 curriculum in other schools, and introduce the blended learning program across the district. However, although the district lead expressed optimism about offering professional development to support implementation of the pilot’s programs, there was also concern about the ability to do so at a level comparable to that of the pilot. As another way of examining pilot success, teachers were asked in the survey to identify the success criteria they felt were most important, and then to evaluate whether those criteria were being met by the pilot. The New Haven teachers identified both student and teacher outcomes as important success criteria, though by spring 2017 they were focused more on student criteria than teacher criteria (Table 15). Table 15. Percentage of New Haven teachers identifying success criteria Fall 2016 (n=10) Spring 2017 (n=16) EL students access academic content in English 60 81 EL students feel more connected to the school 50 81 EL students’ social and emotional needs are met so that they are able to learn 40 69 Teachers working with EL students feel more confident in their abilities to meet the unique academic needs of EL students 30 69 Sixty percent of the teachers in the fall, and 81 percent of those in the spring, felt that the pilot was somewhat or very likely to meet those criteria, with teachers identifying the “thought and effort put into the program” as a key reason for success. The percentage of teachers who felt very confident that they could meet the academic needs of their EL students increased from 10 percent in the fall to Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 23 31 percent in the spring. When including those teachers who are both somewhat and every confident in their abilities, the percentages increase to 60 percent in fall 2016 and 75 percent in spring 2017. While this increase is encouraging, the lack of confidence among teachers participating in the pilot could indicate self-awareness that more professional learning is warranted. Windham: Supporting Newcomer Students in Grades 6–12 District description. Windham Public Schools is a small district with 3,296 students in 2015–16. Of these students, 68.4 percent are Hispanic or Latino, 3.6 percent are Black or African American, and 24.8 percent are white. Students who are Asian or of two or more races make up the rest. Twenty-four percent of the students are EL students (CSDE, 2016d). Identified need. The district has been experiencing an increase in EL students who are new arrivals to the United States, particularly EL students of middle and high school age. The district lead noted that newcomer students arrive throughout the school year, not just in September. Based on these observations, the district decided to focus its pilot on supporting the needs of its newcomer population in grades 6–12. The specific needs were identified as supporting the development of English proficiency and attending to socio-emotional needs, including acculturation in the educational system and to a new culture. Intervention. To support the linguistic needs of newcomer students, the district chose to expand its current newcomer program through the implementation of ELLoquence, a blended learning program designed to develop and enhance English proficiency. The program offers students and educators a wide variety of interactive activities. As part of the implementation, teachers participated in webinars and received personal coaching from the program developers. In addition, the district offered various professional development opportunities designed to build the capacity of all teachers in the district to work with EL students in mainstream classrooms. Among these professional development opportunities were workshops on best practices for increasing the English proficiency of EL students. In addition, the district offered to support teachers in acquiring their ESL or Teacher of English to Speakers of Other Languages (TESOL) certification. To meet the socio-emotional needs of the newcomer students and assist in acculturation into the educational system and U.S. society, the district implemented several initiatives. Teachers were offered the opportunity to take Spanish classes to improve communication with their students and their families, and paraprofessionals, secretaries, and family liaisons were supported to become trained as interpreters for school-related issues. The district planned a summer Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 24 program for newcomers and an event for parents of graduating seniors attending the summer program. The district also developed a survey to assess the socioemotional needs of the students. Intervention goals and implementation. Windham’s logic model included the following three goals: (1) improve students’ English and academic skills to increase their success in mainstream classrooms; (2) build the capacity of K–12 teachers to work with EL students in mainstream classrooms; and (3) improve students’ socioemotional skills so that they feel connected to the school. Fifty teachers participated in trainings related to ELLoquence and used this program in their classrooms. In addition, about 80 teachers across the district participated in professional development on best practices to increase English proficiency in mainstream classrooms. Nine teachers received their certification in ESL or TESOL. More than 50 teachers participated in Spanish classes over the two years of the project, and several district staff were trained as school interpreters. A summer program for newcomers was offered between the two years of the pilot and an event for parents of graduating seniors in the program was held. With the assistance of EASTCONN, a survey on socio-emotional issues was developed and administered. Intervention evidence base. ELLoquence is a comprehensive digital curriculum for grades 6–12 that blends high-tech instruction with hands on interactive activities. It claims to accelerate the development of English proficiency of middle and high school students in one to two years, including newcomers and other students at all levels of English proficiency. The results of a study done by the developer at one high school (http://www.thesysintl.com/elloquence/) report that after one year of using ELLoquence for one period a day students improved their scores by one proficiency level. Similar results are also reported for students at a second school. Inclusion of external expertise. BELL-RL at CCSU provided two types of trainings for teachers. The first training was an introduction to best practices for instructional strategies for working with EL students. The training was provided to 80 teachers, which included a combination of new teachers and returning teachers. The second training was a workshop on supporting EL students’ writing skills. This training was only offered to teachers in the EL department. In addition, teachers from the district attended a cross-district training offered by BELL-RL at CCSU regarding working with dually identified special education and ELL students. EASTCONN worked with Windham on the development of the socio-emotional survey and provided professional development opportunities on the use of technology in the classroom during the first year of the pilot. EASTCONN did not play a role in the pilot during the second year. According to EASTCONN they submitted a proposal Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 25 for onsite coaching to new high school teachers, but the district did not accept the proposal. Thus, it is not surprising that fewer than 50 percent of teachers in either the fall or spring stated that they had received support from their external partners (Table 16). Table 16. Percentage of Windham teachers somewhat or strongly agreeing with statement Fall 2016 (n=46) Spring 2017 (n=59) I feel that I have adequate support from the school administration and the district to do my part of the pilot program 24 39 I have received support, including but not exclusively professional development training, from our local RESC 20 39 I have received support, including but not exclusively professional development training, from CCSU 20 29 Adaptation to district context. As described above, the Windham team focused its pilot on newcomer students in middle and high school. Teachers who completed the survey in fall 2016 were less likely to say that the goals were relevant, realistic, or achievable. By spring 2017, however, the majority of teachers somewhat or strongly agreed with the statements (Table 17). In addition, the percentage of teachers who felt the pilot addressed an important need in the district increased from 57 percent to 68 percent. Table 17. Percentage of Windham teachers somewhat or strongly agreeing with statement Fall 2016 (n=46) Spring 2017 (n=59) I am aware of the goals of the pilot program 37 58 The goals of the pilot program are realistic 39 56 The goals of the pilot program are relevant to our students’ needs 39 66 The goals of the pilot program are achievable, given the timeframe and resources available to us 39 51 The pilot program addresses an important need in the district 57 68 Successes of the Windham pilot. The district lead observed that the professional development that teachers received increased their expectations of what EL students could accomplish, as well as deepening their understanding of how to offer differentiated instruction that responds to a student’s individual level of English proficiency. According to the district lead, LAS Links scores seemed to Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 26 have increased more quickly than expected when looking at scores from previous years for similar students. She also noted that academic growth for these students seemed to be steady. With respect to the socio-emotional status of the students, results from the districtadministered student survey indicated that students felt supported, developed a strong connection with the newcomer program and the program’s teacher, and felt connected to mainstream culture. The factors contributing to these outcomes on the survey appeared to relate to teachers having learned enough Spanish to better communicate with and understand their students, as well as to other types of professional development that teachers had received. A number of teachers noted the benefits of learning Spanish, with one teacher commenting, “I am using my limited Spanish language knowledge in the classroom more frequently and with more confidence, and that helps my EL students.” From the fall to the spring, the percentage of teachers who felt that they had benefited professionally and those who felt that EL students benefited both increased (Table 18). Table 18. Percentage of Windham teachers somewhat or strongly agreeing with statement Fall 2016 (n=46) Spring 2017 (n=59) I feel that I have benefited professionally from my involvement in the pilot program 35 51 I feel that my EL students are benefiting from the pilot program 37 53 As another way of examining pilot success, teachers were asked in the survey to identify the success criteria they felt were most important, and then to evaluate whether those criteria were being met by the pilot. Interestingly, teachers chose parent engagement as the most important criteria in both the fall and spring (Table 19). Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 27 Table 19. Percentage of Windham teachers identifying success criteria Fall 2016 (n=46) Spring 2017 (n=59) Parents of EL students are more engaged with the school and their children’s academic work 57 61 EL students acquire basic English language skills 54 59 Teachers successfully implement new or revised curriculum(a), educational program(s) (e.g., blended learning program) or bilingual model(s) (e.g., biliteracy model) that meet the academic needs of EL students 43 58 EL students’ social and emotional needs are met so that they are able to learn 48 56 Fifty-seven percent of teachers in the fall, and 76 percent of those in the spring, felt that the pilot was somewhat or very likely to meet those criteria. Among the reasons given for saying that the district was not likely to meet the pilot goals were that the practices need to be implemented districtwide to be successful, and that the strategies need to be supported over time, rather than moving from one initiative to another. The percentage of teachers who felt very confident that they could meet the academic needs of their EL students increased from 9 percent in the fall to 29 percent in the spring. While these numbers are quite low, the percentage of teachers who felt somewhat or very confident increased from 41 percent in fall 2016 to 66 percent in spring 2017. Windham did not report clear next steps to continue the pilot interventions due to the district lead’s decision to leave her position in the district. It is unclear who will take on that position and whether the interventions created during the pilot will continue. On the other hand, it is expected that there are tangible successes that will remain such as the increase in the number of certified ESL/TESOL teachers, the presence of trained school interpreters, and the many teachers who received professional development on best instructional practices for EL students. Discussion: Implications from the four districts The four case studies show that while each district approached the CT EL pilot program from a different context, identified different needs, and found different programs to implement, they shared common experiences as well. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 28 The four districts, and the pilot overall, met the three requirements set out in the legislation, but did not limit their pilot to meeting those requirements. Each district succeeded in implementing a research-based program to improve English acquisition skills of EL students, working closely with technical assistance providers, and adapting the program to the specific needs of its EL population. Each district also went beyond the minimum requirements to implement a research-based program; they adapted the program to meet the specific district needs, and also added additional components to expand the impact of the program. In addition, three of the districts developed sustainability plans for how to continue the successes of the pilot beyond the funding period. Across all four districts, there was agreement that having the autonomy to choose the most appropriate program for the district contributed to its success. In some circumstances, the districts would also have preferred to have had more flexibility in choosing their technical assistance providers as well. Three of the four districts chose to work in some way with newcomer students (though across different grade spans), and all of them worked with both general education and EL teachers. The districts identified both specific student populations, as well as specific teacher groups, to benefit from the pilot. Despite the methodological limitations in the design of the pilot program and evaluation, all participants, both teachers and pilot leads, felt that the pilot had improved instruction for students and expertise of teachers. This was the case in each of the four districts, and two districts pointed to evidence of promise as demonstrated by district reporting of strong LAS Links scores. While the pilot programs were not designed to be able to measure their impact, the overwhelmingly positive responses from all involved demonstrates that programs such as these, that allow district and school leaders to identify how to best meet the needs of their student population, are well-received by local leaders and can potentially contribute to improved student outcomes. Among surveyed teachers, from 75 percent (Bridgeport and Windham) to 100 percent (Hartford) of teachers who completed the spring 2017 survey felt that the pilot program would be somewhat or very likely to meet its goals. Across the four districts, the percentage of teachers who felt somewhat or very confident in their ability to meet the academic needs of their EL students increased from 55 percent in fall 2017 to 69 percent in spring 2018. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 29 The four pilots described similar challenges. Delayed release of funding. While the pilot was designed as a two-year project, districts received their funding in the spring of 2016 rather than the fall of 2015. This meant that they had only a few months to begin pilot implementation in Year 1, rather than a full academic year as originally designed. Districts reported that this delay became a significant challenge in launching the pilot program and reduced the time they had for actual implementation. Districts also reported that the delay in receiving funding put pressure on them to begin implementation right away, without allowing time for thorough planning prior to implementation. Resistance from administrators and teachers. Some of the districts reported struggling to get the buy-in of their administrative leaders—particularly district superintendents and school principals. In some cases, superintendents did not sign off on some of the proposed elements of the pilot, thereby requiring the districts to scale down the pilot program or reduce the number of activities. BELLRL at CCSU also reported some resistance from superintendents to allow them to offer professional development in districts. One district reported resistance from some teachers early on because the pilot required a significant time commitment. However, the same district indicated that those teachers eventually gained appreciation for the program and adopted it enthusiastically. Time limitations. Several districts reported that finding time for pilot activities, particularly professional development for teachers, was a challenge. In some cases, this challenge included finding substitute teachers as well. BELL-RL at CCSU also reported that districts’ lack of time for professional development was a challenge for them to offer all the professional development they felt would have been beneficial to the districts. Staffing limitations. Some districts reported challenges identifying the right teachers to implement strategies. In particular, districts reported difficulties finding certified bilingual teachers. Districts indicated they preferred to support existing teachers within the system who they can support to obtain certification rather than bringing in already certified teachers from outside the system. Difficulties in implementing with fidelity. One district reported struggling to ensure that the program was implemented as planned. The challenge in particular was that teachers had to “build the plane while they were flying it.” In other words, they were juggling being trained on new strategies and their teaching responsibilities at the same time. That meant that the implementation of new strategies did not always meet program fidelity, although as teachers had time to absorb the new lessons they were more able to apply them in the classroom. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 30 Developing sustainability strategies. Both districts and RESCs spoke about the challenge of sustainability. While the pilot funding helped to implement new programs for EL students, the districts want to continue with the successes. In particular, some districts hired additional staff to implement new programs (e.g., newcomer centers), and now there is concern that they may not be able to keep those staff long-term. In addition, there is common interest in providing professional development to more teachers to scale up and even sustain the impact of the pilot. Three districts developed sustainability plans to ensure that most if not all successful aspects of the pilot will continue to be implemented, but funding continues to be a concern. Communication between districts and technical assistance providers. Two of the RESCs reported challenges related to communication, particularly at the beginning of the pilot. There was also confusion about the role of BELL-RL at CCSU. None of the district leads had previously worked with BELL-RL at CCSU, and they were not familiar with what BELL-RL at CCSU could offer. Some people interviewed wondered if there could have been more autonomy in choosing technical assistance providers. Limitations of the evaluation. As with all studies, there were limitations in the ability of the evaluation to fully address all issues of interest. The evaluation began data collection a full year after initiation of the pilot, and almost nine months after districts had begun implementation. This led to a number of limitations. It was not possible to get baseline data from teachers or pilot leads. The logic model workshops, which could have contributed to the overall project design and tightened implementation, took place at the beginning of Year 2, after districts had already begun implementation. While each of the four pilot districts set a longterm goal to improve English acquisition for EL students, this was impossible to measure because the pilot design did not allow for a rigorous evaluation design to measure impact. Despite these limitations, the evaluation results clearly demonstrate that both teachers and district EL leaders were overwhelmingly enthusiastic about the pilot and the pilot results. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 31 Recommendations We provide here recommendations in two areas; the first are recommendations for implementation of similar pilot programs, or programs that provide special funding for district or school-level innovations. The second are recommendations for continuing to improve EL student education CT based on the lessons learned from the four pilot districts. Recommendations for Future Pilot Programs Maintain and increase flexibility in choosing the innovation. Each district chose different ways to implement the pilot and all were able to take full advantage of the flexibility in the legislation. This also helped the districts to meet the third requirement of adapting the program to their specific district and student needs. Provide greater flexibility in choosing external support partners. While districts demonstrated appreciation for the professional development that was offered by RESCs and BELL-RL at CCSU, some of them mentioned that they would have liked to be able to select which organization to collaborate with to provide technical assistance for their districts. In addition, both BELL-RL at CCSU and one of the districts recommended greater involvement from institutions of higher learning to support districts, given the expertise on language acquisition and other related topics that exists in such institutions. Provide adequate planning time and technical assistance to choose the specific intervention and the expected outcomes from the intervention. Districts recommended that in future pilots, the grant include a period of planning time. In this pilot districts felt compelled to begin implementation as soon as funding was received, which left them with little time to plan the pilot program, including to map out the different activities, develop a timeline, and determine professional development needs and who would provide them and when, and connect all of this to outputs and outcomes. Ensure districts develop sustainability plans. Sustainability plans ensure that successful practices from pilots are incorporated into the district’s education structure, rather than have the success lie in the hands of specific individuals who, were they to leave the district, would take the lessons learned with them. In addition, sustainability plans ensure that the district plans on continuing elements of the pilot program with existing district funds, rather than see successful elements fade away when pilot funding ends. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 32 Provide formal structures for pilot district participants to be part of a professional learning community to share challenges and successes throughout the life of the pilot. District leads indicated that they would have liked more opportunities to interact with their colleagues in other districts throughout the implementation of the pilot. There was one such opportunity at the start of the pilot, and a second opportunity during the logic model workshop offered by EDC. Districts found these experiences helpful, and would have liked more opportunities to interact and network with each other. Furthermore, districts would also benefit from an opportunity to meet at the end of the pilot to share lessons learned, perhaps including teacher participants, which would serve to further promote teacher collaboration—a helpful element to get teacher buy-in and to support teachers in their day-to-day work. Design pilot programs to include follow up that builds on the results of the pilot. This could include extended funding for additional years for each district to revise their intervention and implement it a second time with a more rigorous evaluation design that would provide greater evidence of impact. Districts found that 1–2 years is not enough time to pilot new programs and see their results. Therefore, they recommended that in the future districts be given more time to plan, implement, and evaluate the impact of the pilot. One district, for example, suggested that funding levels can be reduced in later years, but even some funding would be helpful to ensure that districts can sustain the intervention over time and find ways to measure impact. Provide greater clarity around evaluation and monitoring requirements for individual districts. This should be part of the adequate planning time described above, so that districts are able to take full advantage of the opportunity to implement a pilot by planning both the intervention and its evaluation. Require greater involvement by superintendents. Interventions that are supported by superintendents are more likely to be successful, and state-funded initiatives have the ability to require superintendents to be more involved. Recommendations for next steps to continue to improve EL education in Connecticut The successes of the pilot program demonstrate that providing targeted funding that allows districts to innovate to meet the needs of EL students has evidence of promise and would be worth continued support. EDC believes that if the CSDE builds on the learnings from the pilot, it will provide EL students with higher quality instruction and make good use of the investments made in the pilot. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 33 Provide pilot districts with formal venues to share their innovations with other districts. Each of the four districts that participated in the pilot have important findings that are worth sharing beyond their district. The state should find a way to build on the successes of the pilot by promoting cross-district sharing. Several districts felt that the programs they implemented and their experiences with them would be useful for other districts with similar EL populations and needs. For example, Bridgeport spoke very enthusiastically about Project GLAD and indicated that it strongly recommends other districts consider the program as well, and went as far as to recommend that the state consider supporting Project GLAD across districts. Similarly, Hartford recommended that districts share with each other their experiences with newcomer centers, and recommended cross-district visits to observe different types of newcomer programs. Provide EL-related professional development to general education teachers and to administrators. All of the districts found that general education teachers benefited greatly from focusing on building skills and instructional strategies for EL students. Many also noted that when teachers build skills and learn strategies to help specific groups of students like ELs, all students benefit. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 34 Continue to incorporate logic model or similar planning tools as part of district planning. While districts were not able to fully take advantage of the logic model workshops, they demonstrated that engaging in activities that require planning teams to think through how specific interventions are connected to desired outputs and outcomes can help clarify thinking, and implementing, of interventions. These skills go beyond the development of EL-focused interventions and can become standards of practice for districts as they consider their districtwide planning. Logic models are helpful tools that make theories of change visual, help identify specific program elements, match them with the district contexts and needs and desired outcomes, and identify specific ways to measure those outcomes. These types of workshops can help districts better articulate their theory of change related to language acquisition, select research-based programs that match with their theory and with their context, and ensure that the districts are collecting data on relevant milestones and outcomes. Not all programs can include a formal external evaluation, and using logic models can help both school and district leaders develop the habit of incorporating progress monitoring as part of the daily practice of implementing innovations. Continue to invest resources in encouraging innovations to improve teaching for EL students, while also maintaining support for innovations that demonstrate success. As demonstrated in the case studies, each district valued participation in the pilot. By having the autonomy to choose specific interventions to meet identified needs, both teachers and district leads found that the pilot program was successful with their students. Despite the challenges faced by each of the four districts as they implemented their pilot programs, they were successful in targeting programs to meet specific identified needs of their EL populations. Whether by expanding the pilot programs to larger-scale implementation studies, or providing rolling funding for small pilots across multiple districts, the state should continue to support district innovation. CONCLUSION The CT EL pilot program provided four CT districts with resources to innovatively address the English language acquisition needs of their EL students. Each district identified key needs of a subgroup of ELs in their district, identified programs and strategies based on educational research, and implemented those strategies with the support of their local RESCs and BELL-RL at CCSU. Each district fulfilled the requirements of the legislation while also learning lessons from the pilot that will contribute to ongoing practices in the district. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 35 By requiring only that the districts choose interventions that are research-based, the legislation provided a level of autonomy that allowed each district to tailor the pilot to their student needs. At the same time, the criteria of being “researchbased” is less clear than it may seem. Very few interventions have had rigorous impact evaluations, and when they have, the studies are often with populations that are different from those in any one district. This highlights the fact that while districts are wise to do a careful review of existing research to assure themselves that they are using interventions that have strong evidence, they should also include some form of evaluation to carefully look at how the programs work in their specific context. Building a culture of progress monitoring can help improve the quality of implementation. The autonomy districts were provided in choosing their specific program was not echoed in the procedures put in place for technical assistance. Each district was directed to collaborate with the RESC in their region, as well as the Connecticut Bilingualism and BELL-RL at CCSU. Some districts found the support more useful than others, and a number of people involved in the pilot expressed a desire to have more autonomy in choosing technical assistance collaborators. The evaluation of the four pilot programs found a number of unexpected successes, some of which could be even more successful with slight modifications to the overall program. All of the districts struggled with the tight timeline, which became even tighter due to delayed release of funding. If innovative programs such as these are allowed to have more time—both at the beginning, to do careful needs assessment and development of logic models, plans and evaluations, and at the end, to analyze evaluation results—the results will be even higher quality interventions with greater evidence of impact. Even more importantly, additional time will allow district leaders, teachers, parents, and even students to better understand the programs and incorporate them into ongoing practice long after the funding cycle has ended. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 36 References CSDE. (2016a). Bridgeport district profile and performance report for school year 2015–16. Downloaded from http://edsight.ct.gov/Output/District/ HighSchool/0150011_201516.pdf CSDE. (2016b). Hartford district profile and performance report for school year 2015–16. Downloaded from http://edsight.ct.gov/Output/District/ HighSchool/0640011_201516.pdf CSDE. (2016c). New Haven district profile and performance report for school year 2015–16. Downloaded from http://edsight.ct.gov/Output/District/ HighSchool/0930011_201516.pdf CSDE. (2016d). Windham district profile and performance report for school year 2015–16. Downloaded from http://edsight.ct.gov/Output/District/ HighSchool/1630011_201516.pdf Deussen, T. (2014). Professional development for mainstream teachers of ELLs: Project GLAD® and beyond. Portland, OR: Education Northwest. Downloaded from http://projectgladstudy.educationnorthwest.org/overview Germain-Rutherford, A., & Martinez-Lage, A. (2012). Research to proficiency: The curriculum design of Middlebury Interactive Languages’ high school fluency courses. Middlebury, VT: Middlebury College. Downloaded from http:// www.middleburyinteractive.com/sites/default/files/Middlebury_Interactive_ Research_Proficiency-WhitePaper-080515.pdf Honigsfeld, A., & Dove, M. (2008). Co-teaching in the ESL classroom. Delta Kappa Gamma Bulletin, 74(2), 8. Jacoby, J. W., & Lesaux, N. K. (2017). Language and literacy instruction in preschool classes that serve Latino dual language learners. Early Childhood Research Quarterly, 40, 77–86. doi: 10.1016/j.ecresq.2016.10.001 Kuo, L.-J., Ramirez, G., de Marin, S., Kim, T.-J., & Unal-Gezer, M. (2017). Bilingualism and morphological awareness: A study with children from general education and Spanish-English dual language programs. Educational Psychology, 37(2), 94–111 Murawski, W., & Swanson, H. (2001). A meta-analysis of co-teaching research: Where are the data? Remedial & Special Education, 22(5), 258–267. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 37 Murphy, R., Gallagher, L., Krumm, A., Mislevy, J., & Hafter, A. (2014). Research on the use of Khan Academy in schools. Menlo Park, CA: SRI Education. Downloaded from https://www.sri.com/sites/default/files/publications/2014-03-07_ implementation_briefing.pdf Neely, C. (2016). The benefits of co-teaching in the ESL classroom. In Second language writing in elementary classrooms (pp. 33–46). London, UK: Palgrave Macmillan. Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 38 Appendix A: Baseline Interview Protocols District Representative We are with Education Development Center, Inc., contracted by CSDE to conduct the evaluation of the cross-district pilot program. As part of our role we will be doing the following: »» Conducting interviews with you, other districts, relevant RESCs, and a representative from the Connecticut Bilingualism and English Language Learning Research Lab at CCSU, now and again in June. »» We will also survey teachers (more about that later) »» Conduct a logic model workshop (more on that later) »» And provide you with ongoing support on developing and applying your logic model. »» Please keep in mind though that we are not evaluating individual district’s pilots The interview will take about an hour. We will interview you again towards the end of the academic school year. We have reviewed your pilot description and first annual report »» Can you please briefly describe your pilot, particularly any aspects of it that have been modified since you submitted your description? ºº What school(s)/grade(s) ºº What strategies will you use (e.g., professional development, blended learning, etc.) ºº Who is involved in the coordination, implementation, and evaluation of your pilot? »» Your documents indicate that you are targeting [insert target audience] because of [Insert justification of need]. Can you describe this in a little more detail? ºº What are the specific characteristics of the students benefiting from your pilot? ºº Why was this the right target population(s) to select? ºº What are the unique needs of this population(s) and how have you designed the pilot intervention to meet those needs? ºº How were these needs identified and prioritized? »» How was the pilot developed? ºº Who was involved in developing the pilot? Were principals, teachers, others involved? ºº How were these different individuals/groups involved in developing the pilot? Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 39 ºº How did you involve, if at all, other stakeholders such as CSDE, BELL-RL at CCSU, your local RESC, and/or other experts in the field in developing the pilot program? ºº How are you involving these stakeholders during the implementation phase? ºº Who else would you like to have involved, if you were able to involve additional stakeholders? (parents, teachers, students (for high schools)) ºº How were the school(s) and grade(s) selected? ºº How was the strategy(ies) selected? ºº Based on your understanding, how are the strategies supported by research? »» What do you hope your pilot will accomplish? ºº What are your specific goals for the pilot? Specifically, what are the goals for students, teachers, principals? ºº How were these goals selected? ºº How are you planning on measuring these goals?  Who will be responsible for evaluating your pilot?  How will data related to goals be collected? When?  How will your data be used to inform pilot implementation and possible modifications? »» You have been working on your pilot for a few months so far… ºº What have you been working on and what have you accomplished so far? ºº What challenges have you faced? How are you overcoming these challenges? ºº What changes or modifications, if any, are you thinking about making to your current plan? »» Any additional comments/thoughts/questions to add? Thank you! Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 40 RESC Representative We are with Education Development Center, Inc., contracted by CSDE to conduct the evaluation of the cross-district pilot program. As part of our role we will be doing the following: »» Conducting interviews with representatives from the four districts, the other relevant RESCs, and a representative from the Connecticut Bilingualism and English Language Learning Research Lab at BELL-RL at CCSU, now and again in June. »» We will also survey teachers (more about that later) »» Conduct a logic model workshop (more on that later) »» And provide districts with ongoing support on developing and applying their logic models. »» Please keep in mind though that we are not evaluating individual district’s pilots The interview will take about an hour. We will interview you again towards the end of the academic school year. We have reviewed the pilot description and first annual reports from the different districts. »» What is your role in supporting the pilot of [relevant district]? ºº What services or type of support can you provide? ºº How have you communicated your role and services to the district? ºº What type of support, if any, has the district asked for? »» What role, if any, did you play in designing the pilot? ºº School(s) and grade(s) selection? ºº Selection of specific strategy(ies)? ºº Settings goals, benchmarks, outcomes, etc.? ºº Any other role? ºº Based on your understanding of the pilot, how are the strategies being implemented supported by research? »» Can you please briefly describe your understanding of the program being piloted in your corresponding district? ºº What school(s)/grade(s) ºº What strategies are being used (e.g., professional development, blended learning, etc.) ºº Who is involved in the coordination, implementation, and evaluation of the pilot? Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 41 »» Who does the pilot target? (e.g., all EL students, newcomers, etc.)? ºº What are the specific characteristics of the beneficiaries of the program? ºº Why was this target population(s) selected? ºº What are the unique needs of this population(s) and how are these needs addressed through the pilot intervention? ºº Were you involved in any way in the identification of the pilot’s target population(s)? If so, how? »» How will you support the relevant district moving forward? ºº How often do you plan to connect with the pilot coordination/implementation team? ºº In what way(s) will you engage with them? »» Based on your understanding of the pilot, what do you expect the pilot will accomplish? »» Any additional comments/thoughts/questions to add? Thank you! Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 42 BELL-RL at CCSU Representative We are with Education Development Center, Inc., contracted by CSDE to conduct the evaluation of the cross-district pilot program. As part of our role we will be doing the following: »» Conducting interviews with representatives from the four districts, the relevant RESCs, and yourself, now and again in June. »» We will also survey teachers (more about that later) »» Conduct a logic model workshop (more on that later) »» And provide districts with ongoing support on developing and applying their logic models. »» Please keep in mind though that we are not evaluating individual district’s pilots The interview will take about an hour. We will interview you again towards the end of the academic school year. We have reviewed the pilot description and first annual reports from the different districts. »» Can you please briefly describe your role in supporting the four districts’ pilot programs? ºº What types of support are you able to provide the four districts related to their pilot program? ºº How did you inform the sites about your role in this project and the supports you were prepared to offer? ºº What type of support have they asked for? ºº Did you conduct any form of needs assessment with each site to determine what support was needed? ºº For each pilot, at what stage of the pilot’s design and/or implementation have you been involved?  Hartford  New Haven  Bridgeport  Windham ºº For each pilot, how involved have you been to date, and in what capacity? »» Based on your research and expertise… ºº Did the four districts select the right target populations? Why/why not? ºº Did the four districts select the right strategies to address the needs of their selected audiences? Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 43 ºº Do you consider the strategies being applied to be research-based? On what basis do you make this determination? ºº Do you have any concerns about the design or feasibility of any of the districts’ pilot programs? »» What are the key considerations the four districts need to keep in mind in order to ensure their successful implementation of their pilot and ultimately, their effectiveness of meeting their ELs’ needs? »» How do you expect to be involved with each site moving forward? How will future support be determined? »» Any other thoughts, comments or questions? Thank you! Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 44 Appendix B: Final Interview Protocols District Representative As you know, we are with the Education Development Center, contracted by CSDE to conduct the evaluation of the cross-district pilot program. This interview is part of the overall evaluation of the pilot program (and not of your specific district’s implementation of your pilot). The purpose of this interview is to find out: »» What strategies you implemented (including any changes from what we had discussed in the first interview and through logic model process)? »» What strategies were successful? »» What challenges did you face in designing and implementing your pilot? »» How did you address these challenges? »» Any other thoughts you want to share about the success and challenges of your program? The interview will take about an hour. »» First, we want to review the strategies that you implemented as part of your pilot. ºº Can you please briefly describe your pilot program? ºº What changes did you make to your pilot program throughout since you first began implementing it? ºº Why did you make these changes? ºº What information/feedback did you use to inform changes? ºº How did the changes fair in terms of helping you achieve your program’s goals? »» Next, we want to discuss the involvement of other players in your program’s implementation. ºº Who were the key players in the program implementation (e.g., school principals, EL educators, general ed. educators, RESCs, BELL-RL at CCSU, etc.) ºº For each of these key players, can you please describe their role throughout the program implementation? ºº How did their role change over time if at all? ºº How did they each contribute to the program’s success? ºº In the case specifically of the RESCs and BELL-RL at CCSU, how did they each support your program’s implementation? ºº What challenges, if any, did you have working with them? Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 45 ºº [If RESCs and/or BELL-RL at CCSU not involved, why were they not engaged in the program implementation?] »» We want to discuss the successes of your program. ºº Can you briefly describe what were you program’s original goals (i.e., those identified in the original proposal)? ºº How did you measure progress towards these goals? ºº How well did you program do meeting these goals? ºº Were there any new goals or successes you observed throughout the pilot’s implementation and/or towards its end? ºº Looking back, what factors contributed most to your successes? »» We also want to learn about any challenges you confronted in implementing your pilot program: ºº What challenges did your pilot program face? (time, resources, staff buy-in, parent buy-in, lack of support, selecting the appropriate program/strategies, etc.) ºº For each challenge, how did you and/or others work to address this challenge? ºº Was each challenges surmounted? »» Now that the pilot phase of the program is over, what are your next steps regarding the program/strategies you implemented? ºº How will you sustain your efforts? ºº What changes, if any, do you plan to make to your program? ºº What aspects of your program/strategy do you think will not be sustainable? Why? »» If you were to make any recommendations to another district similar to yours about addressing similar needs as you faced related to the EL student population, what lessons would you share with them? ºº Would you recommend the same program/strategies? Why or why not? ºº What aspects/factors would you recommend they pay special attention to in order to succeed? ºº Would you recommend that they do anything differently from what you did? »» Lastly, do you have any further comments or thoughts about your experience implementing your pilot program? Thank you! Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 46 RESC Representative As you know, we are with the Education Development Center, contracted by CSDE to conduct the evaluation of the cross-district pilot program. This interview is part of the overall evaluation of the pilot program (and not of any specific district’s implementation of your pilot). The purpose of this interview is to find out: »» What strategies were implemented in the district you serve (including any changes from what we had discussed in the first interview and through logic model process)? »» What was your RESC’s involvement in the design and implementation of these strategies »» What success did you witness and/or contributed to? »» What challenges did you witness, how were they addressed, and what role did you have in helping address them? »» What recommendations do you have for other districts with similar EL-related needs? »» Any other thoughts you want to share about your RESC’s involvement in the pilot, as well as in the success and challenges of your district’s pilot program? The interview will take about an hour. »» How would you describe your district’s pilot program? ºº What strategies were implemented? ºº How did the pilot program evolve over time? ºº If any significant change were made, why, as far you are aware, were these changes made? »» What role did you play in supporting the design and implementation of the pilot? ºº What services or type of support did you provide? ºº What type of communication/relationship did you have with the district? ºº How did your role evolve throughout the span of the pilot? »» We want to discuss the successes of your program. ºº Can you briefly describe what is your understanding of the program’s original goals? ºº To what extent do you feel that the district was successful in reaching these goals? ºº What other accomplishments (other than achieving pre-determined goals) did you observe related to the pilot program? Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 47 ºº How did your RESC contribute to these successes, if at all? ºº What were the successes, if any, of your RESC in supporting your district? »» We also want to learn about any challenges confronted in implementing the district’s pilot program. ºº What challenges did you see the district face in the implementation of the pilot program? (time, resources, staff buy-in, parent buy-in, lack of support, selecting the appropriate program/strategies, etc.) ºº For each challenge, how did the district address thesis challenge, if at all? ºº What role, if any, did you play in helping the district address these challenges? ºº Was each challenges surmounted? ºº What challenges, if any, did your RESC face in supporting the district? ºº How were these challenges addressed? »» If you were to make any recommendations to another district facing similar EL-related needs as the district you supported, what lessons would you share with them? ºº Would you recommend the same program/strategies? Why or why not? ºº What aspects/factors would you recommend they pay special attention to in order to succeed? ºº Would you recommend that they do anything differently from what you did? ºº What, if anything, would you do differently to support such a district? »» Lastly, do you have any further comments or thoughts about your experience implementing your pilot program? Thank you! Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 48 BELL-RL at CCSU Representative As you know, we are with the Education Development Center, contracted by CSDE to conduct the evaluation of the cross-district pilot program. This interview is part of the overall evaluation of the pilot program. The purpose of this interview is to find out: »» What role did BELL-RL at CCSU play in supporting each of the four districts? »» What successes did BELL-RL at CCSU have in this role? »» What challenges did you face in carrying out your role? »» How did you address these challenges? »» Any other thoughts you want to share about the successes and challenges of the pilot program, as well as your role in it? The interview will take about an hour. »» First, can you describe the role of BELL-RL at CCSU in the overall CT EL pilot program? ºº How did you conceive your role at the beginning of the pilot program? ºº How did it change overtime? ºº What factors shaped your role overtime (e.g., requests for support from districts)? »» What specific types of support did you provide to each of the four districts’ pilot programs? More specifically, for each site: ºº Was the district proactive or reactive in engaging with BELL-RL at CCSU? ºº What requests for support did they make if any? ºº What kind of support did you provide? ºº How did your role in supporting each district evolve over time? »» How successful was BELL-RL at CCSU in supporting the four districts? ºº How do you define success in this context? ºº What were some of the successes in district-BELL-RL at CCSU collaboration or support provision? ºº What factors enabled such success? Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 49 »» What challenges did BELL-RL at CCSU in supporting the four districts? ºº What challenges did you face? ºº How did you address them? ºº What factors contributed to the challenges and/or your efforts to address these challenges? »» From your perspective, how successful was the pilot program in general in meeting its goals? ºº What do you consider to be the goals of the pilot program? ºº How successful were the four districts, as a whole and/or individually in achieving these success? ºº Did the four districts select the right strategies to address the needs of their selected audiences? ºº Do you consider the strategies being applied to be research-based? On what basis do you make this determination? ºº Do you have any concerns about the design or feasibility of any of the districts’ pilot programs? »» What are the key considerations the four districts need to keep in mind in order to ensure their successful implementation of their pilot and ultimately, their effectiveness of meeting their ELs’ needs? »» How do you expect to be involved with each site moving forward? How will future support be determined? Thank you! Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 50 Appendix C: Baseline Educator Survey Dear educator, You have been selected to take this survey because of your involvement with your district’s English Learners (EL) pilot program. As you know, the Connecticut State Department of Education (CSDE) is funding four districts (Bridgeport, Hartford, New Haven, and Windham) to implement and evaluate EL pilot programs. The purpose of this survey is to obtain feedback from teachers across all four districts about each district’s pilot program, including its design, progress, and possible outcomes. The results of the survey will be used to improve the current pilot programs in each district, as well as to inform future initiatives related to EL education in the state. The survey should take you no more than 20 minutes. Your responses will be maintained confidential. Individual survey results will only be available to staff at Education Development Center, Inc., (EDC), CSDE’s evaluation contractor. Survey results will be presented to CSDE and to the districts only in aggregate form. In addition to filling the survey now (fall of 2016), we will also ask you to fill out a similar survey at the end of pilot period (Spring of 2017). If you have any questions about the survey, please contact Shai Fuxman at sfuxman@edc.org. Thank you! 1. In what district do you work: ❑❑ Bridgeport ❑❑ Hartford ❑❑ New Haven ❑❑ Windham 2. In what type of school do you work? ❑❑ Elementary school ❑❑ Middle School ❑❑ High School Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 51 3. What is your role in your district? ❑❑ School administrator ❑❑ ESOL teacher ❑❑ Bilingual Teacher ❑❑ General Education teacher ❑❑ Special Education teacher ❑❑ Teacher, specific subject. If so, what subject? ____________________ ❑❑ Tutor or paraprofessional ❑❑ Other: ________________________ 4. Did you have a different role in your school prior to the launch of the pilot program (Spring 2016)? ❑❑ Yes ❑❑ No If yes, what is your role in your district? ❑❑ School administrator ❑❑ ESOL teacher ❑❑ Bilingual teacher ❑❑ General education teacher ❑❑ Special education teacher ❑❑ Teacher, specific subject. If so, what subject? ____________________ ❑❑ Tutor or paraprofessional ❑❑ Other: ________________________ 5. Are you a certified or working towards getting certified to be a Teacher of English to Speakers of Other Languages (TESOL)? ❑❑ Yes, I am already certified ❑❑ Yes, I am working towards obtaining my certification ❑❑ No Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 52 6. Are you or are you working towards becoming a certified Bilingual Teacher in the State of Connecticut? ❑❑ Yes, I am already certified ❑❑ Yes, I am working towards obtaining my certification ❑❑ No If yes, what year did you obtain your certification? The following questions ask for your views regarding your district’s pilot program. 7. How familiar are you with your district’s EL pilot program? ❑❑ Very familiar ❑❑ Somewhat familiar ❑❑ A little familiar ❑❑ Not familiar at all 8. Please indicate to what extent you agree with the following statements: Strongly Disagree Somewhat Somewhat Strongly Disagree Agree Agree N/A The district solicited input from teachers like me when designing the pilot program      I feel that my input was taken into consideration when the pilot program was designed      The pilot program addresses an important need in the district      I am aware of the goals of the pilot program      The goals of the pilot program are realistic      The goals of the pilot program are achievable, given the timeframe and resources available to us      The goals of the pilot program are relevant to our students’ academic needs      Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 53 Strongly Disagree Somewhat Somewhat Strongly Disagree Agree Agree N/A Since the pilot program was launched I have been able to provide feedback on its progress      Since the pilot program was launched, I feel that my input is being considered seriously      I feel that I have adequate support from the school administration and the district to do my part of the pilot program      I have received support, including but not exclusively professional development training, from our local Regional Educational Service Center (RESC)      I have received support, including but not exclusively professional development training, from the Connecticut Bilingualism and English Language Learning Research Lab Central Connecticut State University (BELL-RL at CCSU)      I feel that I have benefited professionally from my involvement in the pilot program      I feel that my EL students are benefiting from the pilot program      9. Which of the following, do you think are criteria for determining if your district’s pilot program has been successful (check all that apply) *: ❑❑ EL students feel more connected to the school ❑❑ EL students acquire basic English language skills ❑❑ EL students access academic content in English ❑❑ EL students’ social and emotional needs are met so that they are able to learn ❑❑ EL students’ are acculturated into school’s culture (e.g., are involved in extracurricular activities) ❑❑ EL students continue to strengthen their native language skills ❑❑ Parents of EL students are more engaged with the school and their children’s academic work ❑❑ Teachers** working with EL students are better trained to meet the needs of their EL students ❑❑ Teachers** working with EL students are implementing effective strategies for working with EL students Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 54 ❑❑ Teachers** working with EL students feel more confident in their abilities to meet the unique academic needs of EL students ❑❑ Teachers** successfully implement new or revised curriculum(a), educational program(s) (e.g., blended learning program) or bilingual model(s) (e.g., biliteracy model) that meets the academic needs of EL students ❑❑ Other: ________________________________________________________________ * These criteria refer to the students, parents, and/or teachers who are directly benefiting from the pilot program. For example, your district’s pilot program might be focused only on newcomer students or EL students in specific grades. Likewise, your district’s pilot program might focus on teachers who work with students in certain grades. ** Refers to teachers involved in the pilot program, whether they are designated EL teachers or other teachers who interact with EL students in other ways. 10. Based on the criteria you selected above, how likely do you think that your district’s pilot program will be successful overall? ❑❑ Very likely ❑❑ Somewhat likely ❑❑ Somewhat unlikely ❑❑ Very unlikely 11. Briefly describe your response to the previous question (Question 10). Why do you think your district’s pilot program is likely or unlikely to be successful? If you think it is somewhat or very unlikely to be successful, what changes would need to be made in order to make it more likely to succeed? _________________________________________________________________________ _________________________________________________________________________ The following questions are about your experience working with EL students prior to the launch of the pilot (prior to March of 2016). 12. How much experience working with EL students did you have prior to launch of the EL pilot program? ❑❑ No experience ❑❑ Less than 1 year ❑❑ 1-3 years of experience ❑❑ 4-7 years of experience ❑❑ More than 7 years of experience Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 55 13. Prior to the launch of the pilot, how many hours a week did you work in settings exclusively with EL students (i.e., in classrooms where all students were considered ELs)? ❑❑ Did not work with EL students ❑❑ Less than 3 hours a week ❑❑ 4-7 hours a week ❑❑ 8-10 hours a week ❑❑ More than 10 hours a week 14. Prior to the launch of the pilot program, how many hours a week did you work in settings with EL students in addition to non-EL students (e.g., general studies classroom where some, but not all students were considered ELs)? ❑❑ Did not work with EL students ❑❑ Less than 3 hours a week ❑❑ 4-7 hours a week ❑❑ 8-10 hours a week ❑❑ More than 10 hours a week 15. In how many in-service professional development instances that focused specifically on working with EL students, did you participate in prior to the launch of the pilot? (A PD instance is defined as up to one full day of professional development training, workshop, seminar, etc. For example, a two-hour workshop counts as one instance; a twoday workshop counts as two instances) ❑❑ Did not participate in any PD instances ❑❑ Participated in only one PD instance ❑❑ 2-5 ❑❑ More than 5 16. Besides, in-service professional development, had you received any further training in working with EL students prior to the launch of the pilot program (e.g., taken courses at a university related to EL education)? ❑❑ Yes ❑❑ No Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 56 If yes, please describe your EL training: ___________________________________________________________________________ ___________________________________________________________________________ 17. Prior to the launch of the pilot program, how confident did you feel about being able to meet the academic needs of your EL students? ❑❑ Very confident ❑❑ Somewhat confident ❑❑ A little confident ❑❑ Not confident at all The following questions refer to the support you have received related to working with EL students (including, but not exclusively with newcomer students) since the launch of the pilot program (Spring 2016): 18. Since the launch of the pilot, in how many instances of professional development (PD) specifically focused on working with EL students, did you participate? (a PD instance is defined as one full day or less of professional development training, workshop, seminar, etc. For example, a two-day workshop counts as two instances) ❑❑ Did not participate in any PD instances ❑❑ Participated in only one PD instance ❑❑ 2–5 ❑❑ More than 5 19. What other forms of ongoing support related to working with EL students have you received or participated in since the launch of the pilot program (check all that apply)? ❑❑ None ❑❑ Individual support/mentoring from individuals in my school (e.g., other teachers or administrators) ❑❑ Individual support/mentoring from individuals outside my school [e.g., local Regional Educational Service Center (RESC), academic center such as the Connecticut Bilingualism and English Language Learning Research Lab Central Connecticut State University (BELL-RL at CCSU)] ❑❑ Learning community (regular meetings with colleagues) ❑❑ Other: _____________________________________________________________ Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 57 20. Which of the following entities offered the PD opportunities and other support you received since the project launched? (Check all that apply). School staff District staff Local RESC BELLRL at CCSU Other Don’t know Not applicable PD training or workshop (including both face-to-face and online trainings or webinars)        Individual support/mentoring        Learning community        Other: ______________________________        21. What additional support do you feel you need to be successful in working with EL students in your district (check all that apply)? ❑❑ Additional professional development trainings ❑❑ Individual support/mentoring from individuals in my school ❑❑ Individual support/mentoring from individuals outside my school ❑❑ Learning community ❑❑ Other: _____________________________________________________________ The following questions ask about your current work with EL students. 22. Currently, how many hours a week do you work exclusively with EL students (i.e., in setting where all students are considered ELs)? ❑❑ Do not work with EL students ❑❑ Less than 3 hours a week ❑❑ 4-7 hours a week ❑❑ 8-10 hours a week ❑❑ More than 10 hours a week Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 58 23. Currently, how many hours a week do you work with EL students in addition to non-EL students (e.g., general studies classroom where some, but not all students are considered ELs)? ❑❑ Do not work with EL students ❑❑ Less than 3 hours a week ❑❑ 4-7 hours a week ❑❑ 8-10 hours a week ❑❑ More than 10 hours a week 24. Currently, how confident do you feel in your ability to meet the academic needs of your EL students? ❑❑ Very confident ❑❑ Somewhat confident ❑❑ A little confident ❑❑ Not confident at all 25. Do you have any other comments regarding your district’s pilot program? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 59 Appendix D: Final Educator Survey Dear educator, You have been selected to take this end of year survey because of your involvement with your district’s English Learners (EL) pilot program (you may have been asked to take a similar survey last fall). The purpose of this survey is to obtain feedback from teachers across the four districts implementing EL pilot programs funded by the Connecticut State Department of Education (CSDE). The results of the survey will be used to better understand the success and challenges of the pilot program, as well as to glean lessons learned that can be applied to future EL initiatives throughout the state. The survey should take you no more than 20 minutes. Your responses will be kept anonymous. Individual survey results will only be available to staff at Education Development Center, Inc. (EDC), CSDE’s evaluation contractor. Survey results will be presented to CSDE and to the districts only in aggregate form. If you have any questions about the survey, please contact Shai Fuxman at sfuxman@edc.org. Thank you! The following background questions will be used only in aggregate to better understand the overall background of responding educators and not to identify specific individuals. 1. In what district do you work: ❑❑ Bridgeport ❑❑ Hartford ❑❑ New Haven ❑❑ Windham 2. In what type of school do you work? ❑❑ Elementary school ❑❑ K-8 School ❑❑ Middle School ❑❑ High School Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 60 3. What is your role in your district? ❑❑ School administrator ❑❑ ESOL teacher ❑❑ Bilingual Teacher ❑❑ General Education teacher ❑❑ Special Education teacher ❑❑ Teacher, specific subject. If so, what subject? ____________________ ❑❑ Tutor or paraprofessional ❑❑ Other: ________________________ 4. Did you have a different role in your school prior to the launch of the pilot program (Spring 2016)? ❑❑ Yes ❑❑ No If yes, what was your role in your district? ❑❑ School administrator ❑❑ ESOL teacher ❑❑ Bilingual Teacher ❑❑ General Education teacher ❑❑ Special Education teacher ❑❑ Teacher, specific subject. If so, what subject? ____________________ ❑❑ Tutor or paraprofessional ❑❑ Other: ________________________ 5. Are you currently certified or currently towards a certification for Teacher of English to Speakers of Other Languages (TESOL) ❑❑ Yes, I am already certified ❑❑ Yes, I am working towards obtaining my certification ❑❑ No Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 61 6. Are you or are you working towards becoming a certified Bilingual Teacher in the State of Connecticut? ❑❑ Yes, I am already certified ❑❑ Yes, I am working towards obtaining my certification ❑❑ No If yes, what year did you obtain your certification? The following questions ask for your views regarding your district’s pilot program. 7. How familiar are you with your district’s EL pilot program? ❑❑ Very familiar ❑❑ Somewhat familiar ❑❑ A little familiar ❑❑ Not familiar at all 8. Please indicate to what extent you agree with the following statements: Strongly Disagree The district solicited input from teachers like me when designing the pilot program Somewhat Somewhat Strongly disagree agree Agree N/A      The pilot program addresses an important need in the district      I am aware of the goals of the pilot program      The goals of the pilot program are realistic      The goals of the pilot program are achievable, given the timeframe and resources available to us      The goals of the pilot program are relevant to our students’ academic needs      Since the pilot program was launched I have been able to provide feedback on its progress      Since the pilot program was launched, I feel that my input is being considered seriously      I feel that I have adequate support from the school administration and the district to do my part of the pilot program      I feel that my input was taken into consideration when the pilot program was designed Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 62 Strongly Disagree Somewhat Somewhat Strongly disagree agree Agree N/A I have received support, including but not exclusively professional development training, from our local Regional Educational Service Center (RESC)      I have received support, including but not exclusively professional development training, from the Connecticut Bilingualism and English Language Learning Research Lab Central Connecticut State University (BELL-RL at CCSU)      I feel that I have benefited professionally from my involvement in the pilot program      I feel that my EL students are benefiting from the pilot program      9. Each pilot program identified criteria for success. In this question, we are interested in your option about which criteria are the best indicators of success of the program you participated in. Which of the following do you think are criteria for determining if your district’s pilot program was successful (check all that apply) *: ❑❑ EL students feel more connected to the school ❑❑ EL students acquire basic English language skills ❑❑ EL students access academic content in English ❑❑ EL students’ social and emotional needs are met so that they are able to learn ❑❑ EL students are acculturated into school’s culture (e.g., are involved in extra-curricular activities) ❑❑ EL students continue to strengthen their native language skills ❑❑ Parents of EL students are more engaged with the school and their children’s academic work ❑❑ Teachers** working with EL students are better trained to meet the needs of their EL students ❑❑ Teachers** working with EL students are implementing effective strategies for working with EL students ❑❑ Teachers** working with EL students feel more confident in their abilities to meet the unique academic needs of EL students Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 63 ❑❑ Teachers** successfully implement new or revised curriculum(a), educational program(s) (e.g., blended learning program) or bilingual model(s) (e.g., biliteracy model) that meets the academic needs of EL students ❑❑ Other: ________________________________________________________________ *These criteria refer to the students, parents, and/or teachers who are directly benefiting from the pilot program. For example, your district’s pilot program might be focused only on newcomer students or EL students in specific grades. Likewise, your district’s pilot program might focus on teachers who work with students in certain grades. **Refers to teachers involved in the pilot program, whether they are designated EL teachers or other teachers who interact with EL students in other ways. 10. Based on the criteria you selected above, how successful do you think that your district’s pilot program was overall? q Very successful q Somewhat successful q Somewhat unsuccessful q Very unsuccessful 11. Briefly describe your response to the previous question (question 22). What do you think were the program’s greatest successes? What factors do you attribute to these successes (e.g., the program was well designed, educators were well trained to implement new strategies, educators were well supported by district staff and school administrators) __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ The following questions refer to the support you have received related to working with EL students since the launch of the pilot program (Spring 2016): 12. Since the launch of the pilot in the Spring of 2016, in how many instances of professional development (PD) specifically focused on working with EL students, did you participate? (a PD instance is defined as one full day or less of professional development training, workshop, seminar, etc. For example, a two-day workshop counts as two instances) ❑❑ Did not participate in any PD instances ❑❑ Participated in only one PD instance ❑❑ 2-5 ❑❑ More than 5 Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 64 13. What other forms of ongoing support related to working with EL students have you received or participated in since the launch of the pilot program (check all that apply)? ❑❑ None ❑❑ Individual support/mentoring from individuals in my school (e.g., other teachers or administrators) ❑❑ Individual support/mentoring from individuals outside my school [e.g., local RESC, academic center such as the Connecticut Bilingualism and English Language Learning Research Lab at CCSU] ❑❑ Professional learning community (regular meetings with colleagues) ❑❑ Other: _____________________________________________________________ 14. Which of the following entities offered the PD opportunities and other support you received since the project launched? (Check all that apply). School staff District staff Local RESC BELL-RL at CCSU Other Don’t know Not applicable PD training or workshop (including both face-to-face and online trainings or webinars)        Individual support/mentoring        Professional learning community        Other: _____________________________        15. What type of support, if any, did you feel you needed but did not receive (or did not receive enough) to be successful in working with EL students in your district (check all that apply)? ❑❑ Professional development trainings ❑❑ Individual support/mentoring from individuals in my school ❑❑ Individual support/mentoring from individuals outside my school ❑❑ Professional learning community ❑❑ Other: _____________________________________________________________ Connecticut English Language Learner PIlot Program Final Evaluation Report EDC 65 The following questions ask about your current work with EL students. 16. Currently, how many hours a week do you work exclusively with EL students (i.e., in setting where all students are considered ELs)? ❑❑ Do not work with EL students ❑❑ Less than 3 hours a week ❑❑ 4-7 hours a week ❑❑ 8-10 hours a week ❑❑ More than 10 hours a week 17. Currently, how many hours a week do you work with EL students in addition to non-EL students (e.g., general studies classroom where some, but not all students are considered ELs)? ❑❑ Do not work with EL students ❑❑ Less than 3 hours a week ❑❑ 4-7 hours a week ❑❑ 8-10 hours a week ❑❑ More than 10 hours a week 18. Currently, how confident do you feel in your ability to meet the academic needs of your EL students? ❑❑ Very confident ❑❑ Somewhat confident ❑❑ A little confident ❑❑ Not confident at all 19. Do you have any other comments regarding your district’s pilot program? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________