Evaluation of the Florida Tax Credit Scholarship Program Participation, Compliance and Test Scores in 2014-15 Zahid Kisa Melissa Dyehouse Toby Park Brian Andrews-Larson Carolyn Herrington Learning Systems Institute Florida State University June 2016 Contents EXECUTIVE SUMMARY........................................................................................................................... v 1. BACKGROUND ....................................................................................................................................... 1 2. TEST SCORE COLLECTION IN 2014-15 ..................................................................................... 2 Data collection protocol........................................................................................................................................2 Private school compliance ...................................................................................................................................3 Score reporting in 2014-15 .............................................................................................................................3 Comparison of students with legible, valid test scores to scholarship population ......................7 3. TEST SCORES OF FTC STUDENTS IN 2014-15 ....................................................................... 8 Average test scores in 2014-15 by attributes of program participants ........................................ 10 4. GAIN SCORES FROM 2013-14 TO 2014-15 .......................................................................... 11 Test score gains for FTC students.................................................................................................................. 11 School-level differences in average gain scores, 2013-14 to 2014-15........................................... 13 Individual school average gain scores, 2013-14 to 2014-15 ............................................................. 16 5. ATTRIBUTES OF NEW PROGRAM PARTICIPANTS IN 2014-15 ................................. 18 Comparison of new FTC students and non-participant students in terms of their characteristics........................................................................................................................................................ 19 Comparison of new FTC students and non-participant students in terms of performances of their schools in 2013-14.................................................................................................................................... 20 Comparison of new FTC students and non-participant students within their schools in terms of performances in 2013-14 ............................................................................................................... 22 ii 6. PERFORMANCE OF PROGRAM PARTICIPANTS WHO RETURN TO FLORIDA PUBLIC SCHOOLS ................................................................................................................................... 23 Comparison of 2013-14 performances of public school returnees and FTC stayers in 201415 ................................................................................................................................................................................ 24 Comparison of 2014-15 FSA performances of public school returnees and low income public school students ........................................................................................................................................ 25 7. CONCLUSION ....................................................................................................................................... 27 APPENDIX ................................................................................................................................................. 29 iii Tables Table 1: Distribution of score reporting percentages: 2014-15 and prior years ................ 5 Table 2: Distribution of percent and number of students with legible, valid scores: 2014-15 and prior years............................................................................................................................ 6 Figures Figure 1: Distribution of national percentile rankings of FTC students, 2014-15 .............. 9 Figure 2: Average test scores of program participants in 2014-15 by their attributes. 10 Figure 3: Distribution of test score gains for FTC students, 2013-14 to 2014-15 ........... 12 Figure 4: Distribution of school average gains for FTC students, 2013-14 to 2014-15 14 Figure 5: Distribution of school average gains for FTC students, 2013-14 to 2014-15, schools with 10+ gain scores ................................................................................................................ 15 Figure 6: Comparison of prior year characteristics of new FTC students to "income eligible" non-participant students, 2014-15 .................................................................................. 20 Figure 7: Comparison of new FTC students in 2014-15 to eligible non-participant students by performances of the schools they attended in 2013-14 .................................... 21 Figure 8: Comparison of new FTC students in 2014-15 to eligible non-participant students by quintile of school mathematics FCAT score distribution .................................. 22 Figure 9: 2013-14 test score performance of students remaining in the FTC program in 2014-15 versus those who leave the program .............................................................................. 24 Figure10: 2014-15 FSA performance of FTC students returning to public schools in Florida............................................................................................................................................................ 26 iv EXECUTIVE SUMMARY This report details the 2014-15 academic year evaluation results of the Florida Tax Credit Scholarship (FTC) program, as required by the Florida Statutes, s. 1002.395(9)(j). The ninth in a series of reports, this evaluation is the second of those conducted by the Florida State University Learning Systems Institute (LSI). This report provides a summary of key findings, details about test score collection, 201415 test score results of program participants, gain scores from 2013-14 to 2014-15, test score gains of individual schools with at least 30 or more students, attributes of new program participants in 2014-15, and the performance of program participants who return to Florida public schools. Similar to the previous reports from 2013 onward this report also does not compare the performance of FTC students to public school students due to the difference in the tests that each group takes. Pursuant to the Florida Statutes, s. 1002.395(9)(j), LSI was designated as the independent research organization and was directed to conduct annual evaluations of the Florida Tax Credit Scholarship program beginning in the year 2014. This report presents data collected by LSI during the years 2014-15 followed by the analysis and reporting of these data. Compliance with program testing requirements, 2014-15:  Compliance with program testing requirements is high in 2014-15. Private schools reported test scores for 95.9 percent of program participants in grades 3-10. This is the second-highest level of score reporting in program history and comparable to the highest level of score reporting (96.4 percent) that was observed in 201112. The main contributing factor for the increase in the percentage of legible, valid score is the decrease in the percentage of missing/unusable tests. In 2013-14, 7.9 percent of the expected test scores were missing or unusable. The rate of missing or unusable scores was 2.5 percent in 2014-15. Moreover, the fraction of students not enrolled during testing was at its lowest (0.4 percent) in 2014-15 compared to prior years. The other categories of score reporting remained at levels comparable to those observed in recent years. The rate of unreported scores due to school closures or student suspension was 0.2 percent, the rate of sick students was 0.6 percent, and the rate of students ineligible for testing was 0.4 percent.  Students whose scores were successfully reported come from families with higher incomes (averaging $26,854 versus $23,423) and with parents more likely to be married (46.6 percent versus 37.8 percent). Moreover, students whose scores were successfully reported are more likely to be white (52.0 percent) and female (51.6 percent), compared to students with no test scores (49.6 percent white and 47.2 percent female). This finding is consistent with previous years’ findings. v Differential program participation rates for different groups of students and families:  As in previous years, new FTC students in 2014-15 were relatively more disadvantaged and lower-performing prior to entering the FTC program than freelunch eligible, non-participant students. Moreover, they tend to come from lowerperforming public schools.  Former FTC students who return to the public schools tend to be those who were struggling the most in their private schools.  Former FTC students who returned to the public schools appear to be lower performing compared to other subsidized-meal eligible public school students who never participated in the FTC program. Test scores of program participants, 2014-15:  FTC students scored at the 47th national percentile in reading and the 46th national percentile in mathematics. These scores are similar to previous years’ scores.  In terms of gain in national percentile ranking points from 2013-14 to 2014-15, the typical FTC student tends to maintain his or her relative position in comparison with all students nationally both in mathematics and reading. It is important to note that these national comparisons pertain to all students nationally, and not just students from low-income families. vi 1. BACKGROUND This report details the 2014-15 academic year evaluation results of the Florida Tax Credit Scholarship Program, as required by the Florida Statutes, s. 1002.395(9)(j). The ninth in a series of reports, this evaluation is the second of those conducted by the Florida State University Learning Systems Institute. This report provides a summary of key findings, details about test score collection, 201415 test score results of program participants, gain scores from 2013-14 to 2014-15, test scores gains of individual schools with at least 30 or more students, attributes of new program participants in 2014-15, and the performance of program participants who return to Florida public schools. Similar to the three previous reports, this report also does not compare the performance of FTC students to public school students due to the difference in the tests that each group takes. While FTC students take a nationally norm-referenced test, public school students take the Florida Standards Assessments (FSA) Test. Because there is no correspondence between the FSA and the nationally norm-referenced tests that FTC students take, the independent research organization tasked with this evaluation, the Learning Systems Institute, holds that it is not valid to make these comparisons. The original independent research organization that was contracted to conduct the FTC program evaluation was led by the Project Director, David Figlio. Beginning in 2007, David Figlio’s team retrospectively collected test score data from private schools for the academic year 2006-07 and collected data directly from the private schools for the 2007-08 academic year. These reports continued each year 1 detailing the evaluation of the program using FTC students’ test scores collected from private schools. The first report in which gain scores were reported for program participants was the 2010 report. Pursuant to the Florida Statutes, s. 1002.395(9)(j), the Learning Systems Institute (LSI) has been directed to conduct annual evaluations of the Florida Tax Credit Scholarship program beginning in the year 2014. This report provides the results of the 2014-15 academic year evaluation of the Florida Tax Credit Scholarship Program. 2. TEST SCORE COLLECTION IN 2014-15 Data collection protocol As mandated by s. 1002.395(8)(c)(2), participating private schools administered a nationally norm-referenced test approved by the Florida Department of Education. Schools had a variety of tests from which to choose, including the ACT/PLAN, Basic Achievement Skills Inventory, Comprehensive Testing Program, Educational Development Series, EXPLORE, Iowa Tests of Basic Skills, Kaufman Test of Educational Achievement, NWEA Measures of Academic Progress, Metropolitan Achievement Tests, PSAT/NMSQT, ReadiStep, Stanford Achievement Test, STAR, TerraNova, or Woodcock-Johnson Tests of Achievement. Alternatively, participating students could be administered statewide assessments at a public school in accordance with 1002.395(7)(e). Data collection took place during the year 2014-15, in which private schools sent students’ test scores to the independent research organization, The Learning 2 Systems Institute. The 1,285 private schools that had participating students in grades three through ten during the 2014-15 school year were contacted by the independent research organization in spring 2015 and again throughout spring and summer 2015 to encourage compliance with score reporting. Schools were provided a roster of participating FTC students, which was obtained in December from the Scholarship Funding Organization. 1 From the 1,285 private schools with participating FTC students, 36,106 of the students were in grades 3 to 10, which are the grades mandated for testing per s. 1002.395(8)(c)(2). Schools were instructed to submit students’ test scores to the independent research organization. If schools had any missing or invalid student scores, they were instructed to provide an explanation backed by evidence, most commonly in the form of a notarized letter, for each missing or invalid student score. Private school compliance Score reporting in 2014-15 The large majority of schools were in compliance with test score reporting for the academic year 2014-15. Regarding test score submission, most schools sent photocopied test score sheets that had been scored by the testing company. In a smaller number of cases where tests had been scored by the schools or hand-scored, schools were instructed to send detailed test administration and scoring procedures. Throughout the spring and summer of 2015 the Learning Systems 1 According to the former Project Director, David Figlio, the December roster is based on actual payments made to schools and is thus thought to contain a more precise representation of participating students than rosters from earlier in the school year. 3 Institute followed up with schools who had sent invalid test score results, including missing or incomplete test scores. Test score sheets were sent to the independent research organization where they were stored in a locked room. As test score data was received, two data entry staff members recorded students’ test scores and test information on a spreadsheet saved to a secure password-protected server. The scores were then reconciled with the hard copy scores to ensure the highest accuracy. Score sheets were shredded after this double-entry and reconciliation procedure as mandated by s. 1002.22(2)(d) of the Florida Statutes. To obtain information about prior public schooling records, the electronic database of students’ test scores, including information from student scholarship applications provided by the Scholarship Funding Organization, was sent to the Florida Department of Education (FLDOE) using its secured file share system. FTC student records were matched to FLDOE records in order to include information about students’ FCAT scores, public schooling history, free/reduced lunch status, limited English proficiency, and disability status. A unique FLDOE identification number replaced students’ identifying information. The FLDOE then returned via secure file share the matched and comparison data that were de-identified and stripped of any personal information. These de-identified data were then used for analysis. There were 1,285 FTC participating schools with students in the relevant grades in 2014-15. The vast majority of the FTC participating schools provided 4 evidence of test administration consistent with the specifications of the program. Eight participating schools, serving 66 testing-eligible students, closed or did not participate in the program following the 2014-15 school year and hence did not provide test scores. Table 1: Distribution of score reporting percentages: 2014-15 and prior years Academic year 06-07 07-08 Legible, valid scores received Not enrolled at time of testing Ineligible for testing School closed/suspended Student sick/absent Missing/unusable test 08-09 09-10 10-11 11-12 12-13 13-14 14-15 72.7 92.7 89.8 91.3 93.5 96.4 92.3 90.0 95.9 19.5 2.7 5.6 5.8 3.5 2.1 5.1 0.8 0.4 0.7 0.9 0.6 0.6 0.4 0.4 1.2 0.4 0.3 1.3 0.2 0.9 0.9 0.4 0.1 0.7 0.2 0.2 3.4 1 1.9 1.9 0.8 0.9 0.6 0.7 0.6 2.5 2.6 1.2 1.2 0.3 0.3 1.2 7.9 2.5 There were 36,106 students in relevant grades participating in the FTC program in 2014-15. Valid, legible test scores were received for 34,469 FTC students. It is thus fair to conclude that about 96 percent of all expected test scores were received. As seen in Table 1, the rate of legible, valid scores received in 2014-15 is the second-highest level of score reporting in program history and comparable to the highest level of score reporting (96.4 percent) that was observed in 2011-12. The main contributing factor for the increase in the percentage of legible, valid scores is 5 the decrease in the percentage of missing/unusable tests. In 2014-15, 2.5 percent of the expected test scores were missing or unusable. This rate was 7.9 percent in 2013-14. Moreover, the fraction of students not enrolled during testing, because they either left before testing or arrived after testing at the school, is at its lowest in 2014-15 compared to prior years. Only 0.4 percent of the expected students were not enrolled at the time of testing in 2014-15. This rate was 5.8 percent in 2009-10, 3.5 percent in 2010-11, 2.1 percent in 2011-12, 5.1 percent in 2012-13, and 0.8 percent in 2013-14 (See Table 1). The other categories of score reporting remained at levels comparable to those observed in recent years. The rate of schools closed or suspended was 0.2 percent; the rate of sick/absent students was 0.6 percent. Lastly, 0.4 percent of students on the official roster were either deemed ineligible for test score reporting pursuant to s. 1002.395(8)(c)(2) or were not enrolled in the school identified on the official rosters. Table 2: Distribution of percent and number of students with legible, valid scores: 2014-15 and prior years. Academic year Number of students Number of students with legible, valid scores Percent of students with legible, valid scores 200607 200708 200809 200910 201011 201112 201213 201314 201415 9,721 10,734 11,508 15,151 17,724 19,284 26,595 30,036 36,106 7,067 9,949 10,333 13,829 16,575 18,583 24,534 27,020 34, 469 72.7 92.7 89.8 91.3 93.5 96.4 92.3 90.0 95.9 6 In 2014-15 the number of students in relevant grades participating in the program is the highest compared to previous years. This is because of an overall increase in program participation in relevant grades as well as increase in the percentage of legible, valid scores in 2014-15. As can be seen in Table 2, the number of enrolled students in relevant grades increased over the years and reached 36,106 in 2014-15. 2 Comparison of students with legible, valid test scores to scholarship population Although the rate of successful score reporting was high in 2014-15 at 95.9 percent, there were about 4 percent of students whose expected scores were not received. Thus, it is still important to examine whether the students whose test scores were successfully reported are comparable to the population enrolled in 2014-15. For this analysis we used data from the families' scholarship applications. We found differences between students whose test scores were successfully reported and those whose scores were not successfully reported in terms of their family incomes, their parents’ marital status, their gender and race. This finding is consistent with previous years’ findings. As in previous years, students whose scores were successfully reported come from families with higher incomes (averaging $26,854 versus $23,423) and with parents more likely to be married (46.6 percent versus 37.8 percent). Moreover, students whose scores were successfully reported are more likely to be white (52.0 percent) and female (51.6 percent), compared to students with no test scores (49.6 percent white and 47.2 2 Although the highest level of score reporting observed in 2011-12, which was 96.4 percent, the number of students with legible, valid scores was 18,583 that year. This is almost half of the number of students with legible, valid scores in 2014-15. 7 percent female). Observing that students with reported scores were somewhat more advantaged than students with no reported scores as in previous years makes sense as highly mobile students are likely to be less advantaged, and are more likely to have not been tested because they changed schools. We should note that we cannot make any claims about whether students with missing test scores would have had higher or lower gain scores than those with test scores available. 3. TEST SCORES OF FTC STUDENTS IN 2014-15 We reported test scores in the form of national percentile rankings as in previous years’ reports. There is variation in the test administered by schools and the time of the year it is administered. Reporting test scores as national percentile rankings is common practice to ensure reasonable comparability across schools and program participants. There is no inherent bias associated with comparing the national percentile rankings of students taking different tests since the national percentile rankings indicates a student’s performance compared to a nationallyrepresentative group of students. Thus, reporting test scores in the form of national percentile rankings provides a common metric across different tests taken by students. Another advantage of using national percentile ranking is the ability to compare this year’s test scores of program participants to the test scores of FTC students in previous years. 8 Percentage of students Figure 1: Distribution of national percentile rankings of FTC students, 2014-15 Reading 1-9 8.8 10-19 10.2 20-29 11.8 30-39 11.6 40-49 10.9 50-59 10.9 60-69 10.5 70-79 10.0 80-89 8.3 90-99 7.1 Math 10.2 11.2 12.0 11.6 10.5 10.0 10.2 9.4 8.2 6.8 Figure 1 presents the basic distribution of national percentile rankings of FTC students participating in the program in 2014-15. The average national percentile ranking for FTC students was 47th percentile in reading and 46th percentile in mathematics in 2014-2015. In other words, the typical student in the FTC program scored at the 47th national percentile in reading and the 46th national percentile in mathematics. Average national percentile rankings in 2014-15 are very similar to national percentile rankings observed in prior years for both reading and mathematics. In fact, since the real-time test score collection began in 2006-07, the average national percentile rankings have varied by about a percentile point in reading and less than a percentile point in mathematics over the years including 2014-15. 9 Average test scores in 2014-15 by attributes of program participants We provided a breakdown of test scores of 2014-15 program participants by race, ethnicity, sex, and family income. Family income is expressed in terms of fraction of the poverty line taking into account the fact that families of different sizes have different official measures for poverty. Students from families who have incomes below 130 percent of the federal poverty line are eligible for free school meals, while those from families with incomes between 130 and 185 percent of the poverty line are eligible for reduced-price meals. Avg. National Percentile Ranking Figure 2: Average test scores of program participants in 2014-15 by their attributes Income 130185% of poverty Income > 185% of poverty All Students White Black Hispanic Male Female Income < 130% of poverty Reading 47.5 53.4 38.3 47.6 45.4 49.4 45.5 50.8 53.9 Math 46.0 52.0 36.3 46.4 46.0 46.0 44.2 49.2 50.9 10 As seen in Figure 2, white participants have higher mean national percentile rankings than minority participants. While mean national percentile rankings of males and females are not different in mathematics, females tend to perform better than males do in reading. Lastly, relatively high-income families tend to score better than relatively low-income families. These figures are quite similar to the figures reported in previous years. 4. GAIN SCORES FROM 2013-14 TO 2014-15 Test score gains for FTC students Test score gains for FTC students are calculated as required by the relevant Florida statutes. Gain scores can be interpreted as changes in national percentile rankings for program participants from 2013-14 to 2014-15 since test scores in both years are measured in terms of national percentile rankings. We should note that this analysis is vulnerable to ceiling effects (where students whose percentile rankings were high in 2013-14 cannot gain much more) and floor effects (where students whose percentile rankings were low in 2013-14 cannot lose much more ground). Ceiling and floor effects are of less concern for students whose initial national percentile ranking falls in the middle portions of the initial test score distributions, which is the case for the majority of students participating in the FTC Scholarship Program. 11 Gain scores were calculated for 18,807 FTC students with legible reading scores and 18,869 FTC students with legible mathematics scores in both 2013-14 and 2014-15. The mean gain score for FTC students is -1.1 national percentile ranking points in reading and -0.9 national percentile ranking points in mathematics. This means that the typical FTC student tends to maintain his or her relative position in comparison with others nationwide. It is important to note that these national comparisons pertain to all students nationally, and not just students from low-income families. However, we cannot make any claims about whether gain scores of FTC students would have been higher or lower if they were compared against only students from low-income families nationally. Figure 3: Distribution of test score gains for FTC students, 2013-14 to Percent of students with gains 2014-15 -40 and below -39 to -30 -29 to -20 -19 to -10 -9 to 0 1 to 10 11 to 20 21 to 30 31 to 40 Over 40 Reading 2.3 3.5 7.8 15.0 25.7 22.9 12.5 6.2 2.6 1.6 Math 2.9 3.5 7.6 14.2 25.1 23.0 13.2 6.1 2.5 2.0 12 Gain scores for 2014-15 are similar with previous years’ gain scores as they range from -1.2 to 0.0 for reading and from -2.4 to 0.0 for mathematics between 2008-09 to 2013-14. Moreover, as it was the case in previous years, considerable variation in individual student gain scores is observed in 2014-15 as well (see Figure 3); 10.4 percent of program participants gained more than 20 percentile points in reading relative to the nation between 2013-14 and 2014-15 (10.6 percent in math), and 13.6 percent of participants lost 20 or more percentile points in reading (14.0 percent in math). This suggests that, while some FTC students gained considerable ground relative to peers nationally, other FTC students lost considerable ground relative to national peers. School-level differences in average gain scores, 2013-14 to 2014-15 We calculated average gain scores from 2013-14 to 2014-15 at the school level as well. As mentioned in the preceding section, there is considerable variation in gain scores of individual students. Both individual level differences and school level differences contribute to this variation. By using gain scores aggregated to the school level, we examined the variation in gain scores across schools. It is important to note that observed between-schools variation doesn’t reflect “true” school-level differences since noise in individual test scores is still manifested as part of the school-level average gain scores. That said, examining school-level variation still provides further insights about the distribution of school gain scores. 13 Figure 4: Distribution of school average gains for FTC students, 2013-14 to School average gains 2014-15 -40 and below -39 to 30 -29 to -20 -19 to -10 -9 to 0 1 to 10 11 to 20 21 to 30 31 to 40 Over 40 Reading 0.7 0.7 1.4 10.4 42.7 34.4 7.2 1.4 0.6 0.5 Math 0.8 1.3 2.2 10.5 40.8 33.4 7.2 2.2 0.6 1.1 At the school level, the distribution of average gain scores is concentrated in the middle of the distribution (see Figure 4). The percent of schools with observed average gains of -20 percentile points or below is 2.8 percent for reading and 4.3 percent for math. These figures are 13.6 percent and 14.0 percent, respectively, at the individual-level. Similarly, 2.5 percent of schools have observed average gains of 20 percentile points or above in reading, and 3.9 percent of schools have observed average gains of 20 percentile points or above in math. This contrasts with 10.4 percent and 10.6 percent, respectively, of individual-level gains. As expected, much of the observed variability in gain scores is at the individual level. 14 Figure 5: Distribution of school average gains for FTC students, 2013-14 School average gains to 2014-15, schools with 10+ gain scores -40 and below -39 to -30 -29 to -20 -19 to -10 -9 to 0 1 to 10 11 to 20 21 to 30 31 to 40 Over 40 Reading 0.2 0.0 0.2 7.4 49.9 38.7 3.5 0.0 0.0 0.2 Math 0.2 0.2 0.2 6.7 49.9 38.8 3.7 0.2 0.0 0.2 The degree to which school-average gains reflect “true” school effects rather than noise increases as the number of students in the school increases. Hence, we looked at the same distribution this time only including schools with more than ten students. As can be seen in Figure 5, school-average gain scores become more compressed. The percent of schools with observed average gains of -20 percentile points or below is only 0.4 percent in reading and 0.6 percent in math. At the top of the average score distribution, the percent of schools with observed average gains of 20 percentile points or above is only 0.2 percent in reading and 0.4 percent in math. Moreover, the distribution of reading and math scores become almost identical. 15 Although the distribution of average gain scores for schools that have more than 10 students are more compressed, there still exists considerable variation. 7.8 percent of these schools have average reading gain scores lower than -10 percentile points and 7.3 of them have average math gain scores lower than -10 percentile points. At the top of the average score distribution, 3.7 percent of these schools have average reading gain scores higher than 10 percentile points. This figure is 4.1 percent for math. These findings suggest that there is a non-trivial between-school variability in the average gain scores, although it is not “true” school-level differences as a result of noise due to small sample sizes at the school level. Individual school average gain scores, 2013-14 to 2014-15 We calculated average gain scores for schools with 30 or more participating students as required by the relevant Florida statutes. It is important to note that average gain scores are not a definitive measure of a school’s performance. They only serve as one among many other indicators, of a school’s performance. The average gain score for a school in a single year can be an extremely noisy measure of a school's contribution to student test scores. This measure is less reliable for schools where a small number of students contribute to the average school gain score. As the number of students gets smaller in a given school, the likelihood of noise dominating the average gain score increases. Examining average gain scores only for schools with 30 or more participating students increased the likelihood of getting a more precise measure of average gain scores of individual schools. 16 In addition to the average gain scores for 2014-2015, we also calculated average gain scores over three years from 2012-13 through 2014-15. This added extra observations for schools and hence provided more accurate average gain scores for individual schools. Moreover, school gain scores calculated by a threeyear moving average of gain scores is less likely driven by “regression to the mean” compared to one-year average gain scores. Regression to the mean is the phenomenon that if a variable, such as a test score, is extreme on its first measurement, it will tend to be closer to the average on its second measurement and, if it is extreme on its second measurement, it will tend to have been closer to the average on its first. In this context, if a school had particularly high average scores in 2013-14, it is likely to observe a negative average gain score for that school in 2014-15. On the other hand, if a school had particularly low average scores in 2013-14, it is likely to observe a positive average gain score in 2014-15 for that school. Using average gain scores across the last three years balance out particularly positive and particularly negative scores over time, and thus helps to lessen the likelihood of making faulty inferences driven by regression to the mean. The risk of having faulty observed results due to regression to the mean is another reason to treat one-year average gain scores for individual schools extremely cautiously. Average gain scores for the 198 schools with more than 30 students enrolled in the FTC program in 2014-2015 are reported in the Appendix. Gain scores are reported for reading, mathematics, and combined reading and mathematics (by averaging schools’ average reading and mathematics scores) for 2014-15 as well as for the last three years’ average. Since a three-year moving average is a more 17 reliable measure of a school's average gain scores than one year’s gain scores, we based inferences on the three-year average gain scores. We identified schools with average gain scores that are statistically distinguishable from zero (at the 95 percent level of confidence in a two-tailed test). We highlighted the cells if the three years average gain score-either positively or negatively-was statistically significant from zero. While interpreting gain scores based on national percentiles, one should keep in mind that an average gain score of zero means that, on average, students in that school are maintaining their position relative to the national average. It doesn’t mean that students in that school are not gaining. If a school has statistically positive average gain, it means that, on average, students in that school improved their position relative to the national average (with 95% certainty). If a school has statistically negative average gain, it means that, on average, students in that school worsened their position relative to the national average (with 95% certainty). 5. ATTRIBUTES OF NEW PROGRAM PARTICIPANTS IN 2014-15 Previous reports revealed that FTC students tend to be among the most struggling students and are more disadvantaged than presumably eligible nonparticipant students. We examined attributes of new FTC students in 2014-15 in order to see whether they were systematically different from eligible nonparticipant students before participating in the FTC program in 2014-15 as well. In order to make plausible comparisons among students who spent the 2013-14 academic year in Florida public schools, we compared students who 18 entered the FTC Scholarship Program in 2014-15 versus subsidized school meal eligible students who did not enter the program in that year but stayed free or reduced-price lunch eligible in 2014-15. We excluded students with disabilities who could participate in the McKay Scholarship Program. We limited the analysis to students who had taken either a reading or math test in public school in 2013-14. We also restricted analysis to students who would be in grade 10 or below in 201415.3. With these criteria, we compared 3,427 new students in the FTC Scholarship program in 2014-15 versus 612, 501 students who remained in the public schools and continued on subsidized school lunches in 2014-15. We used Florida Department of Education records for these comparisons. Comparison of new FTC students and non-participant students in terms of their characteristics New FTC students in 2014-15 are more likely to be black, and less likely to be Hispanic or white than non-participant eligible students as seen in Figure 6. Also, they are less likely to be English-language learners than are non-participants. While both new FTC students and non-participant students were eligible for subsidized lunch in the 2013-14 school year, the share of new FTC students who were freelunch eligible is higher than the share of free-lunch eligible, non-participant students. Lastly, compared to eligible non-participant students, new FTC students had poorer test performance both in reading and math before entering the FTC program. These differences suggest that new FTC students in 2014-15 were 3 Students who were in grade 10 in 2013-14 are excluded since they are not tested in 2014-15. 19 relatively more disadvantaged and lower-performing prior to entering the FTC program than free-lunch eligible, non-participant students. These observed differences are similar to the observed differences reported in previous reports. Figure 6: Comparison of prior year characteristics of new FTC students to "income Percentage/avg. percentile eligible" non-participant students, 2014-15 Percent Black Participants 2014-15 Eligible non-participants 2014-15 Percent Percent Percent Hispanic White ESL Percent Free Lunch Math Percentile (FCAT) Reading Percentile (FCAT) 41.4 32.7 21.6 27.0 88.4 40.6 40.3 30.5 39.4 24.9 30.0 87.0 45.9 45.3 Comparison of new FTC students and non-participant students in terms of performances of their schools in 2013-14 In Florida, each school is assigned a school grade (A-F) based on student performance. We compared new FTC students and eligible non-participant students in terms of performances of the schools that they attended in the 2013-2014 school 20 year. We observed that students who entered the FTC program in 2014-15 came from lower-performing schools. On a scale of A-F, with A being the highest performing schools, 23.6 percent of new FTC students were in schools graded "A", before attending a school in the FTC program, while 29.4 percent of eligible nonparticipant students were in schools graded “A” in 2013-14 school year. At the other end of the spectrum, 25.1 percent of new FTC students were in schools graded "D" or "F", as compared with 17.1 percent of eligible non-participant students who were in schools graded "D" or "F" (see Figure 7). Figure 7: Comparison of new FTC students in 2014-15 to eligible non-participant Percent of schools in each Performance grade students by performances of the schools they attended in 2013-14 Grade A Schools Grade B Schools Grade C Schools Grade D Schools Grade F Schools Participants 2014-15 23.6 17.3 34.1 17.1 8.0 Eligible non-participants 2014-15 29.4 19.8 33.7 12.4 4.7 21 Comparison of new FTC students and non-participant students within their schools in terms of performances in 2013-14 We also examined new FTC students’ performances relative to eligible nonparticipant students in their own schools before entering the FTC program. Regardless of the performance of the school that new program participants were in, they tended to be lower-performing students relative to eligible non-participant students in their schools before entering the FTC program (see Figure 8). 22.8 percent of new FTC students in 2014-15 were in the bottom fifth of their prior public school's mathematics FCAT test score distribution, versus 19.0 percent of eligible non-participating students who were in the bottom fifth of the distribution. Moreover, 13.3 percent of new FTC students were in the top fifth of the distribution, as compared with 17.5 percent of eligible non-participating students in the top fifth of the distribution. Figure 8: Comparison of new FTC students in 2014-15 to eligible non-participant Percent of students in each group students by quintile of school mathematics FCAT score distribution Participants 2014-15 Eligible nonparticipants 2014-15 Bottom fifth Second fifth Third fifth Fourth fifth Top fifth 22.8 23.1 21.3 19.5 13.3 19.0 21.9 21.5 20.1 17.5 22 The same pattern was observed for reading FCAT test score distribution; 23.2 percent of new FTC students were in the bottom fifth of their prior public school’s reading distribution, while 19.1 percent of non-participating eligible students were in the bottom fifth of the distribution. At the top of the reading test score distribution, the gap between relative test performances of new FTC students and eligible non-participant students was 3.1 percentage points, instead of the 4.2 percentage point gap observed in mathematics. These findings suggest that FTC students are more likely to be low performing students in their schools before attending the program. This observation has not changed over time as similar figures were observed in the previous program reports. 6. PERFORMANCE OF PROGRAM PARTICIPANTS WHO RETURN TO FLORIDA PUBLIC SCHOOLS In this section we compared FTC students who returned to public schools in 2014-15 after participating in the FTC program to those who remained in the FTC program in 2014-15. We also compared program returnees to Florida public school students who never left the public schools. It is important to note that one cannot make any claims about the effects of participation in the FTC program based on these comparisons, as there are likely factors beyond FTC participation that may influence students’ performance. These comparisons only provide additional insights about the performance of the students who participate in the FTC program. 23 Comparison of 2013-14 performances of public school returnees and FTC stayers in 2014-15 We first compared FTC students who returned to the public school system in Florida in 2014-15 versus those who remained in private schools under the FTC program in terms of their national norm-referenced test performance in 2013-14. The typical student who left the program scored at the 43.0th national percentile in reading and 42.2nd national percentile in mathematics in 2013-14 while the typical FTC student who remained in the program in 2014-15 scored at the 49.1st national percentile in reading and the 46.9th national percentile in math (See Figure 9). Figure 9: 2013-14 test score performance of students remaining in the FTC Average percentile rank, 2013-14 program in 2014-15 versus those who leave the program 2013-14 reading percentile 2013-14 math percentile FTC stayers in 2014-15 Public school returnees in 2014-15 Public school returnees eligible for subsidized lunch 49.1 43.0 42.0 46.9 42.2 41.4 24 This finding can be an understatement of the difference between these two groups, since all students who remained in the FTC program were still incomeeligible to participate while some students who left the program may not meet eligibility criteria anymore in 2014-15. In order to have more comparable groups in terms of income range, we limited the public school returnees to those participating in the National School Lunch Program in 2014-15. We found that the average returnee who is free/reduced lunch eligible in 2014-15 scored at the 42.0th national percentile in reading and scored at 41.4th national percentile in mathematics in 2013-14, somewhat lower than the performance of all returnees as expected. These findings suggest that as lower-performing public school students eligible for the FTC program are more likely to leave public schools to attend a private school under the FTC program, FTC students who struggle the most in private schools are somewhat more likely to return to the public schools. This is consistent with previous years’ observations. Comparison of 2014-15 FSA performances of public school returnees and low income public school students Next, we compared performance of FTC students who returned to the public schools and performances of subsidized-meal eligible public school students who never participated in the FTC program. As can be seen in Figure 10, FTC program participants who return to the public schools performed worse on the FSA than did other subsidized-meal recipients who never participated in the FTC program. The difference is particularly large for FTC returnees in 2014-15, who performed at the 25 36.5th Florida percentile in English Language Arts (ELA) and 33.9th Florida percentile in math while public school students who never participated in the FTC program performed at the 42.7th Florida percentile in ELA and 43.6th Florida percentile in math in 2014-15. Figure10: 2014-15 FSA performance of FTC students returning to public Average FSA percentile schools in Florida 2014-15 FSA percentile for lowincome students who never participated 2014-14 FSA percentile for FTC returnees in 2014-15 2013-14 FCAT percentile for FTC returnees before 2013-14 ELA 42.7 36.5 37.4 Math 43.6 33.9 37.0 As we mentioned before, based on these comparisons one cannot make any claims about the effects of participation in the FTC program since evidence suggests that FTC students who returned to the public schools in 2014-15 and public school students who never participated FTC program represent two different populations of students. Findings indicated that poorly performing public school students are more likely to participate in the program in the first place. Moreover, FTC students 26 who return to public schools tend to be those who are performing worse than the average FTC students. Based on these observations, we cannot associate poor performance of FTC returnees with possible negative effects of the FTC program on participating students. 7. CONCLUSION This report shares findings on the compliance and performance of private schools that participated in the Florida Tax Credit Scholarship Program in 2014-15. Compliance with program testing requirements is high in 2014-15. Private schools reported test scores for 95.9 percent of program participants in grades 3-10. FTC students scored at the 47th national percentile in reading and the 46th national percentile in mathematics in 2014-15. These scores are similar to previous years’ scores. In terms of gain in national percentile ranking points from 2013-14 to 2014-15, the typical FTC student tends to maintain his or her relative position in comparison with all students nationally both in reading and mathematics. It is important to note that these national comparisons pertain to all students nationally, and not just students from low-income families. However, we cannot make any claims about whether gain scores of FTC students would have been higher or lower if they were compared against only students from low-income families nationally. There is considerable variation in individual student gain scores. While some FTC students gain considerable ground relative to peers nationally, other FTC students lose considerable ground relative to national peers. While at the school level, the distribution of average gain scores is concentrated in the middle of the 27 distribution there is still non-trivial between-school variability in the average gain scores. As in prior years, lower-performing public school students eligible for the FTC program are more likely to attend a private school under the FTC program and FTC students who struggle the most in private schools are more likely to return to the public schools. FTC students who return to the public schools in Florida have substantially lower test scores than other subsidized-meal eligible public school students who never participated in the FTC program. However, based on the available evidence (e.g., selection of students into and out of the FTC program), poor performance of FTC returnees cannot be associated with possible negative effects of the FTC program on participating students. 28 APPENDIX Appendix Table: Average gain scores in 2014-15 and three-year moving average of gain scores from 2012-13 to 2014-15 for schools with 30 or more gain scores in 20134-15, ranked by alphabetical order. NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Abundant Life Christian Academy (ST) Academy Prep Center Of St. Petersburg (ST) Academy Prep Center Of Tampa Inc. (ST) Agape Christian Academy (ST) Alazhar School (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Margate 68 184 0.36 2.76 -2.04 1.88 3.45 0.32 Saint Petersburg 61 150 9.69 8.72 10.66 5.39 4.83 5.94 Tampa 75 194 0.22 0.04 0.40 2.12 2.15 2.08 Orlando 45 141 -4.28 -4.04 -4.51 -3.05 -1.55 -4.55 Tamarac 57 149 6.07 2.70 9.44 2.79 2.40 3.17 CITY READING+ MATH READING COMBINED MATH Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Altamonte Christian School + (ST) American Christian School Art Center (BA) American Youth Academy Inc. (CT) Annunciation School (IT) Archbishop Curley/Notre Dame High School * Arlington Country Day School (ST) Atlantic Christian Academy Of The Palm Beach + (ST) Azalea Park Baptist School (ST) Berean Christian School * AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Altamonte Springs 35 35 25.00 4.26 0.74 2.50 4.26 0.74 Hialeah 49 115 9.91 15.19 4.62 12.26 13.27 11.24 Tampa 101 273 -0.58 0.18 -1.35 2.24 3.63 0.86 Hollywood 44 98 -2.33 -3.45 -1.20 -1.10 -1.35 -0.85 Miami 73 195 -10.53 -9.19 -11.86 -5.53 -4.67 -6.39 Jacksonville 30 80 12.63 10.43 14.83 3.48 3.11 3.84 West Palm Beach 30 30 2.20 -0.40 4.80 2.20 -0.40 4.80 Orlando 41 102 0.95 1.29 0.61 2.79 3.39 2.20 West Palm Beach 45 105 4.40 6.09 2.71 0.89 1.70 0.08 CITY READING+ MATH READING COMBINED MATH 30 Appendix continued NUMBER OF GAIN SCORES OBSERVED AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Opa-locka 45 150 -5.02 -2.67 -7.38 0.27 -1.04 1.59 North Miami Beach 43 101 -5.38 -6.19 -4.58 -2.50 -5.18 0.18 Jacksonville 34 86 4.72 8.15 1.29 -1.73 -0.76 -2.71 Blessed Trinity (IT) Ocala 64 174 -5.06 -6.91 -3.22 -2.30 -1.82 -2.78 Bradenton Christian School * Bradenton 44 106 -3.03 -8.39 2.32 -2.00 -2.66 -1.33 Brito Miami Private School (ST) Miami 55 128 9.74 9.65 9.82 1.88 1.47 2.30 Broward Junior Academy (IT) Plantation 63 151 -4.06 -11.08 2.97 -4.56 -7.74 -1.38 Orlando 64 158 -2.34 -2.31 -2.38 -1.19 -0.53 -1.84 37 100 -0.92 -3.38 1.54 0.61 2.46 -1.25 66 148 -1.41 -0.08 -2.74 -3.04 -1.98 -4.09 SCHOOL NAME Betesda Christian School (TN) Beth Jacob High School Inc. * Bishop Kenny High School * Brush Arbor Christian School (ST) Calvary Chapel Academy (TN) Calvary Christian Academy * CITY West Melbourne Fort Lauderdale READING+ MATH READING COMBINED MATH 31 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Calvary Christian Academy (ST) Candlelight Christian Academy (PS) Cedar Creek Christian School (ST) Cedar Hills Baptist Christian School (ST) Central Baptist Christian School + (TN) Central Pointe Christian Academy + (ST) Champagnat Catholic School Of Hialeah (ST) Children's Rainbow Dayschool Academy (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Ormond Beach 52 131 -0.21 1.35 -1.77 1.99 2.74 1.24 Lake Wales 42 111 -0.76 -1.83 0.31 1.13 -0.73 2.99 Jacksonville 35 104 0.99 -2.00 3.97 0.16 -0.91 1.24 Jacksonville 40 91 2.23 3.75 0.70 2.69 2.31 3.07 Brandon 32 32 -6.63 -2.69 -10.56 -6.63 -2.69 -10.56 Kissimmee 38 38 -4.72 -6.55 -2.89 -4.72 -6.55 -2.89 Hialeah 45 174 5.93 7.60 4.27 13.10 12.07 14.13 Goulds 40 87 3.64 7.38 -0.10 1.59 3.53 -0.34 CITY READING+ MATH READING COMBINED MATH 32 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Christ-Mar Private School (ST) City Of Life Christian Academy (TN) Classical Christian School For The Arts Inc. + (ST) Colonial Christian School (ST) Community Christian Academy (ST) Community Christian Learning Center (ST) Community Christian School (TN) Covenant Christian School * AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Hialeah 30 94 5.52 2.73 8.30 1.46 0.93 1.99 Kissimmee 91 221 -3.58 -4.64 -2.52 -3.12 -2.95 -3.29 Pinellas Park 39 39 1.09 3.05 -0.87 1.09 3.05 -0.87 Homestead 56 146 4.59 5.30 3.88 1.45 1.36 1.54 Stuart 33 33 -0.11 1.24 -1.45 -0.11 1.24 -1.45 Apopka 47 123 -6.52 -5.72 -7.32 -1.07 -1.23 -0.92 Port Charlotte 63 127 -4.58 1.79 -10.95 -2.99 0.60 -6.58 Palm Bay 47 115 -10.59 -9.19 -11.98 -3.52 -3.50 -3.54 CITY READING+ MATH READING COMBINED MATH 33 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME CITY AVERAGE GAIN SCORE IN 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 READING+ MATH READING COMBINED MATH Downey Christian School (ST) Orlando 37 37 -1.36 -2.03 -0.70 -1.36 -2.03 -0.70 Dr. John A. Mckinney Christian Academy (ST) Miami 35 35 3.33 2.20 4.46 3.33 2.20 4.46 Eagle's View Academy (ST) Jacksonville 32 102 -2.39 -1.66 -3.13 -3.37 -2.83 -3.91 Eastland Christian School (ST) Orlando 68 160 -1.60 -1.57 -1.62 -0.39 -0.14 -0.64 Hialeah 100 252 -0.09 1.86 -2.04 -0.38 0.13 -0.89 New Port Richey 59 151 3.26 2.86 3.66 3.26 1.24 5.28 Jacksonville 44 134 -8.59 -3.66 -13.52 -1.67 -0.65 -2.69 Faith Christian Academy (TN) Orlando 109 275 3.06 3.85 2.27 -0.81 0.55 -2.17 Faith Lutheran School (ST) Hialeah 34 108 2.47 1.32 3.62 3.44 1.55 5.32 Edison Private School (ST) Elfers Christian School (ST) Esprit De Corps Center For Learning (TN) 34 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Faith Outreach Academy (ST) Family Christian School Of Clermont (ST) Father Lopez High School (PS) First AcademyLeesburg (TN) First Assembly Christian School Daycare (TN) AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Tampa 46 129 2.41 4.35 0.48 -1.16 -0.22 -2.10 Clermont 31 31 -2.16 -3.00 -1.32 -2.16 -3.00 -1.32 Daytona Beach 31 31 -6.35 -9.58 -3.13 -6.35 -9.58 -3.13 Leesburg 42 99 0.02 -3.74 3.79 -1.16 -2.52 0.20 Ocala 45 110 -9.49 -2.80 -16.18 -1.42 -0.22 -2.62 100 248 -3.41 -1.92 -4.90 -3.54 -2.18 -4.91 Altamonte Springs 44 97 4.47 3.27 5.66 -1.70 -1.33 -2.07 Apopka 42 42 1.05 4.60 -2.50 1.05 4.60 -2.50 Longwood 75 204 5.51 4.29 6.72 0.66 0.88 0.44 CITY First Coast Christian Jacksonville School (ST) Forest City S.D.A. (IT) Forest Lake Academy + (AC) Forest Lake Education Center (IT) AVERAGE GAIN SCORE IN 2014-15 READING+ MATH READING COMBINED MATH 35 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Freedomland Christian Academy + (ST) Garden Of The Sahaba Academy (TN) Good Shepherd Catholic School (IT) Greater Miami Academy (IT) Hampden Dubose Academy + (ST) Hebrew Academy Community School (IT) Heritage Preparatory School (ST) Hernando Christian Academy + (TN) Highlands Christian Academy (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Kissimmee 40 40 1.88 -0.03 3.78 1.88 -0.03 3.78 Boca Raton 43 117 -4.02 2.23 -10.28 -2.41 -0.94 -3.88 Orlando 41 100 -3.93 -7.93 0.07 0.62 0.79 0.45 Miami 93 242 3.86 5.33 2.39 1.33 3.03 -0.36 Zellwood 33 33 -1.88 -1.94 -1.82 -1.88 -1.94 -1.82 Margate 47 92 2.91 2.89 2.94 2.46 2.47 2.46 Orlando 65 166 -4.38 -2.75 -6.02 -1.92 -0.70 -3.14 Brooksville 30 30 -0.88 1.10 -2.87 -0.88 1.10 -2.87 Pompano Beach 61 141 -3.02 -3.05 -2.98 -1.67 -3.10 -0.24 CITY READING+ MATH READING COMBINED MATH 36 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Hobe Sound Christian Academy + (ST) Holy Family Catholic School + (IT) Holy Family Catholic School (IT) Holy Redeemer Catholic School (IT) Holy Rosary Catholic School (IT) Horeb Christian School (ST) I.E.C. Christian Academy (TN) Ibn Seena Academy (TN) Immaculate Conception Catholic School (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 CITY 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING+ MATH READING COMBINED READING MATH Hobe Sound 31 31 3.34 -0.06 6.74 3.34 -0.06 6.74 Orlando 33 33 -12.27 -15.21 -9.33 -12.27 -15.21 -9.33 North Miami 85 220 -3.08 -3.24 -2.92 -0.92 -0.27 -1.58 Kissimmee 72 141 -1.55 -4.26 1.17 -0.55 -1.06 -0.04 Jacksonville 55 131 -5.21 -8.75 -1.67 -2.95 -3.95 -1.95 Hialeah 42 95 8.65 7.67 9.64 6.25 6.27 6.23 Orlando 44 98 -2.65 3.05 -8.34 -1.28 1.47 -4.02 Orlando 37 102 2.39 2.65 2.14 1.76 0.86 2.67 Hialeah 56 139 -3.47 -7.50 0.55 1.12 0.40 1.83 MATH 37 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Incarnation Catholic School + (IT) Indian Rocks Christian School + (TN) Inverness Christian Academy (ST) Iva Christian School (ST) Jose Marti School 3rd Campus (ST) Jubilee Christian Academy (TN) Kingsway Christian Academy (ST) La Progresiva Presbyterian School Inc. (ST) Lakeside Christian School (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Tampa 32 32 -2.00 -1.41 -2.59 -2.00 -1.41 -2.59 Largo 41 41 -0.21 -1.27 0.85 -0.21 -1.27 0.85 Inverness 35 99 2.29 2.29 2.29 1.54 0.96 2.12 Largo 43 102 -5.00 -5.95 -4.05 1.50 0.42 2.57 Miami 56 144 -3.55 -3.55 -3.55 7.52 6.56 8.49 Pensacola 35 85 3.10 3.29 2.91 -2.22 -0.33 -4.12 Orlando 122 343 -1.14 -3.28 1.01 -1.24 -1.25 -1.23 Miami 48 273 11.88 14.96 8.79 3.99 5.36 2.61 Clearwater 47 124 0.59 -0.45 1.62 2.54 2.91 2.17 CITY READING+ MATH READING COMBINED MATH 38 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Leaders Preparatory School (ST) Liberty Christian Preparatory School + (ST) Life Assembly Of God Life Academy (ST) Lighthouse Christian Academy (ST) Lincoln-Marti Community Agency 10 (ST) Lincoln-Marti Community Agency 17 (ST) Lincoln-Marti Community Agency 23 (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Orlando 36 113 -10.65 -9.58 -11.72 0.96 0.06 1.85 Tavares 32 32 3.03 1.81 4.25 3.03 1.81 4.25 Kissimmee 50 150 -0.99 -0.42 -1.56 -2.21 -2.05 -2.37 Deland 45 134 4.89 3.76 6.02 2.63 2.49 2.76 Miami 109 349 -2.99 1.59 -7.56 0.39 1.05 -0.28 Miami 88 267 7.28 7.00 7.57 3.21 0.71 5.71 Miami 67 154 -3.15 1.24 -7.54 -7.00 -5.34 -8.66 CITY READING+ MATH READING COMBINED MATH 39 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Lincoln-Marti Community Agency 28 (ST) Lincoln-Marti Community Agency 76 (ST) Little Flower School + (IT) Masters Preparatory School + (ST) Meadowbrook Academy Inc. (ST) Melody Christian Academy (ST) Miami Union Academy (IT) Monsignor Edward Pace High School (PS) Morningside Academy + (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Miami 88 203 -13.14 -13.51 -12.76 -3.48 -3.69 -3.27 Miami 43 101 8.19 16.3 0.07 2.71 8.88 -3.47 Hollywood 35 35 -4.83 -10.14 0.49 -4.83 -10.14 0.49 Hialeah 59 59 5.12 4.95 5.29 5.12 4.95 5.29 Ocala 43 111 1.91 -1.26 5.07 1.74 1.39 2.10 Live Oak 53 150 -2.36 1.55 -6.26 -2.90 -1.05 -4.75 North Miami 69 259 0.12 -4.91 5.16 -1.59 -2.64 -0.55 Miami Gardens 75 190 -12.37 -10.68 -14.05 -8.88 -7.43 -10.34 Port Saint Lucie 38 38 -3.20 -6.26 -0.13 -3.20 -6.26 -0.13 CITY READING+ MATH READING COMBINED MATH 40 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Mother Of Christ Catholic School (IT) Muslim Academy Of Greater Orlando (ST) North Florida Christian School (ST) North Florida Educational Institute + (ST) North Kissimmee Christian School (ST) Northside Christian Academy (TN) Northwest Christian Academy (TN) Nur Ul-Islam Academy (ST) Oasis Christian Academy (TN) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Miami 44 104 -1.08 -2.95 0.80 -0.93 0.28 -2.14 Orlando 60 144 3.70 4.67 2.73 3.40 2.99 3.81 Tallahassee 45 139 -2.86 -4.04 -1.67 0.58 -0.19 1.35 Jacksonville 50 50 -5.26 -4.00 -6.52 -5.26 -4.00 -6.52 Kissimmee 47 131 -2.03 0.89 -4.96 -0.78 -0.24 -1.32 Starke 38 105 -3.75 -3.42 -4.08 -1.79 -0.70 -2.87 Miami 62 165 -4.85 -0.37 -9.34 -1.92 -0.42 -3.42 Cooper City 101 305 3.23 4.57 1.88 2.45 2.69 2.21 Winter Haven 37 97 -1.11 0.76 -2.97 0.62 0.87 0.37 CITY READING+ MATH READING COMBINED MATH 41 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Okeechobee Christian Academy + (TN) Orlando Christian Prep (ST) Orlando Junior Academy (IT) Our Lady Of Charity School Inc (TN) Our Lady Of Lourdes Catholic School (IT) Our Lady Of The Holy Rosary-St Richard Cath (IT) Our Lady Of The Lakes Catholic School + (IT) Our Lady Queen Of Martyrs (IT) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 CITY 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING+ MATH READING COMBINED READING MATH Okeechobee 33 33 -9.03 -8.09 -9.97 -9.03 -8.09 -9.97 Orlando 64 123 -0.53 3.44 -4.50 -1.31 1.45 -4.07 Orlando 45 117 3.44 1.84 5.04 -0.49 0.44 -1.42 Hialeah 55 143 0.53 -2.15 3.20 -1.35 -0.91 -1.79 Daytona Beach 47 127 -1.85 -3.68 -0.02 1.25 1.31 1.18 Miami 39 96 -0.62 -10.44 9.21 -0.59 -2.36 1.18 Miami Lakes 36 36 -1.15 -4.22 1.92 -1.15 -4.22 1.92 Fort Lauderdale 45 97 -9.20 -11.56 -6.84 -3.84 -3.71 -3.97 MATH 42 Appendix continued NUMBER OF GAIN SCORES OBSERVED AVERAGE GAIN SCORE IN 2014-15 SCHOOL NAME CITY 2014-15 SCHOOL YEAR Palm Beach Bilingual School (IT) Riviera Beach 43 101 -12.84 -22.86 -2.81 -6.94 -13.14 -0.73 Titusville 37 37 -7.28 -8.35 -6.22 -7.28 -8.35 -6.22 Jacksonville 37 101 -3.15 -2.24 -4.05 -2.15 -1.52 -2.77 Orlando 35 119 1.63 1.86 1.40 0.33 1.09 -0.43 Palatka 39 107 -3.53 -0.56 -6.49 -2.58 -0.80 -4.36 Miami 47 130 2.31 -0.77 5.38 4.92 2.29 7.54 Kissimmee 77 77 0.58 -1.13 2.30 0.58 -1.13 2.30 Phyl's Academy * Coral Springs 33 107 -0.74 -0.06 -1.42 -1.29 -1.45 -1.13 Potter's House Academy (ST) Orlando 36 103 -3.51 -1.75 -5.28 -2.99 -1.23 -4.75 Park Avenue Christian Academy + (TN) Parsons Christian Academy (ST) Pathways School (ST) Peniel Baptist Academy (ST) Pentab Academy (ST) Pha Preparatory School Kissimmee + (ST) BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 READING+ MATH READING COMBINED MATH 43 Appendix continued NUMBER OF GAIN SCORES OBSERVED AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 CITY 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Miami Beach 30 82 -0.45 2.60 -3.50 -1.30 0.59 -3.20 Orlando 36 91 2.40 4.39 0.42 0.93 1.43 0.44 Real Life Christian Academy (TN) Clermont 40 95 3.78 0.78 6.78 -1.97 -4.89 0.95 Regency Christian Academy (ST) Orlando 38 90 0.97 3.26 -1.32 1.28 2.12 0.44 Jacksonville 30 30 -0.13 -7.67 7.40 -0.13 -7.67 7.40 Ave Maria 35 35 0.61 -0.80 2.03 0.61 -0.80 2.03 Riviera Beach 49 136 0.36 2.88 -2.16 -5.55 -4.42 -6.68 Niceville 32 87 2.63 -1.06 6.31 0.75 -2.24 3.75 SCHOOL NAME Rabbi Alexander S. Gross Hebrew Academy * Radiant Life Academy (ST) Resurrection Parish School + (IT) Rhodora J. Donahue Academy +* Rj Hendley Christian Community School (ST) Rocky Bayou Christian School Nfcea (ST) READING+ MATH READING COMBINED MATH 44 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME CITY AVERAGE GAIN SCORE IN 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 READING+ MATH READING COMBINED MATH S.L. Jones Christian Academy (ST) Pensacola 46 102 -7.71 -8.67 -6.74 -10.5 -10.86 -10.14 Sacred Heart (IT) Jacksonville 46 137 -7.42 -12.35 -2.50 -0.89 -1.33 -0.45 Pinellas Park 34 34 -5.75 -7.62 -3.88 -5.75 -7.62 -3.88 Lake Worth 35 35 -5.74 -8.03 -3.46 -5.74 -8.03 -3.46 38 38 -9.18 -15.21 -3.16 -9.18 -15.21 -3.16 Orlando 84 157 -1.19 -5.29 2.90 -0.14 -1.73 1.44 Deland 33 33 -4.65 -7.67 -1.64 -4.65 -7.67 -1.64 Miramar 44 112 -2.59 -3.66 -1.52 -1.76 -0.13 -3.38 Miami 40 40 -4.06 -6.98 -1.15 -4.06 -6.98 -1.15 Sacred Heart Catholic School + (IT) Sacred Heart School + (IT) Saint Agatha School Miami + (IT) Saint Andrew Catholic School (IT) Saint Barnabas Episcopal School + (IT) Saint Bartholomew School (IT) Saint Brendan Elementary School + (IT) 45 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Saint Helen Catholic School (IT) Saint James Catholic School (IT) Saint John The Apostle School (IT) Saint Johns Episcopal School (ST) Saint Joseph Catholic School (IT) Saint Joseph Parish School (IT) Saint Joseph School + (IT) Saint Lawrence School (IT) Saint Marys Cathedral (IT) Saint Michael The Archangel (IT) Saint Paul Catholic School (IT) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Fort Lauderdale 54 157 -2.38 -4.85 0.09 -2.15 -0.34 -3.97 Miami 131 329 -0.93 -1.23 -0.63 0.06 1.07 -0.95 Hialeah 90 237 3.11 2.24 3.97 3.58 4.99 2.18 Homestead 41 106 0.46 5.29 -4.37 0.87 2.94 -1.20 Winter Haven 49 130 -2.67 -6.29 0.94 -1.24 -1.71 -0.78 Tampa 45 109 -3.02 -5.73 -0.31 0.37 -1.39 2.13 Jacksonville 36 36 -1.82 -4.19 0.56 -1.82 -4.19 0.56 North Miami Beach 53 125 -0.05 -0.83 0.74 0.43 1.51 -0.65 Miami 137 369 -0.56 -0.39 -0.73 -0.79 -0.24 -1.35 Miami 72 183 -3.27 -3.86 -2.68 0.18 0.80 -0.44 Daytona Beach 55 115 -4.54 -5.38 -3.69 -2.47 -1.89 -3.06 CITY READING+ MATH READING COMBINED MATH 46 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Saint Peter Claver + (IT) Saint Petersburg Christian School + (ST) Saint Pius V Catholic School (IT) AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Tampa 43 43 -2.44 0.05 -4.93 -2.44 0.05 -4.93 Saint Petersburg 31 31 -1.68 -0.32 -3.03 -1.68 -0.32 -3.03 Jacksonville 43 109 -4.73 -5.00 -4.47 -1.81 -2.77 -0.85 35 35 -3.34 -2.77 -3.91 -3.34 -2.77 -3.91 Seffner 40 40 -4.61 -1.68 -7.55 -4.61 -1.68 -7.55 Lecanto 38 38 -7.30 -10.63 -3.97 -7.30 -10.63 -3.97 Hialeah 40 99 11.48 10.25 12.70 8.17 7.92 8.41 Fort Myers 37 91 -0.45 -0.32 -0.57 -1.06 -1.74 -0.38 Orlando 63 197 0.19 1.48 -1.10 3.66 3.90 3.43 CITY Saints Academy Inc. Orlando + (ST) Seffner Christian Academy + (TN) Seven Rivers Christian School + * Snow White The Seven Dwarfs School (IT) Sonshine Christian Academy (ST) South Orlando Christian Academy (ST) AVERAGE GAIN SCORE IN 2014-15 READING+ MATH READING COMBINED MATH 47 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME CITY AVERAGE GAIN SCORE IN 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 READING+ MATH READING COMBINED MATH Southland Christian School (ST) Kissimmee 93 244 3.95 4.47 3.42 1.89 1.93 1.84 Spring Hill Christian Academy + (ST) Spring Hill 30 30 -1.60 -0.47 -2.73 -1.60 -0.47 -2.73 Palm Coast 42 103 -3.18 -5.50 -0.86 -0.95 -1.43 -0.47 Port Saint Lucie 82 144 -2.89 -0.54 -5.24 -3.76 -2.12 -5.40 Saint Cloud 48 109 -5.47 -10.48 -0.46 -1.85 -4.08 0.39 Deland 48 103 -0.98 1.38 -3.33 0.18 0.02 0.34 Miami 137 377 -0.96 -3.82 1.90 -1.53 -5.21 2.15 Havana 48 124 1.61 -1.19 4.42 -0.45 -1.10 0.20 Tampa 38 115 7.76 4.53 11.00 2.57 3.85 1.30 St. Elizabeth Ann Seton Catholic School (IT) St. James Christian Academy (IT) St. Thomas Aquinas School (IT) Stetson Baptist Christian School (ST) Sunflowers Academy (IT) Tallavana Christian School (ST) Tampa Adventist Academy (IT) 48 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Temple Christian Academy (BA) The Conrad Academy (ST) The Potter's House Christian Academy Elem (ST) Thinking Child Christian Academy School + (ST) Toras Emes Academy Of Miami (ST) Treasure Of Knowledge Christian Academy (ST) Trinity Catholic High School + (PS) Trinity Christian Academy (TN) Trinity Christian Academy (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Jacksonville 35 99 -0.60 1.63 -2.83 -1.40 -0.58 -2.22 Orlando 48 109 -4.52 -0.94 -8.10 0.80 2.89 -1.29 Jacksonville 60 164 -0.92 2.78 -4.62 -2.68 1.09 -6.45 Homestead 31 31 7.18 7.19 7.16 7.18 7.19 7.16 North Miami Beach 51 131 -6.38 -5.14 -7.63 -2.96 -1.47 -4.44 Orlando 35 106 0.97 4.03 -2.09 1.66 2.15 1.16 Ocala 31 31 -13.21 -8.10 -18.32 -13.21 -8.10 -18.32 Lake Worth 54 131 -3.26 -2.00 -4.52 -1.06 0.66 -2.79 Deltona 105 256 0.04 1.16 -1.08 -0.42 0.71 -1.55 CITY READING+ MATH READING COMBINED MATH 49 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME Trinity Christian Academy (ST) Universal Academy Of Florida (ST) University Christian School (TN) Venice Christian School (TN) Victory Christian Academy (ST) Victory Christian Academy (ST) Victory Christian Academy (ST) AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Jacksonville 135 327 -0.83 -0.54 -1.12 -0.56 -0.40 -0.72 Tampa 149 345 -0.11 -0.62 0.40 1.48 1.01 1.95 Jacksonville 47 124 0.27 3.49 -2.96 -3.99 -2.35 -5.63 Venice 32 100 -4.73 -3.25 -6.22 -0.48 -0.47 -0.49 Orlando 62 166 -7.41 -7.45 -7.37 0.75 0.99 0.51 Lakeland 60 143 1.33 1.72 0.95 1.17 1.35 0.99 Jacksonville 37 100 1.74 0.81 2.68 2.72 2.34 3.10 51 51 -0.24 1.33 -1.80 -0.24 1.33 -1.80 Melbourne 30 30 1.42 1.13 1.70 1.42 1.13 1.70 South Daytona Bea 105 278 -1.40 -0.37 -2.42 -0.49 0.97 -1.96 CITY Villa Prepatory Miami Academy Corp + (IT) Wade Christian Academy + (ST) Warner Christian Academy (TN) AVERAGE GAIN SCORE IN 2014-15 READING+ MATH READING COMBINED MATH 50 Appendix continued NUMBER OF GAIN SCORES OBSERVED SCHOOL NAME West Hernando Christian School (ST) Westwood Christian School (ST) Westwood Christian School (ST) William A. Kirlew Jr. Academy (IT) Winter Haven Christian School (ST) Yeshiva Elementary (ST) AVERAGE GAIN SCORE IN 2014-15 AVERAGE GAIN SCORE FROM 2012-13 TO 2014-15 2014-15 SCHOOL YEAR BETWEEN 2012-13 AND 2014-15 READING+ MATH COMBINED READING MATH Spring Hill 50 123 -2.22 -4.22 -0.22 -2.59 -3.12 -2.07 Live Oak 35 90 -9.71 -6.86 -12.57 -3.50 -2.68 -4.32 Miami 30 94 -1.02 -3.00 0.97 -0.86 -0.03 -1.69 Miami Gardens 47 108 3.03 -0.60 6.66 0.08 -0.81 0.97 Winter Haven 42 93 0.62 -2.21 3.45 -0.08 -1.06 0.90 Miami Beach 44 108 0.83 -2.14 3.80 -4.48 -4.25 -4.70 CITY READING+ MATH READING COMBINED MATH Notes: Cells report average gain scores. Cells (in the three-year moving average columns) that are highlighted are statistically distinct from the national average at the 95 percent level of confidence. Schools marked with + had scores only in 2014-15. Thus, the three-year average for these schools solely consisted of the average gain score in 2014-15. Performances of these schools should be evaluated with extreme caution given that the risk of having faulty observed results are high for these schools. Acronyms within the parenthesis indicate the test that was administered in that school. Schools marked with * administered different tests at different grade levels. AC=ACT/PLAN; BA=Basic Achievement Skills Inventory; IT=Iowa Test of Basic Skills; PS=PSAT; TN=Terra Nova. 51