Review of Special Education in the Houston Independent School District Thomas Hehir & Associates Boston, Massachusetts Review of Special Education in the Houston Independent School District TABLE of CONTENTS Introduction Page 5 Areas of Strength Page 7 Finding 1: The HISD Special Education Leadership Team is an asset to students with disabilities Finding 2: Students with disabilities predominantly attend their home schools Finding 3: Schools are satisfied with the level of district-school communication Finding 4: There are many examples of quality instruction for students with disabilities taking place in Houston’s public schools Areas for Improvement in Identification Page 11 Finding 5a: African American students in HISD are dramatically over-represented in the categories of mental retardation and emotional disturbance Finding 5b: Limited English Proficient (LEP) Hispanic students are, on average, underrepresented in special education during elementary school and over-represented in middle and high school Finding 5c: African American students in HISD appear to be more likely to be overidentified as having emotional disturbance or mental retardation in schools where they represent a lower percentage of the student body Finding 5d: Although Hispanic students in HISD are under-represented in Special Education categories overall, there is evidence that Hispanic students in elementary schools are more likely to be identified as needing special education in schools where they represent a lower percentage of the student body. This is particularly evident in the categories of emotional disturbance and mental retardation Finding 5e: Students with dyslexia appear to be under-identified as needing special education services 2 Areas for Improvement in Assessment Page 19 Finding 6a: Students with learning disabilities make up the vast majority of HISD Special Education students in taking the TAKS-M Finding 6b: African American and Hispanic students with learning disabilities were more likely to take the TAKS-M than White students with learning disabilities Areas for Improvement in Placement Page 22 Finding 7a: A relatively low proportion of special education students in HISD spend all or most of their day in mainstream settings Finding 7b: A relatively high proportion of special education students in HISD spend little to none of their day in mainstream settings Areas for Improvement in Instruction Page 24 Finding 8a: Average performance on the Stanford Achievement Test tended to be higher for students who spent more of their day with their typically developing peers Finding 8b: Students with disabilities appear to lack access to appropriate technology Areas for Improvement in School-District Relationships Page 26 Finding 9a: High schools report a different type of relationship with district staff than do elementary and middle schools Finding 9b: Staff at higher- and lower-performing schools expressed confusion about special education paperwork Areas for Improvement in Parent Choice Page 26 Finding 10: Parents of students with disabilities did not feel they could access the choice system Recommendations Page 27 Recommendation 1: Develop outcome-based school monitoring systems for students with disabilities Recommendation 2: Hold principals accountable for the issues identified in this report Recommendation 3: Develop effective models of special education service delivery emphasizing effective universal design principle Recommendation 4: Develop specialized inclusive schools for students with significant disabilities 3 Recommendation 5: Develop policies, procedures and practices for effective services for students with dyslexia Recommendation 6: Develop and distribute a comprehensive easily accessible manual of special education policies and procedures Additional Potential Research Page 31 Conclusion Page 31 References Page 32 Methods Appendix Page 33 4 Introduction Superintendent Terry Grier requested an evaluation of the special education program in the Houston Independent School District (HISD). After discussion, we specifically proposed to: (1) Provide an overview of special education identification and service delivery in Houston. (2) Conduct an overview of management of special education, which would delineate central functions as well as school-based responsibilities and authority. (3) Provide a Critical Issues Document that would identify the major challenges facing HISD in improving educational outcomes for students with disabilities. (4) Provide a comprehensive set of recommendations for improving the education of students with disabilities in HISD. Method We collected and analyzed our data during the fall of 2010. As such it represents a snapshot of special education at that time. It should be noted that during this time a new administrative structure was being implemented along with some critical leadership and policy changes. Therefore, this report may not have captured the impact of some of these changes. This report is based on quantitative – or statistical – analyses of district and school data, and qualitative – or observational and interview – data collected at HISD schools and district offices. Quantitative data on students in HISD were provided by HISD district staff. Where possible, we used publicly available data sets and reports to provide comparison information on students in Texas and across the country. To select schools at which to observe, we conducted a statistical analysis of all HISD schools and, using Stanford Achievement Test scores, identified those schools at which students with disabilities were performing higher and lower than would be expected, given patterns of achievement in the district (for more on this analysis, please see the Methods Appendix). From this pool of higher-performing and struggling schools we selected nine schools at which to observe and conduct interviews with special education and administrative staff. In selecting these nine schools, we considered those schools whose populations reflected district demographics and that had large and diverse enough special education populations to provide adequate data. In all, we visited three elementary, three middle, and three high schools. We also conducted extensive statistical analyses examining the student and school level factors associated with the identification, placement and performance of students with disabilities in HISD (for details of these analyses please see the Methods Appendix). We also interviewed district leaders in special education including most of the senior leaders in the department of special education, the superintendent, the assistant superintendent for academics, and the deputy chief academic officer. We further conducted two focus meetings 5 with special education leaders to first ascertain their perception of the challenges facing special education and later to present our preliminary findings. Finally, we conducted a focus group meeting with approximately forty parents of students with disabilities served in the district. We used this as an opportunity to both hear concerns and also to get parents’ reactions to some of our preliminary findings. Finally we reviewed various monitoring findings from the state as well as internal district policy documents. We are greatly appreciative of the support and cooperation we received from staff at all levels of the district. Thomas Hehir & Associates Hadas Eidelman, Todd Grindal & Dr. Elizabeth Marcell 6 Areas of Strength Based on qualitative and quantitative analyses, we have identified four areas of strength in HISD as related to the education of students with disabilities. Those findings are detailed here. Finding #1: The HISD Special Education Leadership Team is an asset to students with disabilities Special education leadership in large urban school districts is complicated and demanding work. We have worked with many large urban districts and were impressed with the quality of Houston Independent School District’s leadership staff in special education. They are knowledgeable about research and best practices in special education and were obviously deeply committed to the students and families they serve. We were particularly impressed with the fact that, when we presented our preliminary findings, they were not defensive but rather evidenced a desire to address the problems we have identified. Finding #2: Students with disabilities predominantly attend their home schools A major principle embedded in special education law and practice is the preference for students to be served in the school they would attend if they were not disabled - their “home school”. Essentially this principle means that if a parent would normally send their child to school X if the child did not have a disability, then that parent should be able to send their child with a disability to the same school. The advantages of implementing this principle are several. First, parents do not have to relate to more than one school for their children. Second, non-disabled siblings are often a source of support for their disabled siblings. Third, children with disabilities can develop social relationships with children who live in their neighborhood. Finally, implementing this principle means that no school is over-enrolled with students with disabilities, thereby increasing potential opportunities to integrate students into general education classrooms and other school options. In order to provide students with disabilities options in their home schools, most schools need to have services for the vast majority of students with disabilities. Houston has implemented home school services to a commendable degree. Our data analysis demonstrated that most Houston schools have developed services to serve the vast majority of students within their home schools. Houston’s success in this area compares favorably with both Los Angeles and New York, where home school placements have been subject to litigation. Los Angeles has made more progress in this area over the past five years while New York is just beginning a major initiative promoting home school placement. Though HISD has exerted a good deal of effort to educate students in their home schools, there have been conflicts with schools over this policy. Several schools in our observation sample reported frustration in the amount of time and the number of processes associated with removing a student from his or her home campus. While this may point to a need for better communication between district and school staff about the ‘removal’ process, the fact that it is not easy to remove a student from his or her home campus is an overall positive finding. 7 School-level staff members were overwhelmingly comfortable with the idea of sending a child with a disability to another school, particularly when that child exhibited more challenging or severe behaviors. That the district has implemented a series of procedures to ensure that such removal is justified is an indication of reasonable and appropriate attempts to educate children at their home campuses whenever possible. Decisions to move students from their home campuses should not be made lightly. However, the process of implementing effective home school placements is dependent upon schools accepting their responsibilities to effectively serve students with disabilities. On this dimension we found much variability. A major finding of this report is the great degree of variability that exists among Houston schools in the extent to which students with disabilities are welcomed, included, and served effectively. Consequently parents whose children are enrolled in “home schools” they view as ineffective express frustration in finding a school in which their child will be accepted and properly educated. As Table 1 (below) indicates, comparative data shows that HISD students with disabilities, on average, attend regular schools at higher rates than similar students in Texas and across the country. Table 1. Comparison of rates of students with disabilities being placed outside of their local school in HISD, the State of Texas, and the Nation as a whole.1 Special Education (all categories) Emotional Disturbance Learning Disability Mental Retardation Other Health Impairment HISD 1.1% 10.0% 0.4% 0.9% 1.7% Texas 1.5% 4.6% 0.5% 2.4% 2.1% Nationally 5.2% 18.2% 2.1% 7.5% 4.0% Finding #3: Schools are satisfied with the level of district-school communication Without exception, every school interviewed described communication between the district special education department and school staff (generally the principal, assistant principal, and/or special education chairperson) as adequate, clear, and frequent. Email was cited as the most common form of communication, while special education chairpersons also mentioned the monthly special education meeting as a time to receive information. In addition, the majority of school staff interviewed described their Program Specialists as accessible and responsive. 1 Placement outside of local school was defined as students who were labeled in HISD data as being homebound, placed in hospital classrooms, placed in residential care and treatment facilities, or “off home campus.” State and national comparison data were drawn from https://www.ideadata.org/IDEAData.asp. 8 Finding #4: There are many examples of quality instruction for students with disabilities taking place in Houston’s public schools Our analysis of schools’ average performance on the Stanford Achievement Test indicated that many schools appear to be providing solid instruction to their special education students. Houstonians need not look outside their city for effective schools serving children with disabilities. On average, HISD general education students’ test scores are approximately one standard deviation higher than scores for HISD students with disabilities on the reading portion of the Stanford Achievement Test. However, there is substantial variability in the size of this gap across schools. While there were schools in HISD where the test score gap between general and special education students was larger than a standard deviation, there were also schools where the gap was substantially smaller. Almost one in five schools in HISD had a test score gap between general and special education students that was one half of a standard deviation. Another 7% of HISD schools had average test scores for special education students that were only a quarter of a standard deviation lower than the average test scores for general education students.2 On-site school visits made clear that quality instruction for students with disabilities is happening in Houston’s public schools. Some of the impressive classroom instruction we observed included: • An elementary class for students with emotional disturbance, staffed by a special education teacher and a paraprofessional, in which the teacher was leading an engaging guided reading lesson. A visual describing the habits of good readers was posted, and the teacher used various questioning strategies throughout the lesson observed. • A co-taught fifth grade class in which the two teachers were both actively engaged in providing instruction and in which all students (four of whom were students with disabilities) were working to complete concept maps during the lesson presentation. At one point, a general education student quite naturally turned to a student with an IEP to help him get started on filling out his map. • A middle school English/Language Arts class for cognitively disabled students in which students were clustered around a kidney-shaped table with their teacher completing graphic organizers that helped them to identify nouns and verbs in various sentences and phrases. • A middle school resource class in which the teacher had clearly worked to develop an environment conducive to learning – low lighting, various comfortable reading areas – and in which students were engaged in a reader’s workshop. One of the students was a student with an emotional/behavioral disability who was being transitioned slowly into the general education setting to prepare him for high school. 2 This analysis was limited to the sub-group of schools in which students with disabilities represent a minimum of 2% of the student body; within this sub-group, analysis was limited to schools in which at least two-thirds of enrolled students with disabilities were tested. 9 • A remedial class in a high school in which the classroom has been set up as a small library, with print and digital resources available, where a small staff of teachers and assistants work with students who need additional academic support in English Language Arts and Math. The constant in all of these classes was high expectations for students’ relative abilities and a clear focus on academic outcomes. In these classrooms, all students were clearly engaged in a learning activity and supportive instructional strategies such as graphic organizers were in evidence. 10 Areas for Improvement in Identification Our analyses yielded several areas for improvement. We have grouped our findings into the following categories: Identification, Assessment, Placement, Instruction and Achievement, School-District Relationships, and Parent Choice. Identification Approximately 8.2% of the students in the Houston Independent School District (HISD) have a special education designation. This percentage, though lower than the average across the country (11.2%), is comparable to the average for the state (9.04%) as well as for similar districts including Dallas (7.8%) and San Antonio (10.7%). This percentage is also consistent with Texas Education Agency guidelines. Our research yielded four primary findings regarding patterns of special education identification in HISD. First, African American students in HISD are over-represented in special education as a whole, compared to their non-African American peers. This overrepresentation is especially evident in the mental retardation and emotional disturbance categories, where African American students are dramatically over-represented. Second, limited English proficient Hispanic students are under-represented in special education during elementary school and over-represented in middle and high school, and these findings are not driven by immigration status.3 Third, there is some evidence that African American and Hispanic students are more likely to be identified as needing special education in schools where they represent a lower percentage of the student body. A fourth finding related to identification of students with disabilities, based on school site visits, suggests there is an under-identification of students with dyslexia as it relates to special education. These findings are detailed below. Finding #5a: African American students in HISD are dramatically over-represented in the categories of mental retardation and emotional disturbance Like many urban school districts across the country, rates of special education designation in HISD vary significantly for students from different racial backgrounds. In HISD, this is particularly true for African American students. After controlling for a variety of school and student level characteristics,4 the likelihood of being identified as needing special education was substantially higher for African American students than for non-African American students. This pattern was particularly notable in the categories of mental retardation and 3 In fact, across the board, immigrant students in HISD were less likely than their non-immigrant peers to be identified as needing special education services. 4 In all our analyses, we controlled for relevant student and school-level characteristics (e.g. gender, free/reduced lunch, school enrollment composition, etc.). If these characteristics were meaningfully related to the likelihood that students would be identified as needing special education, then by controlling for them, we were able to take into account their role in identification to get a clearer picture of the unique role of ethnicity. By virtue of this, we were able to produce a more accurate picture of overall patterns in the district. 11 emotional disturbance. Figures 1 and 2 (below) display the odds ratios for African American students in HISD middle schools being identified as having mental retardation and emotional disturbance.5 Figure 1. Comparison of the odds that African American students in HISD middle schools will be identified as having emotional disturbance versus the odds for non-African American students, controlling for student gender, school proportion of students eligible for free or reduced price lunch, and school proportion of African American students•. Non-African American African American Odds Ratio = 4.16 Note: Data from 37,851 middle school students were used in this estimate. • This finding is statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict the likelihood of identification as having emotional disturbance for African American middle school students in HISD. Figure 2. Comparison of the odds that African American students in HISD middle schools will be identified as having mental retardation versus the odds for non-African American students, controlling for student gender, school proportion of students eligible for free or reduced price lunch, and school proportion of African American students •. Non-African American African American Odds Ratio = 1. 67 Note: Data from 37,851 middle school students were used in this estimate. • This finding is statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict likelihood of identification as having Mental Retardation for African American middle school students in HISD. 5 Odds ratios compare the odds of an event occurring to the odds that the event will not occur. Put more plainly, odds ratios compare the likelihood of occurrence between two different events (for example, the likelihood that an African American student in HISD will be identified as needing special education services compared to the likelihood that a non- African American student in HISD with similar characteristics such as socioeconomic status, gender, school proportion of students eligible for free/reduced price lunch, etc., will be identified as needing special education). 12 The odds ratio of 4.2 (displayed in Figure 1) indicates that, controlling for important student and school level factors, the odds that an African American middle school student in HISD will be labeled emotionally disturbed are more than four times the odds that a similar nonAfrican American middle school student will receive the same designation. (For more information on interpreting odds ratios please see footnote six below).6 Our analyses show that African American students in HISD are substantially more likely than non-African American students to be identified as needing special education at the elementary, middle, and high school levels. The fact that these rates are highest in disability categories associated with greater segregation and lower achievement is particularly concerning. The table below provides a detailed list of odds ratios for African American students in HISD elementary, middle, and high schools. Table 2. Odds ratios comparing the odds that an African American student (vs. a non-African American student) in HISD will be identified into Special Education, Emotional Disturbance, Learning Disability, or Mental Retardation, controlling for relevant student and school level characteristics. Special Education (all categories) Emotional Disturbance Learning Disability Mental Retardation Elementary Schools 1.64 3.61 1.64 2.24 Middle Schools 1.48 4.16 1.22 1.67 High Schools 1.59 2.83 1.44 1.95  All findings are statistically significant at the p<.05 level. 6 A practical guide to interpreting odds ratios is as follows: a) if the value of an odds ratio is less than 1, then the event (e.g., an African American student being identified as emotionally disturbed) is less likely than its comparison event (e.g., a non-African American student being identified as emotionally disturbed); b) if the value of an odds ratio is equal to 1, then the event of interest is just as likely as its comparison event; and c) if the value of an odds ratio is greater than 1, then the event is more likely than its comparison event. In comparing different odds ratios, lower values represent lower likelihoods and higher values represent higher likelihoods (e.g. an odds ratio of 4.2 represents a higher probability of identification than an odds ratio of 1.7, even though both indicate an overrepresentation of African American students in their respective special education categories). 13 Finding #5b: Limited English Proficient (LEP) Hispanic students are, on average, underrepresented in special education during elementary school and over-represented in middle and high school Hispanic students’ level of English proficiency is strongly related to the likelihood that they will be identified as needing special education. This relationship varies sharply between elementary school patterns and patterns in middle and high schools in HISD. In the elementary school years, Hispanic LEP students are less likely than Hispanic non-LEP students to be identified as needing special education (odds ratio of 0.6). In middle school however, Hispanic LEP students are substantially more likely than their Hispanic non-LEP peers to be identified as needing special education services (odds ratio of 1.7). In high school this pattern is even more striking, as the odds that Hispanic LEP students will have a special education designation are over four and a half times the odds for their Hispanic non-LEP peers (odds ratio of 4.6). These high rates of identification for high school Hispanic LEP students are particularly pronounced in the categories of learning disabilities and mental retardation (odds ratios of 4.7 and 5.4, respectively).7 In meeting with different groups of stakeholders in special education at HISD, we noted some plausible mechanisms explaining the distinction between elementary school under-representation of Hispanic LEP students in special education and then their consequent over-representation in middle and high schools. The pattern might be a result of teachers’ desire to keep LEP students in elementary schools outside of special education in order to ensure they receive adequate bilingual support. Another potential explanation would be that students who require special education might maintain their LEP status longer (e.g. into middle and high school) than their general education peers, so the over-representation may be driven by underlying educational challenges, not limited English proficiency. While the data we examined could not tell us which, if either, of these processes might be at play in HISD, they both represent plausible explanations for the finding. Finding #5c: African American students in HISD appear to be more likely to be overidentified as having emotional disturbance or mental retardation in schools where they represent a lower percentage of the student body As reported in finding 5a, African American students in HISD were notably over-represented in special education, particularly in the emotional disturbance and mental retardation categories. We now present evidence that furthermore, the extent of over-representation of African American students in special education is greater in schools with relatively low African American enrollment (schools where African Americans represent a relatively small proportion of the total student population). This means that an African American student in a school with few other African American students is more likely to be identified as having emotional disturbance or mental retardation than an African American student in a school with a higher proportion of African American students. Figure 3 (below) presents an example of this, where the odds that an African American 7 These models controlled for a variety of relevant student and school characteristics, including gender, LEP status, free/reduced lunch, school percentage of Hispanic students, and whether or not the student attends a charter school. 14 student will be labeled as mentally retarded differ based on the racial composition of that student’s school. Here, looking at the right-most circle in the figure, we see that in an HISD elementary school with a relatively high African American enrollment, the odds that an African American student will be identified as having mental retardation are approximately 1.5 times the odds that a non-African American peer will receive that designation. However, when we look at elementary schools with relatively low African American enrollments, represented by the left-most circle in the figure, the odds ratio is sharply higher – in this case the odds that an African American student will be identified as having mental retardation are approximately 2.7 times the odds that a non-African American peer will be so identified.8 Similar patterns exist in HISD high schools for African American students in the categories of emotional disturbance and mental retardation. This finding indicates that while the over-representation of African American students in the MR and ED categories is evident across HISD, the problem is particularly acute in schools in which African American students represent a low percentage of the student body. This relationship, though troubling, is not unique to HISD and is consistent with previous research (Parrish, 2002). Figure 3. Comparison of the odds that African American students in HISD elementary schools will be identified as having Mental Retardation versus the odds for non-African American students, displayed for students in schools with a relatively high percentage of African American students (38.1%) and schools with a relatively low percentage of African American students (3.7%), controlling for student gender, free or reduced lunch eligibility, size of school enrollment, school proportion of African American students, and whether or not the student attended a charter school•. In a school where African Americans represent a relatively small portion of the student body: African American In a school where African Americans represent a relatively large portion of the student body: Non-African African American American Odds Ratio = 1.55 Odds Ratio = 2.67 Note: Data from 113,935 elementary school students were used in this estimate. • All findings are statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict likelihood of identification as having Mental Retardation for African American elementary school students in HISD. 8 It is important to note than in both cases, African American students are disproportionately likely to be identified as having mental retardation – this finding (5c) simply highlights that the over-identification of African American students is especially pronounced in schools with a relatively low African American enrollment. 15 Finding #5d: Although Hispanic students in HISD are under-represented in Special Education categories overall, there is evidence that Hispanic students in elementary schools are more likely to be identified as needing special education in schools where they represent a lower percentage of the student body. This is particularly evident in the categories of emotional disturbance and mental retardation Overall, Hispanic students in HISD were less likely than non-Hispanic students to be identified as needing special education. However, there was some evidence that in HISD elementary schools, Hispanic students who attend a school where Hispanics represent a relatively high percentage (around 93%) of the student population are substantially less likely to be identified into special education than Hispanic students who attend a school with a relatively low Hispanic enrollment (39%). For example, as represented in Figure 4, in elementary schools with a relatively high Hispanic enrollment, Hispanic students are much less likely than their nonHispanic peers to be identified as needing special education (odds ratio of 0.6), but in elementary schools with a relatively low Hispanic enrollment, Hispanic students are equally as likely as non-Hispanic students to be identified (odds ratio of 0.9). Although neither of these odds ratios represents an over-identification of Hispanic students, the odds ratio of 0.9 does indicate a higher likelihood of identification than the odds ratio of 0.6. Therefore, as we observed with African American students, there appears to be a strong relationship between the percentage of Hispanic students who attend a school and the rate at which they are identified as needing special education. Figure 4. Comparison of the odds that Hispanic students in HISD high schools will be identified as needing Special Education versus the odds for non-Hispanic students, displayed for students in schools with a relatively high percentage of Hispanic students (92.5%) and schools with a relatively low percentage of Hispanic students (39.1%), controlling for student gender, size of school enrollment, school proportion of students eligible for free or reduced lunch, school proportion of Hispanic students, and whether or not the student attended a charter school•. In a school where Hispanic students represent a relatively small portion of the student body: Hispanic In a school where Hispanic students represent a relatively large portion of the student body: Non-Hispanic Hispanic Odds Ratio = 0.58 Odds Ratio = 0.93 Note: Data from 111,817 elementary school students were used in this estimate. • All findings are statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict the likelihood of identification as needing Special Education for Hispanic high school students in HISD. 16 This pattern was true across special education as a whole, as well as in the specific categories of emotional disturbance, learning disabilities, and mental retardation. A full table of odds ratios for identification into special education for Hispanic students in HISD elementary schools appears below. Table 3. Odds ratios comparing the odds that Hispanic students (vs. non-Hispanic students) in HISD will be identified into Special Education, Emotional Disturbance, Learning Disability, or Mental Retardation, controlling for relevant student- and school-level characteristics. Special Education (all categories) Emotional Disturbance Learning Disability Mental Retardation Elementary Schools 0.74 0.30 0.78 0.69 Middle Schools 0.69 0.20 0.90 0.62 High Schools 0.71 0.28 0.82 0.54  All findings are statistically significant at the p<.05 level. A Note on Over-Representation of African American and LEP Students: The over-representation of students with Limited English Proficiency (LEP) and African American students in special education is a national issue that has received considerable academic attention as well as attention from Congress during the last reauthorization of IDEA. The federal law now requires states to actively intervene in districts in which over-representation is identified. The Texas Education Agency (TEA) has already cited HISD for this. Research in this area is rather extensive and has come to some important conclusions. First, the genesis of over-placement is heavily influenced by practices in general education. Specifically, weak instructional programs – particularly in reading – seem to contribute to over-representation; and ineffective approaches to school discipline and behavior policies can exacerbate the problem. Secondly, ineffective programs in language acquisition and support have been shown to be related to the over-representation of LEP students in special education. Finding # 5e: Students with dyslexia appear to be under-identified as needing special education services. Several school-level staff as well as parents and advocates expressed concern that students with dyslexia are served exclusively under 504 Plans and do not receive targeted, intensive interventions through special education. As such, services for students with dyslexia are not overseen by the department of special education at HISD, and the Dyslexia Specialists work to coach general education teachers to meet their students’ learning needs rather than provide direct support. While not all students with dyslexia may have a severe enough disability to warrant special education services, it is highly improbable that no students with dyslexia require the type 17 of interventions special education can provide. Thus, in HISD’s current programmatic approach to serving students with dyslexia, students who need intervention through special education are not able to access those interventions with reasonable ease. We recognize that Texas law regarding students with dyslexia may complicate decisions regarding the identification and provision of services to these students. Nevertheless, we find that HISD’s approach is in direct conflict with research that supports intensive and early targeted reading interventions for students with dyslexia (see, for example, Shaywitz, 2003; Fletcher Lyons, Fuchs, & Barnes, 2007) and with federal law, which mandates that students with disabilities be eligible for special education services. Given national estimates of the prevalence of dyslexia at, conservatively, 3%-5%, this lack of service to students with dyslexia may explain the relatively low overall percentage of students receiving special education in HISD. In summary, in the area of identification we found that relatively large numbers of African American students and Hispanic LEP students may be inappropriately identified as needing special education services. At the same time, we identified a relatively large population, students with dyslexia, who may be being inappropriately denied services. 18 Areas for Improvement in Assessment For three consecutive years (2008-2010), HISD has failed to meet the adequate yearly progress provisions of the Federal No Child Left Behind Act, in part because the district allowed too many special education students to take modified or alternate versions of the Texas Assessment of Knowledge and Skills (TAKS). Federal law states that only 2% of students may take modified assessments and only 1% of students may take alternate assessments. However, in English Language Arts, 4.7 % of HISD students took the TAKS-M (modified version)9 and 1.3% of HISD students took the TAKS-Alt (alternate assessment) in 2010. This means that approximately 56% of HISD special education students’ test scores were outside of the standard accountability system. The fact that the majority of students with disabilities are not included within the standard accountability system is not only at variance with federal law but it may well be diverting the attention of school leaders from engaging in instructional improvement that could benefit these students. Further, this extensive use of modified assessments may also reflect a broader problem of lowered expectations for students with disabilities. Furthermore, qualitative data collected during school visits indicated that special education chairpersons adopted a liberal approach to assigning students to the TAKS-M. While most staff members were able to describe or demonstrate a clear process for determining which assessments students with disabilities should take (including, but not limited to, tracking binders documenting students’ past performance on assessments, samples of student work, and the testing guidelines and handbook), they also consistently indicated that if they had any doubts that a student in question might not perform well on the TAKS or the TAKSAccommodated, they assigned that student to the TAKS-M. In other words, the default decision-making was not based on an assumption of capability but rather on one of incapability. Our statistical analyses yielded two primary findings regarding the assessment of HISD students with disabilities. First, students with learning disabilities make up the vast majority of HISD Special Education students taking the TAKS-M. Second, African American and Hispanic students with learning disabilities were more likely to take the TAKS-M than White students with learning disabilities. Finding #6a: Students with learning disabilities make up the vast majority of HISD Special Education students in taking the TAKS-M Students with learning disabilities are the largest group of students who take the modified form of the TAKS (as opposed to the standard or accommodated versions of the test). On the one hand, this is not surprising, given that students with learning disabilities make up more than half of the HISD special education population. On the other hand, the vast majority of students with 9 The TAKS-M modified assessment differs from standard assessment (TAKS-K) or standard assessment with accommodations (TAKS-Accommodated) in four important ways. First, the TAKS-M contains no open response questions. Second, the modified exam includes a greater amount of white space on each page. Third, the TAKS-M does not include any experimental test questions. Finally, students choose from three rather than four answer choices. 19 learning disabilities are assumed to have normal cognitive ability and therefore are likely to be adequately assessed using a standard form of the TAKS with appropriate accommodations. Finding #6b: African American and Hispanic students with learning disabilities were more likely to take the TAKS-M than White students with learning disabilities Our analyses indicate that Hispanic and African American students with a learning disabled designation are systematically more likely to be assessed on substantially less rigorous and modified assessments than White students. This finding is particularly notable in African American student assessment rates in HISD middle schools. In elementary schools, as represented in Figure 5, Hispanic and African American students in the learning disability category are more likely than White students to take the TAKS-M, with similar odds ratios (Hispanic vs. White and African American vs. White) of 2.3 and 2.4, respectively. In middle schools, as shown in Figure 6, the odds that an African American student with learning disabilities would take the TAKS-M were nearly six times the odds of a White student with learning disabilities, while the odds ratio for Hispanic (vs. White) students remained similar to the one in elementary school (2.5). In high school (Figure 7), the odds of taking the TAKS-M were nearly equal for Hispanic and White students (odds ratio of 1.3) and were similar to elementary school odds ratios for African American students (2.2). These three sets of odds ratios, represented in the figures below, provide clear evidence that African American students with learning disabilities are consistently excluded from the standard accountability system at dramatically higher rates than similar White students with learning disabilities, and that this trend is apparent – though less consistently – for Hispanic students with learning disabilities. Figure 5. Among elementary school students with learning disabilities in HISD, a comparison of the odds that Hispanic students will be assessed using the TAKS-M (versus the odds for White students) and the odds that African American students will be assessed using the TAKS-M (versus the odds for White students), controlling for student eligibility for free or reduced price lunch, student LEP status, school proportion of African American students, and whether or not the student attended a charter school •. Hispanic White Odds Ratio = 2.30 African American Odds Ratio = 2.41 Note: Data from 1,090 elementary school students with learning disabilities were used in this estimate. • All findings are statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict the likelihood of being assessed using the TAKS-M for elementary school students in HISD with learning disabilities. 20 Figure 6. Among middle school students with learning disabilities in HISD, a comparison of the odds that Hispanic students will be assessed using the TAKS-M (versus the odds for White students) and the odds that African American students will be assessed using the TAKS-M (versus the odds for White students), controlling for student LEP status, size of school enrollment, and school proportion of African American students•. Hispanic African American White Odds Ratio = 2.48 Odds Ratio = 5.82 Note: Data from 1,572 middle school students with learning disabilities were used in this estimate. • All findings are statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict the likelihood of being assessed using the TAKS-M for middle school students in HISD with learning disabilities. Figure 7. Among high school students with learning disabilities in HISD, a comparison of the odds that Hispanic students will be assessed using the TAKS-M (versus the odds for White students) and the odds that African American students will be assessed using the TAKS-M (versus the odds for White students), controlling for student gender, student LEP status, school proportion of students identified as needing Special Education, size of school enrollment, school proportion of African American students, and whether or not the student attended a charter school . Hispanic White Odds Ratio = 1.33 African American Odds Ratio = 2.21 Note: Data from 2,325 high school students with learning disabilities were used in this estimate. • All findings are statistically significant at the p<.05 level. No other student- or schoollevel characteristics were found to predict the likelihood of being assessed using the TAKS-M for high school students in HISD with learning disabilities. 21 Areas for Improvement in Placement HISD students with disabilities are educated in a variety of settings. Some students spend much or all of their school day in “mainstream” settings while others spend more than half of their time in school in substantially separate settings. The degree to which students are educated in mainstream settings varies substantially by disability category. In HISD, as in districts across Texas and around the country, students with learning disabilities are the most likely to spend large portions of their day with typically developing peers, while students with mental retardation are least likely to do so. However, HISD differs from Texas and the rest of the country in the percentage of special education students who spend most or all of their day with their typically developing peers. Finding #7a: A relatively low proportion of special education students in HISD spend all or most of their day in mainstream settings Approximately one half of HISD special education students spend at least 80 percent of their day in mainstream settings compared to two-thirds of special education students across Texas. This lower percentage of HISD students who are educated in mainstream settings is consistent across disability categories. For example, 40 percent of HISD students with emotional disturbance are in primarily mainstream settings compared to 58% of ED students across the state. We observed a similar pattern for students with learning disabilities (65% in HISD vs. 71% statewide), mental retardation (5% in HISD vs. 8% statewide), and students with other health impairments (54% in HISD vs. 67% statewide). Figure 8. Comparison of percentage of students with disabilities who spend 80% or more of their day in mainstream settings between HISD, Texas, and nationally. 80% 66.68% 70% 60% 58.49% 50.49% 50% 40% 30% 20% 10% 0% HISD Texas 22 USA Finding #7b: A relatively high proportion of special education students in HISD spend little to none of their day in mainstream settings. HISD students are educated in substantially separate settings at higher rates than similar students in Texas and around the country. More than 1 in 5 HISD students with disabilities spend 60% or more of their school day outside of mainstream classes. This compares to 1 in 8 students in Texas and approximately 1 in 7 students across the country. Our analyses also found that African American students and males are consistently educated in less mainstream settings at higher rates than similar non-African American and female students. While school visits did not provide definitive evidence to support this finding, it is worth noting that none of the schools visited demonstrated strong tendencies toward inclusive practices, and all schools had a variety of instructional settings as alternatives to the general or mainstream setting. Figure 9. Comparison of percentage of students with disabilities who spend 40% or less of their day in mainstream settings between HISD, Texas, and nationally. 30% 21.57% 20% 14.90% 12.18% 10% 0% HISD Texas USA While it is not necessarily illegal or inappropriate to serve students with disabilities outside of the general setting, the majority of students with disabilities can and should be served in mainstream settings. However, though some schools in HISD represent notable exceptions, most schools do not appear to be taking steps to appropriately integrate students with disabilities into mainstream settings. While one middle school was taking active steps to acclimate students to less restrictive settings before moving them on to high school, one of the high schools in our sample, in contrast, had many segregated settings in which students with disabilities were observed coloring and listening to iPods with their heads down on their desks. In general, especially in the schools identified by our statistical analysis as “struggling,” instruction in segregated settings – whether resource or self-contained classes – was of poorer quality than instruction in general or mainstream settings. 23 Areas for Improvement in Instruction Findings 7a and 7b indicate that HISD students with disabilities spend less time, on average, in mainstream settings than students with disabilities in Texas as a whole and nationally. Found also was that Stanford Achievement Test scores for students with disabilities tended to be lower for those students who spent less time in mainstream settings and higher for those students who spent more time in mainstream settings. HISD students with disabilities earn lower scores, on average, on the Stanford Achievement Tests than their general education peers, with differences in scale scores between special and general education students ranging from 3/5 of a standard deviation in elementary school to nearly an entire standard deviation or more in middle and high school. The performance of students with disabilities on the mathematics and reading subtests of the Stanford Achievement Test vary substantially within and between disability categories, and we focused our analysis on understanding the variation in the performance of special education students in categories where we observed a wide range of performance. These categories are Other Health Impairment (OHI) and learning disabilities. The data we have are best suited to examining these differences by looking at the proportion of each school day that these students spend in classrooms with their typically developing peers. Finding #8a: Average performance on the Stanford Achievement Test tended to be higher for students who spent more of their day with their typically developing peers Though on average HISD serves students in segregated settings at high levels, there are schools that serve students in a more integrated fashion. It was decided to test whether Houston’s students with disabilities who were more integrated in general classes performed better on the Stanford Achievement Test. Our analyses indicate that holding constant a number of important variables associated with achievement such as gender, race, and free/reduced lunch, students with disabilities who attend classes with typically developing peers tend to exhibit higher performance in language and mathematics on the Stanford Achievement Test. Indeed, the amount of time a student spent in mainstream settings was associated with substantial differences in test performance. It is not surprising that children who spend their entire day in a mainstream setting earn, on average, higher scores on the Stanford Achievement Test. – these are likely the students who have disabilities that minimally affect their school performance. However, given that HISD students with disabilities spend disproportionately large amounts of time outside of mainstream settings, we looked for meaningful patterns in achievement in our examination of the relationship between amount of mainstream instruction and student performance. We found that spending more than 20 percent of the day in non-mainstream settings is associated with dramatically lower test scores than being fully mainstreamed, and that in fact, there appears to be little difference in the effect of separate settings beyond this threshold. We believe that this is because students who spend more than 20 percent of their day in separate settings are likely not receiving instruction in core academic subjects in the general education setting. As noted 24 above, our observations indicate that many segregated special education classrooms at struggling schools provide either substandard instruction or no instruction at all. This finding is consistent with research conducted on a national level. The National Longitudinal Transition Study-2 (NLTS 2) is a 10-year study that documented the characteristics, experiences, and outcomes of a nationally representative sample of more than 11,000 youth who were ages 13 through 16 and were receiving special education services in grade 7 or above when the study began in 2001. NLTS 2 found that while more time spent in general education classrooms was associated with lower grades for students with disabilities, students who spent more time in the general setting scored closer to grade level on standardized math and language tests than did students with disabilities who spent more time in segregated settings (Wagner, et al. 2003). Finding #8b: Students with disabilities appear to lack access to appropriate technology New and not so new technologies have provided greatly increased access to education for many children with disabilities. Text to speech, speech to text, communication devices, and captioning are but a few examples of technologies that can greatly enhance the ability of students with disabilities to perform in school. Yet, during interviews conducted at school sites and the central office, interviewees indicated a relatively low-level of adoption of these innovations. At the school level we saw very little use of technology, and interviews with various school staff members revealed an inability to describe a variety of technologies used to meet the needs of students with disabilities. Instead, when we asked, ‘What technology is available to students with disabilities?’ the only answers we received were: computers, Smartboards, Elmos, and LCD projectors. In addition, several special education chairpersons mentioned they received the “hand-me-down” computers from general education when new computers were ordered. Finally, school-based staff reported that district staff was not up-todate on current assistive technologies, such as those afforded by the iPad. Descriptive statistics indicate that surprisingly few students in HISD receive assistive technology (AT): just 2.5 percent of students with disabilities receive AT, 41 percent of whom have an auditory impairment. Less than one percent – .22 percent, to be exact – of students with specific learning disabilities receive assistive technology, and a mere five percent of students with autism do so. This is concerning, because a lack of access to technology may limit the extent to which students with disabilities can be integrated into the fabric of a school and is likely to negatively impact the academic achievement of students who could benefit from technology. 25 Areas for Improvement in School-District Relationships As noted above, staff at the schools we identified as high- and low-performing were satisfied with the frequency and clarity of communication between district special education staff and school staff. However, two important findings emerged from school site visits related to the nature of school-district partnerships at the elementary and middle versus high school level and to expectations regarding paperwork. Finding # 9a: High schools report a different type of relationship with district staff than do elementary and middle schools Elementary and middle school-based staffs consistently characterize their relationships with district staff as collaborative, supportive, responsive, and helpful. Some school-based staff members reported seeing their Program Specialists regularly and relying on the advice of these Specialists, and noted that they received responses to requests for help or information from their Program Specialists in a timely and efficient manner. Conversely, high school-based staffs characterize their relationships with the district staff, and the Program Specialists in particular, as being limited to compliance issues. They felt the Program Specialists served only to come to the schools to check students’ files and to tell the schools what they were “doing wrong.” This feeling was consistent across lower- and higher-performing high schools. In addition, all three high schools reported having to wait too long for responses from their Program Specialists, unlike their elementary and middle school colleagues. Finding # 9b: Staff at higher- and lower-performing schools expressed confusion about special education paperwork The one area of communication that school-based staff felt was not clear or consistent relates to paperwork. Staff at lower- and higher-performing schools felt the amount of paperwork required of them, particularly the supplements to the IEP, had become more overwhelming in recent years. In addition, they found the paperwork confusing to complete and expressed frustration that they were not able to receive clear answers on how to do so: for example, more than one special education chairperson noted that they received conflicting answers regarding how to complete paperwork from different Program Specialists. Finally, school-based staffs were unclear about changing expectations related to paperwork and were not always sure which documents were current and which were no longer being used. Staff lamented a lack of professional development to support them in completing paperwork and expressed a desire to use training time prior to the start of school to address paperwork expectations for the coming school year. 26 Areas for Improvement in Parent Choice Finding # 10: Parents of students with disabilities did not feel they could access the choice system Houston, like many urban districts, offers parents a number of choices in school assignment. There are many charter schools in Houston and there is a history of providing magnet schools. However, a number of parents in our focus group felt they did not have the same options for their disabled children within the choice system. Parents’ perceptions that children with disabilities are not welcome in charter schools are supported by data. Charter schools in HISD tend to have lower proportions of special education students than non-charter schools in the district. While students with disabilities represent 8.6 percent of students in non-charter schools, in charter schools they represent only 4.1 percent. Across elementary, middle, and high schools, and within special education as a whole as well as the categories of emotional disturbance, learning disability, and mental retardation, HISD charter schools served a much lower percentage of students with disabilities. Of the more nearly 1,900 HISD students with mental retardation, only 46 attend an HISD charter school. Among the 126 HISD students with visual impairments, only 3 attend a charter school. Table 4 below displays the percentages of students with disabilities in HISD charter and non-charter schools. These data provide strong evidence that students with disabilities are much less likely to attend charter schools than similar students. It is important to note here that this study does not include charter schools run by other entities that may be serving Houston students because these students were not in the database. This issue thus deserves further investigation. Table 4. Percentages of the total student population represented by students with disabilities and by specific disability categories in Charter and Non-Charter schools. HISD Charter Schools (N=16, 931) HISD Non-Charter Schools (N=199,735) 4.14% 8.61% Orthopedic Impairment 0.05% 0.12% Other Health Impairment 0.51% 0.71% Auditory Impairment 0.04% 0.18% Visual Impairment 0.02% 0.06% Mental Retardation 0.27% 0.92% Emotional Disturbance 0.27% 0.43% Learning Disability 1.99% 4.12% Speech Impairment 0.64% 0.94% Autism 0.11% 0.47% Students with Disabilities 27 Recommendations We believe the following six recommendations provide powerful directions the district should take in order to move its special education program forward. We resisted the temptation to provide a myriad of recommendations concerning the issues identified because our experience has taught us that such reports are less apt to be effective as they result in fragmented bureaucratic responses that ultimately have less effect at the school level. The aim of these recommendations is for HISD to embark on a few powerful, school-based initiatives that we believe will enable the district to more effectively educate students with disabilities while assuring that appropriate students are being served by the program. It should be noted that these recommendations all require the active engagement of both special education and general education leadership. The issues identified in this report cannot be addressed by special education alone. For instance, the over-placement of African American students and English Language Learners is largely due to ineffective programs and supports in general education. Other issues, such as the ineffectiveness of some special education programs, are largely the responsibility of special education working with principals. What we are calling for here is change in the way many leaders in the district segment their responsibilities between special education and general education: that is, we support powerful, competent, unified approaches. Recommendation #1: Develop outcome-based school monitoring systems for students with disabilities When we were presenting our preliminary findings to special education leadership personnel, one stated, “You know, if the principals had this data concerning their schools, I know they would act on it.” We agree. Though we know principals receive a good deal of data from the system, we do not believe they are getting the best data. It should be noted that many of our findings were revelatory to the special education leadership team; we would expect these findings would be even more so to principals. The fact that we found serious problems that had been relatively unknown to the system was not due to incompetence or mal-intention. Rather, we were asking different questions and using different methodologies in data analysis that enabled us to go deeper into district data than may have been done previously. For instance, much of the data analysis done by the district has been driven by the requirements of the Texas accountability system and is therefore dependant on the TAKS. However, due to the large numbers of students taking a modified version of this test, the TAKS is not a valid measure of special education student performance at the school-level. Fortunately, most students with disabilities take the Stanford Achievement Test and we were able to rely on this test data to determine the relative effectiveness of schools. Further, we analyzed district data going beyond descriptive data using multi-variant methods. This is important in that some schools may appear to be doing well but may have a population who come from a higher overall socio-economic level while another may appear to be doing poorly with a population that might have more challenges. 28 We suggest that HISD use our methodologies for developing a monitoring system for its schools. We have spent a good deal of time “cleaning” district data and developing the statistical models used in the report. We will happily provide these to you. It should be noted that a major finding of this report is the substantial variability among schools’ special education programs. We believe that a relatively simple yet powerful system could be developed that measures the following outcomes: (1) Instructional efficacy (Using the Stanford Achievement Test data) (2) Over-placement of African American and English Language Learners in special education (3) Inclusive practices (4) Drop out and transfer rates. These simple but powerful measures would focus the attention of school-based administrators on important data that has the potential to improve practice. Further, it would allow central administrators to intervene in schools that are not performing well and identify schools engaged in exemplary practice. Recommendation # 2: Hold principals accountable for issues identified in this report A number of people we interviewed, at both the school and district level frustration with the degree to which principals take an active role in their special education programs. This sentiment was particularly strong amongst the special education administrators. A number of people said that special education was largely a “delegated” responsibility at the building level. In the high performing schools we observed, it was clear that principals took an active role in their special education programs. Most of the issues identified in this report require the active engagement of general education school principals. The over-placement of minority children in special education has its genesis in general education classrooms (Donovan and Cross, National Research Council, 2002) and inclusive programming is dependant on having effective integration options in general education classrooms. In order to bring all district schools into better alignment with best practices we observed in high performing HISD schools it is essential that principals and general education teachers share responsibility with special education staff for addressing the issues identified in this report. Recommendation # 3: Develop effective models of special education service delivery emphasizing effective universal design principles. In addition to general education having a role in promoting more effective education for children with disabilities, special education must do its part. We were concerned that many of the practices we observed were less than optimal. Segregated classes where children were doing low-level work, or “co-taught” classes where special education teachers appeared to be serving the role of a Paraprofessional, do not comport with principles of best practice. Broadly speaking, special education for most children should involve providing children with specialized 29 interventions to help minimize the negative impact of their disability while providing accommodations and supports so children can access the curriculum (Hehir, 2005). And, for many, advances in technology allow students to access curriculum in more efficient and effective ways. We recommend that the district provide extensive training to building-level staff on effective universally designed approaches for education students with disabilities (Rose & Myers 2005, Hehir, 2005). These approaches can provide students with disabilities with greater access to inclusive education while preventing the need for others to be placed in special education. Recommendation # 4: Develop specialized inclusive schools for students with significant disabilities There are few inclusive options for students within the district for students with significant disabilities. Though it might be the ideal to have all students with significant disabilities served in home schools, for students with less common and often complicated needs, the expectation that every school can develop the capacity to serve these students needs to be reconsidered. Though the ideal that the child attends the school he or she would attend if non-disabled has considerable merit, the ability of each school to meet the needs of students with complicated needs is variable. As this report has documented the variability among schools in Houston in their ability to provide quality special education services is great. We believe HISD should consider developing a number of highly specialized schools that can provide inclusive programming to students with complex needs. Such schools have been successfully developed in other cities. In Boston, the Henderson School serves a population that includes 20 percent students with significant needs (Hehir, 2005). These students are served predominantly in the general education setting with support and the school has consistently had some of the highest test scores in the city. Dr. Hehir is available to discuss the implementation of this recommendation in more detail. Recommendation # 5: Develop policies, procedures and practices for effective services for students with dyslexia The special education department should develop policies and procedures designed to give students with dyslexia appropriate access to special education services. This guidance should also include information about research-based effective practices for students with dyslexia. Training of school-based staff should also accompany the issuance of this guidance. 30 Recommendation # 6: Develop and distribute a comprehensive easily accessible manual of special education policies and procedures Special education law and practice is relatively complex. Accurate information about policies and procedures is central to the proper administration of the program. A major issue in this report involves appropriate communication about special education to personnel in the schools as well as parents. Well-articulated policies and procedures as well as frequently asked questions available online would greatly improve the administration of the program. Additional Potential Research Though we made an attempt to incorporate parental input into this report, our efforts were limited due to the scope of this report. We recommend a more extensive scientific survey be conducted of parents within the district. Such a survey will assist the leadership in assuring that the program is responsive to its consumers. We also recommend that similar surveys are done of principals and teachers specifically focused on their needs for training and support. Conclusion There are many aspects of the special education program in HISD that are commendable. Like other cities, Houston also has areas where the program falls short. We hope this report will help HISD focus its efforts on improving the program. We appreciate all the cooperation we received from Houston staff in conducting this work and are optimistic about the potential of the district to move forward. 31 References Donovan, M. S., & Cross, C. T. (Eds.). (2002). Minority students in special and gifted education. Washington, DC: National Academy Press. Fletcher, J. M., et al. (2007). Learning disabilities: From identification to intervention. New York: Guilford Press. Hehir, T. (2005). New directions in special education: Eliminating ableism in policy and practice. Cambridge, MA: Harvard Education Press. Parrish, T. (2002). Racial disparities in identification, funding, and provision of special education. In D. J. Losen, & G. Orfield (Eds.), Racial inequity in special education. Cambridge, MA: Harvard Education Press. Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Alexandria, VA: Association for Supervision and Curriculum Development. Shaywitz, S. E. (2003). Overcoming dyslexia: A new and complete science-based program for reading problems at any level (1st ed.). New York: A.A. Knopf. Wagner, M. N., et al. (2003). Going to school: Instructional contexts, programs, and participation of secondary school students with disabilities. Retrieved January 30, 2011, from http://www.sri.com/nlts2/ 32 Methods Appendix Selecting the Sample of Schools for Site Visits When we selected a subset of HISD schools as candidates for site visits, our goal was to select a set of schools that were representative of the diverse make-up of schools in HISD. We wanted to include schools at each level of education (elementary, middle, and high) as well as schools that represented a variety of the compositional and charter profiles that characterize HISD. Within those guidelines, we also wanted to choose schools where students with disabilities were performing well on the Stanford Achievement Test relative to their general education peers as well as schools where students with disabilities were performing relatively poorly compared to their general education peers. We used multiple regression models to estimate schools’ average test scores for students with disabilities, given their average general education test scores and taking into account characteristics such as proportion of students eligible for free/reduced price lunch, proportion of LEP students, school racial composition, etc., which might also be related to schools’ average test scores. By doing this, we were able to see – within the guidelines we had set for representativeness – which schools’ special education students had average test scores that were higher than we would predict based on general patterns in the district and which schools’ special education students had average test scores that were lower than we would predict based on general patterns in the district. From among the handful of schools where students with disabilities most under-performed and over-performed on the Stanford Achievement Test, we created thorough profiles for each school using HISD data as well as information we found on specific school web sites to understand the schools more richly. As a group, we then evaluated this subset of schools and came up with the ten that we felt were most representative of the diversity in HISD schools. Odds Ratios For the Identification and Assessment sections of this report, we used logistic regression models to estimate the probabilities that types of students in HISD would be (a) identified into Special Education categories and (b) tested using the TAKS-M. These models produces odds ratios, which describe meaningful patterns of (a) over- and under-identification into Special Education and (b) disproportions in which HISD students are assessed outside of the standard accountability system. A more detailed explanation of logistic regression follows. Logistic regressions test the probability that an event will occur (for example, a Hispanic student is identified as needing special education services), while taking into account a variety of factors that might play into the event occurring (for example, the student’s socioeconomic status or the financial resources of the student’s school, both characteristics that are known to be related to rates of identification into special education). By using this method, we were able to estimate the 33 likelihood of students’ being identified into Special Education, or into a specific Special Education category, while also estimating the role played by student- and school-level characteristics in the likelihood of identification for different groups of students. This process allowed us to understand, on average, the likelihood of identification that was uniquely associated to specific student- or school-level characteristics (for example, the estimated likelihood, on average, that a Hispanic student in HISD would be identified as requiring special education services) taking into account the role that factors such as socioeconomic status, gender, or school proportion of students eligible for free/reduced price lunch, play in a student’s likelihood of receiving a given special education designation. These logistic regression models yielded estimates called odds ratios. Essentially, odds ratios compare the odds of an event occurring to the odds that the event will not occur. Put more plainly, odds ratios compare the likelihood of occurrence between two different events (for example, the likelihood that a Hispanic student in HISD will be identified as needing special education services compared to the likelihood that a non-Hispanic student in HISD with similar characteristics such as socioeconomic status, gender, school proportion of students eligible for free/reduced price lunch, etc., will be identified as needing special education services. A practical guide to interpreting odds ratios is as follows: a) if the value of an odds ratio is less than 1, then the event (for example, a Hispanic student being identified as needing special education) is less likely than its comparison event (for example, a non-Hispanic student being identified as needing special education ); b) if the value of an odds ratio is equal to 1, then the event of interest (for example, a Hispanic student being identified as needing special education) is just as likely as its comparison event (for example, a non-Hispanic student being identified as needing special education); c) if the value of an odds ratio is greater than 1, then the event (for example, a Hispanic student being identified as needing special education) is more likely than its comparison event (for example, a non-Hispanic student being identified as needing special education ). In comparing between members in a group of odds ratios, lower values represent lower likelihoods and higher values represent higher likelihoods; so an odds ratio of 5.4 represents a higher probability of identification than an odds ratio of 2.2, and an odds ratio of 0.3 represents a lower probability of identification than an odds ratio of 0.7. Achievement Analyses Our analysis of special education student achievement was conducted using multi-level regression models. These models used dichotomous variables to represent 80-99% of a student’s day in school in mainstream settings, 40-79% of a student’s day in school in mainstream settings, and 0-39% of a student’s day in school in mainstream settings, with 100% mainstream as the reference category. This allowed us to estimate the difference in special education student performance on the Stanford Achievement test associated with each setting category in comparison with the fully mainstreamed setting category. The use of multi-level models allowed us to account for similarities in the performance of special education students within any given school, which provided a clearer picture of the relationship between achievement and proportion of time in mainstream settings, taking school-level characteristics into account. 34 In this section of the report, we present our findings in standard deviation units. These standard deviations, which represent how spread out students’ scores were on any given test, are drawn from an analysis of the performance of all students in the given special education designation. By doing this, we were able to compare the difference in average scores for students in different setting categories relative to the range of scores for all special education students within a special education designation. We conducted analyses of elementary, middle, and high schools separately. In these analyses, we controlled for relevant student and school level characteristics, which included student race, LEP status, free and reduced lunch status, school enrollment, school percent African American, school percent LEP, and school percent free and reduced lunch. 35