Page 1 i 2 of ll DOCUMENTS Copyright 2010 The Atlanta Journal-Constitution The The Atlzuita Journal-Constitution Augrst 31. 2010 Tuesday Main Edition SECTION: Pg. IA LENGTH: 603 words HEADLINE: APS test analysis was Bair. rigorous For the AJC BODY: As students return to school this fall, Gov Sormy Perdue and the Govemor's Office of Student Achievement. or GOSA. Continue to criticize the work of my company. Caveon Test Security. The govemor`s position is misguided with respect to Caveon_ but he is calling attention to an ugly reality; Wholesale, organized cheating in some Atlanta public schools occurred and must be addressed. His statements about Caveon, though. do not rest on well-established and scientific cheating detection methods. The governor's public statements about our analyses of 2009 statewide Criterion Referenced Competency Tests. or CRCT, data ignore that we confirmed GOSA's conclusion that cheating occurred at some APS schools. Caveon is the leading test security company inthe field, with experience examining more than 15 million test instances for the largest. most esteemed test orgzuiizations in the world. including 15 state departments of education and the Department of Defense. The state's vendor used a "preliminary screening" method of counting wrong-to-right CRCT erasures. Caveon would never reconunend that our clients launch full-scale investigations solely on the basis of wrong-to-right erasures. Such an approach flies in the face of industry best practices. A key concept in our work is helping clients focus 0111116 "worst of the worst." We also believe that minimizing false accusations is highly desirable. This means every school and class that is investigated shows clear evidence of serious problems. What the governor seems unwilling to recognize is that many of the Atlanta schools flagged in the original GOSA analysis are almost certainly on the list of "schools of concern" because of factors unrelated to cheating. These factors 414 Page 2 APS test analysis was fair, rigorous Tl1e Atlanta Journal-Constitution August 3 l. 2010 Tuesday include common rnissteps such as students getting off track and making mistakes marking their answer sheets. "Losing your place" on answer sheets occurs on a fairly common basis, The CRCT is r1o exception and when a student realizes a ntiseue. erases the misaligned marks and corrects the answer sheet. wrong-to-right erasures are created. No credible rneasurernent expert would be satislied with the original GOSA analyses as a basis for deciding that a teacher or school had serious problems unless the results were very extreme. as was the case with some ofthe schools flagged by GOSA and confirmed by Caveorfs ir1-depth follow-up. To the detriment oI` our ability to analyze tests to the fullest extent possible. the GOSA provided data only for the 58 schools that had been designated as being "of concern," not all APS schools. Most irnponantly. we did not receive student-by-student response data for every question This omission prevented us from conducting these other cheating analyses: Collusion: Which schools and classes had "iurusual agreement" among students, far beyond what could reasonably have occurred by chance alone? This is the most basic cheating analysis. one any expert would insist on doing. Unusual pattems; Which schools and classes had very odd patterns of student responses. such as missing easy questions and answering hard questions? This outcome is very likely to have been caused by students receiving help before or during an exam as well as tarnpering with answer sheets or student records after testing. From Caveon's perspective. we would very much appreciate discontinuation of rrninI`ormed and wrongheaded of the state-of-the-art cheating analyses that we did. Any contention that we jeopardized our personal and professional reputations to please a client has no basis in fact whatsoever. John Fremer is president and COO of Caveon Testing Security. LOAD-DATE: August 31, 2010 41 5 1 ,5l_`iff - A 2 7 - iffil ini-felFfa 'V-fw5f<~ a 2 1 1 "a 21168-ws Test EXaminer's Manual Reading, 1 Grades 1 and 2 Online Shell English/Language Arts, and Mathematics 3 GEORG EEUQATHGN Georgia Depa11'mc|1t of Education Kathy Cox, State Superlnuzmleni of Schools 2009 by df E-iuwiw Payne 03617 Secure Test Materials--Do Not Copy A11 test booklets. and supporting materlals assocmted the Cnterron Referenced Competency Tests are conlidenual and secure ING part of any test booklet may be reproduced or ln any form or by any means but not to electronic meehamcal manual or verbal (e photocopying recording paraphrasxng ress or creating mirror for mstrucuon and/01 copying) CRL1 matermls must rem un secure it all rumen and (excluding the School and System Test Coordmator Manual the Rest Exammer Manual for be bg, any mdrvadual or entity pr10r to or after testmg (lest txammer Manuals for grades 1 and 2 are secure Tb do so IS a dxreet vxolatton oi testing pohcxes and procedures estabhshed by Georgxa law (?20 2 281) and the State Board ofEducat1or1 (Rule 160 3 1 07(2) in addluon to copyright laws and Georg1a professional for educators CRCT malermls may not be provided to any persons except those ccmductmg the test and those bemg tested All test booklets (used and unused) and all supportmg materials must be accounted for and retumed at completlon of the test and the manner ln the School and System 'Ilwt Manual grades 3 through S, and the online shell for grades 1 and 2 Test Examinerir Manuals) cannot Copyright 12' 2009 .by the Georgia Deparrm?m ofEducarlma Pay ns 417 TABLE OF Introduction I Purpose of This Manual Test Materials 2 Test Security 2 - Preparing for the 'lest 3 Scheduling the Test. 5 Administering the 6 Administering the Braille Version of the 8 Administering the Large -Print Version of the 8 Completing the Student ldentification lxxfunrzaliou 9 Directions for Administering the Reading CRCT Directions for Administering the English/Language Arts CRCT Directions for Administering the Mathematics CRCT Post-Testing Activities . 12 Encoding Student Information . 12 Completing Sections 9-14: "For Teacher Use Oniy" 1 Completing Sections '15 and 16: "Slate-Directed Usc Only" . 17 Completing the Group information Sheet 1 8 Returning Materials to the School Test Coordinator ., 20 Checklist for Test Examiners . 2 Directions for administering Llie Georgia are omitted lrum this online version. Copyliqllt tl) 2009 by the Georgia Depamamir afiiffncariun 418 THIS PAGE INTENTIONALLY LEFT BLANK Pay ne 0 3 Copyright (C) 2009 by the Gealgrb Deparment ofEducA?on 620 INTRODUCTION Georgia's Criterion-Referenced Competency Tests are designed to measure student acquisition of the knowledge and skills set forth in the states Georgia Performance Standards (GPS). Legislated by Georgia lawmakers, the Reading, English/ Language Arts, and Maihexnaties assessments were developed to yield information at the student, class, school, system, and stale leveis. The primary purpose of the is to provide a valid measure ofthe quality ol` educational services provided throughout the state. The Georgia are comprehensive tests made up of multiple-choice items that map to the GPS. Each Stmienr Test Booklet contains ail three content areas. Each content area test is made up of two sections; each section is timed for up to 70 minutes. Students will work through the the Test Examiner, using no resource materials during testing. Students in grades 1 and 2 should mark only their answers in the Student Test Booklets (using Nu. 2 pencil), and may not otherwise write in the booklet or use a highlighter to mark words or passages. Calculators are not allowed during any part ol' the Mathematics test. Scratch paper, provided by the Test Examiner, can be used during the Mathematics test only. Punt-oss or THIS MANUAL This 1@stExcmtilter's Manual provides procedural infomiation as well as detailed instructions for administering the lt is imperative that the be administered according to directions in this you have any questions, contact your School or System Test Coordinator. Uniform test administration is needed to ensure high-quality assessment data. To ensure this uniformity, the Test Examiners responsibilities are clearly outlined and detailed in this manual, Step-by-step instructions for administering the test are designed to protect its integrity und security and must be followed exactly. It is recommended that TestExnn1i11crs read this manual and become thoroughly familiar with all CRCT administration procedures before admi[LiStel'iI'1g the tests. Payne 33621 Grades I-2 Online Shell 1 Test Manual Copyright (C) 2609 by /liz Department of Education 420 TEST MATERIALS lu addition to this manual, you will receive all test materials from the School Test Coordinator. The Student Test Booklets will he presorted into class packs of 2.0 and 5 and must remain in their shrink-wrapped packs until they are distributed to Test Examiners. Student Test Booklets for grades 1 and 2 are machine-scannable; therefore, it is important that all pages remain free of stray pencil marks or loose eraser bits and that they not be folded, clipped, stapled, banded, taped, or torn. Students in grades and 2 will use one Student Test Booklet to record their answers. Each consists of multiple-choice questions. Sample items are provided to acquaint students with the correct procedures for recording answers, and directions for filling in answers are provided in the Student Test Booklet. NOTE: Materials are shipped to arrive a few days before your scl1ool's nine-day testing period. Contact your School or System Test Coordinator for your scheduled test window. TEST SECURITY The are secure tests. Maintaining the security of all test materials is crucial to obtaining valid and reliable test results. Therefore, test materials must be kept in locked storage, except during actual test administration. Access to secure materials must he restricted to authorized individuals only Test Examiners and the School Test Coordinator). It is the direct responsibility of all individuals who administer the test to follow security procedures. Student Test Booklets and the items therein are to be used solely for test purposes and may not be disclosed or used for any other purpose. Student Test Booklets must remain in their shrink-wrapped packages until they are distributed to Test Examiners. Shrink- wrapped packages may be broken only for partial-count needs or for distobution to exarninees. Student Test Booklets from opened shrink-wrapped packages must remain individually scaled until cxaminecs arc directed to break the scal during administration. The must be administered by a certified educator; All three content urea tests for grades 1 and 2 are read aloud to students by the Test Examiner. Test Examiners must follow all directions and read the scripts for all test items word for word as they appear in the Test Examiners Manual. For information on Accommodations, see pages 12-16 in this manual. Durin test sessions, Test Examiners are directly responsible for the security of the tests and must account for all test materials at all times. When the materials are not in use, they must be kept in locked storage. At the conclusion of each day's test session* Test Examiners must return all used and unused test materials to the School Test Coordinator. Test Examiner 'r Manual 2 Grades l-2 Online Shell (C) 2909 by :ke Georgia cf Education Payne 421 Test Examiners must make sure that students write their names on their Student Test Booklets on the First day of testi ng. Students should also write their names on their scratch paper (for the lylathematics test) to heip account for materials at the end of eneh test session. Testing conditions, especially the supervision and seating arrangements of students, should be designed to minimize the potential for cheating. The Test Examiner must supervise the test administration at all times, All instances of test security breaches and testing irregularities must he reported to the School Test Coordinator immediately. If questions arise, or if any situations occur that could cause any part of the test administration to be compromised, please have your System Test Coordinator contact Assessment and Accountability as soon as possible at (800) 634-4106 or (404) 656-2668. non THE TEST 1. Materials should be distributed to Test Examiners early on the Erst day of testing in order to allow suftieient time for completion of the student identification information. Test Examiners must apply student pre-ID labels or, in cases where there is no pre-ID label, fill out all of the student and school information on the Student Test Booklet for each student participating in the CRCT. Sections through must be completed according to the directions in "Completing the Student Identification Information" (see page 9). 2. This 1?stExamiI1er's Manual will be used to administer the in Reading, Englishilanguage Arts, and Mathematics, Each content area test consists of multiple- choice All content area tests are included in the Student Tiest Booklet. Students in grades and 2 will usethe Student Yes! Booklet to record their answers. 3. One preetor for grades and 2 is recommended. When more than 30 students are to . be tested in one location, the assistance of proctor is required. Proctors may help Test Examiners with morn preparation and monitoring. 4. Before achninislcring the make sure that you have the following rnaterizrls: I A copy of this Test Manual I One Student Test Booklet for each student labels with preceded student information Two No. 2 pencils with functional erasers for each student Blank scratch paper for the Mathematics test (ut least two sheets for each student) 0 A timing device, such as a clock or Watch, to keep truck of time during the test administration A "Testing-Do Not Disturb" sign to post on the classroom door Payne 03623 Grades I-2 Orriine Shell 3 Test Manual Copyright (C) 2009 by the Georgia Uepartmern ?4fEclucatil/lx 422 5. Arrange test materials so that they can be distributed in an organized and cfticicnt manner. 6, Systems will receive pre-ID labels to pre-identity demographic information on Student Test Booklets. Follow these instructions carefully to provide the correct student demographic information on each Student Test Booklet: For enrolled students, place pre-ID labels on the front covers of the Student Test Booklets, in the space marked STUDENT LABEL Verify that the inform ation printed on the pre>>1D labels is accurate (system name, system ID, school name, school ID, teacher name, student name, FTE Student ID and GTID numbers). If any errors are found, do NOT use the incorrect label. Instead, notify the School Test Coordinator. If the information on the pre-ID label is incorrect, Test Examiners should fill in the student demographic information on the Student Test Booklet following the directions in this manual (see page 9). Any incorrect information on pre-ID labels should be corrected in the system's student records. Do not change the labei in any way. Apply only correct pre-ID labels to the Student 1'2'stBoaklets. Not all information included in the pre-ID label is visible. Only specified iields may be seen on the label, as indicated above. Student demographic information (including ethnic group, gender, and date of birth) is included in the pre-ID label and should nol. be filled in on the Student Test Booklets. Student demographic infomation on the pre-ID label will override any hand-filled information for all students. 0 Even with a pre-ID label, the Test Examiner will still need to complete Section I on the front cover for all students, as Well as the "For Teacher Us: Only" section on the inside front cover, if applicable. NOTE: Student lest Boa/cters are designated by a Form number, which is printed on the front cover of each test booklet. 0 If the student is new to the or for any other reason does not have a pre-1Dlahel, follow the instructions in this manual to till in the student demographic information on the Student Test Booklet (see page 9). All fields should bc completed accurately (including the FTE Student ID and GTID numbers). Test Examiners should have a complete list of FTE Student ID and GTID numbers prior to testing. 2' Payne 03624 Terr Examiner? Manual 4 Gm-des I-2 Online Sheff Copyright 0 20179 by the George: Department qf Education 423 7. Before beginning the test, make sure that each student has the following: One Srudertr Test Booklet Two No. 2 pencils with functionaft erasers 0 At least two sheets of blank scratch paper for the Mathematics test 8. Each student's workspace should be large enough to accommodate an open Student Test Booklet. should he cieared of nil other materials. During testing, students should be separated by a reasonable distance to encourage independent work and to prevent collaboration, Posters, charts, and other instructional materials related to the content being tested shouiri not be displayed in the classroom or otherwise made available to students during test administration. Take down or eover all such material, 9. You may want to plan an activity for students who finish early. Students may read a book or do other work if they iinish early, but this activity must be unrelated to the content being tested and students should not be permitted to open their Student lest Booklets again. SCHEDULING THE TEST The Reading, Eugl_isl1fLa11guage Arts, and Mathematics wili be administered to students within the state testing window ofA;Jri1 6 through May 8, 2009. Systems will choose one nine-clay testing period within this window. AH students shouid be tested in surroundings that will provide them with the opportunity to do their best work onthe test. lu schools where students in several classes are being tested, each classroom of students must take me same test at the same time. The must be administered in the order prescribed: Reading, English/Language Arts, and Mathematics. Each section of each lest must be administered in one block of time. Approximately halfway through the administration of each content area test (between Sections and 2), students should be given a ll]-minute break. Alternatively, students may take a lunch break between Sections 1 and 2. Both sections oi' a content area test must be administered on the same day. Under no circumstances may a content area test be split across multiple days-to do so is a direct violation of test security. The administration time for the as indicated in the following table, should bc snfticient for students to complete each test section. The scripting process may increase the adrninistration time In such cases, testing should continue as long as students are productively engaged in completing the test. Students should be given a few minutes at the end of each test section to review independently any items they have not answered. Time may be called before 70 minutes if all students have completed the respective section. 03625 Grades l~2 Online S/lei! 5 Test Exarninerli Mauna! Cupyriglu (C) 2009 /gy the Georgia Dspm-rmen: of education 424 CRCT ADMINIQTRATION SCHEDULE, GRADES 1 AND 2 CONTENT AREA Reading Arts '\/Iatluzmatacs TASK Yrudent Te rr Booklets and read directions TestAdmll119trat|on-Section I Break Test Administration-Section 2 Total Time Dmlribuln. and read dlrecnons Break 1estAdmn-nstration Seeuon 2 To?alAdm1nistrat|on Time D1str|b11le Student les! Bookietr and read dxrecnons lest Administrat1on~5ect1on Break TIME Approximately 10 I5 minutes Approximately 45 minutes (li) minutes or lunch) 45 TE) Approximately ll) 165 minutes l0 15 minutes (EU minutes or lunch) Approximately 45 70 minutes Approximately ll0 !65 minutes Approximately 10 15 minutes .Approximately 45 "rn mmus?=< (10 minutes or lunch) ADMIMSTERING THE Test 2 Approximately 45 70 minutes l0{nIAdnun1sLrauun Timo ll() |65 This Test Examiners Manual shoiald be used to administer the Reading, English/ Language Arts, and Mathematics Before you administer the make sure you have ex1:a No. 2 pencils available and that students' desks are cleared of all other materials, Test Examiners must make sure that students write their names on their Student Test Booklets 011 the Iirst day of testing. Students should use the same Student Test Bookle! on all subsequent days of testing. Because the primed Manuals for grades I and 2 contain secure test items, they must be treated in a secure manner and retumed to lhe School Test Coordinator at the conclusion of each test session with all other test materials. Test items must be read exactly as they are scripted in printed Ylesr Excaminerir Manual. Under no circumstances may Test Examiners deviate, in uny way, from the script--to do so is a direct violation ol' test security. 'lest items should be read at a normal pace and in a normal Lone of voice. 1 I Payne eaeza ks! Examiner? Manual 6 .l-2 Online She!! (C) 2009 by rife Geaagia of Educmian 425 Some students may not be familiar with how to fill in answers. To familiarize students with correct procedure, sample items ure provided at the beginning of each content area test. During each test session, circulate around the room and check to see that students are recording their answers in Coi'1'cCt Section of their Student Test Bookleltc This process will be referred to on subsequent pages as "circulate and check." Students in grades 1 and 2 should mark only their answers in the Student Test Booklets (using a No. 2 pencii), and may not otherwise write in the booklet or use a highlighter to mark words or passages. All content anca for grades and 2 must he read aloud to students. All information to bc read to students has been scripted for Test Examiners and must be read exactly as it appears in the printed lest Exgminerir Manual. Any graphics (such us tables, pictures, or charts) associated with items should not be described to students. Use your professional judgment in responding to student questions that arise during the testing session. You may clarify directions; however, under no circumstances should you reword test items, suggest answers, or evaluate student work during the testing session. You may repcat (verbatim) a test item one time after your initial reading, but each test item may be read no more than two times. 'lest Examiners are cautioned that voice inflection can clue students in to correct response or make other answer choices more confusing, Questions and response options should be read in such manner as to minimize the impact of inlleclion, word emphasis, or reading style. If neces sary, take time before the administration to practice reading appropriately. Road word for word all text that is boxed and printed in boldfucc type and preceded by the word Say. Text that appears in regular type or italics should not be read aloud to the students, For the Mathematics test, students should be given atleast two sheets of blank scratch paper. Students must print their names in the upper right-hand comer of each sheet of scratch paper they receive. Students may ask for more scratch paper during the test administration, if necessary. Ail scratch paper must be accounted for and returned to the School or System Test Coordinator with the other test materials. Unless specified by a student's IEP, calculators are not allowed on the Mathematics test. lf testing is interrupted at any time during the test session (by at tire emergency or a child becoming ill, for example), note the time the interruption begins so that you can calculate the amount of reserved time the studcnt(s) will need to complete the test session. If students must leave the room during testing, be sure to lock the door to ensure the security of the test materials. lf you have any questions about an individual situation, please contact your School or System Test Coordinator. I i Payne 03527 Us-ode! 1-2 (Inline Shell 7' Terr .himnzinerlr Manual (C) 2009 by the Georgia Murata" 426 ADNHNISTERING 'run BnAu.Ln or Tm: The directions in this manual also apply to the administration of the Braille version of the Additional Braille instructions are as follows: Disregard references to class packs. The Braille test books will be individually packaged. References to specific page numbers in the standard Form 1 Student Test Booklet may be incorrect for the Braille version. In order to supply the correct page numbers and other references, Test Examiners need to review-prior to testing~all test materials that accompany the Braille test book, including the supplemental instructions, the standard Form 1 Student Test Booklet, and the standard Form Test E'xaminer's Manual. The Test Examiner must complete the student information for each student on a standard Form 1 Student Test Booklet. The Test Examiner must also print the student's name, teacher, school, and system on the front cover ofthe Brailie test book. Indicate on the inside front cover of the standard Fonn 1 Student Test Booklet that the student was administered the Braille version by tilling in the circle next to Word "Braille" in Section 12. Because extra time may be needed for administering the Braille version, it is recommended that students be tested individually or in a small-group setting. Test Examiners who administer the Braiile version will use the standard Form Test Examiners Manual throughout testing. The Test Examiner or proctor must fill in student responses on the standard, Fonn Student Test Booklet exactly as the student dictates. Under no . circumstances should a student's answer be altered or edited-to do so is a direct violation ol' test security. Keep the transcribed standard Form Student Test Booklet with the other used Student Test Booklets from the student's class. The School Test Coordinator should retum the Braille test books with the nonscorahle shipment. THE LARGE-PRINT VERSION on THE The directions in this manual also apply to the administration of the Large-Print version of the Additional Large-Print instructions are as follows: Disregard references to class packs. The Large-Print test hooks will be individually packaged. The Test Examiner must complete the student identification infomation for each student on tt standard Fonn I Student lest Booklet. The Test Examiner must also print the student's name, teacher, school, and system on the front cover of the Large-Print Lest book. 75:51 Eraminefx Manual (lradas I-2 Online She-fi Cupyrighl (C) 2009 by the Georgia Department of Education Payne 03628 1 Indicate on the inside front cover of the standard Form 1 Student Test Booklet that The student was lhe Large-Print version by filling in the circle next to the words "Large-Print" in Section 12. Because extra time may be needed for administering the Large-Print version, it is recommended that students be tested individually or in a small-group setting. I Students who use a Large-Print version should record their answers directly in the Large-Print test book by circling the letters for their selected answers, writing their responses, or by using a similar marking system. The Test Examiner or ptoctot must then transcribe student responses into the standard Fomi Student Test Booklet exactly as they appear in the Large-Print test book. Under no circumstances should a student's answer be altered or edited-to do so is a direct .violation of test security. I Keep the transcribed standard Form Student Test Booklet with the other used Sfudenr Test Bookfem from the student's class, The School Test Coordinator should return the Large-Print test books with the nonscorable shipment. COEVIPLETING THE STUDENT IDENTIFICATIQN If a student has a pre-ID label, only Section will need to be completed on the front cover of his or her Student Test Booklet prior to test administration. If a student docs not have a pre-ID label, all the student identification information will need to he completed on the front cover of his or her Student lest Booklet prior to test administration. Please follow instructions below. Section 1: Student Name, Teacher, School, System Using a No. 2 pencil, print the student's name, the teacher's name, and the school and system names in the spaces provided. Section 2: Last Name, First Name, Middle Initial In the boxes below "Last Name," print the letters of the student's last name, starting in the first box on the left, Do not leave any spaces between the letters in the last name. If the last name does not fit, print as many letters ns you erin. In boxes bclow "First Name," print the letters of student's first name, starting in the first box on the left. This should be sl'udent's full first name, not 3 nickname. Do not leave any spaces between the letters in the first name. If the first name does not fit, print as many letters as you can. Ln the box below print the initial letter of the stndent`s middle name, if applicable. Complete Section 2 by completely filling in the corresponding circles under the letters that you printed. For example, if you wrote you should fill in the circle containing "Rf" Palfne 03629 Grades I-2 Online Shall' 9 His: Manual 2017? hy the Gemgiku Depnmuen! 428 Section 3: Date of Birth Fill in the circle beside the month in which student was born. Then, print the students birthday and year in the appropriate boxes, and Fill in tire corresponding circles under "Day" and "Year" Days under 10 should begin with "zcro." For example, if the student's birthday is on lifth day of the month, you should prim and fill in "zero-tive." Section 4: Gender Fill in the circle next to the appropriate gender for the student. Section 5: Racial!Ethnic Background Fill in the circle next to ethnic group that best describes the student. You should fill in only one circle: Islander, Black/Non~Hispanic, Hispanic, American Indian/Alaskan Native, White/Non-Hispurlic, or Multiracial. Section 6: GTID Print the S't\1dent's GTID number in the boxes, and fill in the corresponding circles under the numbers. NOTE: A GTID is a Georgia Testing Identifier that is used to track and maintain a student's data within the Georgia Department of Educatiorfs data warehouse and the local Student information System. All PK-12 students that attend a public school in Georgia must have a GTID. The GTID is the key identifier used in all state tiara collections. Section 7: FTE Student ID Print the student's FTE number in the boxes, and Fill in the corresponding under the numbers. NOT E: It is imperative that the student ID number printed in this section be the same as the number that school or system uses to report FTE-type information to the stale. Locally assigned student ID numbers should not be used if they do not correspond with state-reported information. Section 3: Form Because the Form number is printed on the front cover of each Student Test Bookfet. Test Examiners for grades I and 2 do not have to fill in any information in this field. All Srur1'entTest Booklets for a classroom or group should have the same Form number. Sections 9-16: Complete these sections after all content area tests have been completed. Directions tor completing these sections are found on pages 12-18 in this rnanual. Payne 03630 i 72st Exnn|iner's Mrrnual 10 Grades Online Shell Copyright (C) 2009 by the Georgia Depar1memnfE1uCaIiur\ 429 DLRECTIONQ F011 ADMINISTERLNG THE GEORGIA GF THE TEST la MANUAL ARE FROM THIS ONLINE VERSION 03631 Grade: I-2 Online Shell 11 Es! Examiner? Manual Capyrighr 2009 by the This section describes what a Test Examiner should do after testing in all content areas is complete. Read this information carefully and follow all directions. If you have questions, contact your School Test Coordinator. ENCDDING STUDENT IDIFORMATION Federal and state regulations require accurate coding of student demographic information. Follow these steps to verify the coding of this information: l. For Student Test Booklets without a pre-ID label, verify that Sections I through 7 have been completed with the correct information and the appropriate circles have been filfed in for each section. Test Examiners must ensure that each student's Student Test Booklet is correctly identified with both his or her nine-digit state FTE Student ID number and his or her ten-digit GTID number. No other student identitication numbers may be used. 2, For Student Test Booklets with a pre-ID label, verify that Section has been completed with the correct information. 3. Use the following directions to complete the "For Teacher Use Only" section on the inside front cover of the Student Test Booklets. Pill in the appropriate circles in Sections 9 through I4 as applicable for each stu dent. . After you have completed the "For Teacher Use Only" section for applicable students, you will to check each Student Test Booklet to make sure it is free of any stray marks. ONLY answers filled in properly (with a No. 2 pencil) will be scored and reported. Retuni the Student Test Booklets' to the School Test Coordinator. Malte sure materials are stacked in the order prescribed in this manual (see page 20). COMPLETING SECTIONS 9-14: "Fon TEACHER USE It is imperative that information be filled in correctly for each student taking the Federal and state regulations require complete and accurate coding of student information for reporting purposes. Sections 9 through 14 should be blank if they are not applicable to the stedent. Payne 03632 Test Exnminzr's Manual 12 Grader l'~2 Online Shell Copyright (C) 2009 by the Georgia Department af Education 431 Section SRC lhe information provided below will help you assign the correct State-Required Codes (SRCs) for students who receive special education Services! If the student receives special education services, fill in the appropriate c_irc1e(s) next to the snidenfs Please refer to the students Individualized Education Plan (HEP) for the appropriate elassification(s). NOTE: For the sluclent's primary classilication, only one response circle should he filled in for SRCS 1-12 or 14-35. Additional response circles for SRCs 13-14 and 16-19 may be filled in as applicable, SRCS are listed in the following tableGeorgia Migrant Education Agency has identified the student as Migrant Certitied, fill in the appropriate circle (18). The four regional Georgia Migrant Education Agencies have provided each system with a listing of students who are Migrant Certified. Only students should be coded. If the student is classified under Section 504, [ill in the appropriate circle (14), A student is eligible to be coded as.SecLion 504 if he or she has a current individual Accommodation Plan (IAP) on nie. If the student is identified as an English Language Learner (ELL) according to Board of Education Rule 160-4-5-.O2, till in the appropriate circle (13). A student is eligible to be coded as ELL if the student's native language is not English and he or she is eligible for English to Speakers of Other Languages (ESOL) services in accordance with Rule 160-4-S -.02, regardless of Whether or not the student receives ESOL services, Students who are not eligible for ESOL services (as specified in Rule may not be coded ns ELL. Fill in circle 19 for those students who have exited the ESOL program hut who are still being monitored, Payne (xra-dis 1 -2 Unliue Shall' 13 Tim' Manual (C) gow gy me Depwmmii qrgducarwn 432 Accommodations (requires at least one SRC) If the student was given tt test administration accommodation based on his or her Individualized Education Plan (IEP), Individual Accommodation Plan (IAP), or English Language Learner/'Iesting Participation Committee Plan, till in the appropriate circle in the Accommodations section, A current IAP, or Plan must be on iilc for each student who receives an accommodation. Only state- approvecl accommodations may be used, following the guidance issued by an accommodation is included in a studcnt's plan that is not on the approved list, it must be reported as an irregularity, Students who are served by Student Support Teams (SST) but do not qualify for services under IDEA or Section 504 status may not be granted accommodations. Failure to provide appropriate accommodations or providing accommodations to St`l1dBI\tS who are not eligible is a direct violation of state regulations. Accommodation Type If the student was given a test administration accoinmodation based on his or her IEP, IAR or till in the appropriate circle(s) under Typo to identify it as a Setting, Presentation, Response, or Scheduling accommodation. Conditional Administration Conditional administrations result from the use of conditional accommodations, Conditional accommodations are more expansive than standard accoinmodations, and atc intended to provide access to students with more severe disabilities or very limited English proticicncy who would not bc able to access the tests without such assistance. Only students meeting strict criteria (outlined in the 20082009 Student Assessment Handbook) are eligible for conditional accommodations. Arty use of conditional accommodations be coded. Per State Board rule, only a small number of students should participate in conditional adniinistrations, ELL-M students are NOT eligible for conditional accommodations. Definitions of Setting, Presentation, Response, and Schedoiing accommodations (botii standard and conditional) arc provided in the following table. I Payne 03634 TEP LEARNERS: ,mn SECTION 504 IAP PLAN Setting Accommodations 3 Special or adapted lighting Preferential seating Standard West Exnmirrerlv Manual 14 Grades Online Copyrigiiz (C) 2009 by the Georgia Department 0,f'Educafiou ACCOMMODATIONS Sound Held Adaptive slant board) or study Carre) Test by cerutied educator fasrulmr to student Preseniatxon Acxommndatnom Large Pr; nt the test questions Slgil readmg passages Explatxr ox paraphrase the dsrnettons tor only) Color overlays templates or place markers Use ul' by Rtudent Oral of test qnestxons only by reader or technology Oral reading ot teadmg passages tu onlv by reader or assrsuve technologv Low VISION aids (e CCTV eqmpment) of daredtzons (mEngl1sh only) Matenals presented contrast and tacttle cues us-ed mantpulatne Lse drrecttons been marked by teache1 Andto on no|se lruffer/lmtentng devrces Response Technology such as Bratller word processor or olher A devtce all grammar and spell check devtces dlsahled Student marks. answers ut test booklet Student pomts to answers Verbal rupoxlse ln only cmd" 1-2 outing Shell 15 WITH ENGLISH LANGVAGE LEARNERS AND Suction 504 IAP ELLITPC PLAN Standard tandard Standard Standard Standard Standard Standard Condmonal (restricted to students nt grades 3-8 only) tandard Wtaltdard tandartl Conddtonal to ell tn gaacles 5-8 only) Standard Standard Standard Standard Standard 5 luntl Standard tmldard GD ZUU9 by the Gemjgirr ofhliucariun 434 etanciard Standard Standard Standard 'ltandard (testmled tu students grades 3 only) btcutdard Standard Standard Terr Manual Payne 03635 STUDENTS ENGLISH ?lSABil.. rrtEs: IEP Leanwensr Accownwonnions Braille writer Basic function calculator or adapted basic calculator Scribe diameter pencil) Wold-to-word dictionary Scheduling Accommodations mn: SECTION 504 PLAN Standard Conditional (restricted to eligible students only) Standard Standard Landard Adapted writing tools pencil grips, large- Frequent rrioeilured breaks Optimal time of day for testing Extended time 17leX1l:|1hly tn thi. order of for content areas Extending sessions over multiple days Section 10 First Year ELL Deferred Siundixrd Standard Standard landard Standard Standard ELL students enrolled for the first time in a school rn the United stares may receive a one time deferment from assessments in content areas other than mathematics and science lf the student was deferred from any CRCT content arms fill in the appropriate response circles in this section Section 11: PTNA If the srudeni was present for the test administration but did not attempt any items in one or more content areas, this must be coded in the appropriate response circle under Section 12: If the student was tested with a Braille or Large-Print version of the this must be coded in the appropriate response circle under Section 13: Served I the student has received services from tho Early Intervention Program (EIP) at any time during the curieni school year, fill in the "Yes" circle under Served" (relevant to grades l-5 only), Section 14: Local Optional Coding Systems may choose to use this section to code special program students to gangs their performance on the System Test Coordinators will designate a code to be lilled Test Exrrminefs 11/Ianual 16 Grader I-2 Online Shel' Cupyrig/iz (C) 2909 by the Georgia Depurlmenr of hklucuriun Payne 03636 435 in for this section; E11 in the code number starting with the First box on the left. Note that systems will not receive any special reports based on this coding. The coding will generate inforrnatiora on each systeufs data tile for the system to extract and miisaggregale based on their requirements. CONIPLETING SECTIONS 15 and 16: USE Sections l5 and I6 should generally be filled in by the School Test Coordinator only. Directions for hlling in these sections are as follows: Section 15: SDUA: Georgia Network for Educational and Therapeutic Support (GNETS) Only The following table is to be used by GNETS programs to report their assigned faciEities code. The two-digit code should be Filled in under Section 15 for all students who are served at a GBEETS facility. This information is being collected in order to provide reports to each center. GNETS PROGRAM NAME 02 Rutland Academy Sand l-li?ls Program South Metro Program - Gmdes 1_2 Simi: 17 Tis: riraminerkr Manual (C) 2009 by the Georgia Department of Educalion ir Payne 03637 436 Section 16: School Test Coordinators should complete this section only as directed by The procedure for reporting testing irregularities to Assessment and Accountability is described in the School and System Rs! Co0rdinamr's Mumuzl. If GHDOE that an Irregulaxity (IR), Invalidaiion (IV), or Participation Invaliniation (FIV) has occurred and should be coded in Section 16, the Program Manager will provide the proper codes and instructions for completing this section. For each affected content area, fill in only one bubble on the rep line: IR, IV, or There, writc in the five-digit codc number provided by and Fill in corresponding circles. COMPLETING THE GROUP INFORMATION The Test Examiner should Complete a Group Infom1ation Sheet for each classroom or group of students. in Samet ~1 'i"`Fw 'l I. .. ., Hill: 4 .L . 5 'Levi ::il 1,55 ,152 -..- .-tl - :nt mb 'UPayne 03638 Test Etrmrinerlr Manual 18 Grader Shell Copyright (C) 2009 by the Geolgia It is critical to the reporting process that the Group Information Sheet be coded properly and that it remain with the used Test Boo/dats it represents. If a blank Group Information Sheet is used, it must be filled out in the same way as the preceded Group Information Sheet. Test Exurnincrs should use a No. 2 pencil. If any preceded information is incorrect on a Group Information System Coordinator should contact the CTE GA CRCT support team toll-free at (866) 282-2249 to order a replacement. The directions for completing the Group Information Sheet are as follows: Teacher Name Print the name of the Test Examiner and in the corresponding circles, School Name Verify that the school name has been prccodcd. Number Students 'Resting Print the number of Student Test Boolcletx to be scored. This includes the transcribed Student Ties; Booklets from the Braille andior Large-Print versions of the test. The count number should be r_ighL1iuslil'icd. For example, if your classroom contains 25 students, starting in the first box on the left you will fill in "zero," "tvs/o," and "five" Complete this section by filling in the corresponding circles under the numbers that you have printed, NOTE: Unused Student Test Booklets containing pre-ID labels or completed demographic information (for students who were not administered any part of the exam) must he marked and returned with the nonscorable materials. Do not include VOID Student Test Bookfets in the Group information Sheet count. Grade Fill in the appropriate circle. Special Codes For CT use only. ORG-TP (CTB USG) For CT use only. STRUC/Element (CTB Use) For CTB use only. Payne 03639 Grades I -2 Online .S`l|ult` 19 Te6r Emmincr's Manual 2099 by me Georgia qfsducmren 438 RETURNING MATERIALS TO THE SCHOOL TEST COORDINATOR The Test Examiner must return all used and unused Student Test Booklets, scratch paper, the completed Group Infomation Sheet, and the Test Examiners Manuaf to the School Test Coordinator. For soiled documents, Test Examiners must transcribe answers and student demographic information onto a new Student Test Booklet following procedure, and then submit the transcribed Student Test Booklet with the scorable materials. NOTE: Once the transcription is complete, the Schooi Test Coordinator should mark the soiled document on the School Security Checklist as and und then destroy the soiled document. Soiled documents should NOT be shipped back with Scoruhle ui' nonscurabh: materials. It is imperative that materials he organized and returned to the School Test Coordinator in the following order (top to bottom): TOP on STACK Compieted Group Information Sheet Used Text Booklets (du not need tn he zilphilbetized) Scratch paper for Mathematics CRCT Large~Pr%nt lest books and VOID Student Text Booklets but not inC|1.zdin1, SOILED Hnnirfeld) Test Examiners Manual All remaining Student Ext Booklets (including Braille test books, BOTTOM OF STACK IMPORTANT NOTES: The Group Information Sheet must be completed in its entirety. It is critical that it be placed On top of the used Student Test Booklets it represents, Reminder All and responses to those items are secure material and may not be copied, duplicated, or retained in the school after testing is completed. WITH THE ADMINISTRATION OF THE THANK YOU VERY MUCH FOR YOUR ASSISTANCE 1 Payne 03540 Test Fxalninefs Mrmufti 20 Gmfles F-2 Oniine Sheff Copyright (C) 2009 by due Gent;-In Department r1fEduc?1Ii0ll 439 Fon FFEST EXAMINERS BEFORE TESTING: DURING TESTENG2 Assist School Test Coordinator with nolitications to students and parents about the testing program. Read directions for test administration in this Test Ewminerk an ua! Meet with the School Test Coordinator to review testing schedule and procedures. Secure No. 2 pencils and scratch paper (at least two sheets per student for the Mathematics mst). Secure needed information from the School Test Coordinator to coirectly complete State-Required Codes (SRCS) foreach student. Inventoiy test materials from School Test Coordinator, and sign out all secure materials on School Security Checklist. Arrange for an additional proctor, if necessary. Ptlepare the classroom (remove or cover posters, etc.). Remind students to bring something to tend. Ensure that ail Student ks: Booklets have a pre-ID label. lf there is no pre-ID label, ensure that the studcnl idcntitication information in Sections l-7 is filled in on the Student Test Booklet. Do not apply incorrect labels. Write the teacher; school, and system name on the board. Post at "Tesling-Do Not Dislurh" sign on your classroom door. Be sure that all students have a comfortable and adequate workspace. Monitor students' handiing of Student to keep the materials in good condition. Circulate monitor :ill students during the testing session. AFTER Verify that you have collected and counted a Student Buoictet and scratch paper (at least two sheets for the Mathematics test) for every student in your classroom. Check to see that Student Test Booklets are properly filled in and are in good condition. Complete the "For Tcachcr Use Oniy" seclitm in lin: Ties! Booklets, as applicable. Verify that the student demographics and school-identifying, information have been completed. Count and return all test materials to the School Test Coordinator at the conclusion of ei-'cry test day. Date the School Security Checklist each day. On Lhe final day of testing, assembie all the test. materials according to the instructions provided in the "Post~ Testing Activities" section of this manual, and return the-fn to the School Test Coordinator. Notify the School Test Coordinator of the names of any students who did not take any part of the test. Payne 03641 Grades 1-2 online shaft 21 mr Examiner's Maman Capyvigln tn 2009 by :iw Gewgia Department Test Examinefs Manual Grades 1 and 2 Online Shel! 2! 168-ws 2 CW,-rash/ 2009 by nw nf Ham-"frm 0 364 441 "ef 1' iw? - ;s-'Grades 3 through 8 Reading, English/Language Arts, Mathematics, Science and Social Studies 7 1 GEURG i 1 a or EDUCATEON Georgia De nnment oF'Education Kathy Cox, Slate Superintendent o? Schools 2370 . my by me Gear;-0327 (U CU S_ecw'e Test Materials-Do Not Copy AEI test booklets and supporting materials associated with the Criterion-Referenced Competency Tests are coniidemial ami secure, No part of any test booklet may be repmdaced or rraasmiired in any form or by any means, including but not limited to electronic, mechanical, nianual, ur verbal (eg, pholoc?pying, recording, paraphrasing-#rewording or creating mirror items for instruction-audio: copying). CRCT materiais must remain secure at all times and (excluding the Sc/1001 and Systerlt Test Courdinatmis' Marzuaf, the Test Manual for grades 3 through 8, and the online shell for the grades 1 and 2 TestExaminer1v Manuals] cannot be viewed by any individual ur entity prior to or allcr testing. Manuals fur grades. 1 and 2 are secure) To do so is a _direct violation of testing policies and procedures established by Georgia law and the State Beard ufEclucntion (Rule in addition tovcopyright laws and>Georgia professional ethics for educators. CRCT materials may nor bc provided to any persons except those conducting test adminislmlion and those being tesled. AH test boqklets (used and jumised) and all supporting materials must'be accounhtidfor and al LliC nf the ISSE administration and in the manual' prescribed in the I School and Tex! 1 capyilrmr an 2009 by me u?~wigf?i aj mruffurwi 443 . UI. 000328 TABLE or CONTENTS Inlruduclicni. Purpose of This Manuai Test Materials. . . Test Seisurity Preparing for the Test Scheduling the Test Administeiing the Administering the Version of the Administering the Large-Print Version of the Distributing the CRCT find Completing 'the Student Infonnation Dircsurions liie Reading CRCT Directions for Administering the English/Language Arts CRCT . Directions for Administering the Mathematics CRCT Dimci:ions for Adn1iniste1'i.ng the Science CRCT Directions for Administering the Social Studies CRCT Post-'lbsting Activities . . I Encoding Student Information. . . . .. Completing Sections 9-14: "For Teacher Use Only" .. Completing Sections 15 and _16: Use Only" campieung the Group Infomation sneei . . .Q Returning to tim School 'Ibst Courdinator Checklist for Test Examiners Cqnvighr 21109 by nw amen 444 ,2 APS-OIR 02584 0329 INTRODUCTION Georgia's Criterion-Referenced Competency 'lests are designed to measure student acquisition of the knowledge and skills sci fonh in the slaLe's Georgia Performance Standards (GPS). Legisiated hy Georgia lawmakers, the Reading, English/ language Science, and Social Studies assessments were developed to yield information at the student, class, school, system, and state levels. The primary purpose of the is to provide at valid measure of tho quality of edilcatiunai services provided throughout the state. The Georgia are-comprehensive tests made up ot' multiple-choice items that map to the GPS. Each Student Test Booklet contains ali five content areas. Each content area test is nntde up of two sections; each section is timed for up to minutes. Students Wi?l work through the independently, using no resource materials during testing. Students in grades 3-3 may write in their test booklets, but only answers marked on StudeitiAr1.twerS.?reet.v will be scored. Students should use only No. 2 pencils when taking the may not he used to mark words or passages in the Student Terr Booklet. Calculators are not allowed during any part of the Mathematics test. Scratch paper, provided by can be used :luring the Mathematics test only. Punroso or Tins MANUAL This Terr Examiner'.r Manual provides prooedural information as well as detailed instructions for administering the Ii is imperative that the be administered uucoriling to tiiloetiolis in this nnuiunl. ll` you have any questions, contaut your School or System Test Coordinalon - Uniform test administration is needed to unsure higiuquaiity assessment data; To ensure this uniformity, the Test responsibilities are clearly outlined and detailed in this irmnual. 'Stop-by-step instructions for adininistering the test are designed to protect its integrity and securiiy and must be followed exactly. It is recommended that Test Examiners read this manual and become thomugiily with all procedures before administering the tests. - Grader 3-s nn omline-'S #fume :Wright in :rms by in; omgn. Qi mlm-n1f>> TEST In addition to this manuai, you will receive ell test materials from the School Test Coondinator. The Student Ter! Booklets will be presortecl into class packs of 20 and 5 and muse remain in their shrink-wrapped packs until they are distributed to Test Examiners. The 5 auzler1rAnswer Slzeels are also presorted and shrinks/rapped inio class packs of 20 and 5. The Answer Sheets are rnachinescarurable; therefore, it is important that all pages remain free of stray pencil marks or loose eraser bits and that they not be lbldecl, clipped, stapled, banded, taped, or torn. Students in grades 3-8' will use one Student Answer Slfteaz to record - Each Eestconsists of multiple-clioice questions. Sample items are provided to acquaint students wid; the correct procedures for recording answers, and dixecrions for filling in answers are provided on the cover of the Student Answer Sheet. NOTE: Materials are shipped to arrive a few days before your school's nine~day testing period. Contact your School or System Test Coordinator for your scheduled testing window. TEST SECURITY The are secure tests. Maintaining the secnriiy' of all tesi materials is crucial to obtaining valid and reliable Lest results. Therefore, test materials must he kept in locked storage, except during actual test administrarion. Access to secure materials must ine restricted to authorized individuals only Test Examiners and the School 'lust Coordinator). Et is the direct responsibility of all individuals who administer the test to follow security procedures, A Studeru Test Booklets and the items therein are to be used solely lor test purposes and may not be disclosed or used for any other p1.u'pose_ Student 1Ie.vtBooI 5 ?E0r.n 51 . 51 O5 on id- a.2o? gmgg" ==gg2_g -Q 3-S gag., .92 rnc.?- greg; 5 3.5553 -2 E-$3-io :r 7:-Qon U--wa 1-5| .Q 5 <'?cnC a ngemn; Www; 90 Una? C73-gfr Qfiiff, 3:35 E?ggwa egg 5 ?aag. 5 cv, 235 Worn 'inn 33? 53' @3563 af 5; r: 55 a .O g:-35 Seri* rc Q2 Egg; EL eg 5 ew; sew- --gi .mg 3 tag; gg 33,322 ga'-5. Systems will receive pre-ID labels to pre~irlentify demographic information on Sfudenir Answer Sheets. There will be three labels provided foreach student: a text label that can be attached to the Sfudemf T?st to ease distribution in the classroom, and two labels with preceded student infomation (one is extra). Feilow these instructions caretirlly to provide the correct' student demographic on each Stuzierzt/lmzwer Sl':eel: 4 - For enrolled students, place pre-ID labels on the front covers ofthe Sheets in the space marked LABEL 0 Verify that the information printed on the pre-ID lnizcls is nucurale (system nttme, system ID, school name, school ID, teacher name, student name, FTE Student ID and GTID numbers), If any errors are found, do ure the incorrect label. Instead, notify the School Test Coordinator Students who do not receive a correct pxe>>lD label will need to till in their demographic information on the Student Answer Sheet by hand, following the directions in this manual (see pages lG~l 2). Any incorrect information on pre>>ID labels should be corrected in the system's student records. - Do not change the label in any way. Apply only correct pre-ID labels to the Student Armver Sheets. Not all inlinmation included in the pre-ID label is visib?e. Only specilied fields may be seen on the label, as indicated above. Student demographic information (including ethnic goup, gender, anclalate of birth) is included in the pre-ID label and should not be Hlled in on the SlzraEenrA:r_rwer Sheets, Student demographic irilbrruutiun on the ure-Il) label will override any hand-filled information lin- all students. I Even with a pre-ID label, all students will still need to complete Sections and 8 - on the front cover, andthe Test Examiner wit] need to complete the "Fo1' Teacher Ure Only" section on the inside front cover, if applicable. NOTE: Student lbs!-Booklets are designated by Form number, which shouid bc titled in by students in Section ll of their .S'rua'enrAnswer Sheets. I If the student is new to the school/system or for any other reason does not have pre-ID label, follow the instructions ie this manual to till in the student demographic information on the Szuderit/lnsiver Sheer (see pages 10-12). All fields should be compieted accurately (including the FTE Student lD and GTID numbers). Test Examiners should have a cornp?ete list of FTE Student ID and GTID numbers prior to testing, I Become familiar with the sections on the Sheet, You will want to have some of this iltforxnalioir written on the board prior to the beginning of test administration i I lest Fifrnual 4 Grarifes 31? copy,-nrt' L1 was by Jive canst.. Depmhurnr 1 4418 ozses i Lo ua 00 6. Before beginning the test, make sure that each student has the following; I One Bootie! is On .litulienr Answer Sheet "I\/vu No. 2 pencils with funcli onal erasers I At -least two streets ofblunk scratch paper lor the Mathematics test 7. Each students workspace should be large enough to accommodate an open .Student Test Booklet and a Student Answer Sheer. Workspaces should be cleared of all other materials During testing, students should be separated by ai distance ig encourage independent Work and to prevent collaboration.'1'osters, charts, and other instructional materials related ?0 the-content being tested should not be displayed in the classroom or o?herwise made available to students during test adnlinistration. Take down or cover ail such material. 8. You may want to plan an activity for students who finish early, Students may read a book or do other work if they finish early, but this activity mustbe unrelated tn the content being tested and students should not he permitted to open their Student Booklets ugnin. SCHEDULING Tim TEST 'The Reading, English/Langtwage Arts, Mathematics, Science, and Social Studies will be administered to students within the state testing window ofApril 6_th.rough May S, 2009. Systems will choose one nine-day testing period within this window. All students should be tested in surroundings that will provide them with the oppominity to do their best work on the test. In schools where students in severnl ciussos arc being testedt each classroom of students must take the same test at the same time. The CltC"ls must in the order prescribed: Reading, Arts, Mathematics. Science, and Social Studies. Each section of each test must be administered in one block of ti1ne.Approxim/alcly halfway through the udministratiim of each content area test (between Sectinnsl and 2), students should be given a 10-minute break. Both sections of 3 content area test must he administered on the some day. Under no circumstances may a content area test be split across multiple days-to do so is a direct violation of test security The administration time for the as indicated in the following table, should be sufficient for students to complete each test section. All students should bc given at least 45 minutes to compiete each test section. If all students iinish the section before 70 minutes have phased, the Test Examiner :Tiny end tlie session early. However, if.tl'Jo1'e are students who are still productively engaged in completing the section, testing should continue for the full 70 minutes. Unless speciiled in an TEE I/xii or El] Plan, students shouid have a minimum of 45 minutes turd a rnaxiruum of 70 minutes to complete eoch -- section. Grades 3-8 5 Terr Mamuri o`:1on9 by mi 449 S-OIR 02589 AP 00 0334 Page 7 Caggidential Nor:-Pubfic Personal Information _Bminess Decisions In/brmation, Inc, DBA Business Informaiion Services has a strict privacy policy concerning the personal information including but nos limited ta,_ applicants, claimants, and policyholders. Federal and State laws prohibit us #nm disclosing non-public personal information to nanajliliated third parties unless otherwire pzrmifted nr required by icrw. Nonpublic personal in/brmalian is generabja in/'inrmation that personalb= zbiermjies an individual and which is not otherwise available ?0 the public. Non-public personal information includes, but is limited ln, financial and health "Privilcged and Corfidcnlial Attorney Work Product" 450 00259 3-8 Coawmwr AREA TASK . ALLOTTED Rllading Distribute Sraedcnt 7211 Bu:/Mm 15 Q0 and 5`rude1ztAnswerShee/x, and read directions the. student idantk?cation sectifm (for siudems withnuia pre-ID label) _'IbstAdminist?ationTSeQtiun 1 45-70 minutes Break IG minutes 2 45-70 minutes Tola[Adzninis\rmion 'lime 115~170 minutes - English/Language Arts Distx?bute Smdenr Tex! Baalciem 15-20 minutes and Siudcnl Aruwsr Shears, and read directioxs 1 minutes. 10 Section2 3 Total Administra\io11Time 115-170 itainuiss Malheniatius Distriimto llnokfefs minutes and Srudenl Amwer Sheets, and read directions 4540 minuies i Break 10 minutes Test Adminis;ration>>Section 2 45~70 minutes Tomi Administration 'Kms 1 15- 170 minu?e-9 Scienc? 15-20 minutes 1 and Str44ierL!Aruwer Sheets, and road directions Test Administration-Section 1 45-T0 minuies Drunk 10 :ninulcs 2_ minuws i 1l5>>17D:ui:1u1es sms; studies Dixtriimte slum; 15-20 minums und and read directions - TestAdlninlsirntiun-Faction 1 4-5--TU minutuu 'I1=sl Mllxillis - 5 'Thtal Administration 115470 minutes mr Manual 5 Gmfses 319 CJ 21109113 Gear;-141 rg' 451 02590 S-OIR AF H335 THE This lbs! Emmirterli Manual should lie used to administer the Reading, English/ Lang,-uagc Arts, Matlxeuiutics, Science, and Social Studies Beloie you administer the sure you have extra No. 2 pencils available and that students' desks are clcaucrl of all other materials, On the first day cftesiisxg, it is very important that students write their names on the Student Test Booklets and iill in Seetihn 8 on their Student Answer Sheets with their par?cular lest Form number. Students should use the same Student Test Booklet and Sheet on all subsequent days of testing. Some students may not be familiar with howto fiil in answers. To familiarize students with the correct procedure, sample items are provided at the beginning of eachconient area test. During each test session, circulate around the room and check to see that students unc recording their answers he the correct section of their .S'tur1`en1Answer Sheets. This process will be referred to on subsequent pages as "circulate und check." the script provided in this manisal for administering the Reading, English/Language Arts, Mathematics, Science, and Social Studies You may repeal any pait of diese directions as many times as needed, but you should not modify the words used. Use your professional judgment in responding to student questions that arise during the testing session. You may clarify directions; however; under no circlimstances should you reward test items, suggest answers, or evainate student work during the test session. Read word for Word all text that is boxed and printed in hold type and preceded by the word Say, Text that appears in regular type or italics should not be read aloud to the students. . For the Mathematics test, students should he given at least two sheets ofhlauk scratch paper, Students must print their names in the upper right-hand corner of each sheet of scratch paper they receive. Students muy ask for more scratch paper during the lest administration, if necessary. scratch paper must be accounted for and returned to the Schooi Test Coordinator with the othertest materials. Unless by studcnt's calculatorslar? noballowed onthe Mathematics test. For students with an IEE IAR or ELIJTPC Plan that clearly spccihcs st "Read Aloud" Accommodation, use at Student Test Booklet to administer the accommodation. It is prohibited to test any student with st "Road AI0ud"Accon1modution unless the accommodation is specified in the students EEP, IAP, or Plan. 4 Students in grades It-8 may underline und circle key Words 01' Work problems in their test booklets (using as Nc. 2 pencil), but only nmrked on Sheets will bc scored, I smart 3-s 7 ns/ Eiominzr'r Manual NKP9 by Jin* Henryk; Dr/iufllrienr uf Eihuxrriwl I APS-OIR 02591 C3 0336 If testing is interrupted at any time during the test session (by it irc emergency or child becoming for exatntile), note the time the interruption begins so that you can calculate the amount of reserved time the students wil! need to complete the test session. If students must Eeave the worn during testing, be sure to lock the door to ensure the security of the test materials. If you have any questions about an individual situation, please contact your School or System Test Coordinator. ADMINISTERING THE BRAILLE VERSIQN or THE The directions in this manual also appiy to the aclministration of the version ofthe Additional Bmille instructions are ss follows: 0 Disregard references to ciess packs. The Braille test books wilt ine individusily packqed I 1 References to specific page numbers in the standard Form 1 Student Test Booklet may be incur-root for the Braille version. In order to supply tho correct page numbers and other references, 'Pest Examiners need to reviewaprior to testi ng- all test materials that aoeompany the Braille test book, including the supplemental instructions and the standard Form 1 .Student Test Booklet. The Tlest Examiner must compietc the student ideutitlcation information for each student on a StudenrAnswer Sheer, The TestExanii11er most also print the students name, teacher. school, and system on the front .cover of the Braille test book; . Indicate on the Student Answer Sheet that the student was administered the Braille version by Iiiling in the circle next to "Braille">>in Section 12. 0 Boenuse extra time may be ucodeti for adnlinistering the Braille version, it is recommended that students be tested individually or in a small-group setting. I Test Examiners who administer the Braille vtarsion will use the standard Form Student Test Booklet throughout Resting. Tho 'll:srExamine'r or proctor must fill in sludmit responses on the SludenrAn.rWer' Sheet exactly as the student dictates. Under no circumstances should a stude:it's answer he altered or edited-to do so is direct violation of test security. I Keep the transcribed Sheer with the other used Student Answer Sheets from the S{liElBllI'S class. The School Test Coordinator should return the Braille test books with the nonscorable shipment. . Test ilrrmuzzl 3 Glade: .7-8 9 2009-by me Gratgia Deparnm-ni of 453 APS-OIR 02592 OG 0337 ADMINISTERING THE VERSION on "mn The tftrections in this manual also apply to the administration of the Lnrgedlzint version oi the Aclditionn! Large-Print instructions are is follows: Distegaxd references to class packs. The test books Will be individua?ly packaged. The Tcstiixaminet' must complete the student idcntiduation information for each student on a Student Answer Sheet. The Test Examiner must also print the students =1ame,feacher, school, and system onthe front cover of the Large-Print rest book. Indicate on the that student was administered the Lange- Print versien by iiling in the circle next to "Large7Print" in Section 12. time maj.-' hc rteetlecl for ndttiinisterirtgihe version, it is that sm?ents be tested individually or in a small-group setting. '.-vho use at Latge-Print version shuuld record theix' answers directly in the Large-Print test book by circling the letters for their selected Writing their responses, or using il sitniiar marking system. The Test Exsnniner or proctot' must then transcribe student responses onto the Sfudent Answer Sheer exactly as they appear in the Large-Ptint test book. Unrlar nc should tt answerbe altered or edited-to do so is a direct violatisn of test security. Keep the transcribed Stedenri?nswer Sheet with the other used Student Answer Sheets from the siudenfs class. The School Test Coordinator should return the Large-Print test: books with nonscorable shipment. If Hindus 3-8 'Iks/ Mamuri capyi-lym uzoov ti- me (Imrgin (D. EL. 006338 THE CRCT NLXTERIALS AND COMPLETING THE STUDENT IDENTIFICATION INFORMATION 1, Write your name, the name ofthe school, and the name of the sysoem on the board. 2. Say; During the next few days, you will he taking tests in Reading, English/Language Arts, Mathematics, Science, and Social Studies. Some ol' the questions may be easy, and some may be dillicult. You are expected to answer each question as well as you can. You must fill in your answers-on the Stud'entAnswer Sheet with a No. 2 pencil. If you do not have a sharpened No. 2 pencil, please raise your hand. [Supply 2 pencils rolrturients who need the/rt] - write on your answer sheet until I iosiruct you to do so. I 1 am now going to give you your answer sheet. Please do not open or 3. Distxibute the appropriate grade-level Student Answer Sheer to each student. Monitor 4* ily 2" Look at the front cover of your answer sheet Look ut Section 1 Using, your Nu pencil, print your name, teaches name, and the school and system names in the spaces provided The inform alum that you need to complete this section is written on the hoarci [Czrculate and cherlc] pencils If you do NOT have a pre ID label, [ook at bechon 2, "Last \l imc" and "First Numa" In the IJUXE below punt the letters of your last name, stan ting in the ins: box on the iett. Dc not leave any spaces between letters an your last nfum lfyour last name docs lit, write as many letters as you can [Pause to allow .students lzme fo WTIIB their lm: names] In the boxes helow First Name: print the letters ot' your name. This should be your full tirstnarne, not a nickname. Do not leave any spaces between the letters in your first name. If your first name does not lit, write as many letters as you can. lPause to allow students time to write theiryirst nan:es_] . In the box below print the initial iettcr of your middle name, il' ;fou have name. [Pause zo to write :lm initial leirer of their middle name! - 3 Now, Section 2 by completely filling in the corresponding circles letters that you have printed, For example, if you wmte "rg" yon should fill ln the circle containing LCircu!cire ami checkj ssndents Lo make sure they :lo not open or Write on their Sheet. Fur those students who have a pre?ID label, please put clown your I _p . . . - 1551 /emma: 10 Gralcilrs 3-<9 Capo-fylr on znan by uwfgre 5 455 APS-OIR 02594 03 39 5- -Su Look at Section 3, "Dnto of Birth." Fill in the circle beside the month in which you were born. Then, print the day and the last two numbers ofthe year in which-you were born in the appropriate boxes. Fill in the corresponding circles under the day and yearn Days under lil should begin with zero. For example, if your is on the fifth :lay of the month, you should print and till in "zero~Iive." iffircutafe and check] Look at Section If ill in the circle next to "1'emu1e' if you are a female or the circle next to "Male" if you are a male. [Ctrculcue and check.) . Look nt Section 5, "Racial/Ethnic Backgronnrl." Fill in circle next tothe ethnic group that best describes you. You should fill in only one circle; Asian/Pacific Hispanic, American Indian/Alaskan Native, or Multiracial. If you have any questions about which circie you should till ln, please raise your hand. [Circutute and check] Look ut Suction 6, Print your ten~rliglt GTLD number starting - in the iirst box on the left. Fill in the corresponding circles under the you have if you do not lcuow your GTID number; please raise your hand. ?Cimut=zte and chest; providing GTID numbers to have hands miseal] Look at Section 7, Student Print your nine-rligit Student ID number starting in the first box on the left. Fill in the corresponding circles under the numbers you have printed. If you do not know your Student ID please raise your hand. [Circulczte and c/Leek, providing Student ID numbers to students who have their hands Grades 3-8 [When students are really distribute Student TestBooltlets ra students] For those students who hove a pre-Ii) label, please pick up your pencils and continue to tfotiow aiong. On the cover of your Student Test Booidet, please locate your Form number. On the Sheet, print the Form number that matches the test booklet you have received in the box for Section 8, "Fo11n." Fill in the npproprintc circie under the number. [Clrcutaze and checFc.] Leave Sections 9 through 16 blank. I- will complete these sections for you. [Complete diese testing in all content areas has been completed. Directions for completing these sections' begin on page 37 af this manual.] . . Are there any questions before we continue? 11 . rm Mama 53 21709 bji'-'lm Georgia 456 PS-OIR 02595 00 i i 0340 a 6. Answer all questions. Walk around the room to see if students have completed the demographic information correctly. NOTE for Sections 6 and 7: A GTID is a Georgia Tbsting lclentilicr that is used to track and maintain a student's data within the Georgia Department of data warehouse and the local Student Information System. All PK-12 students who a public school in Geoigia must have a GTID. The GTID is the key identifier used in #lil state data collections. For the PTE Student ID (Section T), it is imperative that students use the same student ID number he-re that the school or system uses, to report FIB-type inibnnation tothe state. Locally assigned student ID numbers should not be used if they do not correspond with state-reported information. mianmiifm-'S Manual I2 Gimm as CJ 2009 by me (lnvigia L3fEr111ni|lfw| 457 APS-O 02595 5 0341 DIRECTIONS Fon ADMINISTERING THE READING CRCT Reading-Section 1 Estimated Lime: TO minutes (plus lil minutes for the break) Materials needed: Student Te.st'Boo.lclets and Student Answer Sheets - i NOTE: Follow lhe directions on pages 10-1210 complete the student demographic information on the ,Sheets prior to the storl of the Reading test. Sharpened No.2 pencils with erasers 1. Make sure that all desks are clemsd and that all sluclenlx have their own answer sheet. Confirm that the Form number of she test booklet corresponds to Elie Form number that siudenm have written down in Section 8 -of their answer sheets. 2. -Say Now, look at the test booklet. Print your name on the line provided on i the front cover of your test booklet. Do not open your test booklet until you are instructed to do so. [Pause] When you have finished printing your name on the front ?078.12 set your test booklet to the side. [il/air until all students have completed nits task.) -I Iwill now read some opening to you. 1 The Reading Criterion-Referenced Competency Test will he in two sections. Both sections ofthe test are made up . of multiple-choice questions. When answering the multiple-choice . questions; select the best answer and lil! in the circle for the answer you have chosen. Do your best to answer nach question. - jg Now, break the seo] on your test booklet by pulling the blue tub away from the edge ol' the lmoldet. Pull the blue lab carefully so that you do not tear the pages. Do NOT attempt to remove the clear tal: that remains. - [llizit until all .s~tun'erm' lzave completed oils rrm%.] Open your test booklet to "Reading" on page 4. [Pause] Remfl the 'General Instructions silenny as I read them aloud. Today you will he taking the Reading Crilei-ion~Referenoed Competency' Test. The is made up of mulliplechoice questions. A sample ims been -included. The sample will show you how to mark your answers. ren Manual 14 I Gfnafs 3-as "copy/>>1mo may by rn, coolo Dgpn-nom of 458 0259? us -D i 3. Shy There are several iinportanl things to remember. g- 9 Read each question carefully and think about the answer: You may look back to thereading passage as often as necessary. Answer ull questions on your answer sheet, not in your test booklet. You may write in your test booklet, boi only answers marked on _y your answer sheet will be scored. I For each question, choose the one best answer and iill in the circle in - the space provided on your uuswcr sheet; - 1 If you ?o not know the answer to a question, fill in your best choice. You may return to the question later if you have time. 0 Ifyou finish the section of the lest that you are working on early, you . may review your answers in that section only. You may not review I 0 Read each passage carefully. any other section or go on to the next section ofthe test. Are there any questions? [Ariswer any questions the sturfents may have] Pick up your answer sheet and find Section 1 for the Reading test. This is here you will till in your responses to the questions. [Pause] Look at the sample question ou the next page test booklet and follow along as we review the sample question. lf. The sample test question below is provided to show you what the questions in the test are tike and how to mark your answer lo each question. For each question, choose the one best answer and #ill in the circle in the space provided on your answer sheet for the answer you have chosen. Be sure to mark all of your answers to the questions on 1. your answer sheet. Read the sample passage now. 4. Allow time for students to reaclithe sample passage. Go over the sample question With' students, and he sure they E51 in the answer to the sample question in the appropriate place on their answer sheet. Ask students if they have any questions about the sample Test item, and answer any p_ruecr.iLuul questions. Grades 3-8 15 F-fanuai CWIYHHUF 2009 by I/ll! Ueufgfli cf 459 98 APS-OIR D25 CE ?1343 5. Soy You will have 70 minutes to answer the questions in Section 1 ol' the _t space provided on your answer Sheet. Stop when you reach the stop sign after question 25 in your Student Test Booklet. This will match the Eine With the word "stop beside it on your SLudent.4nswer Sheer. If y0ll - 1 finish this section before time is called, you may go back ami check your work in this section only. You may not review any other section or look ahead to Section 2. Do you have any questions about what to do or where to fill in your answers? {Ansv>>=er any procetfural qnesti/m.s the _rtudenfs may have] - Reading test. Read each question carefully and lill in your answer in the Now, turn the page to Reading, Section l. You may begin. 6. Walk mound the room lion: time to time to tnakc su|\e students are Filling in their answers appropriately. All students should be given at least 45 minutes to complete Section l, ll` all students lin_Esl1bei`ore 70 minL1Les passed, go on to However, if any students ore slill productively engaged in completing Section 1, _ailow them the full 70 minutes hefoto continuing with #1 7 rl Please bookiet You may now take a break You may stand and stretch for 10 minutes Your lest booklet should remain closed nn von! desk. Please do not talk - 15' 'ni 3f- Insert your answer sheet into your test booklet and close your test 8, At this time, allow students to standend stretch for lt) minuses Make sure their answer sheets are inserted into their test booklets and their test booklets me closed. After no more than 30 minutes, continue with 9. Please sit down again. "1 Keep your test booklet closed, und walt quietly as we get ready to move on io the neil; section. ffm/ Test .lffanual 15 Gmrfes 3-8 - cw;-rgnl 2009 by Aho qrst/turmoil 460 02599 CD 0344 TRANSCRIPTION OF RECORDED STATEMENT BDI INVESTIGATOR AND THOMASVILLE HEIGHTS ELEMENTARY Dr. Stacy Brondage Investigator - I'rn Stan Williams representing Atlanta Public Schools- and- the Ofticc of Internal Resolution. Today's date is Friday, May 2nd, 2008. I'n1 at - Thomasville Heights Elementary School and- preparing- to take a from Dr. Stacy Brondage. DL Brondage, would you please state your name? Dr, Brondage It's Dr. Stacy Brondage. Investigator Dr. Brondage, do you understand this statement is being recorded? Dr. Brondage Yes sir. Investigator Dr. Brondage, do you affirm to answer trutl@l1y and to be the best of your - knowledge? Dr. Brondage Yes, I do._ Investigator Dr. Hrondage how long have you been employed by Atlanta Public Schools? Dr. Brondiige I'vc been employed by Atlanta Public Schools since 2004. Icame here in 2001 from the State Department of Education. So I've been here several years, Investigator How long have you been at Thomasville Heights Elementary School? Dr. Brondage Seven years.. I I I Investigator Dr. Brondage And, what is your position at the school? I'm the Instructions Specialist and the Upper Literacy Coach. Investigator How many CRCT Testing's have you been in involved with? Dr. Brondage I've been involved in testing since 2001. I usually assist the Test Coordinator. Investigator In what capacity have you participated in the student testing? Dr. Bronclage As monitors who help distribute the test, help with the insiders, and to collect Investigator Dr. Brondage the test, Occasionallyl have administered the lest bei`ore. Where did you administer a vest? - I administered the test to our First Grade this year because the teacher was 'out due to death in the family. "Privileged mm' Attorney Work Producti' 461 APS-OIR 00260 R0uding-Section 2 Estimated ilme: TO minuaes 1. Say - Now you will take Section 2 ofthe Reaoin Criterion-Referenced I Competency Test. Yon will have 70 minutes to answer the questions in 7 Seclion 2. Stop when you reach the stop sign after question 50 in your Student Test Booklet. This will match the line with the word "stop" beside 5 1 it on your Student Answer Sheet. If you finish this section before time is called,-you may go back and check your work in this section only. 'situ 1 -- may not review any other seciion or look hack to Section 1. When you - are finished, insert your answer sheet into your test booklet, close your lest booklet, and eil quietly or read st book. Now, tulce out your answer sheet and find Section 2 tbl' the Reading test. I Open vour test booklet to Reading Section 2, question 26. [Pause while correcf page] A 5_1 i You may begin. [Make sure stndenm begin filling in their ci/Lover.: in the correc! place on the answer sheen] 'fi' 75131 .students are opening their test booklers; make sum all s1nf??m1_y are rm me 2. Walk around Lhe room from time to time to make sure students are filling in their answeis appropriately. All students should be given ai least 45 minutes to complete Sectios 2. lf all students finish before 70 minutes have passed, go on to However, if any students are still productively ungaged in completing Sectlo412, allow them the full 70 minutes before continuing with dose vour test booklet. 3. Please stop working. Insert your answer sheet inio yotu' Lest booklet and 4. Collect Lhe test booklets and answer sheets. As you collect them, check each leant' booklet to make sure the studenfs name is printed on the cover. Make sure the test booklet Form number matches the Form number filled in' under Section 8 ofthe Sheer. Count the lest booklets and answer sheets to conhrm that all leaf materials nn: accounted lor, and return them to the School Test Coort'lina.tor at the conclusion of each testing day. - . Grazia: 3-8 1-1 Examine/'ir Manual - -- Copyright 2009 by me songs.. 462 02600 IR APS-O U3 45 DIRECTIONS Fon ADMHQISTERING THE ARTS CRCT Englisil/Language Artsg-Section 1 Estilnated time; T0 minutes (plus lil minutes forthe break) Materials needed: Student Test Booklets and Stucient'Am'wer Sheets No. 2 pencils with ernaeres zay I am gomg to you your test hookict and answer sheet When you receive them, cheek to make sure I have given you the correct test bookiet ami answer sheet and not another student's Do not open yeur test booklet eu mark on you; answer sheet until I tell you to [Dutrrbure .Student Test Booklets and .'i`tudem'Am'wer Sheets Set your test booklet to the side [Pause] I new read some opemng uctmm to you The Arts Cnterxon Referenced Pompetencv Test be ln two seetlons Both sections of the test are made up of multiple-choice questions. When answering the multiple-choice questions, select the best answer and till in the circle for the arswer you have chosen. Do your best to answer each question. Open your test booklet to Englislm/Language Arts. Read lhe General Instructions silently as I reall them aloud. 'iloday you will be taking the Arts Criterion- Competency Test. The English/Language Arts test is made up of multipie-choice questions. A sample has been included. The sample will show you how to mark your answers. 1. Malce sure that all desks are cleared. - Glade.: 3>>8 I cafiyf-f,n1 ea by nn- affmin mp..-1-mnPS-OIR O26 so A ua UN asp/ Read each question carefuliy anti think about the answer. Arswer all questions on your answer sheet, not in your test booklet. You may write in your test booklet, but only answers marked on yoas' answer sheet will be scored. For each question choose the one best answer and till in the circle ln the space provided on your answer sheet. If you do not know the answer to a question, till in your best choice. You may lettun to the question later ifyou have time. lt' you finish the section ol' test that you are working on early, you may review your answers in that section only. You may not reyiew any other section or go on to the next section of the test. Are there any questions? [Ansn er any questions' the students may have] Pick up your answer sheet and ind Section I for the Englislx/Language Arts test. This is where you wili iill in your responses to the questions. [Pause] Look at the sampie question on the nest page of your test booklet and follow along as we review the sample question. The sample test question helowis provided to show you what the questions in the test arc like and how to mark your answer to each question. For each question, choose the one best answer and fili in the circle in the space provided on your answer sheet for the answer you have chosen. Be sure to mark all of your answers to the questions on your answer sheet. 3. =Stly, There are several important things to remember: I 0 I . 4. Have students read the snmpic item. Go over the sampie question wilh students, and be sure they fill in the answer to the sample question in the appropriate place on their answer sheet, Ask students ifthey have any the sample test item, and answer any procedural questions. Arts test Bead each question carefully and [ill lil youx answer zu the space provided on your answer sheet. Stop when you reach the stop sign after question 30 in your Student Test Booklet Tins wlli match the line with the word "stop" beside it on your Student Answer Sheet Ifyou finish this section before time IS called, you may go hack and check your work In this section only You may not renew anyother section or look ahead to 2. Do you have ony questions about what to do or where to fill in your answers? [Answer any procedural questions the students may have] 5. You will have T0 minutes to answer the questions in Section 1 oi' the Now, turn the page to English/Language Arts, Section 1. You may begin. Grzules 3-S 19 Ten Manual capynx/K row by as owing of eaicfnrof02602 APS-OIR OG 0347 6, Walk around the room from time to time to make sure students are filling in their _answers appropriately. All students should be given at leasa 45 minutes to complete Section 1. If ail students finish befoze TU minutes have passed, go on to However, if nny are still productively engaged in completing Section I, allow them the full 70 minutes before continuing with $57. 7. S2 Please stop working. Insert your answer sheet into your test booklet and close your test lmuklet, You may now lake break. You muy stnud and slretcll for - Please do not talk. I 10 minutes. Your iss( booklet should remain closed on your desk. 8. At this time, ailnw students to stand a11d-stretch for IU minutes. Make sure their answer sheets are their test bookfets an? their test booklets are closedr After no more than T0 minutes, continue with 9. Say Plvusc sit down on to the next sectmn I Keep your test booklet closed, and wait quietly as we get ready to move "Ext-Exafarlnerlr Manual 21] Gmdex .1-8 $2 2009 by me Geargm ly606348 English/Language Arts-Suction 2 Estimated time: TD minutes 1- -F _ill Wil] take 'lm-action 2 ol' the Arts Criteriouy Referenced Sompetency Test. 'lou will have 70 minutes to answer ?he - questions in Section 2. Stop when you reach the stop sign after question .. 60 in your Student Text Bookler, This will match the line with the word "stop" beside it on your Studem'Ansu cr Sheet. If you Hz-iish this section before time is called, you may go back and check your work in this Section only. You may not review any other section or look back to Section 1. When you are iinished, insert your answer sheet into your Now, take out your answer sheet and ?nd Section 2 for the English! Arts test. Open your test booklet to Arts Section 2, question 31. [Pause while students are opening their res! booklets' make sure are on the correctpage] You may iiegin. [Make sure studerils beginfillirqg in their answers in the correct 35; place on the cmswer shea] Q1 i ?est booklet, close your test booklet, and sit or read a book. 2. Walk around tiieroom from time to time to mdke surestudeots are iillingin theig answers appropriately. All students should be given at least 45 minutes to complete Suction 2. If al! students iinish before 70 minutes have passed, go on to However, ifany students are still productively engaged in completing Section 2, allow them the full 70 minutes iiefoue continuing with 3. Say Plenseslop working Insert your shoot into your test booklet am] close your test booklet. 4, Collect test booklets and answer sheets. As you collect them, check each lest booklet to make sure i?1e student's name is printed on the cover. Make sure the test booklet Form number matches the Fonn number filled in under Section 8 of the Studentfinswer Sheet. Count the lest booklets and answer sheets to confirm that all test materials are accounted for, and return them to the School Test Coordinator at the .conclusion of each testing day. - Grades 3>>5 21 'Roi Enm=iner's ofanual Cupylighr 2009 of 466 D4 APS-OIR 026 CJ C5 -5 Fon ADMINISTERING THE MATHEMATICS CRCT `Matl1ematics--Section 1 Est?mated time: Ti] minutes (plus 10 minutes for the break) Mabcrizxls needed: Siudent lest Booklets and StudentAn.rwer Sheen Slraapened No. 2 pencils with erasers Two sheeis of blank scratch paper por NOTE: Calculators are not allowed during any portion of this lest. l. Mako sure that all desks are cleared. 2. Soy - I am going E0 give you your test bookiet and answer sheet. When you J) receive them, check to make sure I have given you the correct test booliiet and answer sheei and not another studen?'s. Do not open your test booklet or mark on your answer sheet until I tell you to. EDistributc Studem Test Uaoklers and Smdena Answer Sheets.) I nm providing you with two clean sheels paper to use during ?he Mathematics Criterion-Referenced Competency Rst. You can use scratch paper Lo help you solve the math problems, but he sure io record all your answers on your answer sheet. [Disrribue two sheets of blank paper to each If you need more scratch paper during the test administration, please 'raise your hand. Now, w11te your name in the upper right-hand corner of each sheet of scratch paper: I will collect the scratch paper at the end ol' testing, whether- you use it or not. {Allow students lime ro wrire their names on the paper] Set your test booklet to the side. [Paine] I will now mul some opening instructions to you. - The Mathematics Criterion-Referenced Competency Tesl will he administered in two sections. Both sections of the test are made up of muliiplecooice questions. When answering the multiple-choice questions, select the best unswer and Ell in the circle for the answer you haveclrosen. Do your best to answer each question. r\ ,-ev - 17:11 Examinerir 22 34? . cepyfigm 1: 2009 by me 467 Ps~o|R `o2e05 2- A i 0350 3, _Say Open your test booklet to Matheniotlcs. Read the General Instructions silently as I read them aloud. . 'lbday you will be taking the Mathematics Criterion-Referenced Competency Test, 'i11e Mathematics test is mode up of multiple-choice questions. A sample - has been included. The sample will show you how to mark your answers. .- There are several inipurlaut things to rememben Read each question carefully and think about the answer: Answer all questions on your answer sheet, not in your test booklet. You may write in your test booklet, but only answers marked on your answer sheet will he scored. For euch question, choose the one best answer and fill in the circle ln the space provided on your answer sheet. ll' you do not know the answer to a question, [ill in your best choice. - You may return to the question later if you have time; If you finish the section ol' the best that you ure working on early, you . may review your answers in that section only. You may not review . any other section or gn on tu the next section ol' the test. jg Are there anyquestiuns? EAnswer any questions the students may have] Pick up your answer sheet and find Section 1 for the Mathematics test. 1 This is where you will ull in your responses to the questions. [Pnmm] Look as the sample question on the next page of your test booklet and i follow aiong as-.we review the The sample lest question below is provided to show you what the questions ln lho test are like and how to mark your answer to each 3 question. For each question, choose the one best answer and HIE in the .circle ln the space provided on your answer sheet for the answer you i have chosen. Be sure to mark all of your answers to the questions on your answer sheet. 4. Have students read the sample item. Go over the sample question and be sure they fill in the answer to the sample question in the appropriate place on their answer sheet. Ask students have any questions about the sample test ilem, and nnswer any procedure] questions. If you are administering die grade 3, 4, or 5 CRCI1 skip ahead Lo ifym; are administering the grade 6, 7, continue wilh 5. Look at the hottorn of the page, under the heading "l*formulas." ltelouf art',Drrmulas you may find usefui as you work through the probiems.-_ Howeveiysome of the formulas may not he used. You may refer to this . - Guide: 3-E 23 ivianuul - 468 APS-OIR 02606 (R) <3 0351 6. Say- You will have 70 minutes to answer the questions in Section 1 of the Mathematics test. Read each question carefully and till in your answer in the space provided on your tuiswer sheet. Stop when you reach the stop sign after question 35 in your Student Test Booklet. This will match the line with the ward "stop" beside it on your SmdenrAnsw|:r Sheet. If you finish this section before time is collect, you may go back and check - your work in this section only. You may not review any other section or look ahead to Section 2. Do you have any questions about what to do or where to iill in your answers? [Arnwer any procedural questions the students may have] - Now, turn the page to llrlatlleulatics, Section 1. You may begin. 7. Walk around the room from time Lo time to niake sune students are filling in their answers appropriately. All students should be gh/en at least 45 minutes to complete Section l. If all students finish before 70 minutes have passed. go on to However, if any students are still productively engaged in completing Section 1, allow them the full 70 minutes beforc continuing with #Sn - 8. Shy; Please stop working. ~1jI'f _Insert your answer sheet and scratch paper into your test booklet and close your test booklet. You may now take a break. You may stand and I stretch for 19 mimttes. Your test bookiet should remain closed on your - - desk. Please do not talk. 9. At this time, aliow students to stead and stretch for IG miuules. Make sure their answer slieets and scrateh paper are inserted into their test booklets and their test booklets are closed. After no more than 10 minutes, go on to #l0. bay Please sit down again on to the next section I .I Keep your test booklet closed, and wait quietly as we get ready to move I. . . mf Mmm: 24 e' cfm; 1-s 2009 it me Gcunklu CL. 0(J{l352 2 Estimated time: minutes 1. "Say: Now you will Section of the Mathematics C1'ite1'ioxl-Referenced Competency Test. You will have 70 minutes to answer the questions in - Section 2. Stop when you reach the stop sign after question 70 ln your Student Tex: Booklet. This will match the line with the word "stop" beside - it on Sheet. If you finish this 'section before time is called, you may go back and check your work ln this section only. You 1 nwy not review any other section or [ook hack Inu Section 1. When you are finished, insert your answer sheet and scratch paper into your test imoklel, elose your test booklet, sit or read a book. .2 - 5 "1 Now, take out your answer sheet and find Section 2 for the Mathematics 5' - test. Ogen your test booklet to iVl8tI1EURl'l13I1ifS Section 2, question 36. [Pause while students areopening their make sure all students 3 are our/ze correctpage] 2. IF you are a.dn1iniste1~ing the grade 3, 4, or 5 CKUL skip ahead to If you are administering the grade 6, T, or 8 CRCT, continue with 3. Say When you Section 2 in your test booklet, tum back one page and look under the heading "Forinolas." Below are formulas you may find useful as you work through the problems. However, some of the EQ lhrmulas may not be used, You may refer to this page as you luke I the test. . You may begin [Make were in fherr ruzswers the 4__S_y2 - 'fi 2 correcr place on the answer sheen] 5. Walk around the rooan lrom lime to time io make sure students are filling in their answers approp1'iate1y.All students should be given at ieast45 minutes to compiele Section 2. lf all students finish before 70 nliuutes have passed, go on to However, if any students are still productively engaged in compieting Section 2, allow them the l`nl1 70 rninutes before. continuing with #Ee 1: Grader 1-s 25 mr emfunef-e copy-rg/if 2009 by no nr.-login Denfowuunf vfolumum - 470 006353 6. Say Pleaqe Stop close vour lest booklet Insert your answer sheet and scratch paper into your test booklet and 7. Collect the test booklets zind answer sheets and the two sheets (minimum) of scratch paper, As you collect them, check each test booklet to mnlce sure the student's name is printed on the cover. Make sure the test booklet Form number matches the Form number Hlled in under Section _8 of the Student Answer Sheet. Count the rest booklets - and answer sheets to confirm thai all LMT. materials are accounted for, an? return them to the School Test Coordinator at the conclusion of each testing day. 'est ExmJli1la/"J Mammi 2,6 Grades 34? @7009 ag me ,geuiwnnair 471 02609 <0354 2 Investigator Dr. Brondage Investigator Dr. Brondage Inv estigtltot' Dr. Brondage Investigator Dr. Brondage Investigator Dr. Bronclage- Investigator Dr. Brondagc Investigator Dr. Bronclage Investigator Dr. Brontiuge Investigator Dr. Brondage Prior to the 2008 Testing at Thomasville Heights, what was your responsibility? Well I administered a test for a First Grade eiass but before I did that, in the moming I would assist the Special Reading materials out of state in order to make sure the tests never left alone prior to the distribution; and I did help collect them. - Are you familiar with the state mles for adniinistering CRCT Testing? Yes sir, Did the I'1'inoipal: at Thomasville Heights meet with you' prior to this yeur's testing to discuss the upcoming test? Yes sir. When :iid that meeting occur? It occurred a few days before the testing- I ca11'l give you the exact date- but We had an 111-service prior to testing. Who conducted the In-service? Tracy Fischer. And was that a dif`t`erent-meeting from the meeting with the-Principal? A separate meeting? Yes. The meeting_wit1'1 the Principal, when did it ooeur? We have Administiulive break-through so that would have occurred on it Monday morning probably. In the meeting in which the CRCT Testing was discussed, specifically what was said, as best you can Just to make' sure we were monitoring' the students that everyone actively monitors, just to ensure that if the students got oEtralek that they get them back on track. Just to encourage the studentslo do their -best; and pretty rnueh just motivational; as one prepared for the kids and was ready to go. Have there been any discussions at the school about the need to make the Superiuteiidoiifs target? Yes sir_ Wc talked about the targets; at the time so we know exactly what we _have to do in order to meet the targets. "Priviieged und Confidential A ttorney Work Product" I 472 APS-OIR 00261 2 DIRECTIONS Fon THE SCIENCE CRCT Science-S action 1 Estimated time: minutes (plus 10 minutes for the break) Materials needed: - 0 Student Ter! Booklets and Student Answer Sheets Shargenect No. 2 pencils with-erasers Sill' 1 nm going to give you your test und sheet When you receive them, check to make sure I have given you the correct test booklet and answer sheet and not lnother student's Do not open your fest booklet or mark on }our answer sheet 'until I tell you to [Distribute Student [Pvt Booklets and 'itudertAnswer Sheets Set your test booklet to the snie [Paine I will new read some opening instructions to you. Tlx. Quenee Criterion Rcfciemed CuXnpet|.ncy Test w1ll be admlnish-:red f' In two sections. Bath of the test are made up of multiple-choice questions. When the multiple-choice questions, select the best answer and fill in the circle for the answer you have chosen. Do your best to answer each question. Open your test booklet to Science. Read the General Instructions silently - as I lead them aloud. Today you will be taking the Science Criterion-Referenced Competency .- Test. The Science test is made up of multiple-choice questions. A sample I. Mnke Sure that all desks arc cleared. has been included. The sample will shoyv you how lu your answers. Ties! Exnmincr"k' Manual 2,3 G1-rides 3~3 - Copyriglrr 9 2009 tly :006355 3. Shy- There me several important things to remember. - Read each question carefully and think about the answer: - 0 Answer all questions on your answer sheet, not in your test booklet. You may write in your test booklet, but only answers marked on your answer sheet will be scored. 1 For each question, choose the one best answer and fill in the circle in the space provided on your answer sheet. If you sin not know the answer to a question, till in your best choice, You may return to the question later if you have time. if you finish the section of the test that you are working on early, you may review your answers in that section only You may not review any other section or go on to the next section of the test. - Are ihere any questions? [Answer any questions the students may have] . Pick up your answer sheet and tind Section 1 for the Sdence test. This is 11 where you will fiil in your questions. [Pause] Look at the iv '3 sample question on the next page of' your test booklet and foilow atong as we reyiew the sample question. The sample test question below is provided to show yon what the 1- questionra in the test are like and how to mark your answer to each question. For each question, choose the one best answer and fill in the circle in the space provided on your answer sheet for tl1e answer you if have chosen. Be sure to mark all of your answers to the questions on ff" your answersheet. 4. Have students read the sample item. Go over the sample question with studetlts, and be sure they tilt in the answer to Lhe sample question in the appropriate place on their answer shoot. Ask students if they have any questions aboua the sampte test item, and answer ony iarocedusal questions. 5. You will have 70 minutes to answer the questions in Section 1 of the 31: ff' Science test. Read eneh question carefully and filf in your answer in the space provided on your answer sheet. Stop when you reach the stop sign' after question 35 in your Student Test Booklet. This wili line ll finish this section before time is called, you may go back and check your work in this section only, You :nay not review any other section or look ahead to Section 2. Do you have any questions about what to 'do or where to fill in yog answers? [Answer any piruuezlural questions the students may with the word stop beside it on your Sheet.. If you - Now, turn the page to Science, Section 1. You may begins Grades 3>>8 29 Thx! Eiuminars Mantra! -Q 212091/)e rllu ey 5 474 02511 PE-OIR 6356 6. Walk around ihe room from time to time to make sure students me filling in their answels appropriately. All students sho uid be given at least to complete Section 1. If all students finish before T0 minutes have passed, go on to However, if any students are still productively engaged in completing Section 1, allow tlsem the full 70 minutes before continuing with 7. Say". Please stop working. Imert your answer sheet into your test booklet and close your tes; 1 booklet. Hsu may now lake break. You may stand und stretch 10 minutes. Yogi" test booklet should remain closed on your desk. Please do not tnlk. ll. At this time, ullow students to stand and stretch for 10 minutes. Make sure their answer sheets are inserted into their test booklets and their test booklets are closed. After no more than JO minutes, continue with 9. Say Please sit down again. on to the next section Keep your test booklet ciosed, and Wait quietly as we get ready to move 4 gm Gzuzlizx 3-iz Lap)/ight -F9 20179 by the Dcparrmertl ty Mlm-ntluw 475 02612 sLo|R 3 A 0357 Science-Section' 2 Estimated time: T0 minutes i. Say Now you will take Section 2 of the Science Criterion-Referenced . Competency Test. You will have 70 minutes to answer Lhe questions in Section 2. Stop when you reach the stop sign after question In your I Student Test Booklet. This will match the line with the Word "stop" beside .y it on your SI.ll@ll? Answer lf you ilnish this section before time is 2_ called, you may go back and check your work in this section only. You may not review any other section or look hack to Section l. When you are linislied, insert your answer sheet into your test booklet, close your - test: booklet, ond sit quietly or read book. Now, take out your answer sheet and find Section 2 for the Science fest. Open your test booklet to Science Section 2, quesslon 36. [Fame white students are opening their test booklets; make sure all sttadenis are on rim - correct page] YOU may begin. [Malta .mfr l2ogin]?llirtg ir! their in the correct place onthe answer sheet] 2. Walk around tiie room from time to time to make sure students are iilling in their answers appropriately. All students should lie given at least>>?5 minutes to complete Section 2. If el! students iinish before 70 minutes have passed, go onto However, if any students are stiil productively engaged in completing Section 2, allow them the full 70 minutes before continuing with 3. Say Please stop working I Insert your shcci into your test; booklet and close your test booklet. fl. Collect the test booklets and answer sheets. As you collect them, check each test booklet to make sure the studenfs name is printed on the cover. Make sure the test boolqU WUWWOEW WQQUMQL WIWWPQU EIWQPQU ?_mgmw_N _wgnuqg 6 NMONDNRE 6 Ran?mm?m 6 5 3m;NhmN 6 AW 6 wmG_Rg" 6 FOQQEQP 6 EUR Dim? 5 wm_mD?Q; FNONDNRE 6 6 DQEQMCQ 6 DQWWEHMOQ 6 ONEZHCQOQ EUR mgaggmw WMWTEWC (R) VQWT, MH (R) (R) (R) QEGGQ Qrm?mw (R) 'ww Nqmu OQHDN M) 0 MMO WWNN Rg_wmHmU 033 $023 gg? $3 Ogg mw Eggw I im i iam iq iv _ug WMUOEWME _ug 3% ii 3 i 0 ii" gage <2 <2 Oxaqg _wmmga 'mo NQWUW mg HEWNWN EU @m_)< mU?m>< wm5_w>( 52 w0< 5 wm5_w>( m0sh_m>< mU?m>< 5 I I NETCMUE I MEIUEM I Nm_\Gw$ I YEIUHQ I Nm_\Gw$ I I MEIUEQ I ~mIU&m I METGEE I ~mIHu&m I NETGEE I ~mIHu&m I I NEIUHE I NETGEI I ZMIUHQ I EIQEE I NEIUHM >m2d`D4 >m_2m_2m_2(m_4 >m_2(m_4 >m_2(m_4 >m_2(D4 >U_2(m_4 >m_2(D4 BENZ _Dgum DOI Qui ODI m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ OWU )_wU JAJ UMU )_wU '/wU Um}_ QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ Wo< Wo 0750> 0239"Em "Em "Em 0230> 0230> Z9 mem 4 6062 4 UOJQL <4 2 2 EE EK E32 532 ZOWSE A 2; EEZ DZ/ju) DZSW) DZ/ju) DZ/jm) DZ/ju) DZ/jm) DZ/ju) DZ/ju) Dzim) HZ/ju; Dzjm) DZ/ju) Dziw) DZ/ju) Dziw) DZ/juTTS C4'vv<'v OP a Eg! Oda( D154 DOQSNE D154 DOQSNE D154 I I 2012416 ZOPZSM I I ZOPZSM (PZ (PZ (PZ (PZ (PZ SP2 (PZ (PZ (PZ (PZ (PZ (PZ (PZ (PZ (PZ (PZ VAOUU XOOU OUU XOOU XOUU XOOU XCIC XOUU XOUO XOUU VADUU VAOUU VADUU VAOUU VAOUU XOUU VAOUU XOOU I \li ll 4 1 NENIIWHE 35" OOIUW OOIUW 002% m_3n_ QE Cciiwb F4 OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIUW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia (C) NQNEWE Eagug m?H2& NSRW 6 mu NEWER (R) ?5373 $NNm2w MW OWCWW WQOMUWNQ SAEEH my $2Ng QEQEE mu R265 KNQBEU 3 (R) MW gmws? Egbm 03% man* mu 2 (C) UQNN 9 EEQEWQ my Nh my "Ewa WQEWW ESRNS 3 mm Ed NINEL Wmgg' mv CCKU 328 33% ?gw 6 mamg? QWSE ?mw QTMW Mm S66 $3065 _pmma hmswuw Hu _Nm 2m5?5 aw? ERAMT 6 mN?`m $36 mmuqg? $36 Maw 6 Tig? Awwmm? 3 WWNEJ ?mw_g 2] gmg; DWGINNC DWG MQNEWH QQE 6 TUQWIMN 6 mga? WQQNEQUU mlmmix ?_mgmw_N QNWDEE mWmO(C)m biwmig b?wi? 6@g?DmW #@g%nW QEUWWV Nag; 25% hr 25% hr NENON CGW WP 32% #Gmm@5 6 $u@mCmW_" 4; _aw EEMIHZ ESNVC 6 %&m2mw NQHIRQN (C)m;NhmN 5 MW~d_gmv_ 6 DQWWEHVWOQ 6 6 EUR 5 ZQQUWQWW 6 6 Smmomqqu @Dg?3"gW 6 SQMEUQAU QWNQENWD 6 RGHNUWPN 6 $mm2mw Mm EUR $mm2mw Mm EUR wwgamv 0 4" 6 6 @Wwm@mw NQHIRQN SQEQMOQ 6 DQEQMCQ NEW mghag MH (R) NA NWWG CO GQNOU SNCQUQ 35 QHQEN QF QE NH Q?QKQN QNM6 gms; (C) %m%Dn (R) _am ww (R) FCKE Em 3 QWQ2 3 FQWW hm _Swan 5 Wg? (R) mgmmu WEN mu NA g?mwg GWNO GENCY MH MU WWE SQ <33 6 $`hSahW QNWWUGDB Ngmiig QZSEDWN 0 Emmwm?% _1?l?;SlN hm SAJQSQU MW NMONIDHSN hm# hm ig ia 3; m_2d_ A E35 EEW MEEMM 2385> vs_; 155 55 ISS Vg; D250 mg E3 ES 55 EH 55 di/_ M5 mg In 230_img jm 5; WDDOD VFECC mm_m_OD EDGE 7m_m_OD EDGE mm_m_OD WEEGD mm_m_Gn_ mm_m_OD mm_m_On_ EEOC mm_m_On_ WIIOD WMMOD mm_m_OD WMMOD mm_m_OD 34,2 300>>v_u_ D494 Dba( D154 DOQSNE D154 DOQSNE D134 Doggzm D504 D424 DDOEME D504 D504 D504 D154 D504 DOOEEM D504 D002/zu DOQSME D504 DOOEFE DOQSME Dba( D154 DOQSNE Glu( EUGENE D154 Dig( D504 0754 D504 D504 D154 OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia EONES EW Raw _qk;_w_Q mil; Mm g_gNW RGQN gba? mW EUNNQN Mm Am TWNNEU S526 _wgsg MW (C) _Usa Snag* E36 my gm" Nm _kuumg 3 E3 6 0 QR E525 SNNQEU QSG MH NBOQWQW S2 bg W9 ?0 EW mu Qmuwgu m0$E= mm mu H2553 gba Wam 2 m&N? tugusw DN TW @3366 fm ww?mn SJAEQ migmq 6 QUHEW m?mga MU 35% ?mwn; $'nmu_wn'_ 3 0 @mQ_mm mW_N@@q q#m;T QHEHEV ami Evwa 6 $2196 QM >gm_ VA Zggm! I $05 1 Em _/Om 3 OEED MIUZ IE in LEWSK VZEZDD EEZDD 1412 DD 1412 DD EEZDD 1412 DD EEZDD EEZDD VZEZDD 1412 DD EEZDD EEZDD EEZ mm_m_OD EDGE 7m_m_OD mm_m_OD WEEUD mm_m_OD mm_m_GD mm_m_OD EDGE mm_m_OD WIIGD OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia Ei`Hii Nam BEM UE _Eg UNL E255 BENZ 35" Qwgugg gN?_ ww my gg=N DN NE _la 5556> ON NRBEW HN 2; E06 _m EBNQW Sq EH MU UESQW mb ESNWN mzxig 802% sm_2u__U 3393 _wM?_m ?gn MW Eg" Egg Rm M3890 Sung Egg 253; Egg NIE QGIVC 6 2`NnmD gmnamn mW?Nmmf MDQES mmigq Or' @gdm5 Q2 WUNNWO 33% (R) gwmam AMNWWEQ QWQEW mag 6 mam; qwmaw 6 MH "mqmm_2 PQNQ6 RUNCP @0095 mm? QA WNWWEQWH XUFWCUQ Wuwdgw QWQOMR QWQEMX QWWSOQQ D?qgq mq\DW" QEOQTQ WWC QQE #gg?gw TUQQWW TUQQWW ;uUwr"5 EIMHEU ?_mgmw_N wummugq bgmwig MOQOMMW QUQMMW QUQMMW Ngma (R) 6@g?DmW TUQQWW BEMIU 6 6 6 gmuwv?mm? 5 Dmmuomqmw m??gEv_m? 6 Qhuglgg? gDgui;g 6 WOQGSEN 6 EUR wm DD A 2/ 53 4 zo/_ 1 4 KDZES (ISU 4 E235 (Img 5_3 5 2 Eg 4 IK is Img) ZOE ZOE E4 Og gk _*gm _*OPZSM OPZSM OPZSM OPZSM OPZSM A 2 OPZSM OPZEM OPZEM OPZEVM M24 ZOPZSM ZOPZSM ZOPZSM ZOPZSM V73 Pmgd) VZ.) PQ gd) VZ.) Pm< gd) PQ VZ.) Pm< gd) PQ VZ.) Pm< gg PQ OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia nazi: 556; Wm; m_ Qug Kung; 3 RQMS EHQQEW gems *Ugg *hw NEECEU gg: UWN m_ _uxxag gumma mu mu mm (R) S268 EQQN _#hmm 3 Q2 EB HWW ERB Egg" 5 gm mm WQOQHH gh QW 8 QN NN Saga mu ??hN NN ON 2; E36 WE gm; _'Ed RAQNQ Qummd REWE @my&q CQ Mm MQNEWH N@mmD N@mmD 6@g?DmW QUCOMQQ N?ma #@g%nJ TUQQWW 25% hr QED 603% m5 Wumg? Wumg? qW6"mq 6 AMMS6 26g Mg; Rag; Mmm; MWOWQQW wxxsa 5 25 2 5 25 WVU _"Om ZQ2 4m 2 (R) dwta Mm dm 5 (D ESZOI EVVG EVVG M24 OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia laii NINEL BENZ _Dgum Saad _"gig m_ 25:3 EUR SQEN g?m_E sm_wsgm ww Egg swag $033 MW Wg MW Emggr mu WNNEGW gag Eg WQQEEM _gba _w 5 NEWT $625 SUNEQW Mm 3_a@g MW gNh;g :Egg 5 SME NIE Hg 6 (R) _mmdq NWQERW MMQAQD TMDEI Mm DWXOT amp? qm6q__$ gg? mwn?? A mm_"mn_m@ Ewmgm 6 n(R)mWr\ 6 ?mw Nw QAEKQ QOQQW 6 WWQOH 5 WWC @3536 NNmn(R)q GEQ 253 gmagw XUFWCUQ q?Qm@N (W q?gm@ MOQOMMW wNmm;v_u bu Nuqwgs (R)`3qm@ FWMWO 6 SEDW QQE 6@NggW 6 6 6 ;uUwr"5 SND5 TUQWIQF ?23 EUR Qgagm XUFWCUQ Wuwdgw qwmguiw Egmlum wmaggq? wmaggq Ran?mm?m 6 QQNEWDMOQ 5 DmEc?v&W 6 gmuglnm 6 gumwgmimw (C)m;NhmN 6 MEQRQN MEQRQN 6 wm9Rg" EUR WQUQRQ mm 6 qu 6 6 QGDUWEW 6 gmuwv?mm? 5 6 $Dguiii" SQKOQN @2wm%% MH _agw gguqg DV "Msn 3315 mug" QW muggwm 3 (R) mm; hm WCQEFMU (R) Wag; NV EQNCQ _Ng SAW Wm OQKE QQWOAW WMQN (R) ww? XWS Up mw mu QNWQE 3 M) wo gn Wm gaymg ?2 QQSTO EQSME (R) m??mE MN MW $Nmq WCFENEQ gmw?kqn Ewing; WGN WN nw 5 NWWEQWDNE (C) Ag $y#Nmm QEWQD Wmll! liN'i !l lii'i !l Ni EiIl li EiHli Sega! $25! I (I I bgu WDOOZ, WCEIU EIU mm 5_5 1 A M5 mi 70964 REG ZOE hog hog bg! E255 2/_md MSI I MZCUE iv WIEI I I HWEU I N166 I HWQU I I N140 IUZ IUZ IUZ IUZ IUZ IUZ IU2 IUZ IUZ IUZ IJZ IU2 IUZ IUZ IUZ IJZ IUZ IUZ IUZ IUZ IUZ IJZ IL2 IUZ IUZ IUZ IUZ IUZ IUZ IUZ EEZ 35" OOIUW Cciiwb F4 OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia 1 4 5 mba; Egg; NN meg; DN 5 22% an AN gg; ?535 $6 QQNEU EUOONW _"Nga "Um 3 _gb NQNQU mu MRM (R) ZNQQ 3 BN $393 NQNN NN QUQHE gkgtu? EU EU Eaga QNNC gag WGNQ au NSNEWW :gag RQENS _Eg ;h2w? mm WEEWQ 2:63 mm _U_Ug_mw_U gg SNEZXW wg: ERNQW QM EW wg HE r_ Emgg _Nw QQW (R)?Gmd@ gm 8 PENS 6 hkx?n awk MMNEQD wawgmw 2Qmm3 Eww 6 WWQOH 5 WWGQ: BN MQ 3; QW gms; mmga QWHEG gmmimw mmga QWWHED SR WW Egg; MWMOQ: QQSQE D?miv? QNEAMWM bu Nuqwgs (R)`3qm@ FWMWO 6 SEDW MOQOMMW N?ma TUQQWW WWF Wm SND5 TUQWIQF ?23 EUR 603% m5 WW 6 wumm;W mamma? Wymmis $u@mCmW_" UMWS GCUEN 6 6 6 5 gq$c1Om0 >Om0 >Ol0 >Om@ >Dm@ >Dm@ WZO li liI'i `iEiI'i iEiili BENZ 35" OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia Wsaggd WN 3 mu; Eamwg WQKNE gN2wg Ezggv QQN5 g3&g fg?? gg 9 EUN QW mm WQNENWW Nw _gnu my QB ww mu SHENW 3 gmh(R)g 362; Wang" GENE Em_2m gg WEBWW3 D25 mm NWOQN Am EK WW ?h2__v_U QES my RNQWEQ _gaze (R) 3 $663 WNG WWQ Hg Hg (R)m6iN_m WSSN 392% qU_g6(C) NWMEMMV %mm?wn "Maur $26 m?W@ SQONW 6 WQNE WQDQ MREWQ @aw2 _"wg $32 qw#WmD Egg HANWQD $28 33 mg; $23 32 Egg ang mWmO(R)m(R)d QQSQEN bu?ig 9 6@g?DmW 6 SND5 TUQWIWF 5 mwgawg 9 WSEQB wymmis $u@mCmW_" momma? D?qgq 5363; QNDNAMWO UWNUWW MOWOHHV QQE QGQUETQ N@mmD TUQWIM5 6 4; W_ummO2\ QOEEQEA SESXEW 6 SEWc?v_mc 6 Rcmaomg 6 6 (R) EUR $359 BQ mm 6 3 6 6 QQNEWDMOQ 6 DQEQMCQ 6 WOZGSEN 6 EUR WOZMEQHW wm_mD?Q; 6 SNMENND 6 6 5 AW MW~d_gmv_ DQEQMCQ 6 'Iii WNDN _umxo nw (C) N@g MH (C) MH ;&mw 9% 9 m_ COQFC AQWWQ (C) QN 9% 9 GCQEWG CEQNNO m?%2mW Egwgnumn?? MW _bn WSNUWW I MNW WNWQ 6 WN QWGEEB H6 WG W3 6 WX gwawmbw AN Nm MWWDUHWNQO ii ii ii mm _w 2; Qu; 5:25 E302 >vm~ZQMZ IUPUI 292 IJPDI 292 IUPUI 292 IJPDI ZOWZ IUPUI 292 IUPDI ZOWZ IUPDI 292 ZOMZ IUPDI 292 ZOMZ IUPDI ZOMZ IJPDI 232 I PDI ZQMZ I PDI 232 I PDI ZOWZ PDI ZOWZ IUPUI 292 IJPDI ZQMZ IUPUI 292 IJPDI ZOMZ IUPUI ZQZ IJPDI 282 IUPUI 292 IJPDI ZOMZ IUPDI ZOMZ IUPDI 282 ZILZDI I ZIREDI Q23 MIQZDI KILZDI I KILZDI ZILZDI ZIRZDI ZIRZDI ZIRZDI ZIQZDI OOIUW Cciiwb F4 OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIUW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia I liiEiN'i i wiNEEZ BENZ 35" g02_? (C) WQQE *w EEN mm 2B__g mu gm* gn hmm: EN 6 E365 _g?g MW EN _Wg EBNNQ my gag? SGQNEU QXEN mm EE $362 3 Nags $32 (C) WGN Nm gaguz WQQUQK EEHN VVQFIP Anim illmw 3 SNCAWN SQDNE MQWEE @(C)mgE Mm @mDQd ?130 N359 $523 QENQ A gm WQDQ *Emma WUDUNQ mm 4;mm;(R) MQUJNW QREQ wish; 5 WQBQ QSENQ QRQ Mwg?? Mm @aQ 2Qmm3 wawgmw G55 5 25% H5 32% mwgawg N?ma $u@mCmW_" OWQEW #@gmDW QEOWTN EUQQWF Wm mwgawg QQSQE 6@g?DmW #@g%nW WQQGEB EUMHOQN MOWOHHV QQE 6 TUQWIM5 TUOQWW WWC PFW g;mO?; 6 3 6 5 3m;NhmN 6 gmumgnm 6 6 EUR %W(R)Wqrg 5 mm 3 6 6 SEWUQEU SQMEUQAU QWNQENWD 6 OQQEQP 6 FOQQEQP 6 EUR gm CQFCEE 6 SNMENND 6 NQHIRQN DQEQMCQ 3m;NhmN 6 AW WWQES Ami (R) OQQOB WN (R) MH WOGNNQO (R) W0 m&Q" (R) (R) MENQW (R) WDOQUDN 303% I (R) MHQU MW QHEMB mNm@ mWh;t>@Wg _:Eng 3 mm SQU mu gmsma mm ZNNQEW mm gN25 gg; gm ?5025 sag; Vents WW@mhmm 5 WQSE my BEN WW Am EQES mm ZNQRE (R) mm Th; EU 35? umm; Dam Wan; Emm QQMI ?32 2Nm?\ 6 6 wawgmw 2Qmm3 Eww Egg; P33 MH mx 2 QWNN N?giq Mm @aQ WWQMEWMW @_uqv&5W QWMOQM QWQEMX OqNm #@gmDW QEOWTN FWMWO 6 SEDW 6@NggW 6 TUQWIM5 6 TUQQWW WWF Wm WQQUMAC wlmmix ?_mgmw_N EIMHEU ?_mgmw_N Qgagm XUFWCUQ Wuwdgw MOQOMMW 6@g?DmW TUQWIM5 6 TUQQWW Zgawgwg 6 6 gmuwv?mm? 5 Dmmuomqmw _Amiv?sm? 6 RQMEGWRW 6 6 EUR 5 6 6 wgmimw QQENNMWEN 6 AW MEQRQN 6 wm9=T" 0 EUR ONEZHCQOQ EUR 6 qu 6 6 Ran?mm?m 6 5 SQENQWEN (R) QEMH (R) (C) ga mv (R) QUCNR3 OTA DN (C) (R) 0 mu Oimg my QAEN nw MH Vmwmamu ((C) (C)mNm;Qu QW (C) mwg?ym (R) maqb Umm(C) MH mv N333 MU QUQEQN mga QW mv QNNQ (R) 2* WE MN NME WNWQ (R) DNETQOWMM Nga SEV 6 BW WUGEQQW AN Nw_D_WmN%mw QQDAEHD _bn mv wig; MW GCGOUGQE QQNUHKOQ Qm?mmgn hm Eiggw nggmun `q2_q?m2 UDUDUDQE NquDWmN_'m Quadmbamw m>/5 SQ gi 25 ED ig! im! QDIW Egm 1 QOIWM Z3 SI _Low OI S525 gg! fig; 295 XUDQ VUE Og; SJ ZOMZIO (Z SJ 3_?ZOQ 20% 203 ZCVI I ZOQ 203 I ZOQ ZOQ ZOQ ZOQ QEIE ?wil; ED (R) ER wg OOIUWU CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia li EiIlii NIE BEM UE _Eg UNL 1:5 EEZ BENZ 35" EU ENEN gb an GN __UE\mw WW Am PEW nw ghmu gm_2H_H QN gm" _Wag Wmunah gm_E QONWN EQ: 9 $52 EWHEBN gg GQE $626 ma RGEHE WCB QW NN gig; mwgy; WQMCU mug mu WEQWGS Wx; Zagm nw GQMN _wgm _Eg _ana Ewmutx RAQW RUNCP @0095 mm? ?mmD amp; NWQEQW umm; Mg WWONQ ?130 N359 DIDN3 WQNWAE QNOTU ?mw ggi Qilmm qwmaw 6 WQBQ QSEEQ 5 WWQQOQW N26 VV @3536 Wan; 6 $23 g#mmq buqmwig q?gm@ D?miv? QNEAMWM #@gmDW QEOWTN QEOQTQ WWC QQE Qn?wm 6 TUQQWW WmN?wm TUQQWW ;uUwr"5 EIMHEU ?_mgmw_N Qgagm XUFWCUQ Wuwdgw wumm;W $u@mCmW_" OWQEW UC CVSEW @n_mD?mO_" 6 6 wmaggq SmmUm_a_U 5 gq$c1>gm_ PLUUW PICQV PIUUW PIUUW PIUUW A PIUUW PIUUW PIUUW PIUUW PIFQV Pmmlou Zo; Pmmiol Zo; PMMKOM Zo; Pmial PWEOM Zo; PWEOM Zo; PWEOM Zo; PWWKOM 20; OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia i wi`wi !l EiNli Ni I NINEL NC Eu ESM EW 25; UNL 1:5 UEMZ BENZ 35" ggmwa gba? 0 NME mm EHEQE gmma 6 EWEWWS 2 _m 39243 Wh" Eg Ngamm TWSGEWB EGEUEU VNGEGW 582 Nw S5 MESH DN Rm; Eaw (R) mm ENWNE 3 Nga EU mu GQWE 8 NW Qngu WWOQWW "wg 0 W?hNh@ 5 _wgss mu g0(R)(R)g Shaun gb VW 6 Nav 302; mu B525 NINEL Tig? 6 ?_3x :lam Nga q??Wm WOWQQW $22 (C) QQ TUNNQO 3 muni?w $25 awk 4153 MWQNAQW 6 QWWEE QWEE mag; Qvmag 6 6 Qighg mm6nNm QU NQEDV I DWGINNC DWG IE CGW WP 6 SNDQO 6Omm|~& 32% mga? BEGSNW DQDNAMWJ QNDNAMWO MOQOMMW QUQMMW N@mm(C) TUQQWW TUQWIWF ?23 EUR 25% hr 53 mwgawg Wuwdgw w`mmP_w_U QWMOQM QQWQOQQ m_\mmO EEMIHZ 4" mm MWQENNQN 6 6 DQEQMCQ 6 DQWWEHVWOQ 6 6 $"_NhDd3? 5 Samcwga wm_mD?Q; NWONDNRE 6 6 6 6 RUHKUMNMW NMWONDNRE 6 6 DKENDQN 6 WQUQRQ mm $mm2mw Mm EUR wwgamv 0 mXDmEc?v&W 6 gmuglnm 6 Dgawqmw m(R) QUWQEN ww Q5 m(C) mw& Qing M) GOQE 4% WVAQQZ QWQQE 602560 MH WW (R) CQ m(R) gm* QW FUCENVW 3 (R) (R) SEWNQU WFT TW (R) QAWEE mu mm Ik nw VXWEW mm ww i QSQU TN Ewgmw (C) $3830 ww (C) mDuD_uDmOm QFD DREW QD MW 2 EFQGQE 3 3 WMWWUQWQW QQUNWQ 2 mW _bn ww Bmw?;BWw RFNINBIN $925030 (R) $533 (R) NUW5 QONEE 3 > FK 4 MEDU QKOU m<_2O I OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia $0 SQ Shag ta m(R)m(R)g gm& mu EN NN 9 2_am; WNWG 3 TNQ CN QW gm: _Rea YN gxar mN&m(R) WBON Rmgug 9 R253 MW Wh00mWm WNQSE 592 5 my "aww 5365 6 Egg; aw NND 6 SQAWM fgmm@N QQEW QNODWW PQEW m?_Nq QAQQW 3 mm_"mn_m@ "Dar 6 NUQDM EUQQ6 Mxiqm? AMQHENQ H3060 _Ango NNmn(R)q MUHENQ MEDWWV _w CGQW immn; @_uqv&5W Dgm__&V_r g?gmq (W bu?ig QNEAMWM QUQMMW N@mm(C) (R) 6@g?DmW #@gmDW QEOWTN FWMWO 6 SEDW SNDQO 6 mwgawg 5 WSEQB wymmis $u@mCmW_" momma? ;uUwN5 WQQGEB 0235; QEWQB Egg; MWMOQ: qwmn?m bu wmaggq SESXEW 6 SEWc?v_mc gq$c1 Z3 wKmOOIUW CCI wb F4 OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA OOIUM D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia 0 liwEi iwi"i !l 'iEiNwi !l Ni Nam EEMU UE _Eg UNL 5; EEZ BENZ 35" 1 5 5 EKNEW gnsgu 2* QBQUW Am EU QQW _m gmga NN Eanm AN WEQQW MW ?U2_e? _wgg gumw EU Gm Stu E525 QENT sta _lamwg mm QNB (R) KNEW MW Rg?g gnEannQNQH ON Emu NN 2625 Emhgm ?mBm_ MUN QWN gn NINEL MW VNWQUW mg?m hkx?n QQEW @m2_mm :Qma m$m1 WMNOWG sg; QSND _amqgu 6 3%m@ 4%mm_N@ qW@n;m MWOWQQW MQWDOQC q?gm@ mqDN@m@ NU TT QGWOWWV FWMWON 6 MOQOMMW TUQWIM5 6 TUQQWW WWF Wm TUQWIWF ?23 EUR 25% hr XUFWCUQ mwgawg Wuwdgw mamma? qwmn?m QEDOQQ QWQEW GCUEN 6 6 wmaggq SmmUm_a_U 5 gq$c1 Zh; _>mA_mN_ OOIUW CCI OOIUW 002% OOIUW 002% 810% 810% OOIUW CD26 OOIUW OOIJW OOIJW OOIJW OOIJW OOIJW OOIJD OOIJW OOIUW 002% OOIUW 002% QJIJA D025 OOIUW OOIJW OOIUW OOIJW OOIUW CCI OOIJW OOIJW OO JW OOIJW OOIJW OOIUW CCI OOIUW 002% OOIUW 810% OOIUW OOIUW CD26 OOIUW OOIJW F4 m_3n_ QE QE <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia mia /?24 m_Dn_ Ill( F4 m_3n_ QE QE 4:44 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ DDQ /?24 m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia WQDQEHW gm 256 3 5 QD q_UNm? mu S026 g"mQU_m nw NN E@?m SQNN R36 Egmg gg gum UWQGWEU Ex; SNNQ Sh my mu g?N? 3 R@?m W_"M_hm__2w Em??w MH WEOWW $632 EN WSW gawk* mm 9 Qabm gba ww mm 3 $023 MW NQEN Egg; mu Ehnwm Am PFVIGQ 6 NW6@mq 0 $286 NMQAD amp; WSG QI ggi# PWFTE wma; A EMDNW $025 umm; Rawmmw mugs _Bad 353 5 _Hx mHmm>` _ag?r MW 6 N7 @_uqv&5W mWmO(R)n QWQOMR bu?ig D?miv? QUCOMQQ 6@g?DmW 6@g?DmW wlmmig QQSQE QWQOMR 6@g?DmW #@gEDw QEOWWV QQWOAMQN FWMWO 6 SEDW CGW WP w`mmP_w_U mamma? m_\mnO Zgawgwg mg 6 6 SESXEW SmmUm_a_U 5 Rumammkw 6 Rcmaomg SEWUQEU Ramammkw 6 6 EQNUWQNE 6 EUR EUR WWOWWNBOW 4" 6 %&m2mw g;mO?; 6 6 DmEc?v&W 6 gmuglnm 6 Dgawqmw mggmuq (R) QRW (C) MWVAN mx iw (C) "bw _Nrqma 91DQWQU my MHOQWV QON QM MH 5 Vw DNN 253% _5 (R) GQEN ((C) Wg' NDN N596 (C) A355 VND WNW %Rmw (R) ?"mWiQg 2 F5425 Nmmg;N@m 7 MB mw QZN NN6 WLNEOU 0230> ESSOI _?mxg my Emvz; Fm OU WWE m_/_mg WP gi ZQRE _/_Ogg ZOQO m_ ZOMXO gum( 9_5 Dm A m__2_E 3 E25 ggi E5 E6 EIB mUjm5_ _/_Tag 723 Z_&g EE 8_6 Big; __62m_ IU IU tis! E73 QESWM EVE _Lg JSMEU IP JSLEU SU W5 N5 WD E5 20800; ZOWDOQS 20800; ZOWDOQS 20800; ZDEOOE ZDWDODE ZDWDODE ZDWDODE ZCUDOOE 20800; ZOWDOQS ZOWDOQS 20800; 20800; ZOWDOQS 20800; 20800; ZOWDODE ZDWDODE OOIUWU OOIUWU QJIJAJ OOIUMU OOIJWU CCI VL OOIJWU OOIJWU OOIJWU OOIUWU CCI Viv OOIUWU OOIJWU OOIUWU OOIHBU 8108 OOIUWU OOIUWU OOIUWU ODIJWU m_3n_ QE QE 4:44 <5/3 (514 mia <5/3 mia <5/3 mia I /?24 m_Dn_ m_Dn_ m_3n_ F4 m_3n_ QE DDL 4:/3 <5/3 1:24 mia <5/3 mil mia <5/3 ml; mia <5/3 mia i EiIl I NIE GQUINN BEM UE _Eg UNL 1:5 EEZ m0mImD ;Im QUEW >mv_5__m (IEDM xg; FW_/_gm FW_/_gm is VEMHS L75 3 4 5; Em gm a ZOQOU 5 Mig N_mT;mm 9_0 Emm EO 5 9_0 3/)5m@ CA YEPKOIM NEPIOIW WEP _2 <4 MMPMO KEMO E53 "gag mg_y_m>< 5 g< WQEWZ m0<5ZQOU ZQOU DD 2 DD 2 MFE I 1410 WED WED CC I WED EDD i2mIUZ3m EDD i2mIU_/_Dm WED EDD mIU_/im DD 2 DD _2 Z2/Oxm DO 2 Z2/Oxm DD DD 2 DD Z>>Omm DD DD 2 Z2/Oxm DD _2 Z2/Oxm DO 2 Z2/Omm (E45 425; u0< mO?m>< OOIJA EZ/jg DOIUM n_ (EEZ CD13 gl SZ/jg ODIUW OOIUW mg g_/_iz OOIUW OO Um HDL ODIUW OOIUW mg (EEZ OOIUW gl EZ/jg OOIUM n_ (EEZ CD13 gl SZ/jg CD13 A S24 OOIUW mg g_/_iz OOIUW OOIUW (EEZ gl 525g OOIUW OOIUW mg (EEZ OOIUW OOIUM mg (PZ/jg OOIUW gl 52/jg DDIUM mg 525; OOIUW mg g_/_iz OOIUW gl (Egg OOIUW OOIUW mg (EEZ OOIUW OOIUM mg (PZ/jg 331% gl Z5 CD13 mg 525; mo< $55933 GN (R) (R) MU 6 303% xxahwmg mm N330 A500 2 _numb hh Eggs m_Km5_'? 06255 mv WQNEEWNM R38 mg? UQ 0 mm EEUDQ DQ gaamsw NN ma QAM mv X5 NQNG au NDSU tw N225 226 m&NmOm *mom _m gag @Bmh6 MWQOO 382 gm gg; Eg; Egg mga MWQNOQ Egg OMJONEN Nga ggi Egg Egg @0025 333 3 MBSN $8 (R) MKRW QOWNNB *Ewing EORMN as _Rm WUUQW 3* mm mg 6 0(R)0gm *m@@3m nmNm@ gg* Em gm: nm(R)3@ nmNG@ gg *Wm nmNE@ mm? mg mg mg 3 qgg#Ng QQNRA NB mmgg; _Hu giags 6 $3 RWD 6 JW QAEWQ 6 S6323 _Hu 6 6 m_:mmWT\ gsm? 3m;g< HQGW 3333; magma Q&m?Wd"Mau EWWUEL iggmmu (R) _'iqmg mn NC imgmi iggw 3 WOQEW igm HWBEQ NO WQENQ i QEWE iwmgm imw EO Aww ii; i330 VWNEQ Wagga my i Egg; igog NH iggmg wmW%mJmG gg ww gamgg mn mu @m m(R)m pg? pump ZQQEQSQ WN mm Da WQQ mm ?02 w?Du_/_Zu 2/122 UIED ZSZZ BOD ($21 $21 Q21 A51 agi $21 $41 $21 NEIUMZ NEIL 4 WEIUMQ I 1 N_m__Gm< WEIUMQ M2614 EIU: 1 NEIUMQ I N_m__Gm< WEIUMQ $21 $21 1111 A51 agi KmIHvm< _Gm 2_5 EVOUW (tml OU FDUM (Fwd OU DOIUM CD13 ODIUW OOIUW OOIUW OO U00 ODIUW OOIUW OOIUW OOIUM CD13 CD13 OOIUW OOIUW OOIUW ODIUW OOIUW OOIUM OOIUW DDIUM OOIUW OOIUW OOIUW OOIUW OOIUW OOIUM 331% CD13 OOIUW OOIUW Uj HE ODIUW UI NE OOIUW UI OQ _Um Oj 52514 n_ (EEZ gl SZ/jg mg S2514 HDL (EEZ mg (EEZ gl EZ/jg n_ (EEZ gl SZ/jg A S24 mg S2514 gl (EEZ mg (EEZ mg (PZ/jg gl 52/jg mg 525; mg S2514 gl (Egg mg (EEZ mg (PZ/jg gl Z5 mg 525; mg SZ/jg gl (Egg mg 525; NNW i!NINNE ig *mg mu|_DO__2 HDL 3 0 Eazwkz gmggamh gO_=Mw_$ 2_ $6 nw ON MU 382; 33 gg (R) XENN SN m_m2_?mKm mu 6 Ewbakz SGW MUSE 2% NUWQHEQ m_ SQNSEG gganuma VND EERBSAM NNW NgN$$h mgmwam mv mv M: _mmamku QEUNO 3 $3 GQRS 2; wh 3 WQUNWE 3 3 MU DENQHQ RQ an Ah gW_mm_x_ SN QQWEWRN Av ma m_ WNSRE gQ?_gNq_\ NINNE 33; N058 WEWNDW gg; QED Gag MWQOO gag QQDW ggi 3_3 gm WMWODN ga QM m(R)mNm wma? ?s Gm Egg EOWN 6 gm Gm MVSN m; gag gnu WB WQREW 333 msgi 3 ma qg?@N mg Ugwzm nmNm@ Ugwzm ER Dm Em 5 3% nmNG@ HNQW ?23 nmNE@ nm(R)3@ SGD mm hmmBQEE @m_"gA0_H QONROWH NHWN5 6 JW E3 RWD 30m 200 mgagg aww EGO Wagga WWOQQWS Ram Q90 Nam _am mwpim QAEWQ 6 S6323 _Hu 9;gW? 6 JW mw_&8W? 6 WOOWQQWOW ggi? Egg; NHWN5 EWWQQUQD gD`0Ngm_O igg UQEN i?mgw ig gm HWQQ i "ming iggiw i $30 iigmg Am ig? 6 iggmg UNNQA $533 (R) WDW 6 guwm 6 3 gpg; in-Ogg iqwgw Jw (R) i 03 EQ isgg igmgw mg HSEE A I Hai iaigsgm i NN img QN m_ i ixgbq i iqgmm 3 gi WSQ3 ig $33 3% iiqmg 2 EIQQE img Om; img NH 2 i i 550303 @m 3 gm igmgm UE Wm; <3 m0 ONWLAWDM ng; mbam (R) g_m0p?m& NN m_ mph; $5 256?Wm$ m6%&g gumghn QW 2 QEQSQ EUWEEV Emdgm;__" mm EQQEQWHA @ng2mmd D3 4 I 2 <54 my* 20mm AZNxl! Zim!! ic! $75 mg EUCZ 4 ggi!! 4\ U52 tr wZ<> MBE QU Nm Wa Wm Hv>mJ UCOQZ %Qild MED U55 MED EDD 311BOD w?ld MED gig EDD 311; Wig gig M50 <5 Kg EDD BOD Qili MED U55 MED 9E>test) were Included, as field test items were all embedded in CRCT. The basic idea underlying the procedure is a statistical test ofthe null hypothesis (Hu) that the mean number of erasures for a class constitutes a random sample from the state distribution of erasures. The hypothesis is tested against the {right~slcled} alternative that the mean number is too high to be explained by random sampling. Classes for which HD has to be rejected are flagged for further scrutiny. A well-known central limit theorem in statistics tells us that the sampling distribution of mean numtierof erasures for class lmi) is normal with mean and standard deviation (1) SD(rn] (Z) JT 592 where n, and m, denote the size and mean number of erasures for class I, respectively. ln acidition, si and cr denote the mean and the standard deviation ofthe distribution of the number of erasures of the population of individual students in the state of Georgia, It is evident in the formula forthe state standard deviation that the class flagging criterion for each class is adjusted for the number of test tairers in a classroom. For example, ifthe state mean and SD of erasure count are and 2.11, respectively, the flagging criterion for a class size of 20 is - 2.i? .M adjusted to 3.15 3755 - 3.15). This adjustment ensures that the flagging criterion is equally stringent for classes with considerably different numbers of test takers, ln addition, minimizing the probability of false positive (Type ll errors in this statistical test ls crucial in this analysis. cr The ciasses were flagged iftheir m, was larger than ,u Statistically, the flagging criterion 11. set at or above 30 is conservative. The standard normal table shows that under random sampling the probability ofa sample mean being more than three standard deviations above the population mean is around 0.001 However, rejection of Hg only telis us that the observed mean number of erasures is unlikely to be the result of random sampling. 593 iernody l\/titidloticf ni 225 Jann -3 GA School Teari- ti Ki ?3 Davis Williams 5; rii Pirro May 8, 2009 MEMQRANEUM To' 5 Jackie Boyce, SRT 't L.T . From: i ff "sharon Davis Witliams, Executive Director Re: Memorandum of Understanding Dr. Betty Tinsley brought to my attention a situation that occurred ether schoo? on Friday, May 1, 2009. She reported to me that shortly after twelve noon. Ms. Yolanda Coleman. ai second grade teacher, reported that you asked her students it she had given them answers to items on the CRCT. Although Dr. Tinsley immediately caged you upon hearing this situation, you had apparently left the building. In our conference, you indicated that you were engagirg students in trite chit chat because the teacher was called to the office. Both Dr. Tinsley emo* were concerned about your rationale for initiating a conversation with students regarding the CRCT, sihce the testing had been completed. The allegation brought forth with your questioning of students has been investigated and determined unfounded. However, any conversation regarding possible irnpropriety should have been reported to the principal, You aid not report the conversation. lt is important that persons assigned to my team clearly understand their role in support of schools. Yoor job is to support the principal and teachers with technology concerns and needs. if there was any need to discuss anylihlng with students, it should have been about technology. Again, if you ever obsense or feet that any imprc-eriet;-' exists, you should report it immediately. it is important to rne that you receive this Mesnorandurn of Understanding in the spirit in which it is given. make certain that members of my team know without question whore I stand on compromising issues. I expect you to do yourjob with dignity and integrity as well as with cleerludgment. Your judgment was flawed ire this situatwn, and it is my responsibility io aiert you. i mid you acmuniable for nc-t han-'ing a similar situation occur in the future, The iiechnoi-sgy needs for Herndon Elementary have been assigned to Ririuen Barkiey for the remainder of this school year. Cc: Ms, Andrea '-Naters-Winston, Director of Learning Teermriculogiws Dr. Be?iy Tinsiey I 595 im-mm iz ri/lay cmaem rd mbers Dr Hall QFFIGE GF THE Although Dr. Hall oalb on all employees to idly eocrperatewilh ihe inv Director Tamara cuiman recently rnetwith twelve principals in herschools writlfhigh erasure marks, to speak dlsparagingly aboulthe QBI and the erasure investigation. _'During her so called Professional Leaming Community (PLC) meeting she spent an being operiy hostile and crilloalrahoul the Governor, IWKB Bowers and li1einvestigatian_ At one point she even distributediso each principal a sheet of paperihat had printed at the top ofthe page "Go to Hell'. She direrried each principal to write a to I-hell' memo in the She then asked each principal to share aloud their memo. is this the spirit Di: ie expeding from her leaders, orjust anuthenexemple of a cl'rer:l:ors'P iokic eqllure that fillers down to lhe schools by executive I Should principals he pulled out of their schnois to participate in those intimidating praelioes? Will this he addressed internally or will the Professional Standards Department need to he involved in enforcing professional, legal and ethical practices? Conoemed family member employee 596 0 5699 APS-EHALL ;m 1 gg( ig ,Eg sw- numnnz maniac Henman uns: nlsanunus Bunn nnaum uhrlaas 1-Q wht: nm-in naman milzu-I .4 -mug mpg- muh masnhiw mms-ru-| nl-imBHALL APS- 598 ever1y_Hal1, Superintendent Dr. is <1 QQ, QQ 3.1 AFS-BHALL 015701 fCEl.Lf? may April LB. Vis: mad 4534-$92- ,201 "mf 12524 aftlur Ailzm?a Eimr? ami Mr. }Gmm?m Sixemzr Mrs" Ms. M. Mrisier Mrs. i,aChafadr;1 D. Yniamia K, .lcehrefgon Mr, EURfou1"muy Fnggikim Mx; EH Mr. Qmmeu D, E313 Trinity Ave" SW Aihxma, LSA 39303 Rc: Gazer (Bn Frifiay", Apail 22. I ww anivissd hy my Mis. '*>>felc1cr" and that Dirisclm cx_E` Humem Yew. Kimi my woulai bs: to an As?iaaum1 Cuunsci iw me FY 22022 Budger arid my wiih tha Districi zu; Deputy xmz; wrirzirmatmi immeciiaicziyl Ms, Xfiazycia ami Nw asked thai I rcugm my in an ilziziwdiaia pay Qui .E ?cc?imcd t?n5i?' request 2 haw milx?ng to iviggef a lawfui aimed Uma 3 \/wx zcrmirmtefl aw an abwili in ta tim Distriafs plara, I time razmsomf; Ms, ;1riic;uE;1tu;i ibm" fha ami ihfs iezrmiml??un E1>>EURliL?H5i3i :is 1 hzwq? pairfumted md Qu: 05 *dial jab ui ?i.lZ"liTi."F` ?n my iham Ibm yfzars iw Lhf; F2225 never issuafd any uazgatiw diz?xipllzuirjg anions mr fhrmul washing my job Pririlwrmamcei. This iimhcr imc reagm $51 ihv is ami *Coazmiaint In Uciohcr 2010, :mc (af the ;15sisEUR;mt general skmauwtd wifi; me his bfslim ibm Ms, Iwiafiy uk Lrcziivd him dif??renniy bsazause iaix marc and The si?}ega??ox1 was by an cuumzcna Zifiis, mafia is abc AGC regarding her tim "women make better mafigers than men," Sac AGC ,iriicmslutexi fcqucsi MS. made cf ?@im io me ifomneirzwm regzirdiugv one im Qin: in one ihe Copy ??f1{fEf? aitachcd), Alihough Ms. Z\4a.cyck was; my immcdiziia it fm' me ta repcvri ihe A?i?ifs ccnriccrm of discrimin;stiprEURziIQ Qs? me to Elf: tha lk# uf" Human H:mLz1'cc:; and I Shou?d heave fiilc? liar: complzaim mil; Ms, Pew, Ma ww agwafc of iha ,$aGC`s rcgur&?a1g MS. pnlantial viiwlmirm of M5 335 'Jie wax s;ubm?ti23 'sm Hia Humsm am 25, 2852). Ms. officf 2320 Qi' ihxx wmpiznns io an outside %ew:stigmm, Ezsterviexxfs of cmployiaes of size rfiiffr EURf?31,snsc? were in 25530 imfi 2030. On QV abc;-an ZUEEX WM a ;??wx1<2 Qzdi Lim mf ;cpz?r;1iiam Aprii 22, Lu mis; matic? imve been and to he cozwmnad from M54 and mixers Rc ucsi The 0? this corrcapevuz?ciiac is ta pm tin: L>i5t_riQt on matics tha Ms. ami Mm. Efmx ragazxiing my sepzarm?ou are gzmirsxixzai mad in viulzuiun mf`Titie $2 EBSQ. Sections E931 and 5333, 'fide VU of E313 ACI zimemied, and Fazicrzil and whiszle; thi; rcasom: icrz?czwf? flu' my to ezowr up Qc ?ntcm to rf:t:1Ei;ue: againar I rammed md ?n the izwegrigntimx of 21 raw and lagigezi against MSW becziuse 1 sngageai in that tim Ifxwa, 1 rcqucsa flmi tie ur ;1ppmwe4 LE1: tin; Gaim am idizssosi Eiegmri and hefme ihcf Q: 1115 exbcd?gimeng, Q: of the pmsiticvn wif ihe Deputy ikswizmi ifounssi in the FY EGU) budg>>;? ihal mmdxaci 3315 iswessiigaiicm Emo ail time tim: 2 am to zuirawer any and apprsuyiaieg aitcsw?im ii; fhig nlatim, su?>zz?1izmd_ Curiis Dr, (Hamm. E3> wii?w ik: iwiim 601 fCEl.Lf? may April LB. Vis: mad 4534-$92- ,201 "mf 12524 aftlur Ailzm?a Eimr? ami Mr. }Gmm?m Sixemzr Mrs" Ms. M. Mrisier Mrs. i,aChafadr;1 D. Yniamia K, .lcehrefgon Mr, EURfou1"muy Fnggikim Mx; EH Mr. Qmmeu D, E313 Trinity Ave" SW Aihxma, LSA 39303 Rc: Gazer (Bn Frifiay", Apail 22. I ww anivissd hy my Mis. '*>>felc1cr" and that Dirisclm cx_E` Humem Yew. Kimi my woulai bs: to an As?iaaum1 Cuunsci iw me FY 22022 Budger arid my wiih tha Districi zu; Deputy xmz; wrirzirmatmi immeciiaicziyl Ms, Xfiazycia ami Nw asked thai I rcugm my in an ilziziwdiaia pay Qui .E ?cc?imcd t?n5i?' request 2 haw milx?ng to iviggef a lawfui aimed Uma 3 \/wx zcrmirmtefl aw an abwili in ta tim Distriafs plara, I time razmsomf; Ms, ;1riic;uE;1tu;i ibm" fha ami ihfs iezrmiml??un E1>>EURliL?H5i3i :is 1 hzwq? pairfumted md Qu: 05 *dial jab ui ?i.lZ"liTi."F` ?n my iham Ibm yfzars iw Lhf; F2225 never issuafd any uazgatiw diz?xipllzuirjg anions mr fhrmul washing my job Pririlwrmamcei. This iimhcr imc reagm $51 ihv is ami *Coazmiaint In Uciohcr 2010, :mc (af the ;15sisEUR;mt general skmauwtd wifi; me his bfslim ibm Ms, Iwiafiy uk Lrcziivd him dif??renniy bsazause iaix marc and The si?}ega??ox1 was by an cuumzcna Zifiis, mafia is abc AGC regarding her tim "women make better mafigers than men," Sac AGC ,iriicmslutexi fcqucsi MS. made cf ?@im io me ifomneirzwm regzirdiugv one im Qin: in one ihe Copy ??f1{fEf? aitachcd), Alihough Ms. Z\4a.cyck was; my immcdiziia it fm' me ta repcvri ihe A?i?ifs ccnriccrm of discrimin;stiprEURziIQ Qs? me to Elf: tha lk# uf" Human H:mLz1'cc:; and I Shou?d heave fiilc? liar: complzaim mil; Ms, Pew, Ma ww agwafc of iha ,$aGC`s rcgur&?a1g MS. pnlantial viiwlmirm of M5 335 'Jie wax s;ubm?ti23 'sm Hia Humsm am 25, 2852). Ms. officf 2320 Qi' ihxx wmpiznns io an outside %ew:stigmm, Ezsterviexxfs of cmployiaes of size rfiiffr EURf?31,snsc? were in 25530 imfi 2030. On QV abc;-an ZUEEX WM a ;??wx1<2 Qzdi Lim mf ;cpz?r;1iiam Aprii 22, Lu mis; matic? imve been and to he cozwmnad from M54 and mixers Rc ucsi The 0? this corrcapevuz?ciiac is ta pm tin: L>i5t_riQt on matics tha Ms. ami Mm. Efmx ragazxiing my sepzarm?ou are gzmirsxixzai mad in viulzuiun mf`Titie $2 EBSQ. Sections E931 and 5333, 'fide VU of E313 ACI zimemied, and Fazicrzil and whiszle; thi; rcasom: icrz?czwf? flu' my to ezowr up Qc ?ntcm to rf:t:1Ei;ue: againar I rammed md ?n the izwegrigntimx of 21 raw and lagigezi against MSW becziuse 1 sngageai in that tim Ifxwa, 1 rcqucsa flmi tie ur ;1ppmwe4 LE1: tin; Gaim am idizssosi Eiegmri and hefme ihcf Q: 1115 exbcd?gimeng, Q: of the pmsiticvn wif ihe Deputy ikswizmi ifounssi in the FY EGU) budg>>;? ihal mmdxaci 3315 iswessiigaiicm Emo ail time tim: 2 am to zuirawer any and apprsuyiaieg aitcsw?im ii; fhig nlatim, su?>zz?1izmd_ Curiis Dr, (Hamm. E3> wii?w ik: iwiim 604 Vi warning signs of CRCT cheating ignored by APS bccausc they wa11Lcd1nirac1cS'? Page oi4 Sta* 3rd "it hi; FYEE Jia. JT it 'd tteltisfsifa Get Schooled Were warning signs of CRCT cheating ignored by APS because they wanted miracles? 11:21 am February 12, 2E1O, by Maureen Downey in responding to what appears to be epic levels of cheating at her schools. Dr. Beverly Hall has to answer this question: Did she allow principals to present her with miracles that she failed to question hard enough? Did APS ever look to see whether students and classes with sudden, unexpected surges in their CRCT scores maintained those levels of proficiency in high school? For example, the state audit found compelling evidence of answer sheet tampering in BD percent of Parks Nliddle School classrooms. Has APS ever followed Parks students through to high schools to compare scores? Consider this post from someone who says they are an APS high school principal: (Whether he/she is a pi?ncipal is not critical; it's the comment that I think is credible and worthy of exploration.) (Peso, here is a new map of the possible cheating sites statewide.) UNDERSTAND HOW PARKS MIDDLE SCHOOL CAN BEAT OUT INMAN AND SUTTON OR EVEN BUNCH. LOOK AT THE FEEDER SCHOOL DATA AND THEN LOOK AT HIS DATA, THE FEEDER SCHOOLS ARE DOING POORLY AND THEY GET TO PARKS AND BECOME SMART. THEN CHILDREN FROM PARKS COME TO lVlY SCHOOL WITH LEVEL THREE AND READ. Many of you have commented that you suspected cheating for many years and that complaints were made. Indeed AJC reporter Paul Donsky wrote this story in 2001. Now, it seems prophetic. if only Dr. Hall had acted on this issue in 2001. Her reputation and that of her system may not have been on the line to-day. (Nor did the state respond to the allegations at the time, even though it could have sought an erasure analysis in which, bythe way, is a minimal review and does not catch all instances of cheating.) Of the 68 elementary schools tested last year. 30 had gains of 30 or more percentage points in one or more CRCT subjects. Ten of those had gains of 40 or more points. And Dobbs is one of 17 Atlanta Public Schools that old wel! enough to come oitthe 2000 failing list. Atlanta school ofhciais say new reform efforts and old-fashioned hard work by teachers and students helped push the districts scores up. But amid the cheering. there are some questions. A rise in scores is expected the second year any standardized test is given, as was the case with the CRCT last spring, because students and teachers become more familiar with the test. But sudden 40-_ 50- and even 60-point spikes are not common. testing experts say. "Either somebody is doing a at something or theres something inappropriate going on, said Gregory Cizek, associate professor of educational measurement and evaluation at the University of North Carolina. Some ot the numbers are astonishing: 1 0/ 02/'1 1 605 Wore warning, signs oi`CRC`l` cheating i gnnred by APS because they wanted miracles? I Page 2 of 4 At Dunbar Elementary School near downtown Atlanta, three-quarters of fourth-graders passed the reading portion ofthe test, compared to about one~quarter last year. At M.A. Jones Elementary in west Atlanta near the Atlanta University Center, 88 percent of fourth~ graders passed in math compared to 34 percent the year before, a 54~point increase. At Thomasville Heights Elementary School in sout'1eastAt|anta. T3 percent of fourth-grade students passed in reading, compared to 19 percent the year before. Atlanta does about-face Atlanta's results are notable because many schools that posted huge gains have student populations that are almost entirety minority and low-income. groups whose test scores historically have lagged far behind. At several schools that posted huge gains, including Thomasville Heights and Cook Elementary schools, nearly all students live in public housing. Atlanta's CRCT scores remain below the state average but are in Zine with neighboring systems in the metro area. However, Atlant.a's scores are rising much faster. For example, 72 percent of Atlanta students passed in English in 2001, a 15 percentage-pointjump from the year before. ln Fulton County, 80 percent passed in English, a 2-poirttjump. Clayton County posted a 71 percent pass rate, an S-pointjump DeKalb County had a 70 percent pass rate, a 3-pointjump. For Atlanta Public School officials, the scores help validate reforms that have been put in place in recent years, such as an intensive reading program designed to boost literacy in early grades. Kathy Augustine, deputy superintendent for instruction, said the district also made sure teachers knew the topics covered onthe CRCT. Principals checked regu?arly to see that those areas were covered in class. Atlanta school oflicials say the rise in CRCT scores should not raise eyebrows, pointing out the districts SAT scores jumped 16 points this year. Still, Atlanta school ofhciels say they double- and triplechecked the CRCT results, at times examining scores of individual students, to make sure the numbers added up. That wasn't enough tor Atlanta school board member Jean Dodd, who stormed out of a school board meeting Sept. 17 after expressing concern about the validity ofthe scores. Her comments came during portion of the meeting that was closed tothe public. The meeting was held to decide the size of Superintendent Beverly Ha]l's bonus, which is largely tied to test score results. She received $47,520 bonus. "Over a period of 30 years, I taught every grade of elementary school, Dodd said in an intenriew. "1 had just not ever seen scores like that before. and so ljust made my concems known." Thircl-party audit sought Gary Henry, who has studied state testing for years and serves as director of Georgia State University's Applied Research Center, said AtIanta's results shoutcl be independently verified. "There will be some folks celebrating At|anta's turnaround, and others shaking their heads at these results until you have a third party at arm's length validate these results, "said Henry. The most likely organization to do that is the state Office of Education formed last year to grade schools as part of the state's education reform efforts, CJEA Director Davis Nelson said he will be drafting a policy in the coming months to determine when to launch such investigations. Dramatic get-schooled-b1ogi'2ll 02/"1 2/were-warnin 1 606 Wore waming signs of CRCT cheating ignored by APS because they wanted miracles? Page 3 of4 test score jumps could be one trigger of an inquiry. Atlanta school olfcials defend the scores and say theres no need for such an audit. "We are proud and our students are proud ofthe scores, said spokeswoman Pai Bowers. "And we believe the results should be allowed to speak for themselves, particularly when reinforced by other test scores over the last year." Cheating and manipulation on standardized tests has occurred in other cities, including New Orleans New York and Fairfield, Conn., where officials in 1996 discovered an unusually nigh number of wrong answers erased. Two Atlanta high schools were investigated and ultimately cleared last year after allegations arose of cheating on the high school graduation test. Georgia States Henn; said there are several possible explanations for Atlanta's test scores. you look ai these numbers. he said, "the reaction is that yourjaw is going to drop." However, he said a much higher percentage of East year's fourth~graders were enrolled in pre- kindergarten programs than the previous year's fourth-grade students, making them better prepared. A?so, last year's fourth-graders were also the Erst in APS to benedi from a new district reading program that targets early grades. But, he added, "no single factor is likely to explain" the results. (Also, here is a new map ofthe possible cheating statewide.) 12 3 N926 ?DQll@ Tell us what you think about the site iSend us your i suggestion is abou: :his page. Send a feedback technical issue lYour e-mari address (wird Su bscrihe Tits: L-fm: gli, 6, i Customer Care Vacation stops, manage subscriptions and more AJC Services Siaffcontacts Reprints and permission Contests Submit event listings a Send us news tigs Q_ar_wrs._atAJQ goose!-r ooled-bio g./201 0 607 Perggnan-Garrett, Damaris From: Few, Miflicent Sent: December 22, 2005 2:92 PM To: Hail, Bcwerly (Sum) Cc: DeFrain, ?ame|a; Augustine, Kathy Perryman~Garr-stt, Damaris Subject: RE: [Fwdz Principal Waller ei Parks Middle School] "7 Good Afternoon, We will have an external investigator begin in the new year. Millieent D. Few, fc) Chief Human Resource Officer Human. Resource Services Division q_ 130 TriniLy Avenue, S.W. hv Atlanta, GA 3 303 Qu! 1dfew3atlahLa Original k12_gn_us Message From: Hall, Beverly L. Sent: Thursday, December 22, 2065 ll:l5 M11 Augustine,-Kathy Few, MillieeUL Cc; DeFrain, Pamela Subject: FW: [Fwd: Principal Waller at Parks Hid&1e School] Importance: High Please lnvestigate. Thanks - Message From: Sent: Thursday December To: aftl555@bel1south.net Subject: (Fwd: Principal Waller at Parks Middle School] Please see the intormaticn belnw as another document generated from Parks as per Moral issues, Thanks. Subject; Principal Waller at Parks Middle School Verdaillia, per our conversation yesterday: On Monday, December 19, several Faculty members of Parks Middle School met with Anna and :J,,to discuss many of the aL Parks l/1.3. Listed is an outline cf several of the Caneerns raised: E5 There exists a sunshine fund that was taken up by the staff for various purposes i nding illnesses and time OFF. To date, there has been no use of the sunshine fund and :h have bee: at least Lhree isstances where it should have been used, Additionally, when the staff paid into the fund they were Lold that checks would not be accepted. when teacher Kelli 5 il enquired about the finds use she was netified that the account was frozen ecause some staff had wr;tte: honnced checks. But checks were not supposed te be accepted. Furthermore, there is no account of the cash tha: most of the staff used to pay into :he 2, Mr, Waller expresses a lax attitude on many issues related to :he school because, 35 he has expressed ts both faculty and parents, he'li "be leaving Parks socn." This leaves the stair with both confusion and frustration because it seems he is not committed to the school, but only tc use it as a stepping stone. Furthermore, it insults the tcachers who have committed themselves to serving the community and the students when the principal does not seem te Care to either. 608 APS-OIR 06501 3. Prior Lo Mr. Waller's arrival there was no assigned parking. Without so much as informing andfor Consulting the staff there were 5 assigned parking spaces at the beginning of the year for principal and other administrative staff. 4.l Mr, Waller's wife held a birthday party for him on school premises in the conference for more then 3 hours where only selected staff was asked to atteno. Some :eachers were allowed to leave their classrooms where paraproffesionals were placed as subs so that they could purrakc in iestivities, while olhern ncvor received an invitation. After the party was complete. Mr. Waller got on the loud speaker and told the staff if they wanted some cake. it would be in the conference room. >{hf There is a strong rumor about the school that Mr. Waller has both a cousin and a br ther-in law employed at povks who are viewed hy he resL of the as fav;fite5_ By Sandra Ward's [his alleged cousin] own admission, if there was a proslem between her and another staff member he would probably side with her. 6, Many of the teachers were able to pinpoint specific staff members who are said :o be in "Waller's Circle." These are the individuals he favors and spends time with, He has been observed telling these staff members not to associate with many of the veteran teachers. He explicitly told a staff member, "Don't hang out in [Kelli] Smi:h's circle." On Friday, December 16 Vicki Johnson wa called into a meeting with Mr. Waller and Sw" director- Mr. Pitts in mernher aprsreci?ltiw HL- Slice. T-here was fi note on a fly?ffigi ffen by a staff encouraging the staff :o come to the appreciation and share any of their concerns. This was a private event. However, Vicki Johnson was held in an intimidating meeting whore Mr. Pitts told her if sho had any "issues" she should come to him and not B. That sane morning Vicki Johnson says her desk drawers appeared to have been .ramsecks Manv other tgaghers said the same. One teacher admitted her invitation to the appreciation dinner wa?"mTs??HTfrom her desk. 9. Sandra Ward, Mr. Wallor's alleged cousin, although an AFT member, was not invited to the membership appreciation dinner by Vicki Johnson but Came anyway, without an invited quests and was said to be a spy for Mr. Waller by all the present staff members. 10. The staff also reported an us/them mentality among staff where Mr. Waller works with new staff and not old and is attempting to divide and conquer the seasoned staff. New staff also carries an attitude that they now more than seasoned staff because they are in Wsller's good graces. ll. Mr. Waller frequently intimidatcs the staff by telling them that the School will Qithgr be g]Qging nr be taken over by :ho ssaff creating mn atmosphere of uncertainty by the staff on their job security. 2 609 O2 APS-OIR O65 Denture JAN 1 5 2005 Dem Dr. Hall, UFFICE OF THE This letter is written concerning corruption, misuse and mishandling of federal funds for Supplemental Educational Services (SES), abuse of power by certam administrators and designated personnel, and an overall lack of care and concern for the students at Walter Leonard Parks Middle School. To begin, Mr. Christopher Waller, principal of Parks, appointed Mrs. Sandra Ward, SFA Facilitator (and an alleged relative) to be the director of the after school program for the 2005- 2006 school year. Parks is a Needs Improvement (NI) school and must provide supplemental services to its students as well as send a letter home to parents informing them of their option to select the provider of their choice. However, when Mrs. Ward and Mr. Waller gave the letter to the students, they instructed the students to take the letter home to obtain their parents' signature but to leave me space indicating choice of a preferred SES provider blank. Rey told me students mat the school would handle that particular portion ofthe fomu. This method was used because Mr. Waller decided he only Wanted to use a SES provider named Preferred Care Services. It appears that Mr. Waller and Mrs. Ward has a cormection to the gentleman who is affiliated with Preferred Care Services named Mr. Bill Selmon. There has been dishonesty occurring with the pay and the submission ofthe time sheets. Mrs. Ward, the director, receives the largest amount of pay at $40.00 per hour and is ultimately responsible for the submission of the time sheets; however, lvirs. Ward has only been present for the 2-ther school program approximately live full days since the program started on November 14, 2005. Mrs. Ward may remain for the after school program only on the days when there is a nnmdutory after school meeting occurring such as a Design Team Meeting or alfaculty meeting. Mr. Waller is very aware of Mrs. Wai-d's non-attendance in the after school program. Mr. Bill Selmon is also very aware of Mrs. Ward's because he has stopped by the school on several occasions and l\f[rs. VVa.rd was not present. Additionally, there are other individuals who are falsifying time spent working with the after school tutorial program. For instance, there is Mrs. Sonja Thompson (Special Education Program Assistant) who regularly leaves at 5:30 p.m. although the after school program does not end until 6130 pm. There is also Ms. Kyia Halloway who signs for days she did not work, and Mrs. Ward knowingly submits time for days and hours that Ms. Halloway is not present and does not work the utter school program. Mr. Gregory Reid, the Assistant Principal, also submits time as a tutor although he does not tutor kids during the after school pro gram. On most days, there are approximately 20 - 30 students total who remain for the after school program. However, the program has one director, approximately sixteen teachers, three clericai persons, and two support personnel. Many days, teachers have two ro zero students. Yet., all of the staff members are told they can still sign for time although no students are showing up and 610 06506 being serviced. The number efpersonncl staying on a daily basis and signing for time far outweighs the number of students who are staying after school to be serviced. Mr, Waller and Mrs, Ward received a heads up that the program may potentially be audited; therefore, the week of December 5, 2005, hh. Waller and Mrs. Ward summoned students, over the PA system, to come and sign attendance sheets dating back to the first day the aiter school program began through December 22, 2005, the last day ofthe first semester before the Holiday break. Mr. Waller and Mrs. Ward had approximately all ofthe students who returned forms tor the alter school supplemental services to engage in the falsihcation ofthe attendance sheets They had both students who had been attending the after school program and students who had never attended the pro gram io sign the attendance sheets. l\/lr. Waller had different color pens, markers, and pencils spread out on the table. They had the stud ents to sign different dates using various colors ofink and pencil so the attendance sheets would look authentic and believable. When some ofthe students questioned Mr. Waller and Mrs. Ward as to Why they had to sign the sheets, they told the students that their names were going to 'ce entered Z1 drawing for a Play Station garnc or a bicycle. Mr. target date for giving out a few prizes to the students was the last day before the Flolid ay break He knew that he must follow through on this promise to the students to keep down any Oflf]JLiI suspicions, Mr. Waller did rattle during the Holiday program held at the end ofthe school day on Thursday, December 22, 2005. The student names that were entered in the raffle are those who they summoned to falsify attendance sheets. Speak to some of me stndenm and you will discover that many of them never remain for the after school services. Also, look into the funds that were used to purchase the raffle prizes. Furthermore, the SES after school services was canceled on both Wednesday, December 21, 2005, and Thursday, December 22, 2005. Check to see if studenm signed the attendance sheets for these two days and whether the SES staff time sheets reflect the staff worked these two days. The week oi' January 9, 2006, the SES after school services were canceled Without giving parents or students any piior notice. Many students remained after school and missed their buses. Mrs. Ward claims that she canceled the program because everything is too chaotic. Mrs. Ward and Mr. Waller arc suspicious ofpcople talking about the alter school program, They are utilizing this week to try to get their information/docirmentation in order and cover up all of their A diagnostic assessment was administered to the students on the iirst two to three days ofthe after school program, however, Mrs. Ward never graded the tests and actually rnispiaoed students' answer sheets. Mr. Waller forced teachers (who Work for the after school program) to give up their planning period during the regular school day in order to re-administer the after school diagnostic test to students. Mr. Michael Jackson, the Opporiunity teacher, helped orchestrate the second administration ofthe diagnostic test for the after school pro gram. He Went around pulling students out of their connections ciasses (and sometimes core classes) to retake assessment. The second ofthe diagnostic test was done to help cover up the mishandling of the program. Also, they wanted the number of diagnostic tests to match the of students who would now appear to have been present in the program (as a result of the falsified student attendance sheets). The week of December 2l, 2005, teachers were also 611 06507 made to complete CRCT forrns that were used make it appear that Mrs. Ward, the after school program director, had done the appropriate analysis on the students needs. Mr. Waller and Mrs. Ward also began forcing student athletes to report to Dr. Alfred Kiel, for the first 30 minutes alter school, for mandatory tutoring. Many ofthe athletes did not even choose the SES services. Most ofthe athletes being forced to tutorials do not need the services. Many of these students inet and a lot of them exceeded on all parts ofthe G.C.R.C.T. Mr. Waller and Mrs. Ward are forcing the student athletes to attend uttering so it will boost the student attendance number on the attendance sheets. Mr. Waller and Mrs. Ward informed the students that they could not pmicipate in their sport or ride the utter school/activities bus unless they complied with this request. Although Dr. Kiel only tutors shldents for the tirst 30 minutes of after school, time is still being submitted for him as i_t` he works until the program ends. In addition, the teachers have not been provided with an after school program curriculum, scope and sequence, teaching resources, or any instructional direction. - I I I The SES provider, Preferred Care, is supposed to provide a complete snack for the studenm each day. There have been days where the students did not have a snack at all. On other days, when the students were provided snacks, they were not provided vritli a drink. Other Abuses: Mr. Waller attempts to persuade, intimidate and coerce teachers to cheat onthe upcoming spring 2006 G.C.R.C.T. Ile constantly makes statements such as, "We know that they are cheating at the Elementary Schools. It is no Way that those elementary test scores are real, Unless we [Parks] do like them fthe Elementary schools], we will continue to look bad." Or he makes comments such as, "Price [Middle School] had to cheat They [Price] are smart. Iley, if you ca11`tbeat them, job; them." Mr. Waller is always questioning teachers how many of their students are going to pass the G.C.R.C.T. If a teacher respond sixty of their students [out of apx. l00], he makes comments such as expect no less than eighty or ninety students passing." Then he goes further with similar comments like those above such as, "You need to teach the test. Do what you've got to do." Ita teacher questions or disagrees with the suggestive comments that Mr, Waller makes, he usually follows by asking, "Are you a team player? Are you on my team?" or by maldng comments such as need people who are on my team." These examples provide the context in which the attempted persuasion, intimidation, and coercion generally take place. Mr. Waller also attempts to intimidate teachers to not talk to or associate to other staff members. For instance, if he observes Teacher A speaking to another staff member he considers "not being a part of his team," he will make comments to Teacher A such as, "You were doing good by staying away from that "click". Dun't get caught up with the Wrong "click" Or he will say, "If you disassociate yourself from that "crew" you won't have to worry about any repercussions from us [his administrative team]_ Be wise, and stay away from that "crew" Additional intimidating comments are, observe you talking to your friends again. were you, I wouldn't be caught talking to them anymore." M.r. Waller is constantly harassing staff members with these types of 612 APS-OIR OE 508 warnings. He has created a very tense atmosphere at Parks in which there is absolutely no trust among colleagues and everyone is paranoid and afraid to speak to or hold cordial conversation with one another because no one wants to be perceived as being associated with the wrong "crew" and not being on Mr. Waller's "tea:n". Further intimidation occurred during a faculty meeting held on Tuesday, January 10, 2006, Mr. Waller passed out ajob description document to all staffmernbers, highlighted only the last bullet ofthe document, and read it aloud, and it states, "Perform all other tasks and duties as assigned," and Mr. Waller added, "By Mr. Waller targets teachers who he feels are not on his "team" or non- compliant to his dem an ds. He targets teachers by having several members of tho administrative team such as Mrs. Soma Thompson, PEC Program Assistant, Dr. Kiel, Mr. Reid, and Mrs. Ward, go into their classuooins (ou the sainc day or oltcn dunn the same Week) to observe their classroom. He instructs these individuals to document everything they performed Wrong lesson. These ruhsertfations are not for the sincere pmose of helping teachers improve because rnost times, the teachers do not receive any feedback or suggested strategies improving teaching. These "targeted observations" are conducted for the mere purpose ofintimi?ation or obtaining negative documentation against a teacher. Another tactic used to target teachers is to submit their names for needing assistance with the implernentation of instructional initiatives to personnel outside of Parks. The teachers' names are submitted without a formal internal conference with the concerned admiuisuatitfe team member or without informing the teachers what they are not doing properly and without giving the teachers effective strategies for their improvement Also, Mr. Wallerhas Mrs. Sonja Thompson, PEC Program Assistant, to Write letters to both special education and regular education teachers reprimanding the teachers for whatever she deems. These letters go into teachers' personnel file. Mrs. Thompson signs her letters "Adn1inistrator". Mr. Waller makes teachers who are assigned morning duty report to work at 8:00 am. This reporting time is forty-five minutes earlier than APS scheduled reporting time for middle school students, which is at 8:45 a.rn. Mr. Waller instructed teachers to no longer take homeroom attendance in Class XP (linked to APS SAST system) for months. actually questioned a teacher during a meeting why was the teacher still taking attendance in Class XP alter he instructed not to. Instead, Mr. Waller forced the attendance clerk to manipulate the student attendance in SASI to ensure the attendance target for AYP is rnet. It was not until Monday, December 19, 2005, that Mr. Waller rrutdc Ms. White, the attendance clerk, begin making announcements that it was mandatory for teachers to talce homeroom and period attendance in Class XP. He also Coerued Lisa Specialist assi to support Parks, to monitor attendance daily and send teachers e- mails (hegimtitlg this same week) if they were not using Class XP for tdneir attendance. Ms. Hills' involvement was strange yet obvious, Ms. Hill had not been reporting to Parks to assist teachers for over two months. For her to suddenly begin reapp eating at this time Wm obvious to all that Mr. Waller Was utilizing Ms. Hill so that he could claim that teachers were not properly taking attendance either due to their ignorance or due to technical difficulty, when in fact, Mr. Waller instructed teachers not to take attendance. Again, this was another tactic used by Mr. Waller to cover his tracks, 613 05509 Mr. Waller flirts, make inappropriate and sexual gestures to many female staff memb ers. He threatens that it is the female teachers' word against his word. During the week of November 28, 2005, Mr. Reid, assistant principal, planned a surprise birthday party for hir. Waller and invited select staff members. The party was held in Mr. aller's conference room during regular school hours. The invited guests included Mrs. Melanie Clark (who was gone off campus the entire rnoming purchasing the food, cake, and decorations for the party), Mrs. Sandra Ward, l\/irs_ Som a Thompson, Mr. Michael Jackson, Mrs. Cheryl Hunley (leadership facilitator from the state department of education), Mr. l-lenry Coleman (CIS), Mrs. Laverne Grant (special education paraprofcssional who is constantly given the okay to leave her classroom to handle o_r asist With matters not related to her immediate responsibilities. An example ofthis is helping in the front ofticc), Ms. Kyia llalloway (who was released during her regular Sm grade math class through coverage provided by another special education paraprofessional pulled out of her assigned class), and Coach .Timmy Lipscomb. The party lasted approxirnately two hours from 1:00 pm. until 3:00 pm., and the attendees remained in the conference room for the entire period with die door locked. Teachers at Parks paid $25.00 for Sunshine Committee dues (checks were not accepted). There have been three staff members ill, Ms. Marshall, Mrs. Watson, and Mrs. Smiley (two out sick for a long period of tirne). When it was questioned Whether or not the Sunshine sent flowers or gifts, the response was that there was a freeze on the Sunshine Conimillee account. The staff is curious were the funds have been spent. Mrs. Melanie Clark is chair ofthe Sunshine Committee. It is also alleged that Clark is Mr. Waller's mistress and that Mr. Waller gives Mrs. Clark money from petty funds such as the Sunshine Committee and money raised from school functions such as sporting events, dances, and bake sales. There was a Homeroom contest to raise funds for families of students who attend Parks that relocated due to the Hurricane Katrina disaster. Over two thousand dollars was raised by the students and staE at Parks. No one knows how this money has been spent. To everyone's knowledge, none ofthe students or families affected by llurricanc Katrina has received any money or donations. lt is alleged that Mr. Waller purchased Walmart and Best Buy gift certificates with the money raised to give to the families. It is said that these gin cards have significant spending lhnits. Also, there have been allegations surfacing that Mr. Waller and Mrs. Clark have been utilizing the gift cards for personal use. Mrs. Clark is hired as a special education paraprofessional; however, at the beginning of the school year, Mr. Waller removed Mrs. Clark 'dom a special education classroom and assigned her to assist the media specialist. Mrs. Clark was never in the media center. She always ran special errands off campus or around the school for Mr. Waller. Eventually, during the second semester, Mr. Waller hired another paraprofcssional, Ms. Ferndinand, to assist in the media center with Tide I funds (even though the school needed a grade math teacher and a grade science teacher at the Lime). Now, Mrs. Clark works in the main office as an oflice assistant, although she is hired as a special education paraprofessional. Parks has live special education teachers -Ms. M. Jones, Ms. V. ohnson, Ms. J. Watson, Ms. A. lohnson, and Dr. Flagler - however, there are only three paraprofessionals assigned to assist the teachers - Ms. A. Hadley, Mr. Judge, 614 S-OIR 06510 AP and Ms. L. Grant (who is never in her class and always hanging in the CIS roozn or helping Mrs. Clarke in the iiont office). I Mr. Michael Jackson was hired as an opportunity teacher; however, he has not had an opporturiity class since the beginning ofthe school year. Mr. Waller Lttilizes Mr. Jackson as if he is another Assistant Principal, allowing Mr. .lackson to roam the halls all day with a radio. In addition, Mr. Jackson is getting paid to Work in the after school program, and he is not a certiied teacher and never tutors any students. Ms. Kyia Halloway, an Sm grade mathematics teacher and a second alleged mistress of Mr. Wa.ller's, is constantly pulled out of her classroom during the regular school day to fulfill personal requests for Mr. Weller such as creating "Club Day" schedules and assisting with system and state reports/documents. Ms. Halloway is usually provided with a supply teacher for the entire day or Mr. Waller directs ii special education pampro fessional to cover Ms. HaIloway's class for the entire day. When Ms. Halloway shows up to work in herjogging suit, it is understood that Mr. Waller is her coverage for that day to utilize Ms. Halloway to complete his requests 'ms he deems. This has been a frequent and Mr. Waller docs not code this in system. lt most recently happened on Tuesday, January l0, 2006. Any ofthe 8" grade teachers can to this. As a concerned constituent, I hope these issues are addressed. As such, I will be forwarding a copy of this letter to the following people: Dr. Beverly Hall, Superintendent Ms. Kathy Augustine, Assistant Superintendent Mr. Michael Pitts, Executive Director SRT-2 Ms. Sheryl Freeman, Acting Director of Office of Internal Resolution Sincerely, A very concerned constituent 615 APS-OIR 05511 ATLANTAPUBLIC Of?c? hi Hug 130 Avenue, - Phone: (4041) 80211803 rv, D?h? Tm 311 eqYseH?q?l A For Yqur E!-i CorivqrsatijDiEP05if`iCI1fJAN17 ZUU6 Amxrm SCLIDTI >>-fuwmm nasuupzca 616 APS-OIR 06505 it ATLANTA PUBLIC SCHOOLS Osnceurmn 'f f/ ll Human Resource Services Division ni 'ro= 'Par M-s. "Few The attached is seuttu you fm' the following: El Please handle and inform me -GQ E1 For your information 1: Per nur conversation El Further discussion needed rs\\ii\9? More infomation needed - Referred to: Cm IJ Please schedule nmeeting with me Cl Please respond and copymebyf Prepare a response for my Copy Us by eminem /Please sumea DATE: (16504 Qgohtvt. January 16, 2006 if Qepigsczan 1 A sh I4 Dear Ms. Freeman, This letter is written regarding dishonest and unethical practices occurring at Walter Leonard Pinks Middle School in the Atlanta Public School system as it pertains to the upcoming 2006 Georgia Middle Grades Writing Assessment that will be administered on January l8 - 19, 2006. The writing booklets are being secured in Dr. Alfred Kiel's office, the instructional Liaison Specialist, Mr. Christopher Waller, principal, and Dr. Kiel, looked at the official writing prompt for this years' assessment. The Week of January 9, 2005, the eighth grade teachers and other designated teachers/school personnel were given the task of conducting last minute writing workshops with the eighth grade students in hopes of better preparing the students for the upcoming test. These staff members were given a packet with the title Tips for Passing the 8" Grade Writing Test. On the front ofthe second page, there are ton sample writing topics listed for practice (enclosed). Aitcr Mr. Waller und Dr, Kiel tucked at official writing prompt, it was emphasized to the teachers on Friday, January 13, 2006, to have the students concentrate on ofthe sample writing topics located in the writing packet. Topic states: "Think ofa rule in your home, school, or that you is unfair. Tell what the rule is, explain why you think it is unfair, and describe how you would change it." The teachers were informed that sample question above was very close to the official writing prompt; however, instead of the word rule, the ofhcial writing prompt will ask the students to think of a that is unfair. The students were also told that rumor has it that the real writing prompt is going to ask them about a Law that they think is unfair, thus, the eighth grade students were instructed to on particular Laws that they think are unfair. The students were also told to think of persuasive arguments as to why the particular law [thought of by the student] is unfair and what they [the student] would do to change the law or improve the law. To tbllow up on these concems, I would advise that you speak to both the eighth grade teachers and eighth grade students to verify the claims written in this letter. Sincerely, An Ethical Individual 618 APS-OIR 06503 Phoenixkesearch Investigations Memo Il. Ili. To: Ms. Damaris Pesryman-Ganett Fran: Reginai Dukes i Date: March 15, 2006 Re: Preliminary findings I Parks ll/Eddie School The following ls a preliminary Ending into improprieties at Parks Middle School ouuineo in an anonymous Ietterreceived by your oftioe on January 13, 2006. Summary ofA|\egatlnn The Supplemerrlal Educational services (SES) alter school progran is being abused, in operation and by participating teachers and administrators. The Georgia Middle Grades Writing Assessment (2006) may have been compromise alter Mr, Waller persuaded, coerced teachers to cheat on the test. Mr, Waller has began Iritimidallhq, threaten and urging teachers to reveal any infomialion they may have about this investigation. Person interviewed During the course ofthis investigation Regina! Dukes interviewed nine persons, including Mr. Waller and Mr. Pits Preliminary Endings Since the beginning of this investigation, working conditiors at Parks Middle School have become very difficult for some ofthe teachers and other staff that are viewed as having intimate Knowledge of some 'l 61 9 ofthe allegations. There should be some consideration tu alleviating the anxiety Gurrentiy existing. The infomiation acquired thus far certainly indicates thatihem were some improper conduct This investigation will require many more imerviews uf staffas weii as some parents and Students. 0 Fage2 620 000002 Confidential Draft Report of Investigation For Atlanta Public Schools Parks Middle School Prepared by: Investigator Reginal J. Dukes Phoenix Research Investigations, LLC May 5, zoos 52 l'1'oprirstary lin' the Public Sclmol System Office of Inlezrnzil ltesciltilioaa, compiled by Pliocilix and Rcsearcli, is a conli?cntial Liraft of findings as work product. 621 06519 2 Table of Contents Page A. Summary of Investigation 3 B. Investigative Details 4 C. Conclusion 7 D. Attachments 1. Summary of issues from initial investigation of first communication 2, Outline of issues from investigation of subsequent communications 3. Interview of Christopher Waller 4. Interview of Gregory Reid 5. Interview of Sandra Ward 6. Interview of Fabiola Aurelien 7. Interview of Anthony Tiller S. Interview ol' Tamika Butler 9. Interview of Felicia Phillips 10. Inten/iew of Kelley Collins 11. Interview of Melanie Clark 12. Interview of Sujuena White 13. Flow chart of student attendance documentation process 14. Sujuana White email to SASI Project Manager on ooncerns 15. Sujuana White Statement of change in duties and resulting injury 26. Sujuana White statement of Mr. Reicl's call to someone to interfere with Ms. White's pastffuture unemployment status 17. of Alfred Kiel 18. Foilow-up interviews with Bm grade teachers 19. Interviews of Students ZD. Inten/iew of Mr. Pitts 21, Interview of Ke?li Smith 22. List of PCSS studenls 2005 (first semester) 23. of PCSS students 2006 (second semester) 24. Pending documents not received from PCSS Private Pro]ni'ietary information Atlanta l-'ublic Sichool System - Office uonipilud investigation Research, This is a confidential draft offindings as zitturneymlicrit work nmduet. 622 APS-OIR- O6 520 Report of Investigation Parks Middle School Summary of basis for investigation This inquiry commenced on January 10, 2006 with the receipt of an email from t'ne Atlanta Federation of Teachers, sometime in late December 2005. The ooncems expressed at l:hat lime were vague and nonspecific and did not rise to Uwe level of any policy violations. Initially, it was decided that an inquiry be made to determined if tl-rere was more information that would give us an understanding as to exactly what was occurring at Parks Middle School. Sometime around January 13, 2005, a second piece of communication (letter) arrived, via U.S. Mail, for Ms. Sheryl Freeman in the Ofnce of Internal Resolution. This letter had some very specihc allegations: citing some operation discrepancies in the Supplemental Educaljonal Services (SES) after school tutorial program at Parks Middle School; allegations of manipulation of the SASI system attendance (class and numerous instances of questionable managerial practices by Principal Waller and other staff members. The third letter was received sometime prior to January 18, 2006. It oontained information of possible cheating on lhe eighth grade-wriljng exam. Both letters were anonymous and the email from did not reveal their source or author of information. Copies of these communications were provided to Investigator Dukes, after which he held interviews with members of the faculty and staff at Parks Middle School. A summary of each of drose interviews is included in the attachments to this report. i>fi'J3ll2 _-*rilanlar I'r.rtrlic School by Plrocnix Investigation and Research. 'Lois is a conlitlential dralir of findings as attor'ney-client work prodr;ct, 623 APS-OIR 06521 4 Investigative Details The investigation initially focused on the issues raised in the first oornmunicationl, but was refocused to concentrate on the four issues in the second and third anonymous communications: Supplemental Educational Services After School Tutorial Program - SASI data system - Bu" grade writing test - managerial practices ?upglemental Educational gnlices (SES) Afterschool Tutorial Program The investigation found that parents were not given a choice of providers for this program: During the selection process of me SES providers, students were not given the complete package, which should have included: a letter to parent(s) with instructions, a list of providers the request form for supplementai services (on the Request for Supplemental Sen/ices forms for Parks Middle School, PCS5 is the and only choice, It is written in with a different handwriting from the rest of the form in all cases.) - This practice has continued with the distribution of application forms on March 9, 2006. The investigation found that teachers did not have written instructions or curriculum for participating in the program: No written instructions could be found. The ad hoc program coordinator stated that no written instructions were provided to the teachers. - It was determined that Assistant Principal Reid is me paid program coordinator for the afterschool program, but Ms. Sandra Ward was the unpaid coordinator for the program. Ms. Ward was not full-time and her lack of availability during the program time was of concern to several teachers and staff. Ms. Ward was available during the school day and for a short time after school for some coordination. The teachers and staff participating ln the program considered her the program coordinator and did not approach Mr. Reid on their concerns. Ms. Ward was sent to the initial program overview by the Principal and has been working with this specific provider since October 2005. See attachment #1 Sl. for Lili: Allante Pxiblic Syalcril Rcsolulion. Lrg.-' Ph0c:1izr. Inx-'estigzcion and Research. This is ai confitientizd driiftof findings as work product. 624 APS-OIR 05522 5 The teachers did not work with the same student from one day to the next, making progress reports difficult. Some teachers working in the program did not always remain on campus during the after school hours of the operation, however, dwey were allowed to sign in. Some were encouraged to sign in and allowed to attend faculty meetings and design team meetings during those hours. There appeared, according to records, to be a large number of support persons working the program. These persons had little or no roll in providing tutorial sen/ices to students. The investigaijon found that student paiticipation was not regulariy documented: Student attendance was not documented on a daily bases, attendance sheets for the 2005 sessions were carried to regular class sessions by staff members (December 2005) in an effort to get reports to the site provider (Preferred Care School Services). Students were asked to sign the attendance sheets without any verilication of their attendance or enrollment in die program. iimesheets for teachers and students' attendance sheets were submitted to Atlanta Public Schools, External Programs for December 20" and 2152 2005, diese were days that the sessions were canceled, and no one participated, student(s) nor teachers(s). Of the 96 students on the ofncial list of certified students, approximately half were not eligible to participate. The records on student participation have been "lost." There is a lack of consensus on the number of students that are participating inthe program. There is concern raised by staff and students on documentation of participation in the program. There is an allegation that students were given prizes to incent them to falsify documentation about participation in the program. During the December pep rally, names of students eligible to participate in the program were pulled from a hat and prizes awarded included Play Stations. The students had eadier been called to the ofnce to sign an attendance spreadsheet, allegedly with different ink pens. in vate for Public Schott] System Oliice ofliilcnliil C0l\'11JilCd hy' xc tn. ion and Research. This is a confidential draft nffindings as attorney-client wot? product. 625 APS-OIR 06523 6 SASI data gsbem The investigation found allegations of manipulation of student attendance and student grades in the SASI data system. Specinc allegations are that: - Suspended students are not properly documented as absent in all classes - The SASI clerk was directed by line Principal to change attendance data to reduce absences - The SASI clerk was directed by the Instructional Learning Specialist to create student grades for data missing from teachers in order to meet the report card deadline - The SASI clerk has been changed but her access number is being used grade writing test The investigation found that a tip sheet was provided to all Bm grade students, as early as prior to the Christmas break for the January writing test. The tip sheet included sample questions that were the same or very similar to the questions on the actual test. The tip sheet was given to the students by their language arts teachers, most noted was Mr. Rodgers. One teacher stated that a student came to her about the Principal asking him to change one of his answers on the test. PrinciQal's Managerial practices The investigation did not focus on managerial style or practices. The investigation did rind that: - Some employees feel they have been threatened by the principal, assistant principal, and/or the educational instructional specialist in regards to the after school program performance reports and the SASI data input - There is general discontent among the teachers and the students about the perceived disorganized and dishonest manner in which the alter school program has been managed, and HWEY feel that the Principal has been involved in the process One employee has been hospitalized for two weeks due to high blood pressure. She indicated this medical condition was due to stress from a hostile work environment. This is the same hourly employee who has indicated she was required to work overtime without pay to maintain her job. - Private Propri<:Lury information for Lin: Atlanta 'Public School System Office of internal Resolution, compiled by l'hoeni>; Im-estigasion and Research. This is a confidential draft ofindings as attorney-clioni work product. 626 05524 7 Conclusion Based on these investigative Endings, we conclude that: Preferred Care School Sen/ices Principal Wailer is aware of and involved in the daily operations of the SES Afterschool Program run by Preferred Care School Services. He was involved in: the selection of the ad hoc program coordinator, Ms. Sandra Ward the selection of some ofthe teachers as tutors - the supervision of the support staff, during school hours and alterschooi purchase of gifts for a raffle to incent students to attend to program - Principal Waller is aware of the operational issues with the PCSS program. He has been involved in: - Meetings with teachers Meetings with support staff 0 Meetings with PCSS staff 0 Purchase of gifts to help improve attendance in the program - There was no evidence that Principal Waller is being compensated by PCSS. - The current after school program run by Preferred Care School Senrices is poorly organized and poorly implemented. This has caused frustration with teachers and students. The documentation for teacher participation and student participation is not based on actual participation by either. The poor program implementation makes it impossible to detennine who participated and when. SASI data system Based on the investigative findings, we conclude that there is reason for concern about the accuracy of the attendance and grades input into the SASI system at Parks Middle School. Further computer forensic investigation would be needed to determine the veracity of this concern starting with: tracking how suspended students have been entered for the school year and back-up documentation on all grades entered for the first semester. BP Grade writing Test Based on the investigative Endings, we conclude that students were coached on the topics that were actually on the formal writl ng exam. The language arts teacher most mentioned in providing this information is no longer at the school. The investigation did not determine how he gained access to this information. We do also conclude that there is reason for concern about the securing of these tests before the exam at Parks Middle School. We also conclude that further investigation is needed on the second-hand information that the School Principal interceded during the test to make students change test answers. This investigation did not include addressing the managerial practices of the school Principal, but the original and expanded issues that were investigated raise concerns about the climate at Parks Middle School. Private i'i'upi'lelary lor Liu: Atlanta Public Scliuo! System Odfice of1m'e|'11al ltesoluliorm cuinpilcil ty Plioontx investigation und Research. Tux is aconfricntlai draft of findings ns attorney-clicni vvoriz product. 627 APS-OIR 06525 8 Attachment 1 Issues _raiseg_ Allegations Persons intenriewgl Checks received for death or illnesses ln the family were - 1/10 Chris Waller 1' Sunshine fund bouncing because 1/10 Kelli Smith . TUBE- Certain teachers were asked to leave class during 1/10 (hrs Waller 2' Birthday party instructional time to attend a birthday party for the School al. The school principal hm hired his bother-in-law and 1/10 Chris Waller 3' Nepotlsm cousins as employees at the school in violation of i stem oil.. 1 A flyer was circulated from the teachers union at the 1/10 Chris Waller 4` ATF flyer school for a meeting to daacuss Concerns at the school, fund Investigators interviewed PMS staff to determine who manages me sunshine fund and how it operates. The focus was on deiernilning if checks received from the sunshine fund for death or ln the family were returned from funds. During the intervlew indicated that Melanie Clark and April Hadi ey were managing the program, while me scwool principal indicated that Ms. Hide and Ms. l'~'lcCoilugh were managing the program. There was consensus that the sunshine fund comes from voluntary contributions by school staff for distribution to school staff in times of illness or death in the family. No written procedures were found. This investigation was terminated at the direction of APS given that the sunshine fund ls not a system program and to focus on the issues raised In second oommunication. Birthday Qagty Investigators interviewed PMS staff to determine if and when the party happened, and who was invited. The focus was on determining if teachers were allowed or required to leave class during instructional time and If lhere was selectively in teachers invited. Durng the interview ofthe school principa it was determneri that a "surprise" birthday :iarty for him had been coordinated bv his wife at the school on December The school principal indicated that only 'ion-lnst'uctional staff attended the pany and Teachers were invited after he party to come to the confe-ence room during their breaks to share in the refreshnienis. This was terminated at t'|e direction oi' APS Lo iucue on the issues raised in the second o:\mmuni:atlon_ Nggotisg Invatigators Interviewed PMS Principal to determ ne if he had hired any relatives as staff at Parks Middle School. The focus was to detemiine if APS system oollcy was violated on hiring or During the interview ofthe principal he stated that he had not hired his zirother-in>>law, oousins, or any other reiatives at Parks Middle School. He also stated of cousins was not in violation of APS system policy. The person specincally considered to be a relative was Ms. Sandra Ward. This invstigetlon was terminated et the d=rection of APS lI> focus on the issues raised in the second communication. ATF Flyer Investigators interviewed PMS staff to determine lf the flyer was circulated and what were the underlying concerns. The Vows was Lo determine if a meeting was called and held, and what were die reasons for ctmcern. It was determined lhat the liver was circtlated at the school for holiday party. It was determined that a handwritten note was added to the printed flyer' indicating that the party would Include a discussion ol 00r\ce'r\$ at the school, It could not he determine: who acfletl the handwritter-r note, but it appears to have l:-een on all flyers distributed, Tire party/meeting was held on December 16"- There was a discrepancy on statements regarding who attended the December meeting with the union representative to discux the flyer. This investigation was terminated at the direction of APS to focus on the issues 'alsed in the second [iz lb? the l"t|blic of lry lnx-'estigatiini an: Research. 'lfhis is a confidential draft as 5-ttom-13--client work product, 628 O6 526 2 letter issues 1. After school tutoring program 2, SASI documentation 3. Study tips with writing prompts Alle ations a. Parents did not have a choice in b_ The person in charge of managing the person, was not the same person being paid to manage the program. selectin the afterstilodl tuturlnt C. did l'lOf Instruction on Opdratluhs. 9 Attachment 2 2/12 SuJu~na White 7/15 Sandra Ward 2/1? Saiualia White 3/1 Gregory Reid Alfred Kimi 2/12 Fahiola Aurellen 2/17 Chris Wailer 1/17 Fabiola Aurellen 3/1 Sonja '|'hompson 3/1. Anthony 'lilier Reviewed parent fonns 3 1 Melanie Clark 12 1? 1/ .315 2/12 d. The dudent Program Darticipaeon was falsified. e. Teachers and sla?f parljsipants were not oaid equally no' based on actual artid ation hreats a suspended students are 'lOl documented ln the attendance em students are suspended more 9 da fs er c. siaff to manigulate SASI data ds staff threatened io manipulate SASI e. student scheduled not updated properly regarding in confusion on whore students are and problems with attendance doodmentation Students were given actual test questions prior to formal tesling. 1/1? 2/1? Faolcia Aureiien Chris Waller Kelley Collins SuJua:1a White Chris Waller Felicia Phillips 3/1 Alma Kiel 3/7 Tanlika Butler 2/12 Fabiclla Aurelien 3/1 Gregor,-' Reid 3/ll 2/12 1? z/11 2/12 2/12' 2/12 3/ 1 Fablola Aurelierl Suluana Whize SUJLE ne White Suluana Whilne Suluarla Whme SuJuana White 2/17 Fanlola Aurelien 2/17 Felicia Fatxiola Aurelien 2/17 Suluana White 3/1 <;~agary Reid 3/1 Anihony'lTEler 3/16 Kelley Collins 3/11 5 3/1 Sonja Thompson 5 3 1 Melanie C|a'k Sulltlalia Willie Fablola Aurel len Alfred Klel 3/7 Tamika Butler 3/1 Students 2 These persons were questioned about issues relaiing tu the allegation, not wlletller they agreed or disagreed allegation, 81 1"el'tl'le Public Sclluul Office l.1OlliPiiEUR7{i by ilu-'vssligaiion and Resaarcii. 'This is a draft of findings as attorneywlien? work product. 629 DE-527 10 Interview of Christopher Waller Investigator Reginal Dukes met with Mr. Christopher Waller on January 10, 2006 and February 17, 2005 at Parks Middle School. Mr. Waller provided the following information: Mr, Waller is employed for one year with the Atlanta Public School System as Principal of Parks Middle School. On January Mr. Waller was asked about four issues arising out of the first ATF email: (1) Mr. Waller was asked about his knowledge about the operation ofthe "sunshine fund" at Parks Micldie School. He advised that he was aware of the existence of the fund, but not aware on how lt operates. (2) Mr. Waller was asked about his knowledge of teachers leaving students during instructional time to attend a birthday party in his honor. He advised that there was a surprise birthday party on Deoember 1, 2005 in his honor. His wife coordinated the party. Mr. Waller advised that only non-instructional staff was in attendance, and that teachers were invited over the PA system to share in the refreshmenw during their planning period. (3) Mr. Waller was asked whether he had hired any relatives as employees at Parks Middle School in violation of APS nepotism policy. He advised that he had hired no direct relatives (as defined by APS), nor had he hired anyone who lived with him that was not a relative. Mr. Waller made the point that cousins are not direct relatives under the APS nepotlsm policy. (4) Mr. Waller was asked about the distribution of an ATF flyer promoting a Christmas holiday party that would also include discussion of concerns about issues at the schooi. He advised that he was aware of the flyer, and that on December 16" ho and Mr. Pitts met with Ms. Vicki Johnson, the ATF representatve al. the school to discuss the flyer. Mr. Waller advised that any concerns are a result of him being the new Principal and his decisions as such. Mr. Waller was interviewed again on February on another issue3, specincally about his knowledge of the supplemental educational services provided by Preferred Care School Services (PCSS): Mr. Waller advised that he was aware of the program. He advised that he had sent Ms. Sandra Ward to an APS orienmtion meeting regarding the SES program. Mr. Waiier advised that Ms. Ward was sent at the East minute to bring the infomwation back, because a substitute teacher would not be needed to backfleld her while she was at the meeting. Mr. Waller stated that he was aware the Assistant Principal Reid worked for PCSS In this program, but he was unclear on who he thought was the PCSS site Mr. Waller stated that he was indirectly aware of operational issues with PCSS. Ho stated that he purchased gifts for a drawing from students who participated in me PCSS atterschoo! prograrn. That drawing was held at a pep rally in December. Mr. Waller stated he was not aware of how the names were determined for inclusion in the drawing. He stated that he was not aware of students signing an attendance form/spreadsheet to participate in the drawing. The last matter discussed in this interview was whether Mr. Waller had ever given cash to employee Sajuana White for doing extra work. He advised he had not. Upon further discussion Mr. Waller did state he had provided cash to Ms. White to help her out with her utility bills on one or two occasions. 3 Investigator Marc Lawson was also present during this interview. l'l'opi'ietaxy inR>iTna1io11 lor thi: Atlanta Public School Ofioe of coiripilctl hy Pltoenix Investigation and Reseanzli. This is a conficlentia? draft ofindirigs as attorney-client work product. 630 28 APS-OIR (JE55 11 Mr. Waller was re-interviewed on May 10m for follow-up issues: - when asked about payments to PCSS, Mr. Waller stated: I W|'lEl1 He is not involved in arrangements with PCSS on classroom rentals. He stated that classroom rentals are handled between Facilities Services with PCSS. He is not involved in arrangements with PCSS on student transportation. He stated that student transportation is handled by the Department of Transportation with PCSS. He is not aware of or involved in any other payments to PCSS. asked about the management of input into the SASI system, Mr. Waller stated: Ms. White has been at Parks for over one year as Secretary to the Assistant Principal and SASI input coordinator Mr. Waller stated that Ms. White is still the SASI coordinator. When asked how Ms. White could still be SASI coordinatoz' without access to the SAS1 system, Mr. Waller stated that her privileges had been removed because of an inappropriate actions and insubordination with Ms. Thompson. Mr. Waller stated that Ms. Thompson is now responsible For SASI input. He was not sure of Ms. W?aite's weel?y hours, but thought they were originally 2? hours per week. When asked about her current hours, he was aware they had been reduced to 27 or 28.4 When asked why her hours were reduced he stated to be in line with Title One rules. Yes, the original and current "reduction" is inconsistent. t'l'op1'iecary information lb: the Atlanta llubtic School Systuui OlTice of Internal ltesolution. compiled by Investigation and Research. This is a collfidential of Endings as anoriley-client work product. 4 631 05529 12 Interview of Gregogy Reid Investigators Reginal Dukes and Marc Lawson met with Assistant Principal Gregory Reid on March 1, 2006 at Parks Middle School. Mr. Reid provided the following informaljon regarding his role with the SES program provider Preferred Care School Sen/ices: Mr. Reid advised that he started with PCSS at the beginning of the program. He was unclear as to whether that was October or November of 2005. Mr. Reid stated that he was the site coordinator and was paid $50 per hour for that role (he knew that teachers were paid $25/hour). He stated his responsibilities included: Assigning class rolls - Assigning students to classes Determining which teacher was assigned for what subject Observing classes Team teaching as needed Filling in for teachers as needed Mr, Reid advised he was not responsible for: - Reports to parents - Student progress reports Attendance records for students or teachers. 0 He advised dnat Mr. Selmon assigned this task to Ms. Thompson and/or i-'ls. \/viaite. Mr. Reid stated that 30-60 students participated in the program on any given day. He advised that Dr. Kiel provided tutoring before and after school under the PCSS program. When asked about the role of Ms. Sandra Ward, he advised that they worked together but he was not aware of her official title. Mr. Reid acknowledged that he did not attend the APS orientation on the SES program. When asked about the December incentive drawing for PCSS student participants, Mr. Reid stated he was not involved in any wav not in the determination of the students included, the drawing, or the gifts. He stated he was not present forthe drawing. Mr. Reid was re-interviewed on May 11"' regarding Ms. Sajuana Whites additional allegations (see page 21 in tab 12). Mr. Reid advised that: He was not aware that her SASI duties had changed, but had directed Ms. White to help with the year-end cleaning out of closets (see tab 15). - He has never had a discussion with anyone at GDOL or otherwise about Ms. White's social security number or her attempts for unemployment during the summers (see tab 16). Private 8; 1"roprierzny los- Allante Public Ulhoe oflnlornel Resolution. compiled by Phoenix Investigation and Research. This is a ooniicentizu drali ufliridrigs as work produc: 632 06530 13 Interview of Sandra Ward Investigators Reginal Dukes and Marc Lawson met with Ms. Sandra Ward, SFA Facilitator, on February 16, 2006 at Parks Middle School. Ms. Ward provided the following information on her role with the SES program provider Preferred Care School Services: Ms. Ward started with the program as a result of Mr. Waller sending her to the APS orientation session forthe SES program. Ms. Ward stated that she is not employed by PCSS and is not paid by PCSS. She was aware that certified teachers were paid $25/hour for tutoring and that non- certitied persons were paid $15/hour for program support. She stated her "role" has been: To bring back the information from the orlentatlon session to the school 0 Give forms to teachers for distribution to students for participation in the program, including a parental permission forrn and list of providers I Handing out application forms to teachers for participation as tutors with PCSS Touching base with Ms. White on program operations I Interfacing with Mr. Selmon at PCSS Ms. Ward slated that her role did not include: 0 Giving instructions to teachers on the programs operation Being present during program hours She advised that 70-98 students participated in the program. When asked about the role of Mr. Reid, she advised that he was the program coordinator. It was her understanding that his duties were to Oversee the building, including monitoring and walking around Back-field as a tutor in math when needed Help Ms. White Ms. Ward said she felt the teachers were not being asked to do anything different from their daily duties as certified teachers and did not see the need to go into details on program operation. When asked about the role of Ms. White, she advised that Ms. White was responsible for: Overseeing the program - Giving out snacks 1 Making sure that transitions took plaoe in the building Ms. Ward was asked about her knowledge of documentation previously requested by Investigator Dukes on two occasions from Mr. Waller: Student sign-In sheets for attendance, including athletes - Original SES provider Forms showing the selection of PCSS by parents List of all teacher applications and acknowledgment letters to teachers for participation 0 Original time sheets for teachers and Blank application to teachers When asked why she participated in the program without compensation, Ms. Ward stated that she knew when she changed schools that she would need to help out without compensation for all support given in order to help the school improve. She stated that the school needs support. Pr ix-are ?t infornraziorr ter the Ar?urta Public School System Ollicc ol' Resolution. compiled by Phoenix lnvesirgatiorr and Keseerolr. I'-ris is 1 confidential draft of iindir' gs as nttorncg-'-client work jorodtct. 633 06531 14 Interviews with Fabiola Aurelien Investigators Reginal Dukes and Marc Lawson met with Ms. Fabiola Aurelein on February 12" at her home and on February 17"' at Parks Middle School. Ms. Aurelien provided the following information on her role with the SES program provider Preferred Care School Services: Ms. Aurelein stated that she has been a Math teacher at Parks Middle School for two years. She stated her role with PCSS was as a math tutor. Ms. Aurelein stated that she thought there were 15 teachers serving as tutors. She stated that teachers signed-in with Ms. White (and later Ms. Thompson) at the beginning of the program each day. Ms. Aurelein stated that she was not aware of any direct supervision of teachers in the program. Ms. Aureleln was of the understanding that Ms. Sandra Ward was the program coordinator, until she was recently chastised for Ms. Ward the program coordinator. She stated that she had earlier shared concerns with Ms. Ward about the program: Never receiving request for progress reports in December 2005 - Asked to provide progress reports ln February 2005 when she did not have the same students each day Never advised to provide documentation of student attendance 0 All participating teachers not included in meetings to review program procedures Ms. Aurelien stated that she was called into a meeting with Ms. Sandra Ward and Bill Selman on Friday, February 10"' to terminate her services with PCSS since she was paid but dit! not tutor students, because she would not provide progress reports. She stated that she advised them she did tutor, but could not do progress reports since she did not see the same students consistently. Ms. Aurelien stated that Ms. Ward took the led in the meeting and was very harsh with her. She stated that she was confused on her status because at the end of the meeting Mr. Selmon asked her not to "resigr|." During the February 17'" inten/iew, Ms. Aurelien stated that at some point Mr. Waller had attempted to solicit her as program coordinator to "get things straightened out." Ms. Aurelien stated that she was a member of the design team and did not tutor on Monday's when the design team met, but did sign-in for PCSS on Monday's along with other teachers. She stated that Ms. Ward was aware of this practice since she was also a member of the design team and in the room when the PCSS sign-in sheet was brought into the design team meeting for teachers to sign-in. When asked about allegations of cheating on the writing test, Ms. Aurelien stated that she was included in a teacher meeting about this subject. At that meeting all persons were asked to sign a sheet on whether they were aware of me al|egations.5 Afterwards she was individually called into Mr. WaIler's orhce as part of his investigation into the allegations. Mr. Waller had a tape recorder. Ms. Aurelien slated that she wanted someone else in the room as a witness and Mr. Waller called in Dr. Kiel. She then answered Mr. \~'Valler's questions. The conversation was not taped. After Dr. Kiel lett the room, Mr. Waller asked some hypothetical questions. Ms. it-is. Aurelien was ihc only person ll'iHi21tl!i1lilS? ro hearing about the allegations. 'Private 3: Fai' the ?'ublir: System Office of Resolution. by Plroerfx and Research. Ybis is a corrliderrtial draft of findings as attorney--clierrt work product. 5 634 APS-OIR 06532 15 Aurelien thought he was trying to see if he could count on her not to say certain things that would incriminate him (Mr. Waller). Ms. Aurelien also stated that an unscheduled staff meeting was held in which the Principal stated that know there are people who are unhappy here and I'l! take care of making sure that these people go on their way." When asked about how someone could access the test prior to test time, Ms. Aurellen stated that there is general access to Dr. Kiel's ofhce and that the tests are not sealed when the teachers come to pick them up. She also slated that the week before the test teachers where given specific instructions on howto prepare the students forthe writing test. Proprietary Aiianlu Public School System Ofiicc oi' Resolution. in'-fesligatinn and Reseezch. 'Vziis is :1 confidential -:trail of Iindings as work product 635 06533 16 Interview with Anthony Tiller Investigators Reginal Dukes and Marc Lawson met with Mr, Anthony Tiller on March 1, 2006 at Parks Middle School. Mr. Tiller provided the following information on his role with the SES program provider Preferred Care School Services: Mr. Tiller is the in-school suspension coordinator. He stated that his role with PCSS was 50% as a math tutor and 50% support. His support duties included monitoring the halls, securing the doors, and escorting students to the bus. Mr. 'lTller stated that his rate of pay was $25/hour. When asked about program operations, Mr. Tiller stated that he understood Ms. Sandra Ward to be the site coordinator. He stated that he interviewed for the position with Ms. Ward and Mr. Waller. His current understanding is that Ms. Thompson has replaced Ms. Ward as site coordinator. He estimated that 5-8 teachers participate as tutors and an average of 20-25 students participated in the program on a daily basis. Mr. Tiller stated that he has not done any student performance reports. Bt iL'f`ol'1natlon f`o1'tl1t' l-'llblic School ltesnlulimi, cmiipilell by and Rcscarch, This is a draft of findings as attorney-clicrir vvoriz product. 636 APS-OIR U6 534 I7 Interview with Tamika Butler Investigator Reginal Dukes met Ms. Tamika Buder on March 7, 2006 at Parks Middle School. Ms. Butler provided the following information on her role with the SES program provider Preferred Care School Services: Ms. Butler is a ianguage arts teacher at Parks Middle School. Ms. Buder stated that she is not a tutor with PCSS. She stated that she is the cheerleaders' coach and her knowledge of the program relates to tne interfacing with athletes and cheerleaders. Ms. Butier stated that athletes were forced to participate ie the program in order to be eligible to play sports, To her knowledge the athletes spent 15-20 minutes before practice in tutoring sessions. Ms. Butler stated that cheerleaders were also forced to participate in order to participate in this activity, "even if they were straight A students." When asked about the December raffle, Ms. Butler stated that she was aware of the raffie. She understood it to be an attendance incentive and that large gifts were awarded. In regards to attendance documentation, Ms. Butler stated that she remembered a time that Mr. Tiller came around for two days with attendance sheets forthe basketball team and the cheerleaders to sign. She stated that she initialed the form for the cheerleaders for January but refused to initial for January 9" she did not think the program was operating that day. It was her understanding that the basketball team members did sign the attendance sheets for attendance on both days. Ms. Butler was also asked about her knowledge relating to the CRT test. She stated that she was not aware of a tip sheet for the at grade writing exarn. She stated that she was vaguely aware of students stating that someone talked to them about what to focus on. Ms. Butler said she does recall that after the test a special education student ran to a teacher (Ms. Johnson) and stated that the test was exactly like the sheet they had studied from. Ms. Butler also recalled that a student (Ashanti White) told her Mr. Waller approached him during the test to question one of his answers, specihcally stating "are you sure you want to do that' and asked him to change his answer. Private Eufoinraliorx lb: the fttlarne E'ublic School System Resolution. compiled by Plxoenix Investigation ood Research. This is draft of Endings as work product. 637 05535 is Lnterviews with Felicia Phillips Investigators Reginal Dukes and Marc Lawson met with Ms, Felicia Phillips on February 2006 at Parks Middle School. Ms. Phillips provided the following information on her role with die SES program provider Preferred Care School Services: Ms. Phillips stated that she has been working with the PCSS programs since October or November, 2005. Her role was as a math tutor. When asked about program operations, Ms. Phillips stated that: After hlling out the application, she did not receive an acceptance letter. When presented with a copy of the letter, ivls. Phillips stated she had never seen the document. She did not receive anything in writing about her responsibilities, no curriculum nor information about her hourly rate for her sen/ices She does not recall being invited to any program coordination meetings She did see the student participation forms signed by parents At times 2-3 teachers worked with one group of students at one time She understood that Ms. Sandra Ward was the program coordinator, but she was not on-site during the program time. She was aware that Mr. Reid was involved with the program because of his involvement in assembling students and doing some team teaching. When asked specifically about her participation between November 28m - December lst, Ms. Phillips was not sure which days she worked. When shown her time sheet with some blacked out areas, she was not aware of why certain parts were blacked out. Ms. Phillips stated that she did get the request for progress reports in late December, but was confused and frustrated on what to do because of confusion on what students were included and what was suppose to be tracked. She stated there were no names of students on the forms provided. Mr. Phillips stated that Ms. Ward had commented on her frustration about "people not doing what they were suppose to do" in regards to the progress reports. At some point in December, Ms. Phillips stated that she made a written list of 10 concerns about the program, but did not share them with Ms. Ward. She did participate in her program coordination meeting on Tuesday, February 14" and though duties were still not clear, it was a positive start. Ms. Philiips stated that some teachers expressed that they were not willing to continue working with the program until the problems were nxed. To her knowledge 20-23 students pardcipated on February Ms. Phillips also stated that she was aware that some parents wanted to pull their 6* grade student out of the program because they were not getting the language arts l:utoring that was promised. de Pr-oiwietary irrformariorr lor Lilo A1lanta`Publiu School System Omte of liriemul in and Resesrch. This is a ccmEdenLial drah findings as axromsy-client Work pmdum. 652 50 APS-OIR 065 33 Document reguests to William Selmon Investigator Dukes has made repeated requests to Mr. Bill Selmon at PCSS for the following documents, which have not been received: Payroll records for PMS teachers involved in the program to determine the Services for which they were paid, specifically For November cl December 2155 The canceled check we were told by Mr. Selmon was given to Ms. Ward to purchase the gifts for the December pep rally. Private Ak E'i-upricluiy 1`oi'tl'ie Oliioo nl`Iiilor1;al Ru1=oluLioi|_ eompiled byPlioe1;ix lm-'estigaiion and Reserercli. E`l1is is a Llrali as aliomey-client work product. 653 06551 Wh5b?2,Ss?ua Fmm: ME 951372 Will Ml TO: WG*-ie, '5\1j\\EUR\'nl CLI WQTEI, Su 8150! HF: NS. of Q2 ren1emi;>>er vnu, Fm ana: have Such but! new I can heip you. There are 3 issues that 1. do want to address wzt?\ you. Fivisi., you miirlwir Suriv: Jriuxisalge. I ix.-Jw with Mr; Wailar to try to gain game rleriry err the s=I;uaticn, and have :opie? mm on th|5 cmail. Rhazfia was doing; fear ish as she has b@n Mr. i*JaHrr1mt\on, me ezuormr 1 can Ealffo anion. Thank your canpcratiun in these ixsues. Mer-eciith E_oEURermat'cn i"-iamger 654 06 552 IR- -O APS "wc 2 oil U, L30 Trinity Avezrlue Fra-ln: While, Siljuana Sent: l/lofiday, Apsii 10, 20% 10:62 AM Tn: Rahman, Subject; Impolinlvcec E-ligh Gfaasl You may nel ffimii mba( my name once you :sad Elvis alrraii you will rernemiser ma an? mf situaiicn, Sujuana Vn?ailew over al Paras Nlisidie School. have been haviilg iss-.seswhli so memr inexpoeienueci users xvixli at my school. Uur imeiidariue is the Tlrsi and slain issue. The lsacleeis aw wining me atuandanea and daily. llowuven i ilzzve noticed an increase in absemzeswhieri lets me know that lie altendance is being lakeri bmuoawe (you and i) mm-:l thai lime teachers do rsoicpsn up their nlassmz imake-llzn attendance it slulornnliciaily makes lhe slullenl pressnl. Al lhis puirfi my SASI right? have been taken uway #mm me and given someone Wham my prinuinal calls 3 (exam player. li can and wil mannmle more wiih you in person) Now I am 5S6;llQ a lolofli?egal things happening ham and am lying low ifsel rw mb is on lho line but I urn being foited In cal! ink: lo your zsalwtlon. My SASI ziglus we-ua snalrzluecl army on Nami; Zvi 2096 and lnund QLLI wriran it in In client support. lherl ca?lsai Rhonda who vexy familiar wilh all my concerns and she was about it and loicl me lhal she gal lheorderto cancel sw; :ighls from my principal on Thuriaifay and lliui six: did noi know w?ratwag; going un, men weni Lu him my principal) and asked was there a reason lor me not being abi?to access SASI . l-le Elan hail me he mold talk Lo me later on annul ll1aL er, wa have people usisvg SASE Sag in and passwords of peupie who nu loingwworla hare amllor have been gone for wasons for 'There ale even issues with FTE ann other programs and ilvfofmaliuri that has been ialsilifsd. Al this poinl I clozfi know win-xl you can do hui ljus! wanted to sizzle ini; iawrnxaliori will: you In nraile me l'ee= irellen Plame me as soon as yols are avfailahla to diswua my r;omfr=rns umm am walling pauanvly with our rllsoipllna and me having a as lc when ami what io do. All year I have been of Rhonda Malik arxl her sdvina and saipprm Dura! mis polnrl dan? :rust anyone and! ll?wd Sumo apoloqizefor well long eurall hui rim $0 uuelwhelmed my what l*=11 seeing and until 1-gmac 1?urcl\ iurtnux' men Rhonda. I :oak forward lu hsailng from asvd.'oi wil: you soon. 655 R- 06553 S-OI AP PHOENIXERESEARCH sl I- CASEACTIVITYLOG CASE NO: Atlanta Public Stlhoul - Parks Middle Sthuul Date Acrivi Hours Atlantzi -GA. - - A 1/17j2006 Intw wg' Mr. Pitts 1090 Windsor Adanta 2 - 2/1212006 _Ei Inrw wr sunuana White her home. RD al ML 12 2006 Intnl _Fabiola Aurelien her homeIntw wr' Sandra Ward 1090 Windsor Atlamz GA. RD 2/1712006 Imrv wr smuana White 1090 Windsor str, sw Adanta GA. Roar-1|_ 2/1712006 RJD Intrv wr Fabiola Aurelien 1090 windsor Str, sw 4 1_ - Intrv w/ wane; 1090 str, sw -2 A Arianna' I Atlanta GA. Atlardix GA. RD ML Atlanta GA. - 3 3/112005 mn 0 STUDENT NAMES REDACTED 1090 Windsor Str, SW Atlaniz GA. . A _Atlanta GA. Atlanta GA. 33 3/15,/2006 Intrv w/ Kelly comms 1090 Windsor Str, Sw 2 33 Atlanta GA I 656 KES 000085 5 5 2006 Pre ared first draft of Investi aljve Re ort Met w/ Dr. Hall Mr. Pitts Ms. M. Few 130 Trinity Ave Atlanta GA. - A I i Wilson-@ 1090 \MndsorStr SW`Atlant/a GA. 5111/2006 Intrv w/ Gregory Reid 1090 Windsor Str' SW Atlanta GA. Prepared final clrafc of Invesl:igal:lve'Report and WD ll 6130/zoos 2 S1 -Z Q- ll Q1 Tam deliveredto 130 Trini Ave Atlanta GANote: Two Lnvestigatprs were authorized by second irnrestigator, Marc Lawson KES 000086 657 Confidential Report of Investigation For Atlanta Public Schools Parks Middle School Prepared by: Investigator Regina! J. Dukes Phoenix Research Investigations, LLC June 30, 2096 Private irslisrmallori Rn- rlle Arlmlie Public System Oliiee Resolution. cumpilad by Phoenix lnvesiigatlcn and Research. This is a cortfidrzlu?al statement of tinflings as is-ark product. 658 S-OER- 06672 AF 2 Table of Contents Page A. Summary of Investigation 3 B. Investigative Details 4 C. Conclusion 7 D. Attachments 1. Summary of issues from initial invesligatlon of first communication 2. Oulline of issues from investigation of subsequent communications 3. Interview 0" Christopher Waller 4. Interview of Gregory Reid 5. Interview of Sandra Ward 6. Interview of Fabiola Aurellen 7. Interview of Antho ny Tiller E. Interview of Tamika Butler 9. Interview of Felicia Phillips 10, Interview of Kelley Collins 11. Interview cf Melanie Clark 12. Interview af Sujuana White 13. Flow chart ui student attendance documentation process 14. Sujuana White email to Project Manager on concerns 15. Sujuana White statement of change in duties and resulting injury 16. Sujuana White statement of Mr. Reid'5 call tu someone ho interfere with MS. 'u'>>lhite'5 unemployment status Interview of Alfred Kiel 18. Follow-up irtcn/ievvs with Bm grade teachers 19. Interviews of Students 20. Interview of Mr. PILLS 21. Interview of Kelli Smith 22. List of PCS5 students ZOC5 (first semester) 23. List of PCSS students 2095 (second semester) 24. Pending dooiments not received from PCSS 25. Check frorr' PCSS for pep rally gifts Private Proprietary information for the Atlanta Public Sclnoc-l System -- uflniemal Resclution. compiled by Plwenix Investigation and Research. This is statement of findings as work product. 659 OSGTB Report of Investigation -- Parks Middle School Surnmagv of ggsis for investigation This inquiry commenced on Janualy 10, 2006 with the receipt of an email from the Atlanta Federation of Teachers, sometime in late December 2005. The ooncems expressed at that time were vague and nonspecilic and did not rise to the level of any poiicy violations. Initially, it was decided that an inquiry be mace to determined if there was more information that would give us an understanding as to exacdy what was occurring at Parks Middle School. Sometime around Januan; 13, 2006, a second piece of communication (letter) arrived, via U.S. Mail, for Ms. Sheryl Freeman ln the Ofhce of Internal Resoludon. This letter had some very allegations: citing some operation discrepancies in the Supplemental Educational Services (SES) after school tutorial program at Parks Middle School; allegations of manipulation of the attendance (class and (C) numerous inslralwces of questionable managerial practices by Principal Waller and other staff members. The third letter was received sometime prior to January 18, 2006. It contained informahon of possible cheating on the eighth grade-writing exam. Both letters were anonymous and the email from AFT did not reveal their source or author of information. Copies of mae communications were provided to Investigator Dukes, after which he held ininen/lei-vs wiiji members of the faculty and Staff at Parks Middle School. A summaly of each of .those interviews is included in the attachments to this report. Private Proprielnry irihzrmaiiun for the Atlanta Public School System Office of internal Resolution, aoinpiled by Phoenir. investigation and llesearaii. This is a cc-niiclential staternentoftindiugs as product. 660 06674 4 Invesfgaiive Details The investigation initially focused on the issues raised in the oommunicationl, but was refocused to concentrate on the four issues in the second and third anonymous communications: - Supplemental Educational Services After School Tutorial Program - SASI data system - Sl" grade writing test - Principal's managerial practices (11 Sugglemental Educational ServiceslS?S) Afterschool Tutorial The investigation found that parents were not given a choice of providers for iiwis program: During the selection process of the SES providers, students were not given the compiete package, which should have Included: a letter to parent(s) with instructions, a list of providers the request form for supplemental services (on afl the Request for Supplemental Services forms for Parks Middle School, PCSS is the first and only choice. it is written in with a different handwriting from the rest of the form in all cases.) This practice has continued with the distribution of application forms on March 9, 2005. The investlgauon found that teachers did not have written instructions or curriculum tor participating in Uwe program: - No written instructions could be found. The ad hoc program coordinator stated that no written instructions were provided tothe teachers. - It was determined thai: Assistant Principal Reid is the paid program coordinator for the afterschool program, but Ms. Sandra Ward was the unpaid coordinator forthe program. Ms. Ward was not full-time and her lack of availability during the program time was of concern to several teachers and staff. Ms. Ward was available during the school day and for a short time after school for some coordination. The teachers and staff participating in the program considered her die program coordinator ard did not approach Mr. Reid on their concerns. Ms. Ward was sent to the initial program oven/few by the Principal and has been working with this specihc provider since October 2005. See attachment #1 Private at infomation f`o1'the Aflania Public School System Office of Infernal Resolution, compited by Phoenix Investigation and Researcli. This is a cnrifldential stutemeilf of Encliigs as altorney~olient work product. 661 APS-OIR 06675 5 - The teachers did not work with the same student from one day to the next, making progress reports difhcult. Some teachers working in tte program did not always remain on campus during the after school hours of the operation, however, they were allowed to sign in. Some were encouraged to sign in and allowed to attend faculty meetings and design team meetings during those hours. There appeared, according to records, bo be a large number of support persons working the program. These persons had little or no roll in providing tutorial services to students. The investigation found that student partlcipatlon was not regularly documented: Student attendance was not documented on a daily bases, attendance sheets for the 2005 sessions were carried to regular class sessions by staff members (December 2005) in an effort 'ro get reports to the site provider (Preferred Care School Services). Students were asked to sign the attendance sheets without any verification of their attendance or enrollment in the program. iimesheets for teachers and students' attendance sheets were submitted to Atlanta Public Schools, External Programs for December 20" and 2152 2005, thae were days that 'me sessions were canceled, and no one participated, student(s) nor teachers(s). Of the 96 students on the official list of certified students, approximately half were not eligibie to participate. The records on student participation have been "lost" There is a lack of consensus on the number of students that are participating in the program. There is concern raised by staff and students on documentation of participation in the program. There is an allegation that students were given prizes to incent them to falsify documentation about participation in the program. During the December pep rally, names of students eligible to participate in the program were pulied from a hat and prizes awarded included Play Stations. The students had earlier been called to the office to sign an attendance spreadsheet, aliegedly with different ink pens. We repeatedly requested and did not receive documentation from Preferred Care School Services regarding payroll records for PMS teachers involved in the program to determine the services for which they were paid. We did receive a copy of the check for $324 given to Ms. Ward to purchase the gifts for the pep rally. In follow-up discussions widr Mr. Waller we were advised that another APS deparlrnent handles arrangements with PCSS for ciassroom rentals and student transportation, and that he is not aware of airy other payments to PMS by PCSS oulsicle of die tutoring fees. Prix-are Kc Proprietary information for Atlania Public System -- Ofiice oflntemal Resolution. compiled lay Pnoenix Investigation and Researrlr. This fs zr stalemeutnftindirrgs as work. product. 662 R- 06676 6 SASI data system The investigation found allegations of manipulation of student attendance and student grades in the SASI data system. Specific allegations are that: - Suspended students ere not properly documented as absent in all ciasses - The SASI clerk was directed by the Prinoipal to change attendance data to reduce absences - The SASI clerk was directed by the Instructional Learning Specialist to create student grades for data missing from teachers in order to meet the report card deadline - The SASI clerk has been changed but her access number is stiil being used We also found that Ms. White had been removed from her duties as SASI coordinator and her access to that system has been removed since early May. Her hours have also been reduced. Ms. Whites interview Statement shows here concems about manipulation of student data. grade Wriifirlg TESIZ The investigation found a tip sheet was provided to all 8" glade students, as early as prior to the Christmas break forthe January writing test. The tip sheet tnclucled sample questions that were the same or very similar to the questions on the actual test. Th tip sheet was given to the students by their language arts teachers, most noted was Mr. Rodgers. In reviewing the website identified as the source of the tip sheet, we could not such document on the website. Four of the six language arts teachers were interviewed: Lewis, Wilson, Arnold, and Draper. We couid not locate Rodgers who has left employment at APS, and Lewis is on maternity leave. In addition, che teacher stated that a student came to her about the Principal asking him to change one of his answers on the test. Principars Managerial practices The investigation did not focus on managerial style or practices, The investigation did find that: - Some employees feel they have been threatened by the principal, assistant principal, andior the educational instructional specialist in regards to the after school pro-gram performance reports and the SASI data input There is general discontent among the teachers and the students about the perceived disorganized and dishonest manner in which the after school program has been managed, and they feel that the Principal has been involved in the process - One employee has been hospitalized for two weeks due to high blood pressure. She indicated this medical condition was due to stress from a hostile work environment. This is the same hocrly employee who has indicated she was required to work Overtime without pay to maintain her job. Private Pmprieisiry info-rmat.io11 for che Atlanta Publi: School System Office uflniemixl Resolution. compiled by and _Ruseurc!1, This is ti cozifideniiul as work product. 663 AFS-OIFP 06677 1' Conclusion Based on these investigative findings, we conclude that: Preferred Care School Services Principal Waller is aware of and involved in the daily operations of the SES Afterschool Program run by Preferred Care School Services. He was involved in: - the selection of the eci hoc program coordinator, Ms, Sandra Ward the selection of some of the teachers as tutors the supervision of Lhe support staff, during school hours and purchase of gifts for a raffle to incent students to attend to program - Principal Waller is aware of the operational issues with the PCS5 program. He has been involved in: Meetings with teachers Meetings with support staff Meetings with PCSS staff I Purchase of gifts ID help improve attendance ln the program - There was no evidence that Principal Waller is being compensated by PCSS. The current after school program run by Preferred Care School Services is poorly organized and poorly implemented. This has caused frustration with teachers and students. The documentation for teacher participation and student participation is not based on actual participation by either. The poor program impiementalion makes it impossible to determine who participated and when. SAS1 data system Based on the investigative findings, \-ve conclude that there is reason for concern about the accuracy of tho attendance and grades input into the SASI system at Parks Middle School. Further computer forensic investigation would be needed to determine the veracity of this concern starting with: tracking how suspended students have been entered for the school year and - back--u documentation on all grades entered for the semester. Bm Grade Writing Test Based on the investigative findings, we conclude that students were coached on the topics that were actually on the formal writing exam. The language arts teacher most mentioned in providing this information is no longer at the school. The investigation did not determine how he gained access to this information. We do also conclude that there is reason for concern about the securing of these tests before the exam at Parks Middle School. We also conclude that further investigation is needed on the second-ha nd infonnation that the School Principal lnterceded during the test to make students change test answers. This investigation did not include addressing the managerial practices of the school Principal, but the original and expended issues that were investigated raise concerns about the climate at Parks Middle School. Private Proprietary infonnatioir Atlanta Public Schooi System Office of internal Resolution, compiied by Pl;oeni>\ lirvcsiigntioo rind Researcim This is cnnlidential smiament oftindir-gs as niiorncy-clicol work piodiicx. 664 75 066 AP Issues_ra\ised 2 Allegations 1 Sunshine fund Chea? recelvazl for death orilbress ln the family were bouncing because of mismanage~r-rent of the 4' party uma to amend a srehaay partyforrne lhe school principal has hired his brother-in-law and cousins as employees at the school In viniatiorr of Certain teaclers were asknd D3 leave class during 3. 4. ATF flyer 5 rm' . A flyer was cLrculateC from the teachers union at the sdwool for a rneeting Lo discus concerns at the school. Sunshine fr.md Persons in Attachment 1 te rviewed 1110 cririsitrallaf 1/1o Kerri smith 1/to chris E'Efi5"` I Chris Waller' 1/13' Mr. Pitts [nv-xhgamrs Interviewed Pl*-15 staff to detennhe who manage the sunshine rurd and _-row it upuruta The rc-:us was on determining rf checks received from the had for death cr illness in the l'a1lly were returned from insufficient funds. During the iniervlew staff members indimted that Mdanic Clark and April Hadley were managing the while We school p-l ncir-ral indicated that Ms. Hide and M5, r-1 xoliugj-| were managing the progrirn. There was oonsmsus that the fund cnrnes from volsnlary by school staff fur clishibutton to school staff in times of inne55 or death ln the l'amrly. No wntten procedures were found. This investigation was terminated at the Clrectlulr gverr that the sunshine fund is not a system program and to 'ocus c\"r lhe issues reise? ln the second Investigators Lrrtervleared PMS staff to determine lf and when the party happened, and who was invloed. The focus was on determining ir teachers we-e allowed ur rcuuircd to leave dass during lfrne and was selectively in invited. During the interview of the school principal it was debermined theta "surprise" party for had been coordinated by his wrfe at the school The school principal Indicated that uriy utturrced the party and teachers were after the party to come to the conference rocm during their breaks to share ln me refredwmenm. This irn-'estigation was terminated at lhe direction of APS to focus on the issues raised in the second communication. Qlegntism Investigators Tntervierved PN5 Principal to deierrnine he had hired any relaives as slaff at Parks Middle School. The focus was to determine if APS system nollcy was virrlaieri on hiring ol' relallv-ras. During the :rf the mincipal he stated that he had nc-t hired his brotrer-in-law, cousins, or any other rdatives at Parks Middle School. ire ergo stale; that the of consms was not ln of APS system ibofrey. The person s|Jeclf'rcally considered tc be a relative was Ms, Sandra Ward. This in'-'estigatinn was terminated at the of AP5 to focus on me issues raised in the sec-and communication. ATF Flyer investigators interviewed PMS staff to determine lf the flyer was circulated and what were the Lmderlying concerns. The focus was to determine lf a meeting was rzlled and hdd, add what were the reasons for concem. It was thatthe ftyer was circulated at the sd-mol a holiday party. It was determhed that a handwritten :rote was added to the printed flyer hdlcadng that the party would include a of concerns ae the sdrnol, It could ns: be deterrnined who added the hancwr'lt'ten note, but it appears to have been on all flyers distrihrrted. The WGS Mid CN Them discrepancy on statements regarding who attended the December meeting with the union represeztative ho dlscu? the flyer. This ifrvatigation was terminated at the direction UFAPS to focus cm the isles raised In the second oommunlcation. Private inforrnaiic-rr for Pub?ic School System Office of Internal Rescrlutiorz. complied by lrrvu-aligptiun and Research. This is rr as p1'ud\rcl_ 665 79 066 2" letter issues 1. After school tutoring program 2. documentation with writng prompts Alle I a, Parerts did nel: have a choice In isehsclirg the afte|S@rLpt5jrlg|JJlovider. Persons interviewed Reviewed parent forms 9 Attachment 2 2 ls. The 1| charge ol muneglng the 2/11 Suluana Wh'te len i person, was not the same person being 2/15 Sandler Ward 2/17 Waller Deldto manage me program. 2/17 Saluana White 2_r'17 Fabiala Aurelien 3/1 Gregory Reid 3/1 Sonja Thompson - 3/1 Alfred Kiel 3/1 Antlicny Tiller 3/1_ _Melanie Clark C. The teachers who participated not 2/12 Fablola 2317 Fehiula Aurellen receive Enstrucljun on program uperatlons. 2/1? 2/17 Fellcla Phllilgxs 3 d. The smdent program was Suluana Whlte '2/12 "Fa't5i'oIa Aurelien falsilied. 2/17 Chris Waller 2517 Suluana Wtfte 2/1? Fellda Phillips Gregory Reld 3/1 Alfred Mel 351 Antho:1ylT|Ier Tamlla Butler 311.6 Kelley Collins TTI Studenls 'see footfore e. Teacters and staff were 2/12 Fablula Aurellen 2/17 SuJuana White not pald equally' nor based on actual 3/1 Gregory 3/1 Sonja Tllumpaun _garticlpation 3/1 Anthon Tiller QQ Meade Clark f. threats 2/12 Fabiola Aureien Snluana White a. suspended sludenls are not Suluana Whiie la. students are suspended more than 9 2/1.2 Suluawa Whlze da Soer incldent staf1'pald/thrleatened so manipulate SuJuana whine SASI alierldalwce dala d. Staff threatened bo manlpulaie SASI 2/12 SuJuana Whibe radlzs data e. sludert sulxeduled not updated 2/12 SuJuana Whibe pwperly regardng in cowfusion on where mnrienls are and problems dltendaltce dccumerllalicln Students were given actual test questlors 2/12 Fabiola Aurelen prlorto formal heating. 3/1 Alfred Klel 3/7 Tamlla Butler 3/1 Students 2 These persons were about issues relating io the allegation, not whether they agreed or disagreed with lhe :1llegal:iD11. Private infommatlon forthe Atlaniu Public School System Office oflntemal Resolution. compiled by Pnoenix lnvenigaticn and Research. This is a cunidential staterient of findings as product. 666 06680 AP 10 Interview of Christopher Waller Investigator Reglnal Dukes mot with Mr. Christopher Waller on January 10, 2006 and February 17, 2005 at Parks Middle School. Mr. Waller provided the following information: Mr. Waller is employed for one year with the Atlanla Public School System as Principal of Parks Middle School. On January Mr. Waller was asked about four issues arising out of the first ATF email: (1) Mr. Waller was asked about his knowledge about the operation of the "surshine fund" at Parks Middle School. He advised that he was aware ofthe existence Oi the fund, but not aware on how it operates. (2) Mr. Waller was asked about his knowledge of teachers ieavlng student: clurihg instructional time to attend a birthday party in his honor. He advised that there was a surprise birthday pany on December 1, 2005 in his honor. His wife coordinated the party. lsr. Waller advised that only non-instructional staff was ln attendance, and that teachers were invited ovor the PA system to share in the refreshments during their panning period. (3) Mr. Waller was asked whether he had hired any relatives as employees at Parks Middle School in violation of APS nepodsm policy. He advised that he had hired no direct relatives (as defined by APS), nor ted he hired anyone who lived with him that was not a relative. Mr. Waller made the point that cousins are not direct relatives under the APS nepotisrn policy. (4) Mr. Waller was asked about the distritzuizion of an ATF flyer promoting a Christmas holiday party that would also lndude discussion of concerns about issues at the school. He advised that he was aware of the flyer, and that on December t6"' he and Mr. Flits met with I-ls. Wcki Johnson, 'he ATF representative at the school to discuss the flyer. Mr. Waller advised that any concerns are a result of him being the new Principal and his decisions as such. Mr. Waller was interviewed again on February 17"' on another lssue3, specifically about his knowledge ofthe supplemental educational services provided by Preferred Care School Services (PCSS): l-ir, Waller adveed that he was aware of the program. He advised that he had sent Pia. Sandra Ward lo an APS orie'1tation meeting regarding the SES program. Mr. Waller advised that Ms. Wa'd was sent at the last minute to bring the infomation back, because a substitute teacher would not be needed to liackheld her while she was at the meeting. Mr. Waller slated that he was aware the Assistant Principal Reid worked for PCSS in this program, but he was uriclear on who he thought was the PCSS site coordinator. Waller Stated that he was indirectly aware of operational issues with He stated that he purchased gifts for a drawing from students who participated in the PCSS elteraschooi program. That drawing was held at a pep rally in December. Mr. Waller stated he was not aware of how the names were determined for inclusion in the drawing. He stated that he was not aware of students signing an attendance form/spreadsheet to participate in the drawing. The last matter discussed in this inten/iew was whether Mr. Waller had ever given cash to employee Sajuana 'Nhite for doing extra work. He advised he had not. Upon further discussion Mr. Waller did state he had provided cash to Ms. White to help her out with her utility bills on one or two occasions. 1 lov esligaior Mari: Luwson was also present during this interview, Priv are Proprietary infornnatioxi for the Atlanta Public School System Umce oflniernal Resolution, compiled by Plioexro; lnwsiigaiioti and This is 11 confidential staiumcnt of iindir-gs as attorney-client work product. 667 06561 1] Waller was re-interviewed on May 10" for follow-up issues: when 1 asked about payments to PCSS, Mr. Waller stated: He is not involved in arrangements with PCS5 on classroom rentals. He stated that classroom rentels are handled between Facilities Services with PCSS. He is not involved in arrangements with PCSS on student transportation. He Stated that the Department of Transportation with PCSS handles student transportation. He is not aware of or involved in any other payments to PCSS. When asked about the management of input into the SASI system, Mr. Waller slated: Ms. White has been at Parks for over one year as Secretary to the Assistant and SASI input coordinator (all). Mr. Waller stated that Ms. White is the SASI coordinator. When asked how could Ms. White still be SASI Coordinator without access to the SASI system, Mr. Waller stated that her privileges had been removed because of an Inappropriate actions and insubordination with Ms. Thompson. Mr. Waller stated that Ms. Thompson is now responsible for input. He was not sure of Ms. White's weekly hours, but thought they were originally 2? hours per week. When asked about her current hours, he was aware they had been reduced to 27 or 28.4 When asked why her hours were reduced he stated to be in line with Title One rules. Yes, the original end cuscnt "redL|cl1nu" is inconsiqent Proprietary inticrmation for Atlanta Pnlniic School System - (limes of Internal Resolution, compiled by Phoenis: lnvresligaatinn This is xi stuiernent as aeorney-client work product. 668 06682 12 Interview of Grgog Reid_ Investigators Reginal Dukes and Marc Lawson met with Assista nt Principal Gregory Reid on March 1, 2006 at Parlea -viiddle School. Mr. Reid provided the following information regarding his role with the SES program provider Preferred Care School Services: Reid advised that he started with PCSS at the beginning ofthe program. He was unclear as to whether that was October or November of 2005. Mr. Reid stated that he was the site coordinator and was paid $50 per hour for that role (he knew that teachers were paid $25/hour). He stated his responsibilities included: - Asigning class rolls Assigning students to classes - Determining which teadner was assigned for what subject Observing classes Team teaching as neeced Filling in for teachers as needed Mr, Reid advised he was not responsible for: Reports to parents - Student progress reports Attendanoa records for students or teachers. I Fe advised that Mr. Selmon assigned tlris task to Ms. Thompson and/or MS. White. Mr. Reid stated that 30-60 students participated In the program on any given day. 0 l-Ee advised that Dr. Kiel provided tutoring before and after school under the PCSS program. When asked about the role of Ms. Sandra Ward, he advised that they worked together but he was not aware of her ofncial title. Mr. Reid acknowledged that he did not attend the APS orientation on the SES program. When asked about the December lncendve drawing for PCSS student participants, Mr. Reid stated he was not involved in any way not in the detemiinatlon of the students included, the drawing, or the gifts. He stated he was not present for the drawing. Mr. Reid was re-interviewed on May 11"' regarding Ms. Sajuana Wh?te's additional allegations (see page 21 in tab 12). Mr. Reid advised that: He was not aware that her SASI duties had changed, but had directed Ms. Vi/hite to help with the year-end cleaning out of closets (see tab 15), He has never had a discussion with anyone at GDOL or otherwise about isis. Vi/hite's social security number or her attempts for unemployment during the summers (see tab 16). Private Proprietary for the Atlanta Public School System oflnterasl Resolution, compiled by Phoerrb; and kescurcll. This is a otudidcriliul sleierneni ng gnome;-client product 669 06683- 13 Intenriew of Sandra Ward Investigators Reginal Dukes and Marc Lawson met with Ms. Sandra Ward, SFA Facilitator, on Februan; 16, 2006 at Parks Middle School. Ms. Ward provided the following information on her rolerwith the SES program provider Preferred Care School Sen/ioes: Ms. Ward started with the program as a result of Mr. Waller sending her to the APS orientation session forthe SES program. Ms. Ward stated that she is not employed bv PCSS and is not paid by PCSS. She was aware that certified teachers were paid $25/hour forgtuizoring and that non~ certified persons were paid $15/hour for program support. She stated her "role" has been: I To bring back the information from the orientation session to the school Give forms to teachers for distribution to students for participation ir the program, a parental permission form and list of providers I l-landing out application forms to teachers for participation as tutors with P355 I base with Ms. White on program operations Interfacing with Mr. Salmon at PCSS Ms. Ward stated that her role did not include: Giving instructions to teachers on the programs operation Elelng present during program hours She advised tl1at70-98 students participated in the program. When asked about the role of Mr. Reid, she advised that he was the program coordinator. It was her understanding that his duties were to - Oversee the building, including monitoring and walking around - Bad<-field as tutor ln math when needed Help Ms. white Ms. Ward said she felt the teachers were not being asked to do anything different from their daily duties as certined teachers and did not see the need to go into details on program operation. When asked about the roie of Ms. White, she advised that Ms. White was responsible for: Overseeing the program Giving out snacks Making sure that transitions took place in the building Ms. Ward was asked about her knowledge of documentation previously requested by Investigator Dukes on two occasions from Mr. Waller: 1 Student sign-in sheets for attendance, inc'uding athletes - Original SES provider list - Forms showing the selection of PCSS by parents List of all teacher applications and acknowledgment letters to teachers for participation Original time sheets for teachers and Blank application tn teachers When asked why she participated in the program wiihout compensation, Ms. Ward stated that she knew when she changed schools that she would need to help out without oompensation for all support given in order to help the school improve. She stated that the school needs support. Private Proprietary information for the Atlar1ta`Public Oitice oflntomal Rosoluiion. Compiled by Phoenix This is as 670 06684 14 Interviews with Fabiola Aurelien Investigators Reginal Dukes and Marc Lawson met with Ms. Fabiola Aureiein on February 12"' at her home and on February 17" at Parks Middle School. Ms. Aurelien provided the following information on her role with the SES program provider Preferred Care School Services: Ms. Aurelein stated that she has been a Math teacher at Parks Middle School for two years. She stated her role with PCSS was as a math- tutor. Ms. Aurelein stated that she thought there were L5 teachers sewing as t|.tors. She stated that teachers signed-in with Ms. White (and later Ms. Thompson) at the beginning ofthe program each day. Ms. Aurelein stated that she was not aware of any direct supervision of teachers in the program. Ms. Aurelein was of the understanding that Ms. Sandra Ward was the program coordinator, until she was recently chastisecl for calling Ms. Ward the program coordinator. She stated that she had earlier shared concerns with Ms. Ward about the program: - Never receiving request for progress reports in December 2005 Asked Lo provide progress reports in February 2506 when she did not have the same students each clay Never advised bo provide documentation of student attendance 0 All participating teachers not Included in meetings to review program prooedures Ms. Aurelien stated that she was called into a meeting with Ms. Sandra Ward and Selman on Friday, February 10"' to terminate her services with PCSS since she was paid but did not tutor students, because she would not provide progress reports, She stated that she advised them she did tutor, but could not do progress reports since she did not see the same students consistendv. Ms. Aurelien stated that Ms. Ward took the led in the rneeting and was very harsh with her. She stated that she was confused on her status because at the end ofthe meeting Mr. Selmon asked her not to "reSign." During the Februan/ Ms. Aurelien stated that at some point Mr. Waller had attempted to solicit her as program coordinator to "get things straightened out." Ms. Aurelien stated that she was a member of the design team and did not tutor on Mondays when the design team met, but did sign~in for PCSS on Mondays along with other teachers. She stated that Ms. Vl/ard was aware of this practice since she was also a rnember of the design team and in the room when the PCSS sign-in sheet was brought into the design team meeting for teachers to sign-in. When asked about allegations of cheating on the w-iting test, Ms. Aurelien stated that she was Included in a teacher meedng about this subject. At that meeting all persons were asked to sign a sheet on whether they were aware ofthe allegations? Afterwards she was individually called into Mr. Wallefs office as part of his investigation into the allegations. Mr. Walter had a tape recorder. Ms. Aureiien stated that she wanted someone else in the room as a witness and Mr. Waller called in Dr. Kiel. She then answered Mr. questions. The conversation was not taped. After Dr. Kie? left the room, Wailer asked some hypothetical questions. Ms. 5 l\/ls. Aurolicn was the only person present the? admitcedto lzearing, about rite. allegations. Prix-atc Proprietary information for the Atlanta Pubiic School System Utica uflniernal Resolution. compiled by Phoenix lnvestigatiorr and This irr rr uurrlidurrliall Stair-Trent oftindiirgs as uterus)--client work product. 671 06685 15 Aurelien tnought he was trying to see if he could count on her not to say certain things that would incriminate him Waller). MS. Aurelien also Stated that an unscheduled Staff meeting was held in which the Principal stated that know there are people who are unhappy here and take care of making sure that these people go on their way." V\/hen asked about how someone could access the test prior to test time, Ms. Aurelien stated that there is general access to Dr. ofhce and that the tests are not sealed when the teachers come to pick them up. She also stated that the week before the test teachers where given specihc instructions on how to prepare the students for the writing test. Private ?roprie1ery information for the Atlanta l'ub`ic Sch:-nl System Office oflntemal Resolution compiled by Phoenix lnvesriggilion find This is El Confidelirlnl Statement nf findings as atoorncy-clicni wcult pmduot. 672 06686 16 Interview with Anthony; Tiller Investigators Reglnal Dukes and Marc Lawson met with Mr. Anthony Tiller on March 1, 2006 at Parks Middle School. Mr. Tiller provided the following information on his role \-vith the SES program provider Preferred Care School Services: Mr. `lTller is the in-school suspension coordinator. He stated that his role with PCSS was 50% as a math tutor and 50% support. His support duties included monitoring the halls, securing the doors, and escorting students to the bus. Mr. Tiller stated that his rate of pay was $25/ hou r. When asked about program operations, Mr. Tiller stated that he understood Ms, Sandra Ward to be the site coordinator. He stated that he interviewed forthe position vv?th Ms. Ward and Mr. Waller. His cerrent understanding is that Ms. Thompson has replaced Ms. Ward as slte coordinator. He that 5-8 teachers participate as tutors and an average of 20-25 students participated in the program on a daily basis. Mr. Tiller stated that he has not done any student performance reports. Private >in IHHTE "Gold cnvciopu" gives tu Jaekszir: Nafw in5:t?7? missing when returned from iilness Lost 77? Prwaae Prapxictaiy Rn" the School Sysv;n1 C-'Ente oflnrerufd Resciutioni compilfzd byPl1oe|\ix and ?lesea1'ri1_ This is a iindings as 680 06594 Page 1 of 2 White, Su'uana From: Kaltman, Meredith Sent: Mun 10:23 AIV Ta: White, Sujuana Cc: Waller, Christopher Subject: RE: Attachm ents: S. tl ite- Of course I rernernhor you. I'n1 sorry that you have had such difficulties, but I don't know how much I can help you. There are 3 issues thati do want to address with you. First, you have made some serious allegations in your message. I have spoken with Mr. Waller to tg: to gain some clarity onthe situation, and have copied him on this email. Second, Rhonda was doing herjob as she has been told. Mr. Waller has full controi over who has access to the SASI data at the school. He tells us who can have access and Whose access should he restricted. We just complete the actions as he requests. It you have any questions about that, please talk with him. Finally, as I said, these are very serious allegations you have made. I need specific incidents of illegaE activities and talsilication of records or data. Please send me dates that these events occurred along with the names of the people involved and the exact description of what occurred. The sooner I have that information, the sooner I can take action. Thank you for your cooperation in resolving these issues. Meredith Kaltman - Student information Support Manager Atlanta Pubiic Schools 681 06695 Page 2 of 2 130 Trinity Avenue 404-802-2735 From: White, Sujuana Sent: Monday, April 10, 2005 10:02 AM TD: Kaltman, Itlareclith Subject: Importance: lligh Good rrorning You may not remember my name out once you read this email you will remember me and my situation, This is Sujuarla over at Parks ll/lidcl'e School. have been having issueswith so many inexperienced Ltsers with SASI at my school. C-or attendance is the and mam issue, The teachers are taking tile attendance ac:curatel'y and daily. However, I have no?oecl an increase: in absences which lets rne know that the attendance is lzieing taken because we [you and l) know that ifthe teachers do not open up their classxp llc- take the attendance it aftomatioally makes the student present At this point my SASI rights have been taken away from lm and given to someone whom my principal calls a team players (I can and will elaborate more with you in person) Now am seeing a lot of illegal things happening here and am lying low because feel my job is on the line but am boing Forced to call this to your attention, My SASJI rights were snatched away on Friday, March 31, 2006 and I found out when lcalied it in to clientsuoport. Ethen called Rhonda tl/lali< who is very familiar with all of my concerns and she was nonchalant about it and fold me that she got the order to Cancel my SASI rights from my principal on Thursday and mat she did not know what was going on. linen went lu aim (my principal) and asked was there a reason for me not being able to access SASI . He then told me he would talk to me later on about that However, we have people using SASI log in and posswor-:is ofpeople who no longer work nere and/or have been gone for whatever reasons (cut sick for months) There are even issues wah FTE and omer programs and information that has been falsiied_ A1 this point I dont know what you can do but just wanted to share this information with you to make me reel better. Please contact me as soo.-i as you are available to discuss my conoems unEl then 3 am waiting patiently with our discipline building and me not having Lx clue as tu when and what tu do, All school year have been cf Rhonda Maiik ana har advice and support but at this point don't trust anyone and need some clearance, apologize for such a long email but I am so overwhelmed by what l'm seeing and experiencing until had to search further than Rhonda, locl-< forward to hearing from and/or meeting with you soon. 682 06695 ZGOLQ my dum. fm 6 8 3 Rl <51 Mm? QQ ixamm ?fuhlmk mf; fum; hm 0% .WW OXWLW Qi? Ls Rf Mm w?ifvua? sw . M5 diwg @m f' gm QM im lm), Uh 685 2. MWF aw aww A ?W0J\~fc} ml A KL if, -Js@i~ 1>@Sm@ QQ wwlu ms giim MLQQ QL wc gm fig bg, gag/v Mi? Ci ww "k?wm mwwi?l own Qi Jt9m_ Q52 ATN MQW 2% ha 687 mm 24 Interview with Alfred Kiel Investigators Reglnal Dukes and Marc Lawson met with Dr. Alfred Kiel on March 1, 2006 at Parks Middle School. Dr. Kiel provided the following information on his role with the SES program provider Preferred Care School Services: Dr. Kiel stated that he is an Instructional Learning Specialist at Parks Middle School, which includes responsibilities for admiriishaiion of the CRT test Dr. Kiel advised he had been with the PCSS program since the beginning in September or October of 2005. He stated that his role in the program was as a tutor primarily for the athletes before 0:30 am to 8:30 am) and after (4:30 pro to 5:30 pm) school under the PCSS contract Dr. Kiel advised that the athletes came to the tutoring sessions before sports practice in the afternoon and their time in the program depended on the sports practice schedule. Dr. Kiel stated that he did aiso tutor non-athletes after school. He stated he did not know what his hourly rate was for this sen/ice. asked about program operations, Dr. Kiel stated that he understood that Ms. Sandra Ward was the program coordinator. Dr. Kiel was also questioned about his role in the administration of the CRT test He stated that he receives sealed packets with the test and maintains them in a locked safe until they are distributed to teachers. Dr. Kiel stated there is not a study guide for the test and that no one other than himself has access to the tests before he distributes them to the teachers. Dr. Kiel stated that no one has approached him about advance access to the test. He stated that he breaks the seal of the test packets the day before the tests are to be administered to sort and label them for distribution to the teachers. When asked about rumors of cheating on the test, Dr. Kiei stated that he had heard those rumors. He also stated that he heard that irrformaton about the test was available on the Internet. When asked about teacher statements that a Study guide was provided for preparing students for the test, Dr. ide! stated that he had not seen nor distributed that document. Pri\-me Proprietary infarrnarion forthe Atlanta Public School System Office Resolution, compiled by Phoenix li\\/esligaitioii and This is a statement of findings as attorney'-clioril work product. 689 06703 25 Follow-ug Interviews with 86' Grade Teachers Investigator Reginal Dukes did follow-up with the grade teachers regarding the Sl" grade writing test. On May individual interviews were held with teachers: Damon Arnold, Draper, and Doreadw Wilson. All three stated that they had seen and received the tips in a grade level meeting at Perks Middle School, We were not able to interview Dorian Rodgers, who no longer works at Parks. We were not able to interview Tanzy Lewis because she is on maternity leave. Private 84 information for due Atlanta Public Schnul System Office Resolution. compiled by Elhoenix investigation and Thix is a c-nnfidenhal sraitcmviil nf finrlin gs as atromey-client work product 690 06704 Z6 Interview with Students at Parks Middle School Investigator Reginal Dukes met indb/lclually with 13 students on March 1, 2006 at Parks Middle STUDENT NAMES REDACTED The students were asked about two issues: their participation in the December rattle and their knowledge about a tip sheet used in preparation forthe CRT writing test. School: December Raffle All students were aware ofthe raffle. Two students stated they were not included in the drawing because of the low attendance in the program. One student said they were included in the drawing even though they had low attendance, but did not win a prize. The majority of the students stated they were not consistent En attending the tutoring sessions, with the majority attending less than 25% ofthe time. Severar students remembered being called into the ofhce or approached during class to sign a spreadsheet. They said they did as they were toid and inltialed all boxes indicated by their names. Those who went to the office stated MS. Ward or Ms. White were the persons with the spreadsheet. Writing Test All students were aware of the tip sheet for the writing test. Most stated that they were directed to practice responding to question #7 on the tip sheet in preparation for the writing - test. They also acknowledged that a very slmliar question was on the test. Some students stated that they received the 'lip sheet before the Christmas holidays and were told to study over the holidays, primarily those who received the tip sheet from Mr. Rodgers. Other students stated they received the tip sheet a week or so before the test. When asked who gave them the tip sheet, all students stated they received the tip sheet from tneir language arts instructors (Mr. Rodgers mentioned most often, also Ms. Lewis and Ms. Wilson). One student did state that they were recently called into a meeting in the cafeteria \-vith Mr. Reid and Ms. Holloway about new tutorial sessions on Saturday. The student stated that this investigation was not discussed in that meeting. Prim-'ate ?ropr-letary Rn' be Atlanta Public School System Office oflniemal Resolution. compied hy Plroenis and R?semtfh. "l"lr?s is za of Endings as attorney-client work 691 06705 27 Interview with Mr. Pitts Investigator Reginal Dukes met with Mr. Pitts on January 17, 2006 at his ofnce. The focus of this interview was his meeting with Ms. Vicki Johnson and Principal Waller regarding the distribution of the ATF holiday flyer at Parks Middle School. Mr. Pitts provided the following information: Mr. Pitts stated that he visit Parks Middle School for a meeting. During that meeting he stated that Mr. Wal er made him aware ofthe flyer and asked whether he (Mr. Pitts) thought it was appropriate for distribution at the school. Mr. Pitts stated that he not it was appropriate to distribute the flyer with the handwritten message at the top for a meeting to discuss Issues at the school, he think it was appropriate for advertising a holiday party. Mr, Pitts stated that he advised Mr. Wal'er to call ln the ATF representative to the meeting to discuss the flyer. Mr. Pitts stated that Ms. Young did come to the meeting and when he questioned her about the handwritten note Ms. Young stated that she did not write the note. He also stated that he advised her she Could tile a level 2 grievance with him if there were major concerns at the school. Mr. Pitts stated when he asked her if she had any concems about the school. Ms. Young did not state any. "She not say a word." When asked if any subsequent concerns about Parks Middle School have come to his attention since the flyer, Mr. Pitts noted only one concern regarding teacher evaluations. He stated this was not within his scope of responsibilities. Mr. Pitts stated that he did Implement a survey after the flyer incident and that survey was "pretty positive." He did not provide a copy of the survey c-r the results. Private St Pmprietnry inferination for the Atlanta Public School System -- CFfice oflntemal Res-elution, oompiied by Phoeni;~; mul _Rnsezu-uh, This is auoizfiilentinl as uilornvy'-cliuill wud; product. 692 AF 06706 ry School Comparison for 2008 to 2009 CRCT Eleme CRCT. Un is equal Lo a decline in performance RE rmance or higher) on CRCT. performance [193-3%} on CF.CTEMF. i ii. ual to constam perfurmance an CRCT. Orange is eq Black Font 25% nr more of the classes within the grade-level are flagged. as sections af #ui wil:|1 Classes Classes Ta YP A ur more jf m_v\rrn Clas SRT Sd\nnlNema . A NT -.ff-_f_ "'l'3"Y3 ssl.gat gk (9 3 yvwm? lr-' mx uv :gg *Ei sr -Wd'-arf- rv, .r 1" Blain fl me .K . . ~v -s-wel "2 "5 -il, 5=_si "Qi __f-5-'-nflJkgxe :never sn- -V-fe. - fs, i 1 57 fry: _s ,assi fswr! c-mn el; 4, _J-li, .1 -i _f . . sA_-s -E sa. ge, ass :asa apaageag; 2 -f 2 r- 2332322533213 E53 35332253 nflasei is; gg mo; - mi- gs., ilusmgs ?525= BEE 693 GOSA AFS MCKEE 000003 upderedz z/18/10 qu MQ QQ mm mtg' 9? D532 viva _:`11_ `z1 9 ,aw 1 1.zs_, 4' nf ,jgj 6 _11f '11_ -L "5 zz\ 5 lb) 8 784% 4U 51 4 USHER ELEMENT wr In 3325 mm me EE mm mm :ana gong DSS uH?U_umE_ gm 3m__Um& A 4_1 _m_Hm_mm "_Aim uzg? 5 E: H5 N: in _uni Nm _Nw Em_?nV_q 'can Wahine" mamma _Em in asa ua 2_2 Ea _Emgc QE M5 mm_mWN_u U5 ECE gm EE _Ego _Lo Gag mu: _gsm mmcmo no ?m?E _gum 520 5 $3 8 _g 5 505 igk _Emu wU?_E?tg 2 _gum _gon OM WSN _anim Charter CRCT Cumplrison for ZIJDS tu EDU9- ll nm;= mn. secm-UNIVERSITY CO GOSA APS MCKEE UOOCID1 GOSA APS 000002 Moy 70| T~loTe To Hle RE: ln The Maher of Parks Middle School- May 5, 2006 Afler reviewing The grade TesT ollegailon To deierrnine il This porlion ol The inveshgolion was properly disposed, I found The following. spoke wiTh Ray Hari, Executive Direclor of Research, Planning and A_ccounTabiliTy, as he is considered in~house subjecl maTTer expert regarding assessments. Dr. Hari provided copy oi The 2005-2066 SysTemwlde Tesiing Calendar and noied The 2006 Sli" grade wriTing exam was odminisierea .lariuaryl8-l9, 2006. Dr. I-larT sTaTed ThaT TesTing maielials are Typically seni i week prior To The adnTinisTraTior1 of a sTandorclized TesT and iT is highly unlikely Thai TesT maierials Tor This lesi would have been senT To schools prior To The ChrisTmas break. Addilionally, Dr. Hari will Search his office Tor documenls which may outline The exacl daTes APS received TesTing rvaierials during This school year. Given Thai iT is highly unlikely Thai Testing maTerials were dislrilouled la APS schools prior To The Chrisirnas break for This exam, Mr. Dorian Rodgers, The Teacher in aue5Tion, would nol have been ohle To provide wiih The acTual wriilng TesT prompi. Dr_ Harl rnenTioned lhaT iT is irnporlanT To nole "coaching" sTudenTs could include Teachers accessing G-OAS and coaching sludenls on sample wriTing prompls approved by The Georgia DeparTrnenT of Educalion. "Coaching" sludenis on pre approved sample W|iTing TesT prompTs is noi considered a TesT inegulariiy. Nicole Lawson Special AssisTani CHRO 698 AFS-OIR C6713 1.9 Ocurbcr I2 October l7~21 Unlober l7>>21January 3 -February 25 January l7-21 larurury Ill-19 Ianuary I8-20 Fehrunn' 20424 ular Calendar Atlanta Public Schools 2005-06 Systemwide Testing Calendar "gust 15-Z6 I2-l 6 0 September 19-23 za . March I ~Apri[ 30 March I - . March 2 o-lu. lfi, I4 - Mrrerr zu-24 - March 3 I I April 3-1 - Aprn I3 mi I4 . April I3 April 15,26 April 14~M1y 5 . Georgia Kindergurren 1 evisad (GCAP-R) Window 1 renzrred un August 25), llelest ibr Guargrh High School' Uradsra-iiuvl Zilla! in urriui ivy day lmidunirlg Monday: /mx Mathematics, Suuiul Studies, and Science ineluriirrg ma\re~up testing Rerest for eligible sturiems who did nur piss uf misled the test previously. Georgia Ermic in reirdlsilg lm] mall). Ailninlslur nnly In who qualify, eleerrcniz: make-up fur ahrdenm missed Spring 2005 tesr. Georgia IHgir School Teri i.r| Wriimg fur grade ll eludenis and eligible sludenls who did nut pas nr previous leslillg. Gem-git: Barre Skilk Tw in Wrirbrg (No make-up]. Admims-vcr nnly lo fnrmer studenrs wha qualify. Gear ir1Hi Schunl Gmdualien Terr in Wrilin {MilkB>>u PSAT (Grades 10 I/uri I il EOCT elecirnnir: make-up for who missed Spring 2005 tm IEES in Gra?er 3 5, and if decision is made for fiall teslirw instead Gmgu mgr. sr/mi mr In nrder iv? day beginning an Monday: Are. sneak" Studies, and Science including makeup testing. Retest for eligible students wha did not ur mg ml previously. End qr'C.'aur.rs Term Week one beginning nu Tuesday with nine les! per day; 'l`uesduy, Amencnn gm] Composition; \Verlnesday_ 9" Grade Lneramre and Composition; Thursday, Economics; and Friday, s_ lqiggon-_ Seir:nce_ daynn lirirlary and Mrmdiy, Georgia Aszessmenr Revised (GKAP-R) Window 2 make-up for srudenis wha missed December Georg.\a Mlm; Grades ll nnly). Terran Wednesday Ind lnrirefup nn . ,ez Wrifir Tm nz Grade 5, Timm Wedneszla with make- cn Yrid EDCI electronic srurlelnx m??rrd_De?_ember EOC1 . Seennd week heginnhg on lr-Iunday; Algebra Tuesday, Genmezry; Wednesday, Biology; and Thursday, Physical . . Georgia Kinds-rgurien Asxesrmeut Revised (GKAP-R) Window 3 Gecryna High School lbs! fm Uligibli Wm did F95 UF 'lid DDI D11-U I-hu l:2iL ggi?-Eguric Skill: Test in Wririrg fur eligible sludents (Nn nreke-up). Administer nnly In tin-mer srudearts who qualify. Georgia High School Graduariau Teri ie Writing [Make-up; Teal (FFBS) Gnrius 3, 5. and il, Alsn Terra Nova lar idmlilied smrlenls in grades Glhed Sureenmg and ESOL Program exit" Hu-'r Kehuul Graduatiuw Tru in br dey b=ei1\Ili\1e Murder" AHS, Social Studies and Science including n:ake~up Leaking. Georgia Basic readmg and math, Arirrinirler nnly to farmer students who qualify. Geargia Gnd: 3 Aueuwrenr Schools remm raring [brms ccunpleteri by grad: 3 members. spring Break REP 'Testi in Mali; fur 9 and IE- REP yzartiuipulris Georgia Kimfergarlen .4.nes.smen1 Program Revised (GMP-R). Schoais resum _#inns completed by - Georgia Campeteney Tu! (CRCT) ii grades I through ll and marh, and in 3-ll, xeicnuu social mldies, Georgia End qfCaur.n: Tests QOCU Week one wirh one lest per dey; Monday, Literature md Culnposillcn; Tuesday, Biu;0gy_ U, S, Hiennry, Thursday, Economizs with Friday dry. Wrelr (wa: Geometry, 'l'uesday, Algebra l; Werlnesrlay, Physical Science; and Thursdag-, 9* Grab Lireralure and Composilion wiLlr Friday make-up day. None: The second week ofl?.OCT will overlap wid-| week ufAd\~sneed [AP)Ex>>1ms, Am,/May . English newsupmenrussessmurr (ELDA) (Dates ta hr M1134 5 I Lernrl Guor GAA) Forms Retumed ar . . il", iz - Summer 2006 Dates rGra\n'es 3 and 5 .rmdem do nm mu: .riandurrie Dam: za be riviznrrinedfl Georgrh High Schual Granimlim Teal in Writing July 17-20 - Georgia Higi>>Sc/mai Graduurrkm Tm! in by ?ay: Ars, Science, Sucizl Studies, Retesrfur eligible students wha :Lid nor page or mixed the ls-larch 2004 ariminisirarimr, Ju End qfCaurse Tears [Tn be adruinlsrererl during high snhnuE summu sehnoL) _wr/Ik #2943 Rermrh, Firming emi 699 AFS-OER- 0671 4 Page 1 of2 Atl IKON From: Penn Payne [ppayne@pennpayne.oom? Sent: Friday, To: Rush S. Smith Subject: FW: Privileged and Gonndential -Attomey>>C|ient Communication Attachments: draft letter responding to GOSA Y-2-O9 PP track changeadoc Penn Payne Penn Payne, LLC Arbitration, Mediation and Investigation 3556 Tuxedo Park NW Atlanta, Georgia 30305 404.841 3295 phone 404.541.3296 tax ppayne@pennpayne.oom h.-t NOTICE: This e-mail transmission (arid all included attachmenis) may oonialn information which is conhdential and legally privileged. This information is intended solely forthe use ofthe addressee. it the reader of this message is not an intended recipient you are hereby notified that any reading, dissemination, distlinution. copying, or other use of this message or its attachments is strictiy prohibited. If you have received this messago in error, please notify the sender immediately by telephone (404-84'l-3295) or by electronic mail gQayne@Qe|3npayne coml and delete this message and all copies and backups thereof. IRS Circular 230 disclosure: To ensure compliance with requirements imposed by the IRS, we inform you that any U.S. federal tex advice contained in this communication (including any attachments) is not intended :ir written to be used, and cannot be used, forthe purpose of (ii avoiding penalties under the Internal Revenue Code or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. From: Penn Payne Sent: Thursday, July U2, 2009 3:24 PM To: Subject: Privileged and Confidential - Attorney-Client Communication Dear Dr. Augustine and Mr. McKee, Attached is my mark-up ofthe letter we discussed today, I have also made some comments in the margin. If you are not familiar with the Track Changes function of Word, please be aware that anyone who had an electronic version of this letter, even after Track Changes is turned off, could see my edits and comments, So I would recommend that any changes to the letter be re>>typed into the original version ofthe letter before I added the changes. I made changes only to the text that was "assigned" to me -I believe Lester will make additional changes. I have not copied Jett Schiller on this email because, upon reflection, I am concerned that sending my draft of the letter along with my comments mightwaive the attomey-client privilege. Unless he is an attorney with an attomey-client relationship with APS, he might not be covered as a consultant leave it up to you to decide how to communicate the content ofthe letter to him. Please let me know if you have any questions. Penn Penn Payne 6/10/2011 Payne_ 67 700 Page 2 of 2 PENN I. .l MEDEATTON AND 3586 Tuxedo Pant NW. Atlanta, GA 30305 (404) 84143295 (Pho ne) (404) B41-3296 (Fax) ppayne@pennpayne.com NOTICE: This e-mail transmission (nd all included attachments) may contain information which is confidential and legally privileged. This information is intended solely for the use of the addressee. If the reader of this message is not an intended recipient, you are hereby notiied that any reading, dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited, lf you have received this message in error, please notify the sender immediately by telephone (404-841-3295) or by electronic rnail and delete this message and all copies and backups thereof. IRS Circular 230 disclosure: To ensure compliance with requirements imposed by the IRS, we inform you that any U.S. federal tax advice contained ln this communication (including any attachments) is not intended or written to be used, and cannot be used, for the purpose of (ij avoiding penalties under the intemal Revenue Code or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. Payne_ 701 tions offered to the State Board of Delelell: online an fmui-gs in nn _l Atherton Elementary School use 1 where nal adinllsiom Mguilrhna been ered off oe|13ns11= Deluli?dl such admission I genres: 11111 This letter is in response to recommen a Education regarding alleged testing viniatinns al Docrwuud Academy during the 1' . zoos summer CRCT Raed. I "1 As a consequence ofan internal review (may put in mme process points here), the _1,f_1_1 district strongly disagrees the allegations, the proposed sanctions to be applied _1 . and the merits uf the Slal,c'? :agonist the Atlanta Public Schools and Deerwood '1 Academy. It appears that the basis for the case rests with the statistical probs-1l.111l.ny of cheating and not specific incidence of clieanng. ?cilhcr Liist1jgl_ -,Lf personnel nor voluntieiihave ad mitted to any tamperinv with or alteration of unnamed: 11111111111 11.for n?1?1111 11. 1111n.1n1.,.,111.1. n1zuui_ Tilt statistical analysis 01| Student papers based on erasurcs cannot: Formatted: prove, as alleged, that an individual was responsible for changing answersthe school, school administrators or the testing 5 wen an nr *ggi 1-1 - Q-1 We mai-afailure reportindigates if that Deerwood experienced an average gain of 41.2 scale score points on the CRCT retest to the main adlninistralion. However, nne Student had a 31 scale 127115321 score point decrease. Ifthe average gain had been calculated correctly, the average 1_1 gain wonld hc 39.2 and would translate into 3.3 Standard Deviations (SD) above the 'v 1. 111111111 1111 state average (of16.4 with a SD of 7). If the Governor Oftice of Student ff 1 11. Achievement used an average gain that was 3.5 SD above the state average, 1' - 11 vulunleerk 1 mind 111111 then would not have mel the established criteria and would not have been fi 1 1 1 considered for further investigation. 'v 1 :1 I I 11n111|1= I . The district has retained an external investigator to conduct an investigation into 'l i 11 wuhthe the alle ation in GOSA rc art that someone had access lo test 1 1 111.111 on 11111 . 111 1 d~ materials after testing concluded changed multiple students' answers on the 5 1 1 1 The ggde CRCT at Deerwood Academy. That inveshvatinn is currently and I _1 111. on 1111111 111 rn. 1111 has not vet been At this stave, however, the investigalion has not I 1 1, uncovered evidence that someone actually altered students' RIISWBQ if d11?11m=1n=1. n11?n>> the mme 11fnn1? P1-111x111=n| pre-id labels, the i on unch answer document Finally, as a regional training pa and host site for Tcan-_h for mnerica (TFA), "Fl pi-111: to 1111: testing pm-11,1i_ 11 prepares aspiring teachers for their school placements by matching meh TFA 1 teacher with a veteran APS Faculty Adviser and providing each teacher pair with wmtrupensiniliq fm- ?111111n111n Uu- student plans based on content and domain area weaknesses. The secure 11>>a1111n 11 teacher pains then deliver targeted instruction designed to uniquely and with the Payne-_ 702 tactically support students to achieve passing scores on the retest during the summer. Given al] the above, the district must disagree with the alleged findings and believes that no consequences should he exncted on Deerwoorl Academy. Decrwood Academy should retain its AYP status and a formal recommendation and acknowledgement nfihis fact slmuld he offered to district by the State Board and Georgia Department of Education. With respect to district-level actions outlined in the 1-eport, Deerwood Amdemy has already communicated with all of its parents/guardians regarding the GOSA allegations and findings and has agreed to keep parents informed as the investigation mova forward. The dis'trict's Comprehensive School Reform agenda requires differentiated instruction based on each student's current academic performance and prior academic history, and as such, the district believes that adequate support will be oifered to the middle school students who have left Deerwood Academy. The district is also engaged in an intensive Math Science reform, funded by the GE Foundation and all math and science teachers are involved in intense professional development to realize mastery of the GPS standards and all related tasks in these areas. These efforts should adequately address the concerns regarding student preparation and success. Vi/'e anxiously await your review ofthe facts and are prepared to discuss these matters as appropriate. Thank you. Payne 0004170 703 'Pag? ua The report also refers to loose security due in part to volunteers supporting the school during the summer. What the report fails to document was that the referenced volunteer is a retired teacher who had conti.uua_lly provided support as a substitute teacher at Deerwood Academy during the school year. District personnel suggested that the volunteer in question was not involved with the proctoring or testing of summer school students and did not have direct access to answer documents. The volunteer supported the test coordinator prior to the test administration by huhhling in student demographic information on answer documents. Due to the absence of state produced student-level CRCT retest pre-id labels, the demographic information on each answer document had to be completed in its entirety prior to the testing period. Also, the report suggests that no one took responsibility for ensuring the materials were in a secure location at all times. The district strongly disagrees. This statement is in direct contradiction with the Principal Certification Form signed by both the Summer School Site Coordinator and Test Coordinator. This APS form {proyided to us by the state) specifically requires certification that the materials were secure at all times. No school monitoring reports or testimony exist to support a breach in district or school-level protocols. Payne_ 704 UE5ce ol' the Superintendent 1 A Beverly Hall, nw Phmennportnrennont 332233353 Making A Difference July 6, 2009 Ms. Kathleen Boyle Mathers Director of External Relations Oilice of Student Achievement 1554 Twin Towers East Atlanta., GA 30334 Dear Ms Mathers; This letter is in response to recommendations offered by the Govemofs Office of Student Achievement tn the State Board of Education regarding alleged testing violations at Deerwoocl Academy during the 2008 Summer CRC'1`Retest. The district retained the services of on external investigator to conduct an investigation into tltB"~? allegation in the GOSA audit report that "someone who had access to test materials after testing concluded multiple sturlents' answers on the grade CRCT nt Docrwood Academy." The portion of that investigation focusing onthe "cheating charge" is completed and concludes tltat there is no evidence, no basis in fact, that someone actually altered students' answers. Neither district personnel nor volunteers at the school have admitted to tampering with or altering CRCT answer ctocttments, Based on time external investigation, and the district's thorough internal investigation, the district strongly disagrees with the allegations, the proposed sanctions and :lie merits of the State's case against the Atlanta Public Schools and Deerwood Academy. lt appears that the basis for the State's case rests with the statistical probability of cheating and not a specific incidence of cheating. Neither district personnel nor volunteers at the school have admitted to tampering with or altering CRCT answer docuzrlertts. The statistical analysis pertlorrned on smdent papers based on erasnres cannot prove, as alleged, that an individual was responsible for changing student answers. In fact, then: was no motive to change the answer sheets. It was not in the interest ofthe school, the school administrators or the testing coordinator to lakethe extraordinary risk of amending student answer documents, when no tangible harm would have resulted from a one-year failure to achieve AYP in 2008 und no tangible benefit would have been lost from that one-year failure. The GOSA report fails to mention the rigorous academic and teachers' efforts that we are convinced are at the heart of the gains. Further, the report indicates that Deerwood experienced an average gain of 41.2 scale score points on the CRCT retest compared to the main administration. However, one Aclancs 'l'rlnicyAvenus S.VV. A:lenba,GA 30303 Payne 00158 705 Ms. Kathleen Boyle Mathers July 6, 2009 Page 2 student had a 31 scale score point decrease, and the average gain should be 39.2 and would translate into 3.3 Standard Deviations (SD) above the state average (of 16.4 with a SD of 7). If Govemcfs Ofliee of Student Achievement used tm average gain that was 3.5 SD above the state average as grounds for review, then Dcerwood does not meet the established criteria for further investigation. Given the above, the district must disagree with the alleged findings and believes that no consequences should be exacted on Deerwood Academy. Deerwood Academy should retain its AYP status. With respect to recommended district-level actions outtined in the report, Deer-wood Academy has already communicated with all of its parents/guardians regarding the GOSA allegations and Endings and has agreed to keep parents as the investigation moves forward The issue of student support will lie adequately addressed through the Comprehensive School Reform agenda that requires differentiated instruction based on each siuder:t's current academic performance and prior academic history, As such, the district believes that adequate support will be oifered to the middle school students who have leti Deerwood Academy. The district is also engaged in an intmsive Math Science reform, funded by the GE Foundation and all math and science teachers am involved in intense professional development ho realize mastery of the GPS standards; and all related tasks in these areas. These efforts should adequately address the concerns regarding student preparation and success and should in no nay be interpreted as a sign ef' agreement regarding the allegations. We anxiously await your review of the facts and are prepared to discuss these matters as appropriate. Thartk you. Sincerely, Wad( Beverly L. Hall, Superintendent ec: Ms. Kathy Augustine Payne 001 59 706 Here is a copy with the text in the email. Thank youf Heather Vogell Reporter The Atlanta Journal~Constitution The Atlanta Journal-Constitution GEORGIA OPEN RECORDS ACT REQLEST Tuesday, April, 14 Dr. Beverl}'Ha1l Atlanta Public Schools Dear Superiutenrleiit Hall, the Georgia Open Records Act. U.C.G A. Sec|;ionA? 50-13-70 el all The Atlania Joumal- requesungrn review l_l_\e follmving public docilireiits with the exception of ihose portions speciizally exeiupied by srate law: *All con lai11ts_ supporting (l0L`llIllC132|Ll0lL findings aucl any and all related clocrumhrs made to Lls: Oflice ofln1emz|lResolurim1 since TP 2005, `l`o assist 3 uu_ I have th: pcrliucul from the Geo1'gifrUpcuRcuords Act cuiupluirns and lm csligalions: The law unlg: subruiucd lu_ or cmiimiimllom: prepared by. a agency and prepared inocnneclinn\\~i1l1 the or hiring ofa public officero1'en1plc-yee; and records consisting of material obtained in lelaierllu Ol public 1' In 1- saihe to tire age/rcy or Qfficf'/'_ actin/1 fir I/zu rv mi/rcizarled or provided f//is /mr hr' ro .-wake .rm/1 Ifynu believe ang, ofthe requested is exeir|pl_ please pr-oi-'icle Llx: cilz1Liorrh'orn|lie law that permits such exem|JliUu_ As you lm;m'_ the Georgia Open Recoirls Act allmr reasonable search and rerri eval fees after the first quarter hour or a cllurgc ul`25 cents per page or lu [llc case ufrccords mzliilaiued -una computer, Llie ai-:tual cost of like tape 01110 is Therefore iffees are assessed and are expected to exceed please alert the newspaper in advance, as the law requires. and explaili llx: h'>l'll1e fees. S0 the can ils request, can be 21| [404J52ri~7l or 'via email fn Please dc 1101 husitale to me \\-illr queclinns, Simerely, Heanher \>>bgel.l Rep-:me 707 From: Fitts, Sharron To: Hall, Beverly (Supl) Sent: 4/14/2-309 926:41 PM Subject: AJC ORA req ues! Interesting. From: Few, Millicent Sent:Tuesday, April 14, 2009 5:23 To: Veieter; Howard, Colinda; Pitts, Subject: FVV: AJC ORA request FYI Sharron From: Tucker, Seana Sent: Tuesday, April 14, 20-39 4:45 PM To: Few, Cc: Manguno, Joseph Subject: FW: AJC ORA reoue?t Hi Millicent, I oe Manguno will follow-up with Ms Vogell tc- find out what she needs spe-sificaily. I will advise you ofthe Endings, Thank you. With Regards: Qfama gavaiizzn- - /?wi Paralegal! Otice ofthe Chief of Staff Atlanta Publi; Schools 130 Trinity Avenue Atlanta, GA 35303 404-802-281 (Direct Line) 404-802-1807 (Facsimile) |2.ga.us 'ii iw i ,.ii 1 wiak?axg From: Sent: Tuesday, April 14, 25309 12:48 PM To: Tucker, Se-ana Subject: AJC ORA requat Public PHX No. :4o4aoa1ao2 nag. aa oszoopn P1 *All complaints, supporting documentation, llndings and any and all reiated documents made to the Oftioe of Internal Resolution since March 2006 concernin allegations of Empioyee Wrongdoing involving all types of parties in the following categories: Grading, Applications, other Instruction), Public Funds/Properly and Improper Remunerative Conduct. The results are as foliows: 2006/2007 Caselog En The Matter of Gwendolyn Bramwell (T) Tull Waters ES 5/18/07 Testing Violation Charge of Misappropriation of Funds at Douglass High School (Referral to Office of internal Compliance) - In The Matter of Corliss Randall-Davenport, Principal, 8. Denise Bell, Secretary Dunbar ES 10l'l9/00 Miseppropriation of Funds In The Matter of Suprenia Mil1er(TeaCher) East Lake ES In The Matteroi T. (Student) Cascade ES Testing Violation In The Matter of Barbara White (Teacher) AD ES 4/17/07 Testing Violation In The Matter oi Thomasville Heights ES 4/ Testing Violation Regina Cuffey-Harris (Counselor) Thorrell HS 8/12/06 (NC) Falsification of Athlete-'s Eligibility In The Matter of Fabrice Aims (T eacher/Coach) Kennedy MS 11/1/06 Falsification of Athletes Eligibility In The Matter of Lesme Richards (Teacher) Cook ES Testing Violation ln The Matter of Gloria Johnson (Teacher) Grove Pk ES Misappropriation of Funds ln The Matter ol Beverly ll/lcCray (Ed Spec.) Career Ed Misappropriation of Funds in The Matter of Continental Colony ES (Testing Wolation) ln The Matter of Chin Chou ,Teacher, Sarah Smith 4i'19/G7 (NC) Testing Violation In The Matter of Delfreda Hancock ,Teachen (Continental Colony) Testing Violation 2007/2008 - In The Matter of Brenda Blake,Teacher, Townes ES 3/14/08 Testing Violation Betty Foster (Program Asst.) Grady HS 11/6i07 (NC) Testing Violation - External Investigator: Thomasville Heights ES: Testing Violation 4/22i0B In the Matter of Cheryl Jenkins, Douglass HS 1/30/08 Testing Violation I In the Matter of Jena Rainey, Crim HS 1/30/08 Testing Violation 709 FREN IFlt1Elr1tEl Pub] ic FFIX NU. 242149221332 Flay. ZS 2269 ZEIBBPF4 ln the Matter of Gwendolyn Carter, Counselor, West End Academy 1/30lD8 Testing Violation ln the Matter of Therese Powell, Teacher, Bolton Academy ES 4/30/08 (EW) Testing Violation Warrkesha Conyers, Teacher, White ES Testing Violation. Testing Violation -r In the Matter of Dwight Hardy, Teacher, Toomer ES. 5/7/08 Testing Violation 710 FREN Qtlarlta Pub?ic Schools FQX NJ. 24943221822 Mag, QE 2339 pg 2008/2009 Anonymous Complaint Hutchinson ES (EW) 6/25108. Testing Violation In the Maher of Pacola Green ,Teacher, Blalock ES Testing Violations. in the Matter of Marcus Barber, Principal, South Atlanta High School of Leadership and Economic Empowerment) ivlisappropriation of Funds In the Matter of Mays High School - Various complaints ln the Matter of Stefanie Woods, Bus Driver, Application In the Matter of Falana Favors, Teacher, Scott ES 915108 Misappropriation of Funds in the Matter of Dr. Lucious Brown, Principal, Kennedy MS 'i/28/O9 Employee Income Verification in the Matter of Carol Dennis, Administrative Assistant. Kennedy MS 1/28109 Employee income Verification Falsitication External tnvestigationz Dr. Angelisa Cummings Principal, Crim Open Campus Misappropriation of Funds in the Matter of Cheryl Jenkins, Teacher, Douglass HS lvlisrepresentation of Certification In the ll/latter of Uliana Gancea, Teacher, Carver HS Tech 10/16/08 Employment Verification Falsification External Investigation: ln the Matter of Karen Riggins-~Taylor, Principal, Turner MS 31 18109 Testing Violation External Investigation: ln the Matter ot Dr. Andre Williams, Principal, Coen MS 3/ 18309 Testing Violation ln the Matter of Bolton Academy. Misappropriation of School Property In the Matter of Darryl Evans (T) King MS Time Falsitication ln the Matter ot Judy Butts-Smith, Counselor, Hill ES 12/10108 Falsification of Medical Documentation ln the Matter of Titfani Stevenson, Media Specialist, Coretta Scott King l/27109 Time Felsification 711 PREM :Quanta mx No. :4@4a@21E|@2 - ATLANTA i SCHOOLS Making A Difference Pages[not|ncluding sm/erj: 5 Pham e: Fira* 'iam For Disease: Com: COMMENTS Nag. En 2299 P4 Atlanta Publix: Schools of (Train H30 '1`r'ir1ity Avenue, FLW. Athilutu, GA 30303 ]Phm1c; Data =r .grga L;i" iemt ease Ficpfy Please 'Fiec:ym!s 712 Bob Wilson From: Clyde, Thomas Sent: Monday, June 13, 2011 2:36 PM To: Clyde, Thomas Subject: FW: An additional OIR complaint did not receive From: Heather Vogetl Sent: Tuesday, August 25, 2009 4:49 PM To: kbromervtiatlanta.l<12.ga.us ?uh]'ect: An additional OIR complaint I did not receive Hi Keith - I havejust learned that there was a testing complaint at Brown Middle in 2008 that I also did not receive in my Open Records Request. just left you a voice mail about it. Here are the names on the complaint: Teacher Imogene Redwine and principal Donnel Underdue. I'm totd its about cheating. need to find out what this complaint is aboutASAPf I'm also growing more concerned that I have not received all the testing misconduct complaints requested that should have been provided under state law. Have you had a chance to discuss this with the I Know you're just jumping in here, and it is SAT day, but please convey my concerns to those who have been involved with this for weeks hovv. Thanks, Heather Heather Vogell Reporter, The Atlanta Journal-Constitution Cell. (404)270-0303 1 713 Bob Wilson From: Clyde, Thomas Sent: Monday, June 13, 2011 2:37 PM To: Clyde, Thomas Subject: Missing cases? Sent: Thursday, August 20, 2009 4:23 PM To: kbromei;yg1.jatlanta.k12.ga.us Subject: Missing cases? Hi Keith, Our request covered the 06~07, 07-08, and 08~09 school years. We asked for all OIR complaints involving testing misconduct. Heres what we have. The question is, what others are we missing? School Individual Date Thornasviile Heights Elementary Principal Janice Kelsey Thomasville Heights Elementary Testing coordinator Tracey Fisher 4/17/2007 Towns Elementary Teacher Brenda J. Blake 3l11/2008 General 4/24/2009' Toomer Middle Dwight Hardy 5/12/2008 Cook Elem Lesma Richards others 4/1832007 Hutchinson Elem List of teachers aug 2007 and may 2008 Continental Colony Delfreda Hancock 4/19/2007 Turner Middle Principal Karen Riggins-Taylor 07-08 and 08-09 Walter Francis White Elem Warrkesha Conyers 1023/2008 AD Williams Elem Barbara White 4117/2007 Blalock Elem Pecola Green 6319/2008 Cascade Elem General 9/1512006 West End Academy Gwendolyn Jones Carter Sarah Smith Chih Chou 4/18/2007 Grady Betty Foster Nov. 2007 Summer Enricl':mentAcad Jena Melissa Rainey 2005 summer/2008 decision Summer Enrichment Acad Cheryi Jenkins 2005 summer/2008 decision Tull Waters Elem Gwendolyn Bremwell 5/23f2007 Continental Colony Michelle Toltiver 4/28/2009 Bolton Academy Theresa Powell 4/30/2008 Thanks, Heather Heather Vogell Reporter The Atlanta Journal-Constitution 1 714 Bob Wilson From: Clyde, Thomas Sent: Monday, June 13, 2011 2137 PM To: Clyde, Thomas Subject: FW: OIR complaint follow-up From: Tucker, Seana Sent: Thursday, July 23, 2009 4:40 PM To: l-leatlaer Vogel! Cc: Yeager, Su Subject: RE: OIR complaint follow-up Ms. Vogell: The information requested is available for review. Please contact me to schedule a day time that is convenient for you. WiLh Regards, @;Mzw gwdan- Paralegal Office ofthe Chief_of_S'raff Atlanta Public Schools 130 Trinity Avenue ALla11ta,G/\ 30303 404-802-28] (Direct Line) 404~802- I 80? (lfacsimile) 2. ga."?fT ikiixkiimg Si sent: 2009 4:03 PM To: Tucker, Subject: Fw: OIR complaint follow-up Hello Ms. Tucker, I spoke with Ms. Yeager and Ms, Howard about these additional complaints on Tuesday, I believe, of Iastweek MS 1 715 Howard indicated the information woutd be located. I was wondering whether it had been. Could you let me know the status? Thank you, Heather Heather Vogell Reporter The Atlanta Journal-Constitution Forwarded my i-leather vegeliiri-let/News/AJclus on nazsa Pit-2 To stuckcr@atlants.k12.ga.us cc 02248 PM Subject OIR contplalntfollow-up Good Afternoon Ms. Tucker, I have finished reviewing the copies of the OIR complaints provided to me and compared them to the log of complaints. There are a few loose ends to tie up. list below. 1) There are two complaints that I requested but did not receive materials for. They are: Gwendolyn Bramwelt Tull Waters, testing violation 5/ 18107 Suprenia li/liller, East Lake, l'll definitely need a copy of the Bramwell complainthsince it pertains to testing, I'm not sum Miller complair1t__l_ don't remember leaving any whole complaints out after reviewing them with you, but like to check whether it pertains to testing. Does it? If it does, l'll need that as well. Otherwise, I won't. I understand additional copying charges will accrue since these were not part of what was copied before. 2) I'cl lett one testing complaint out of the revised request: It's Theresa Powell, teacher, Bolton Academy ES 4/30108 testing violation. l'd like to request that additional complaint under the Georgia Open Records Act as well. 3) And finally, can someone tell me whether the first Continental Colony testing violation listed in the log is the same as the second complaint listed for that school, concerning Delfreda Hancock - which I have? Also, do any of the Mays High "various complaints" pertain to testing? If so, l'd like to request themlit also under ORA (but not, if not). [want to make sure i have all the documents responsive to the original request for OIR complaints about testing violations. However, don't want to make extra work for you all ll some of these complaints that aren't well-described in the log have nothing to do with testing. Please let me know when I can pick up copies of the Bramweil and Powell complaints, and let me know whether the others are relevant to the original request. Thanks, Heather Heather Vogell Reporter The Atlanta Joumal-Constitution 2 716 Atlanta Journal~Constitution, The (GA) October_1B, 2005 . Section.: News Edition: Main; The Atlanta Journal-Constitution Pago: A1 Drastic test swings valid? An AJC analysis EDUS questionable changes in test scores at 19 schools. Heather Vogelt John Parry Staff Statistically unlikeiy state test scores are showing up in more classrooms, suggesting the cheating investigation that has engulfed four schools might be about to widen. An Atlanta Journal-Constitution investigation found 19 public elementary schools statewide with extraordinary gains or drops in scores between spring last year and this year. A dozen were in Atlanta. in West Manor and Peyton Forest eiementary schools, forinstance, students went from among the bottom performers statewide to among the best overthe course of one year. The odds of making such a leap were less than 1 in a billion. This summer, state othcials found strong evidence of cheating at four schools statewide in an investigation that followed a December AJC story about improbable gains on state tests. En the most recent analysis, the AJC again used statistics to look for schools with test score changes far outside the normal range. The newspaper compared students' scores in one grade versus their scores in the next. Some improved astronomically, but others deteriorated sharply. "Changes of that magnitude are just extremely suspicious," said Walt Hany, a testing expert and professor at Boston College. Atlanta officials said they do not believe cheating occurred. Yet questionable scores appeared last school year in more than one in Eve of the districts 57 elemen ry schools, the AJC analysis showed. At some, multipte tesbs stood out, with scores moving up or down erratically. Experts say chilciren's scores are normally fairly stabie between grades. 'There had to be something considerable that happened that you would swing that much in a single year," said Kathleen Mathers, executive director of the Governofs Otlice of Student Achievement Her ofice is scrutinizing state test scores tor the sort of anomalies that could signal test-tampering. lt will use its findings -- due this month - to decide where to audit, she said. This is the tirst time state otlicials have undertaken such a broad search for test cheats. Several Atlanta principals attributed unexpected score changes to factors such as a good instructional programs, talented or struggling teachers or changes in the student population. Atlanta Deputy Superintendent Kathy Augustine said the district has no plans to check the validity of the scores highlighted by the AJC. "i don't have any reasorrto look at that," she said. "We expect outliers every year." She said the districts use of testing data to guide instruction and good teacher training are among the strategies that have helped schools make steady progress. Also, high rates of student tumover at some schools in question could create surprising score iumps, she said, A check of several schools outside Atlanta with similarly high turnover, however, found none with such unusuai test results. 717 Last summer, critics chastised Atlanta for its handling of cheating allegations at Deerwood Academy, one of the schools where state ofticials said they had uncovered evidence of likely test-tampering. Superintendent Beverly Hall said the district found no proof- a stance that drew a rebuke from Gov. Sonny Perdue. Augustine said the distriotwould investigate if it had evidence of cheating. Besides the Atlanta schools, E-leards Ferry in Fulton County was the only other in the metro area to report such unexpected scores, the AJC found. A meteoric jump The AJC examined scores on state reading, math and language arts tests for students in grades 3 through 5. The newspaper compared students' scores from 2008 with how they did in spring 2DO9. The state Criterion;Referenced Competency Tests are Georglas main measure of academic ability through eighth grade, The Atlanta elementary schools in question include one that state Superintendent Kathy Cox praised eifusively in ll/lay as. a hardworking school with an "absoiutely no-excuses attitude." "By the way, knocking the socks off of the test scores," Cox said of Peyton Forest Elementary at a state Board of Education meeting. "They're just a shining star." Indeed, when state test results arrived a few weeks later, some scores' rise was meteoric. Peyton third-graders' math results last yearwere among the lowest in the state. But as fourth-graders this spring, they placed fourth in math out of nearly 1,200 schools statewide, strips-sing dozens of affluent suburban classrooms. The feat was even more surprising given that two months before the state test, 94 percent of Peyton fourth~graders scored at the lowest of four levels on the district's own practice math tests. Peyton Principal Karen Barlow-Brown said the increases were partly due to a former third-grade math teacher who was ineffective East year and a talented fourth-grade teacher this year. She atso said the school doesn't use the practice tests as a predictor of state test results. "That is really an insult," she said when asked whether tests might have been altered. Such dramatic gains in such a short amount of time, however, are abnormal at best, experts said. "lt's very hard to explain these huge gains," said Tom Haladyna, a professor emeritus at Arizona State University and testing expert who reviewed the AJC's Endings, "You have to wonder: ls this the greatest school in the world?" Schools that attribute such rare gains to a successful program have a responsibility to show others what they did, he said. "The whole world wants to know this," he said. "lf we could get this out of every class in your state and every other state, wouldr|'t that be fantastic?" An Atlanta district spokesman asked a reporter to call Michael Casserly, executive director of the Council of Great City Schools. The group, of which Atlanta is a member, supports urban systems. Casserly had not seen the AJC's analysis but said he disagreed with experts who said the scores were questionable. Casserly said some schools might teach their curricula differently from others in the state, of the changes might be random, or another factor such as teacher tumover could differ in Atlanta. "lf you're after one single explanation, you are on the verge of badly misleading the public on the basis of a very bogus analysis," he said. Loapftogging peers If falsihed, scores can disguise serious academic problems, said Eric Cochling. vice president of public policy at the nonprofit Georgia Family Council. 718 Parents need valid test scores to make key decisions for their children, such as whether to change schools or teachers, or get remedial help, Cochling said. "How do they know what their child needs, ultimately, if they can't rely on the test results?" he asked. "lt seems it sets these kids up for failure." Atlanta's West Manor Elementary made some ofthe most astonishing gains this year. ln fourth grade last year, students' poor scores ranked 830th statewide on the math test. This year, tifth-graders not only caught up to their peers but sped past them; they scored the highest statewide. Their average score grew by nearly 90 points year to year, data show. Statewide, the average rise was about 15 points. Practice tests again suggest a disconnect with the CRCT. Sixty percent of West Manor hfth-graders were still scoring at the lowest level in Febn.|ary practice tests. Not only did every student pass the CRCT in April, but 89 percent scored at the top "exceeds" level. Principal Cheryl Twyman attributed gains to "the hard work ofthe teachers and students - that's a given." She declined to discuss the results further. Parent Sharon Shannon Bussie said she has seen West push students to achieve and doe-sn't believe they would cheat. "This schoot is quite different," she said. "If you're an underachiever, you might as well not go here." Another puzling result came from Atlantaks Toomer Elementary, Last year, Toomens fourth-graders scored best En the state on the Englishllanguage arts CRCT, which focuses on writing. Toorner's average score was so high that no other school came within 14 points. But this year, Toomer itth-graders struggled wim the test of concepts such as grammar and sentence structure. Their average score plummeted 58 points. Halaclyna said researchers rarely see such a steep drop. "Kids don't go backward in their learning," he said. Interim Principal Hezekiah Wardlow said school staff realized some of its scores had dipped, but not to the extent made clear in the AJC analysis. He said the school has smail grades, and three or four children leaving can have a big impact on scores. To be sure, test scores can be affected by shifts in a school districts boundaries or other events that change the makeup of the student population. A Fulton district spokeswoman noted that Heards Ferry's attendance boundaries changed last year. Some Atlanta schools have seen deep declines in student enrollment after housing projects closed in recent years. Blalock Elementary, which sented children living in the Bankhead Courts housing project, had four subject tests with astronomical gains this year, the AJC's analysis found. On one, more than 96 percent of fifth-graders scored at the ;'exceeds" level in math, compared with 36 percent statewide. Former Principal Frances Thompson said last year was an unusual one for her school. Steep drops in enrollment as Bankheacl Courts emptied meant more attention for students who stayed, she said, adding she has no concern that cheating might have occurred. "Our otass sues were so much srnatler, and we did use that to our advantage," she said. "We were able to address the needs ofthe students very, very closely," Blalock closed at the end ofthe school year. - Peyton Forest is not the only school in question that has won awards, money or visits from dignitaries because of test scores. 719 Top federal education officials have visited Atlanta's Capitol View and Stanton - which both had tests that were extreme outliers in the AJC analysis. Capitol View Principal Arlene Snowden said she did not believe the gains the AJC cited were unusual. She said factors such as strong teaching programs and stellar staff made the difference. "We accept no excuses from our children. I have a very highly competent staff. We look for teachers who know how to teach the Capitol View way," she said. "We want everyone to be successful," Georgia School Superintendent Cox would not comment on the questionable scores because of the state's investigation, a spokesman said, adding the state would act if cheating were found. The student achievement oftice, which is independent of Cox's agency, is scrutinizing both spring CRCT scores and the results from summer retests taken by students who failed on their Erst try, Mathers said. She said the state may use two approaches in addition to statistical analyses. One examines erasure marks for unusual numbers of answers changed to correct. The other looks for unexpected pattems of responses, such as a class where students get all the hard questions right but the easy ones wrong. This summer, state investigators said an erasure analysis revealed strong evidence that adults at four schools had cheated on CRCT retests DeKalb County police charged two school administrators with falsifying state documents. The state board of education revoked the four schools' status as having met federal standards, or made "adequate yearly progress," Rumors persist Unlike most districts, Atlanta hands out bonuses of up to $2,000 per educator to schools that meet targets for improving test scores. Last week, the district announced more than two dozen schools earned bonuses this year, including eight that the AJC found had highly unexpected score changes. Rumors of cheating have swirted for years in the district. Some teachers have said they are afraid to report problems because they fear retaliation. This summer, Superintendent Hall said she did not beiieve cheating was "pervasive" in the district and attributed anonymous complaints about it to disgruntled employees who resented being held accountable. Former Atlanta teacher Joan Shensky said she reported finding a student with an illicit answer key that a teacher had distributed to other tifth-grade teachers at Collier-Usher Elementary in 2005. "l was horritied, horrified," she said in an inten/lew. District records show a teacher was sanctioned. Shensky said she wasn't punished for speaking up but felt like an outcast aftenrvard. She left for a teaching job in another system in 2007. felt ostracized after that," she said. was not comfortable." Steep gains These charts show the change in two Atlanta schools' average CRCT scores and the average change for all schools statewide. Compare schools' soaring CRCT scores with the results of the districts practice tests, which students took about two months before and did much worse on. On practice tests, "unsatisfactory" is the lowest of four levels and means less than 55 percent of answers were correct. Standard deviation shows how unexpected a score change is. The odds ofa four standard deviation change are worse than 1 in 31 The odds ofa Eve standard deviation change are worse than 1 in 3 million. The odds ot a six standard deviation change are worse than 1 in 1 billion. West Manor Elementaiy School fifth-grade math 720 CRCT gain: 6.2 standard deviations Odds: Less than 1 in a billion Practice test results January 2008 math: 57 percent unsatisfactory February 2009 fifth-grade math: 60 percent unsatisfactory Peyton Forest Elementary School fourth-grade math CRCT gain: 6.1 standard deviations Odds: Less than 1 in a billion Practice test results January 2008 third-grade math: 58 percent unsatisfactory February 2009 fourth-grade math: 94 percent unsatisfactory Souroa: AJC analysis of Georgia Department of Education and Atlanta Public Schools data Unexpected scores Scores climbed or dipped dramatically at 12 schools in Atlanta and one in Fulton County. The AJC compared each ciass score in 2008 with its score in 2009, inthe next grade, Standard deviation ls explained above. Elementary school Grade Test Standard deviation Atlanta Public Schools Benteen 3 Read 5.5 5 English 4.3 Bethune 5 Read ~4.6 Blalock 3 English 5.8 4 English 4.1 5 Math 5.9 5 English 6.9 Capitol View 3 Read 5_8 Dunbar 5 Read 4.5 F. I. Stanton ItRead 4.3 Perkerson 5 Read 5.6 Peyton Forest 4 Math 6.1 4 English 4.8 721 5 Read 4_1 Toomer 4 Read -6.3 5 Read -5_4 5 English -9.3 Usher 5 Read 4.6 Venetian Hilis 3 Read 4.7 3 English 4.8 West Manor 5 Math 6.2 Fulton County Heards Ferry 3 English 4 Note: For a full list ofthe 19 outtier schools, see Sources: AJC analysis on Georgia Department of Education CRCT data. How we got the story To detect unusual CRCT test score changes, the AJC used a statistical technique called linear regression to compare average 2009 scores at each elementary to comparable scores from the previous grade the year before. This analysis found that the 2008 scores consistently explained about BO percent of the ditferences between 2009 scores. The analysis also resulted in a mathematical formula that describes the general relationship between 2008 and 2009 scores, For exampie, an average 2008 iifth-grade reading score of B00 would predict a 2009 score of 802 points. A 2008 score of B50 would predict a 2009 score ot 840. - The differences between actual and predicted scores were convened into a rneasure that can be plotted on a normal probability curve, or "bell curve," to End the probability of that difference occurring bychanoe. A score greater than four "standard deviations," for example, has an approximate probability of 0.0032 percent, or odds of less than 1 in 31,000 A score greater than six standard deviations has a probability of 0.0000130099 percent, or odds of less than 1 in 1 billion. There are limits to this analysis. Data pubiicly available from the state do not pemwit tracking students' individual scores from year to year. And because we were able to look at average scores only, student mobility couici create score variations not accounted for by the formula derived from the regression. This is especially true for schools and grades with smaller enrollments. To counter these limits, we didn't anaiyze cases with fewer than 20 students. We also only singled out schools with a four standard deviation or larger difference between predicted and actual scores. Typically, cases greater man two standard deviations from the average are considered outliers. Copyright 2009 The Atlanta Jourr\al~Constitution 722 Atlanta Public Schools News Page of we Today's Date avednesdey, June -21, 1011 HDME DE PARTMENT5 INSIDE APS MEDIA GALLERY PARENTS Sc STUDENTS SCHOOLS CALENDARS 7 - 11/13/2009 utjf 'ie gi ll :dy ihnmtizr arf ltr-latinas ,rl-Mt rf tl in ,vp uttr=mg,? A .- - Neue Atlanta Edumto Ii]/ws leading Room FS News Releases [Lll-lot [Elm-t Topics Atchives faivem. Online cool Km; National Education Experts Selected for APS Test Review ATLANTA -APS Superintendent Dr. Beverly L. Hall today announced thaitwo top educational exzverts ha'-'e agreed to examine test scores identitieci in recent analyses of state standardized test scores. The national experts are Dr. Ancrew Porter, Dean of Education at the University of Graduate School of Edumticn, and Dr. Douglas Reeves, founder :rf The Leadership and Learning Cerlterin Englewood, Colorado. Porlor will loot: at CRGT data and provide a repdn that gives his perspective on what the ciala means Dr, Reeves has develcoed a hcdy cf knowledge that identifies factors that prorrote or ihlt=bit student learn ng. He will visit classrooms, talka'-'rlh teachers ard administrators and review data. He wil- determine what factora impact student achievement. Dr, Hal said, "l am ver; pleased that two ofthe 'tations topleeting and education experts have agreed tc review CRGT lil 12 of our scliuuls. Eccausc data alone does not tell the full atory, we will have reports from both the statistical rd classroom perspectr.-es. We want to know lf the large gains or declines in student testing are a result factors not considered in recent news reports." Dr. Porter is af' appiiec statlstician anc whose research has locused on assessments and accountability, content and the effects of curriculum policies. He is 2 former p-resident ofthe American Educational Research Association and was elected a member ofthe National Academy of Ecucation in 1994, where he has served aavicu president since 2305 His organization has worked with sfool systems in all SO states anc every Canadla" province, as well as in Europe, Anim, Jlsia, South andthe Middle East. Dr. Reeves is tt-e autnor of more than 2D books and numerous articles on leadership and student achievement. i-le has twice beer named to the Harvard Lnlversih- Authors Series. He was named Ltie Brock lnterratiohal Laureate for l'llS to education He also received the Dlstiriguisliod Award rrom the National Association :lf Secondary School Principals and the Parents for his writing on children and parents, 'lheir reports be marie polfiu. The Allarlta Education Fuhdwill pay forthe rel.-iew. copyrigl-: lo zooe Atlanta Funlic Policy ierms of use iausl he Approvetl Urlvacv Pullof tau nanny Ave. r-tents, 30303 vi-me: race; eo;-xsao S6l lOl 6/ l-'2Ol 723 February 7, 2010 To; Dr. Beverly Hall, Superintendent Atlanta Public Schools From: Douglas Reeves, The Leadership and Learning Center Re: School Visitation Report I. Executive Summary: ln November 2009, was asked by the Superintendent ofAtlanta Public Schools, Dr. Beverly Hall, to undertake si brief inquiry with regard Lo teaching and leadership practices at several Atlanta schools that had displayed si gnihcant improvements in student test scores from 2008 to 2009. During the period December 7-9, 2009, lvisited these schools and since that time have reviewed other documents and field notes related to those \-isits. In addition, reviewed evidence on impact of teaching practices on student achievement. The teaching practices in place at the schools visited, particularly with respect to high expectation, focused curriculum, formative assessment, and extra time allocated to literacy, are all consistent with the professional practices associated with improved student achievement in schools around the world. Indeed, based on the practices in these schools, ii would have been surprising il" scores had not improved significantly. 1 neither sought nor received compensation for this work, not does the organization for which I work, The Leadership and Learning Center, have any financial or contractual arrangement with Atlanta Public Schools. 2. Schools Visited: December Th: `E`oomer Elementary, Coan Middle School Venetian, Peyton, ?\/lanor, Bethune, Perlcerscn, Dunbar, Capitol View, Benteen December 9111, L, Stanton, Usher, Harper-Archer Report of Douglas Reeves Atlanta Public Schools Page February 7, 2010 El. 009053 724 February 7, 2010 Dr. Beverly Heli, Superintendent Atlanta Public Schools From: Douglas Reeves, The Leadership and Learning Center Re; School Visitation Report 1. Executive Summary: Ln November 2009, was asked by the Superintendent of Atlanta Public Schools, Dr. Beverly Ilall, to undertalte a brief inquiry with regard to teachi ng, and leadership practices at several Atlanta schools that had displayed signihcani improvements in student test scores from 2008 to 2009. During, the period December 7-9, 2009, I visited these schools and since that time have reviewed other documents and tield notes related to those visits, In addition reviewed evidence on the impact of teaching practices on student achievement, The teztching practices in place at the school 5 visited, }'>a1TiCularly with respect to high expectation, focused curriculum, fonnative assessment, and extra time allocated to literacy, are all consistent with the professional practices associated with improved student achievement in schools around the world, indeed, based on the practices in these schools, il would have been surprising if iesi scores had noi improved significantly. I neither sought nor received compensation for this work, nor does the orgalriaitiou for which I work, The Leadership and Learning Center, have any financial er contractual arrangement with Atlanta Public Schools. 2. Schools Visited: December Toomer Elementary, Coan Middle School December Sm: Venetian, Peyton, West Manor, Bethune, Perlcerson, Dunbar, Capitol View, Benteen December 9'h_ FL. Stanton, Usher, Harper-Archer Report ol`Douglas Reeves For Atlanta Public Schools P@e I February 7, 2010 EL 009058 725 3. Teaching Practices and Improved Test Sem-cs: Research Summary: The research literature on the relationship between teaching practices and improved test scores is rich and varied. Individual studies can vary in quality and relevance and some i'cS0a1'chc1'S cngagc in at "stL1Cly of studios" - at meta-analysis - in order to bring together the findings of many researchers on the same topic. A more ambitious global approach is a meta-analysis of meta-analyses, and that is the approach recently undertaken by Professor John Hattie and published in the book Learning: A 806| (Routledge, 2009). The thousands of studies in this volume included an intemational representation ofmore than 80 million students. While individual studies and hold varied opinions about the various causes of student achievement, the preponderance ofthe evidence cart in my view, be fairly surnrnarized as follows: While demographic suc-li as poverty, have a significant negative impact on student achievement, other factors within the control of schools - curriculum, assessment, and most particularly teaching - have a significant positive impact on achievement. The importance of Hattie's work is that it quantities the impact of these factors, and that is particular importance in context ol` Atlanta Public Schools, ai high-poverty urban system. 3_2 Measuring Impacts on Achievement; The measurement used in meta-analysis is "effect size" - the percentage of a standard deviation in student results, A simplified explanation ofthe impact ofetifect size is that "tr one standard deviation increase is typically associated with advancing children' achievement by two to three years, improving the rate or learning by 50%, or a correlation between some variable amount of hoinevvork) and achievement of approximately r=.5O is the correlation, or statistical relationship, between two variablesl. When implementing a new program, an effect size of I .0 would moan that, on average, students receiving that treatment would exceed 84% of students not receiving that treatment" 7-8). Therefore, when schools engage in particularly effective practices, such as formative evaluation, that have an average effect size of then the many studies by Hattie suggest that snident achievement increases very si That is not unusual, of cheating it is what happens when classroom practices improved. lnrieed, it was precisely this Report ol' Douglas Reeves for Atlanta Piihlic Schools Page 2 February 7, 2010 EL 009059 726 practice l`ot'1native assessment along with other demonstrably effective interventions - that 1 noticed in the schools that I visited. 3.3 Demographic Impact on Student Achievement: lt is that poverty adversely influences student achievement. study suggests that socioeconomic status influences of a standard deviation of student achievement. Other factors, ranging from parental involvement to low birth weight, also have a significant impact on student performance, In APS, another impact of poverty is unusually high mobility rates for some schools. This is caused by rent incentives that lead low-income parents to move from one school to another' indeed. frorn one district to another - several times within a single school year, The impact on student achievement is devastating Even when teachers and administrators are doing many things right, the effectiveness of their efforts cannot be when students are in those buildings for only a few Weeks before the state test is administered. Despite the pervasive negative impacts of low income on stutlent aclnevement, there are a number of published describing schools with hi gh percentages of low-income students that also have high percentages of students who achieve proficient or advanced scores on statc tests. Readers who wish to explore examples of these can find more than decade ofresearch from The Education Trust and The Leaflersltip and Learning Center The existence of successful high-poverty schools does not .indicate that socioeconomic status is unimportant; it only suggests that schools around the nation have been able to make effective use of specific interventions to improve the performance of students in high-poverty schools. Some state departments of education routinely identity and study these schools in order to identify the inventions that are most effective in their local contexts. 3.4 Limitations of Correlation Research: Just because two variables are corretated :toes not mean that a change in one variable caused the change in the other variable. Nevertheless, there are important reasons to examine correlations, First, correlation leads to important causal discoveries, such as the relationship between tobacco use and cancer, For years, cigarette companies dismissed the relationship as "only a correlation." Nevertheless, the use of Correlation evidence to reduce smoking, saved many tives. Second, there are many simultaneous intlnenees on student performance, and parsing out any individual cause is nearly impossible. Report. ol`l`)ongIas Reeves lor Atlanta Public Schools Page 3 February 7, 2010 EL 0091, 60 727 The best we can do is to examine the relative impact ot` many variables. Third, the manner in which scientists in other fields estahlish causation -- randomized experiments - could he unethical in an educational context. I doubt that readers of this document would want their on-n children randomly assigned to the "no treatment" group, or to a classroom that did not seek every day to optimize learning Given these limitations, researchers do the next best thing, relationships between Classroom practice and student achievement. That is V-*hail have done for more than a decade and it is also what Hattie did in a much larger scale. There are clearly exceptions to the relationships suggested by a correlation. Some smokers never get cancer and live to a ripe old age; some runners die of heart attacks, Some instructional do not work for all Students and some and some adverse impacts on students such as poverty and absent parents - do not have negative impacts on all students. Parents know that a parenting technique that worked for one child may not work for another. This uncertainty does not make us helpless, but should make us humble. We cannot claim that "formative assessment always Works" anymore than we C1111 Claim that is always deadly" t`or student achievement. We can only examine the preponderance of the evidence and draw our best conclusions. 4.0 uc tional Practices in Atlanta Public oolsi 4.1 Formative Assessment: A good deal of assessment in schools is designed to provide a report - perhaps to parents, to school oftieials, or to the public. This sort of "tinal" assessment is sometimes described as "summative." This sort serves an important evaluative purpose, but the results from summative assessments are almost never provided to students and teachers in a manner that is suiciently timely to improve teaching and learning. It is, to use a crude analogy, like an autopsy. lt provides interesting infonnation, but does not help the patient, Formative assessinent, hy contrast, provides feedback lo Students and teachers throughout the year and, as the name suggests, is designed to "infoml" teaching and leaming, This is a. critical point, because it distinguishes schools that give frequent tests and claim to be engaging in "formative assessment" trom those schools that use the infomation that they gather trom student tests to make improved teaching and leadership decisions. Readers may inquire, all Atlanta schools have formative asse.sstnents?" There are few schools anywhere that fail to make Report of Douglas lleeves for Atlanta Public Schools Page 4 February 7, 2010 EL 009061 728 such a claim, But not every school uses the information in a t`om1ative manner. ln schools 1 visited, for example, there was not only evid ence of formative assessment, but also evidence of student work and other data posted on the walls so that teachers and students could use inforrnation about student performance in a consistent and visible manner to improve teaching and learning. Moreover, there was evidence ofa high degree of speciticity of analysis. `1'hatis, teachers not only knew if students were passing or failing, lJut also knew the exact areas in which individual students needed assi stance. almost all schools take periodic benchmark tests or other assessments, not all of them use the data in the thoughtful, constructive, informed. and specific- Way that I observed in the schools that I visited. Finally, in the schools that I visited, teachers and administrators supplemented the APS assessments with t.l1eir"humernade" additional assessments in order to provide immediate and specific feedback to students and teachers, By knowing what students know and do not know, teachers maximized the value of their instructional time, By frequently analyzing data on student achievement, teachers and grew taster and more proficient at the entire process, giving themselves more time to focus on improved teaching and learning. 4.2 Leadership and Coaching: One hallmark of schools that demonstrate academic improvement is an intensive focus on improved instruction. The two primary sources of this improvement is instructional coaching - an expert teacher working directly with classroom teachers - and effective leadership and supervision, While the daily lives of principals and coaches can be overwhelmed with details and interruptions, in schools 1 visited, instructional coaches were working directly with students and teachers and nel, as often happens elsewhere, diverted into administrative duti es. Principals observed entire lessons Aperhaps 30 to 60 minutes and offered immediate feedback for improvements of teachers. This is a muc-h greater level of administrative support for teachers than is typically the case when there are either Superlici al and hri el' observations, or oftieial year-end evaluative ohservations. 4.3 Time Allocation: The most effective instructional intervention will never be implemented ifteachers are not given the time necessary to make the interventions work. in some of the schools that I visited, principals and teachers had doubled the amount ol' time for student literacy every day. For students with the greatest needs. they also provided four days of Report of Douglas Reeves For Atlanta Puinlic Schools Page 5 February 7, 2010 EL 009062 729 additional after school work. `l`hese two interventions alone provided almost 300 hours eveiy Sollool y'CEl1' of additional reading This exceptional level of additional ir1s1Iuctiona.l time is at least one reason that, within a single year, students with significant learning deiicits could achieve proficiency on state assessments by the end of the year. It is particularly noteworthy that schools that over-emphasized literacy were also able to achieve gains in social studios. This denionstrates why it is unwise For Schools lOtl1&it'ttailt El traditional schedule for literacy with the rationale that "we don't have time for reading because we have to cover the social studies curriculum." In fact, covering any curriculum is an exercise in futility if students cannot read their lessons. This is consistent with my research (The Learning Leader, ASCD, 2008) in which schools that devoted signilicantly greater amount; of time forliteracy greater gains in reading, math, science, and social studies than schools that maintained a traditional schedule. 4 4 Appropriate Test l'i'Cpara.tioni Since the dawn ol` the Standards movement and associated state tests, there have been frequent expressions of concem about "teaching to the test" and excessive amounts of time devoted to test preparation. In the schools I visited, I noticed two important trends with regard to the issue of test preparation. First, teachers and administrators did seek an explicit link and assessment. They insured that students were instructed about the content required by the State of Georgia and also knew the format of the tests. Inappropriate test preparation is when students are prepared for specific questions; appropriate test preparation occurs when students are prepared tor any question, having understood tho content and of exam, rather than atternpting to memorize specinc answers to specitic questions. The test preparation techniques I observed - process of eiiniination, re-checking work and changing wrong answers, underlining the question, and so on A are common for all exam preparation programs, trom elementary school to graduate school. They arc the sorts of tecltniques lltat arc noutinely given to students in Wealthy suburban schools because it represents an essential lifelong skill. Second, school leaders and teachers appeared to be aware of the negative impact of test anxiety of students who had, in past, failed to perform well on tests. By beginning early in the school year to think about the expectations associated with the end ofycar tests, teachers reported that students had an increased level of a decreased level oi' anxiety, and the right level of nervousness - that is, the sort ofnervousness Report of Douglas Reeves l`or Atlanta Public Schools Page 6 Februar 7, 2010 EL 1109063 730 that would lead them to check their work twice rather than give up on a challenging problem, I did not observe any behavior or attitudes that suggested inappropriate preparation or the willingness to cross ethical lines to achieve improved student performance. That does not mean that cheating is impossible; it does mean that the practices in these schools are consistent with those in many high-performing schools around the nation that are able to improve student performance as a result of the work of teachers and school leaders, 4_5 Effort and Expectations; There is an extensive literature on the impact of teacher expectations on student work dating to the early l96U's. IL is known as the "Pygmalion Effect," a term taken from the George Bemard Shaw play on which the Broadu-'ay musical, "My Fair Lady" was based, Essentially the evidence suggests that when teachers expect students to do well, it becomes a self-fulfilling prophecy. More recently, my ovm research in more than 2,000 schools with more than 1_5 million students confirmed that when teachers and administrators attribute the causes of student achievement to factors within their control (assesslnen t, curriculum, time allocation, etc), gains in achievement are three to tive times higher than in schools where teachers and students atuibute the causes of student achievement to factors they cannot control, such as student dernographics Learning Ir//0 Student Rem/ts, ASCD, 2010). Moreover, Professor Carol Dweck_ a Stanford ogisl, has assembled impressive evidence to suggest that when students and adults believe that intelligence and success are the product of effort and hard work rather than innate intelligence, then their performance improves significantly Ihe New Ballentine, 2007). This is relevant to APS schools I visited because there was a consistent - even relentless - theme of high expectations and hard work for both students and adults. While the eiaim of high expectations is universally made in schools, it does not take long for at visitor to note the quality of student work, the intensity ofthe feedback, the tocus ofthe teachers and students, and most of all. the willingness of students and staff to 1'eS]Jond Positively to mistakes. 5.0 Sustainability and Variability: While the importance ofcffective teaching and leadership practices is clear from the evidence cited in this report. that research does not necessarily translate into sustainability for student results unless an educational sy-'stein can provide stahility in student population, teaching staffs. and leadership. _lust as student mobility Report ol`l`)ouglas Reeves tor Atlanta 'Public Schools Page 7 February 7, 2010 GOSA APS AEF EL 009964 731 creates disruption and inconsistency in test results, so also does variability in teaching and leadership lead to uncertainty, inconsistent practice, and even cynicism among adults in the system. While the evidence is clear that teaching and leadership have an exceptional impact on student learning, it does not follow that public policies are based upon this evidence and lead to stability in teaching and leadersliip. Un the contrary, schools with the highest needs otten have greatest of tuniover among teachers and leaders. Tlrei'el`ure, the results of this report and, more importantly, of decades of research before it, are of little value without a systemic commitment to consistent and deep implementati on ofthe best instructional practices. In research recently conducted at the Leadership and Learning Center* and to be published later' this year in 21" Skills Solution Tree, 20lO), we learned in studies of more than 100 urban schools that the same interventions, from professional learning communities to instructional coaching to formative assessments, had vastly different levels of impact on student results depending on the degree of implernentarion al the classroom and school level, ln essence, it is not the label of the intervention that is most important, but the degree o|`imp|enicntation of the intervention. That is why monitoring ofteaching and leadership are essential to improving sustainability and reducing variability. 6.0 Rccomulendaliuiisz 6.1 Identity Factors Associated With Improvement: APS would benefit from a public display of data - sometimes called the "Science Pair for Adults," (see ltqiranzirrg lfieric/rar ASCD, 2008) - in which schools display on a board their student achievement data, specific teaching strategies associated with that data, and their own inferences and conclusions regarding the relationship between teaching strategies and student results. While the case ofzm individual school may appear to be only anecdotal evidence, the participation ofthe entire district in such an exercise would help system-level leaders identify trends in effective teaching and leadership practices. 6.2 Articulate Fair and Appropriate Test Preparation Policies; 'lest preparation rernaius a topic for many teachers and Particularly in an environment in which Lest preparation can he equated with cheating. APS should develop a clear and speciti policy, based Report Reeves for Atlanta Public Schools Page 8 February 7, 2010 EL 009065 732 upon the best practices in schools (including at least some of those that observed), that not only pemiits but encourages teachers to link classroom institiction to ultimate assessment. The policy should distinguish between inappropriate test prep (memorizing specihc responses to specific questions) and appropriate test prep understanding subject matter content, assessment format, and test-taking strategies. 6.3 Clarity the Formative Assessment Implementation for Schools: Formative assessment is one ofthe single most effective interventions available in schools. But merely administering periodic assessments is an expensive diversion of time and resources if it is not used to inform Lcattllilig and learning, APS can identify its own bes1.praciices in this are io distinguish effective formative assessment from simply the administration of periodic tests. 7.0 Conclusion: This inquiry began because ofa public suggestion that significarit inipi'oven'\eni5 in LBSI Scores in schools from one to another were unlikely - so unlikely, in fact, that cheating must have occurred. In one middle school that I visited, vigorous efforts were under way to reverse an unacceptable state of discipline, achievement, and morale. The adininistration was making sig,-nilicant improvements and it is not unreasonable to expect that in years ahead, the percentage of students who scoiie at the proficient or higher will double or triple. In the same school, a football team that had won only a couple of games in the previous year recently celebrated an undefeated season - a complete, if highly unlikely, tumaround, Not as unlikely as the New Orleans Saints winning the Super Bowl in 2010, but very unlikely nevertheless. The coaches commented that students this year were not more talented this year, they just worked harder. School leaders suggested that "good coaching" had a lot to do with their success. So it is with dramatic improvements in the fortunes of athletic teams. While there are rare instances of cheating which are punished and iidiculed, the vast majority of athletic- Success is al.l'rihutod to hard work and great coaching. Based on my observations in Atlanta Public Schools, the specific actions of teachers and administrators in the schools I visited represent the equivalent of hard work and great coaching. But my opinion matters much less than the i1il`er'e11ces drawn by citizens and policymakers. When they consider the improved academic of a school in spring. of 2010, I encourage them to ask, "What conclusion would we be drawing if our football team had a similar success?" Report of Douglas Reeves for Atlanta Public Schools Page February 7, 2010 EL 009066 733 From: Bill N|cCargo Tn: CC: BHI McGargo Sent: PM Subject: Atianta studeni achievement data paper Attachments: Dr. Hall, 1 I am fcrwarding this draf: report on behalf of Bill. This is a sonfidential preliminary draft which is not ready release. I Also, Bill wanted yo; to know that LaChandra has requested to attend the 11:30am meeting We agree that it would be beneficial for her to attend, however we before confirming. Melissa PORTER [andyp@gse.upenn.edt] ay, February 18, Seit: Tnursi 'Fur I-Nfl MCC 2010 12:00 PM paper Subject: Atlanta student achievement data Bill, Attached is the draft of :he paper reanalyzi hope yuu find it useful. Let me know what you think. Al Stperintendent Fall. 0, pasa it or to John Riqe and others aa I checked with our consultant and my good rv prcfessor emeritus at the Universit' of Cul-3 investigates cheating is called Caveon and ia' co-owners are Jim also agiee5 that Brien Jacob, who I believe Harvard Uni?ers;ty, and/cr Steven levitt, W: Ba; he good a is at :he University 5 well. Cec:o`t1 apples: An investigation of tie prevalerne the Augist 2003 jssue of the Quarterly Journal of Economics. Andy 734 you deem 5 :end and fellow psyehon rado in Boulder, Ee say Innarra and John Fremer. Bob is eLl1l at the Kennedy nd predictors of teache wanted to check with you ng tae ZCEB and ZGD9 student achievement data. I opcE_lly including etriuian Bob linn, who is a the company that Schsol oi Government, of Chicag: and the American 1 have a piece "Rotten cheating," publi=hed in DRAFT February 18, 2010 ldentifying Atlanta Elementary Schools that Had Unusually Large Gains in Student Achievement Test Scores from the Year 3007f'S to the Year 2005./9 Andrew C. Porter Jennifer McMaken University of ln hi gh stakes student achievement testing, the validity of results is especially important. Validity can he compromised in any one or" a number ofvfays, one of which is through cheating. ln December of2C'U9, the Atlantic Journaf - Constitution raised the possibility of cheating in Georgia schools by noting steep gains at some schools on tests taken lirst in spring and then in sumrner" (June ll, 2009). Apparently, that prompted the Office ot`Student Achievement For the state of Georgia to conduct erasure analyses that identihed four schools in the state where a large number of student wrong answers had heen erased and filled in with the correct answers "causing passing rates on the stare's criteiion-referenced competency tests to spike" (Tune 1 1, 2009). In an October 18, 20239 issue ofthe Atlantic Journal Constitution, reporter Heather Vogell, working with data analyst John Perry, reported on the results from regression analyses using spring 2007/8 results to predict spring results for grades 3, 4, and 5 for each of the tests in English Language Arts, Reading, and Mathematics tor the elementary schools in Georgia. Statewide, I9 schools were with "extraordinary gains or drops in scores betwees spring last year and this year. A dozen were in Atlanta" (October l9, 2009}. Vogell wondered whether cheating might have occurred in these schools, which did substantially better or worse than was predicted from the schools' student achievement levels the spring before. Overview of Analysis Plan The Atlanta Education Fund contamed the authors of this report to ask ifwe would do analyses to detemiine the the validity ofthe Endings and identiFy alternative explanations for found." Our analyses were to be focused on just the 2007f`S and 2008.59 years in just the grades 3, 4, and 5, as analyzed by the newspaper. Vt/e asked about analyses that extended heyond grades 3, 4, and S, the focus ofthe newspaper analyses, to include grades 3 through 8 on which there is annual testing. We were advised to stay focused on grades 3, 4, and 5 We asked the state for teacher-level data so we could look at not only variance between schools, but also variance among teachers within schools, but the _state could not supply, We also asked for item-level responses so we could do some erasure analyses, but again the state could not supply. We asked for grade 2 student level scores but did not receive them, so the Sm grade analyses could not be done. At first, we thought that students were not tested in 2" grade as they typically are _not in other states, That must not be the case, however, as the newspaper analyses used grade, AEP EL EUR'l9ll5 735 On December 3, 2009, we agreed to undertake the requested analyses. We in-in-iediately requested the necessary data tiom Melissa Fincher of the Georgia Department of Education. Data were obtained in a tile suitable for analysis on January 28, 2009. Replication of Newspaper Results Our first analyses sought to replicate the results of the newspaper analyses. The newspaper regressed 2008/9 school-level achievement means on school level achievement means for each of the three grades and each of the three tests; English Language Arts, Reading. and Mathematics, The newspaper calculated residuals for each school in the state with grades 3, 4, and 5 where a residual is deined as the school's actual mean level in comparison to its predicted mean level performance for Spring 2009 based on achievement in 2008 for the grade prior. Schools with residuals larger than 4 standard errors of estimate were identified as outliers. Using the State Department of Education provided analysis die, which contained student level scores for the two years and three subjects in question, we were able to replicate the newspaper results. 'Iwo Analyses to of Newspa.1_Qer_Rest1lt? The newspaper results were based on all students in each ofthe two years. Thus, the results did not control for possible changes in student body composition. To control For possible changes, we completed two analyses. First, we did analyses on longitudinal data. The longitudinal analyses were only possible for grades four and five because we werenot provided grade 2 test scores. The analyses are based on only the students in a grade at 2009 that were in the same school in 2008 and had test scores for 2008, for example, the 5'1" graders in 2009 were graders at the same school in 2008. As yet a third analysis, we regressed 2009 on 2003 data at the school level, just as did the newspaper, but we added as control variables 12 difference variables contrasting the composition ofthe group ofstudents in 2009 to the composition in 2008. These variables were differences in: enrollment, female, white, black, Hispanic, Asian, American Indian, rnultiracial, economically disadvantaged, disability, LEP, migrant. We call these regression analyses "student demographic." Student demographic regression analyses allowed us to investigate outliers at the SQ grade level, which the longitudinal analyses did not, as well as at the 49" and grade levels. As slated previously, there was some confusion about the availability or not of grade 2 test scores. In one analysis tile, we had the data provided bythe state that we requested, which did not have grade 2. ln another analysis til e, we had the data given to us by the newspaper, which is what they analyzed, which in retrospect we saw had grade 2. We had'ea.rlier thought that the newspaper analyses had regressed grade 3 in 2009 on grade 3 in 2008. Instead, they had regressed to gade 3 in 'zeros on grace 2, 2008. So, in all of our results, Une newspaper results are based on grade 2 as a control. We could not replicate those grade 3 results with either of our longitudinal nor student demographic analyses because we didn't have grade 2 in the state provided analysis tile. We believe we could get grade 2 data and do those analyses if that is desired. Lnwhat follows in results, we do report on a student ,dernogiaphic analysis for grade 3, but it uses grade 3 in 2008 to predict grade 3 En 2009, controlling for shifts in demographics. For grades 4 and 2 Aki' EL |39 16 736 5, the student demographic analyses use the prior grade in 2008 to predict the subsequent ,grade in 2009. of ai? Three Table 1 summarizes the results ofthe three regression analyses: newspaper, longitudinal, and student demographic. The rows in the table indicate Atlanta schools with grades 3. 4, and 5 that were identified as an outlier by one or more ofthe three analyses. An indicates that the school residual was four standard errors of estimate or more above (or below) the school' predicted level ofachievement based on 2008 data (a very high standard for statistical significance but reasonable given the number of statistical analyses done). The columns of the table divide the analyses by grade level and within each grade level by test and within each test, by the three analyses. The first set of schools in the table are tor those with positive residuals and the last set of schools are for the three schools with negative residuals. As can he seen in Table l, there were 22 instances of unusually large positive or negative residuals identified by the newspaper analysis, seven at 3" grade, three at 4* grade, and eleven at it grade. Ofthose 22, eight were Fully replicated, meaning that both the longitudinal analysis and the student demographic analysis also indicated that the residual was unusually large and Two were partially replicated. Thus, ofthe 4'h and im grade newspaper identitied large residuals, they often were repiicated in part or in whole, The lim and 5 grade analyses arelaetter than the 3rd grade analyses because they use as a control achievement in 2008 in the prior grade. The eight folly replicated newspaper results are across seven different schools In short, no one school was identified as having unusually large positive residuals across all grade levels and tested subjects. To the contrary, the unusually large positive residuals that were lixlly replicated were uniqueto a specific grade level and tested suhiecx at a specitic school Blalock was the sole exception with two replioations; One ofthe two partially replicated results was in yet a ninth school. Large residuals were not systemic, Table reports the multiple correlation squared For each regression equation and identifies the significant predictors in the regression equation. All of the RES are substantial as one might expect when prior achievement is used to predict subsequent achievement, There is no real pattern to these Rzs, other than that in most cases, roughly half of the variance is accounted for by the predictors. Perhaps surprising, the longitudinal analyses did not yield the highest Ras, yet the longitudinal analyses used the same sruclents in 2008 to predict their performance in 2009. ln all cases, the 2008 achievement level Was a signiticant predictor ofthe achievement level in ZUO9. In the student demographic analyses, there were as many as tive significant predictors in addition to prior achievement. Student disability and economically disadvantaged were thetwo most common signiticant predictor, with gender and enrollment next The general conclusions from these *results are first, the information was a good predictor of performance in 2009 and student demographic anaEysis, using changes in student body composition as predictors, worked in the sense that several ofthese variables were significant predictors and so, helped T0 control for shiis in demographics. 3 AEP EL 09117 737 As has been described, the longitudinal analysis was based on only the smdents that were tested inthe same school, beth in 2009 and 2003. Table 3 gives :he percentages of students tested in 2009 that were tested inthe same school in 2008 for each ofthe Schools identitied as having an unusually large residual, As seen in Tab_le 3, statewide retention rate school-wide was 75.59% and the Atlanta retention rate was 67.50. From grade 3 to grade 4, the analogous percentages are 72.78 and 66.28 and for grade 4 to grade 5. the percentages are 73.42 statewide and 68.73 for Atlanta. Clearly, year to year stability for students attending the same school is greater in the state than it Es in Atlanta Table 3 gives the results for each individual school and the columns for gade 3 to grade 4 and grade il to grade are most relevant to interpreting the longitudinal analyses. In both cases, Blalock had the lowest student stability rate, only 39.34% for grade 3 to grade 4 and 45 .lO% for grade 4 to grade 5. One might imagine with the relatively low stability for Blalock, newspaper results might be less likely to be replicated, but that is not fully borne out in the results. Blalock was identified by the newspaper as having an unusually large positive residual for grade 3 English Language Arts and Mathematics, grade 4 English Language Arts. grade 5 English Language Arts and Mathematics. Of those tive large positive residuals for the newspaper analysis, two were tizlly replicated, and two were in 3"d grade. No other patterns emerge from Table 3. SzeQLReSait1ab When interpreting results in Table l, a good question to ask is how large were unusually large residuals? ln short, while they were unusually large, in terms of being more than four standard errors from what was predicted, were they also large in terms of the scale score metric? In Georgia, scale scores are not vertically equated, making it impossible to compare a score in one grade level to a score in another grade level Scale scores are, however, equated from one year to ?he next so that comparisons overtime within grade and subject are possible. They are structured to range from 650 to 900 or above. For grades 3, 4, and 5, 650 is the lowest possible score for each ofthe three tests. The highest possible score is 920 for Reading, 930 Ear English Language Arts, and 990 for Mathematics, indicating a greater possible range of scores for Mathematics than tor English Language Arts and in English Language Arts than for Reading. For the data in the stale-provided analysis tile, standard deviations for English Language Arts and Reading were about 1.5, regardless of grade level, with the exception of grade 5 Reading where the standard deviation was 101. The mathematics standard deviations were larger, ranging from 8.6 for grade 3 to 19.7 for grade 4 and grade 5. In Table 4, each of the residuals identified by one or another of the three regression analyses that exceeded four standard errors are repoied in bold. The largest residuals are for Sm gade Mathematics, West Manor. residuals, for which the newspaper finding was replicated, are 74.72 for the newspaper analysis, 75,42 for the longitudinal analysis, and 68.72 forthe student demographic analysis, These results are 34 standard deviations in size, which wouldbe seen as large by any standard, For example, in education research, when the size ofthe effect of an intervention is being described, it is often described in terms of numbers ofszandard deviations. An intervention that has an effect size of ,5 standard deviations is seen to be a strong intervention. 4 AEP EL L39l 8 738 For the replicated results, Table 4 indicates which analysis found the Eargest residual. The pattern was for the longitudinal analysis to yield the largest residual. For example, in Perkerson and Usher Elementary Schools Sm grade Reading, the longitudinal analyses got by far the largest positive residual in comparison to the other two analyses. We believe that the best control for shifts in student body composition from is represented in the longitudinal analyses, yet often these were also the largest residuals, Table 5 reports the average residuals tor Atlanta public schools -by grade, tested subject, and each ofthe three analyses. First, statewide. the residuals have to average to zero; that is a statistical fact. Each regression predicts values and the average predicted values equal the average actual value-sf For some schools, the prediction is lower than the achievement level obtained; tor others, it`s lugcier. In Table 5, it is clear that the average residuals for the Atlanta public schools do not differ much from zero. In short, across all schools in Atlanta, they are no more likely on average to have a large positive (or a large negative) residual than schools elsewhere in the state. Second, while there are differences across the three analyses, none are striking. All are less than hve scale score points. Nevertheless the standardrleviations of residuals are systematically larger for Atlanta schools than forthe state, with a tendency for the greatest variance in Atlanta to be for the student demographic residuals. i In Table 4, we can also investigate the possibility that the unusually large positive or negative residuals identified, each ofwhioh exceeded the four standard errors ofestimaie criterion for size, might hide nearly as large positive or negative residuals that just missed the tour standard errors criterion. Not surprisingly, :here are additional large positive and negative residuals, but not as many as might be expected. For 50' grade Mathematics, I-'erlrezson has systematically large positive residuals that didn't meet the four standard errors criterion. The residuals for Usher in 5" grade \/[athernatics are large and positive across all three analyses despite not being flagged in the newspaper analyses. In English Language Arts at the Sm grade level, all three regression analyses showed fairly large positive residuals tor Benteen Elementary School, but only two,of them matched the four standard errors or more criterion. i- I_n 41" grade for Reading. the negative residual For Toomer comes close to being fully replicated as there- was a large negative residual tor the longitudinal analysis as well that again didn't meet the four standard errors criterion. For English Language Arts, the Blalock and Peyton Forest newspaper identified schools could be considered fully replicated, as both the longitudinal and student demographic analyses estimated large positive residuals as well. For grade English Language Arts, the Blalock large positive residual identified by the newspaper could be considered replicated by the szudent demographic analysis even though the residual didn't quite meet the tour standard errors criterion. We looked at the report CRCT Analysis by School Report" on the Govemor's Othce ot' Student Achievement website. In that report, they list the "percent of classes flagged in the erasure analysis." Eight ofthe 12 schools flagged by the newspaper 5 AEP EL e09ll9 739 analysis have 2/3 or more of their classes iagged in the erasure analysis. The erasure analysis was done on the spring 2009 data just same as residual regression analyses. - Summary Over the course ofthe last several months, several analyses have been conducted of student achievement in Georgia pub-lic schools with an toward identifying possible instances of cheating. One set of those analyses focused on changes between results in_ and 2008;'9 tor grades 3, 4, and 5_ on each ofthe tested subjects, English Language Reading, and Mathematics. The analyses reported by Heather Vogell in the Atlantic Journal 4 Constitution used a prior grade in 200810 predict the following grades in 2009 for all ofthe students in the school in those two years and those grade levels for each ofthe schools in the state. The newspaper analysis identified 12 schools in Atlanta for which there were unusually large residuals. The analyses reported here replicated those newspaper results. The analyses went on to explore the validity ofthose newspaper results by conducting analyses based just on students wbo stayed in the same testing in 2008 and then again testing in 2009 at the next grade level. These are called the longitudinal results. We also added in student demograpl-tics as control variables to the newspaper analyses and these were called' the student demographic analyses. Both the longitudinal analysis and the student demographic analysis were meant to provide controls not provided in the newspaper analyses tb: shitts from one year to the next in student demographics that might have accounted forthe unusually Large positive and occasionally negative residuals the newspaper identitied and that might be an' indicator of possible cheating. Newspaper findings not replicated would be suspect", having not had adequate controls. First, the size of the positive and occasional negative residuals were often large, as many as 3 or 4 standard deviations in size. Second, many ofthe newspaper results were replicated with both the longitudinal and student demographic analyses. Not all newspaper results were replicated and as might be expected, the additional analyses identined additional subjects and grade levels in the already identified schools that might also be outliers having unusually large or occasionally unusually large negative residuals. No additional Atlanta schools were identified by these two additional analyses. We weren't given the more recent erasure analyses published by the governor's office of student achievement at the grade and tested subject level, only at the school level. Eight ofthe I2 newspaper flagged schools had 2/3 or more of their classes flagged in tl'e erasure analysis. in summary, controlling tor shifts in student demographics trom 2008 to 2009 through thelongitudinal analyses and the student demographic analyses did not in large part negate the newspaper findings, Schools with unusually large residuals indicating that the school did much better or much worse than would have been expected based on prior achieverhent found in one analysis tended to he replicated in the other analyses, as well as in the erasure analyses. 6 BL. 740 Those results do not prove that cheating occurred, but they do point to student achievement gains and losses that are highly unusual and for which cheating could he one enqnlanation. Nevertheless, unusually large residuals were not systemic across grade levels and tested subjects in a school, suggesting that the unusually large residuals are localized to specihc grades and subjects. Further, the average residuals for Atlanta schools were comparable to the average residuals for schools statewide. To the extent that there were unusually large positive residuals, these were offset by unusually large negative residuals in Atlanta schools. The standard deviations of residuals were, however, larger for Atlanta schools than for schools statewide. In short, there were more large positive and negative residuals for Atlanta schools than was typical for schools in the rest of the state. Finally, it is also true that if a sc-hoo] cheated in 2008 to get better student achievement performance and then cheated again in 2069 in the same way, neither the newspaper analysis, nor our longitudinal and student demographic analyses would flag this school as an outlier with an unusually large residual. 7 GOSA APS AEF El. tJ.)9' 741 School Name Parkersor. Elememary Usher Elementary' Ve welian Pe,-lun F;-rest Elemeniary- Ealnck Elamcniary Eemeen Elernenla 7 Cu ::iluI Flemenfary FB L. Slanhln Elememary Bethune* Tnumul |9s:|rIImIn were ne;jaIIve (SSWOUI performed worse than ELA Grade 3 Rs adin I n.'a Anaivsm - a I Grade 4 Readin a Grade 5 Rs adin -n-'J'l r:r.5r.Imts::' Nm A L0ngIIudInaI SI Dun 742 12 Gre Cub ed Grade Grade Rudd ulalh 'nal read newspaper ongrludrnel news apex' ongrl udrnal aL,l\uu| Jam news aner mal Iongrtudrnal newsua er Ion rludlnal .>Llr00| newspaper schon darn newspaper lonnlludlnal bclruul Jam news a er School dem new aye: ongnun nal |118 U57 El i6 nfa D75 DB7 1 D62 U36 Ll?l OB OB read UB rcad D5 fl gr SBDB read D8 read OB reud OB n"alh rfalh rraln ela 06 ela 08 reed Tal: 2 Predlcm 5 econ I ed female fernale enrullrrenl econ dlsadvantaged 5 Predrcrors ernn 'll 'lCl\/Q econ 5 am. r'nu|tl'ac'l1l econ dlsan dlsa 3llIT'/ .1 . - EI-A I2 "1 U51 ua' 3 .923 - |112 i 56 a DB ELA I school dum ?2.56 e.s DB female mulliracrel econ. Ellsadvalrta ud disauilil LEP EFS c.5o .54 lm- .. 5.69 05 c-.0.55 ELA 5 0.72 |50 ?=ily 0.77 5.73 I Ds' m"1h I US - a math school :lem 01 enrul|m= nl econ ed 743 9 Table 3. Psrcunl of .students testo: in 2009 that wcro in the some Schcol in 2058 School Name STATE RETENTION RATE APE AVERAGE RETENTIDN RATE INDIVIDUAL SCHOOL RETENITTON Elementare School Usher 'l/Vcsl lvlanur Elemantan' School is an Fcresl Elemeniary School Blalock Elementan' School Sclmul - Canitolk-Sew Elementa 'School Qunbar Elememarx' School 5 F. Stanton School 5 Bethune' wlaealoa Gr-2 na to lu og '5r4 ue ee, Gu: on lD overlao overla B7 sn es_2e save 5775 G04 E110 zsos mem 75.35 as 57 eif?? ease -ww G9 #lu 54.75 me FU-U0- 10,42 5263. 55.-.50 51-U5 744 10 155151 gsraqgg sra1e4 GradeE 5_5 we Eu md 111111 ELA Maw School YU A EY A 5 frSTATE AVERAGE RAW 0.3 DLI 0,0 ATERAGL R5s1uuA|_s _1.e 4e_ 2,9 0.1 -14 -5,5 az 1? -44 15 5,5 .55 -5_os 1Nn|v|auAL 1 1 m;51uuAL5 296 11 7 19 7 25.0 13 5 iS 3 Vf4'5 1T.U I '11 21.1 21.4 13.? 9.5 US 13,3 5 25.5 3K2 24.0 25 3 20.5 15.2 1151151 1154 2:3.1 5.5 3.5 5.5 115 15.2 1522.5 55.5 22.5 an 1 45,2 v/1 1/511511211 2555 25.5 15 21.1 115 15.4 5_1 1-.1 2 .- 4.5 5.2 24.5 15 5 15,5 55 15.5 15,5 11.5 155 11.5 15.1 14.5 5.5 2_5 no WPI 1u1=.m 2555 15.5 5.5 15,1 51.1 15.5 45.4 5.4 1; _1.5 7,7 2.1 11.2 4.4 55 25 15.2 15.5 155 10.5 5,5 5.1 75.4 55.1 Peym - rws. 51:55 2.5 15.0 -5.5 1.5 15.5 - 5.5 me 55 1 25,1 <51 71,11 15,5 55,5 5.10 eu; 15; yy 5 151, 15,5 15,1 11,4 -2.5 14.5 5.5 5151555 -1552 I 55,5 21,5 22,4 11.5 511,11 21 ,a 24.5 19 2 24,3 -2_5 :2.1 10,2 5.1 5_5 5.5 40.5 40.5 411.5 15: 5.5 5,3 12.5 51.1 55.5 5551551 5551 - 41.1 -1,2 51.2 4,5 25.5 51.3 14.5 152 15.2 -1.5 15,2 5.1 sa 7.5 25.1 50.1 25 11.2 25.5 5.2 T3 15.5 4.2 1:55151 5 V15-.-1 51154 25.4 -15 55.11 -5,5 25,5 5.1 15.1 -5.5 15.3 17.6 -1.5 25 5 22 15.0 _1 415 21 55 11 5.5 -55 _15 511111551 5555 -5.1 - 1.5 - -5.5 -.4 -- 11.5 ua 251 11.2 _1 4 54 21.5 35.1 55.1 5.5 21 5 -:1.r1 21.1 21.5 25.4 5.5 111.5 121 F. . 5556 4-3.5 1 'Zo ZU.1 19.1 29.5 135 12,7 /3_1 15,4 156 55m11r11=- 11151 _1 1 -- .114 41:1 - -211 15.5 15.1 -1_1 21.5 15,1 15.5 5-1.5 25,-1 51.2 'ruufrg 5551 -4.2 -5.4 4.1 24.1 2.5 251 15.4 -4.5 -5.0 55.9 25.3 54.5 15.2 :x1 21 1 54.5 45.11 -15.5 25.5 22,5 15.1 -5.2 -4_5 4.5 NOTP- indinaw H15 rwsidw?s were i Nm; irvdiciiss 5111555 of 11/5 Ejded GE .5 ar? 4? worse Tan recmed) ll 745 Tame 5: A1-'era jQri11?,Fytlania Schools 0101100101100 Rusiduals Reslduals Average APS Slaie Axferage APS Stain 1405200151 S3 SD 005101101 011 SE -0 P5 1 737 1 0-03 -1 0' 1-1841 0.103 ELA -0.54 1 177 0,905 -4.00 10,000 0.501 0,51 -I 015 1,003 2,00 10,474 5.714 Read 0.02 1.020 0.005 0.10 0.400 5.142 -0,14 1.402 1.004 -1.35 15.051 112.011 1 Grades M0111 -0.20 1.220 0.005 -3.26 14.021 11.200 1.000 19,450 E350 0.20 1.320 1.000 1.69 11.001 0.300 ELF-. -0.455 2.763 0.000 444 27.502 2-.051 -0.00- 1.004 1.002 -0.40 5.355 0.00 1.100 1.000 0.21 12,030 10.100 000 5.1214 0-Zi 11-120 5-SBU 110-00 0 01 -0,00 1.405 1.004 -0.51 -14,000 0.741 -0.07 1.1100 1.000 -0.70 15.419 11.107 Gf0d@_1_ 10011. l_ 3233? -3-25 28-469 _il 0.10 1 2.168 1 000 1,10 12.745 I 5.002 0.05 1.719 1 000 2,00 14.0031 0.143 ELA 0.15 1.050 0005 0.79 12145: 0.100 010 2140 10-00 00-3 101102 4010 0.10 1838 1.000 1.01 11.504 E257 reuc- ssnnol dem 0.05 1.935 0995 0.27 9.714 4.5186 000 1.000 1.000 1.04 10.000 12.171 0.13 1.700 1.c00 1.40 10.001 11.102 Grades mam 0011001110111 -0.04 1.601 0.000 -0.47 17.701 4.000 746 12 1 3 Tame 5' Resix1s|:iu1te12 arms -124; 1955 2519 1. .111 11: 11111 1 2:1-15 21:12 1ssa _#nr 1112 1425 uns 12711 111119 Quo Mah au 41 ws; 's um 11,09 :ns .- 39 a si 111113 _11111 1 1 asm 219, _arm 111 1:1 3 so 15 1 11114 fa 2 use 1310 LA 8 sas E117 111 111 1 12.12 3.75 13114 Juv: 11 111: 1111 1.11 21131 vue ass* 1111115 .sua 1 11 ann wsu :us 1/ 2211 1112 1-1111 1523 021 Aa; Rem 21717 11.11 1251; 119 cs 1 111 11" 1141 1n'3 ww as 24 <13 11117 Jaw 1 1 1014 zz se 21 1:1 105' .212 11111 -1 -1:11-1 194; ,am 11| |11 1115 2112 1 11 sm# Q1 11-ul 32 _11 an se as 1:1511 .1385 111111 1na5 vw sus 64 20 21 .110 :nn 15511 11 5112 2:14 1 -131 361 1.71 2.15 D. :xx 111,21 _um ?1 11,111 MJ 1:1 .12 '_11 4.411 -115 .1 1 (11 511| 3111 -1113 -111 1.1 5_5 2.15 -35 A21 31 111112 DQ 9 3 BB 2.111 -nm' .1112 :se - 1 - .mn .711 1~ .1 11 21:1 us' . :nm 11,2 411- uw- 1 2511 _1 - '-111 . . 55.21 11.21 .41 1 12 5 1115 .asc 114 224_ofa - 2:15 als 2252 _f mm 41.11-' man 1? n1- mm 011.1-1- 3026 711.11 99711 :mr 11' 11111 sw 1111 271.5 -131-1 -11.5 111.11 sm _1m. -use |1111 11.11 -111.11 A ;=s_v1 - 215+ - ?149 . 1.90 . :1 S1391 - _ess 2- 5_m_ - im 175 155 ass 41 7 D72 1541 11 212 id' 1133 101 .11 155- 2 .Bri 5555 us- :Lan n_ >>2_sr 1 -455 -sm _'im _s 2 Q52 sfz- mn - us: 11 use .211 - -|111 1 .114 -um -7.111 2311- _est . - 14 1 1 B11 5.25 ll?" EJ Ren 1.11. A S|:|1n1| STRTEAUIRAGE u>>im1 DT ua 11 AVERAAZE @1111 1 51 um|1au1u.11 111111111111 1111 1111. 1 11.1.1 11.11 me .1111 1 uw .1111 1111 111111 _111 2.1 1-1 1111 Msratinn $3 2217 13B 17111 CAR IQ U21 BB 1 .CIE 2 737 am/2115 :asf are Umm! 111- 112 .1 ww 1:1 1121 15,1 11110 C?wird WJW 2l53 |543 31140 2 i3 1 1055 '1 761i 1 5 I E3 511111. 11.11 1 1111 IIJB B7 15715 HM 3 num" 1111> 1 1 mem 1/1; 111 seas NJ IE. 5 1 j?11mpmn~1111 w1r?1H1uV1 wwf? C\11u1?1r -1 111111151 111-M1-11.19.5311 748 saw sm 51 ur 11119 111 ANDREW PORTER raacnxun From: Beverly Hal? [drbeverlyhall@gmaH.ccm} Sent Wednesday, May 19, 2010 914i AM To: ANDREW PQRTER Subject: Re: Atlanta reporton analyses Andy, I apologize but [just sawthis email. I will read the :eport and get back to you. Thanks, BH On Tuc, May 18, 2010 at 1:04 PM, AINIDREW PORTER Wrote: Beverly. The other day, I sent Bill the final report on our ana directly. I hop: you End this repnrl useful. All lhu best. Andy 1 1 sew. I heard from him, su I'm sending you a copy 000001 749 May 11, zu 10 Identifying Atianta Elementary Schoois that Had Unusually Large Gains in Student Achievement Test Scores Horn the Year to the Year' 2008/9 Andrew C. Porter Jennifer McMaken University efl-'ennsyivania in high stakes student achievement testing, the validity of results is especial?y important. Validity can be eornprornised in any one of a number of ways, one of which is through cheating. ln December of 2009, the Atlantic Journa? Constitution raised the possibility of cheating in Georgia schools by noting "improbably steep gains at some schools on tests taken tirst in spring and t?ten in summer" (June ll, 2009). Apparently, that prompted the governor's Office ol' Student Achievement for the state ol' Georgia to conduct erasure analyses that identined four schools in the state where a large number of student wrong answers had been erased and lilled in with the correct answers "causing passing rates on the stale's criterionfeferenced competency tests to spike" (June 11, 2009). In an October 18, 2009 issue ofthe Atlantic Ieurnal V- Constitution, reporter Heather Vogell, working with data analyst John Perry, reported on the results from regression analyses using spring 2007/8 results to predict spring results for grades 3, 4, and 5 for each of the tests in English Language Arts, Reading, and Mathematics for the elernentary schools in Georgia. Statewide, 19 schools were identified with "extraordinary gains or drops in scores between spring last year and this year. A dozen were in Atlanta" (October 19, 2009). Vogell wondered whether cheating might have occurred in these schools, which did substantially better or worse than was predicted from the schools' student achievement levels the spring before. Overview of Analvsis Plan The Atlanta Education Fund contacted the authors of this report to ask if we wou?d do analyses to the "aceuraey, the validity ofthe AIC's Endings and identify alternative explanations for wlrtat is found." Our analyses were to he focused on just the and 2003/'9 years in just the grades 3, 4, and 5, as analyzed by the newspapet'. We asked about analyses that extended beyond grades 3, 4, and 5, the focus ofthe newspaper analyses, to include grades 3 through 8 on which there is annual testing. We were advised to stay focused o11 grades 3, 4, and 5 We asked the state torteacl1er-level data so we could look at not only variance between schools, but also variance among teachers within schools, but the state could not supply. We asked for item-level responses so we could do some erasure analyses, but again the state could not supply. On December 3, 2009, we agreed to undertake the requested analyses. We immediately requested the nece ssary data lrom Melissa Fincher ol' the Georgia Department of Education, Data were obtained in a Hle suitable for analysis on January ZS, 2009. The tile did not contajn second grade results. data were requested on March 3, 2010 and received on April 15, 2010. ARF EL 014610 750 Replication of Newspaper Results Our lirst analyses sought to replicate the results ofthe newspaper analyses. 'l`he newspaper regressed 200 8/9 schooi-level achievement means on 200713 school level achievement means for each ol' the three grades and each ofthe three tests: English Language Arts, Reading, and Mathematics. The newspaper calculated residuals for each school in the state with grades 3, 4, and 5 where a residual is defined as the school's ae-tual mean level performance in comparison to its predicted mean level performance for spring 2009 based on achievement in 2008 for the grade prior. Schools with residuals larger than 4 standard enrors of estimate were identified as outliers. Using the same analysis file as used by the newspaper, we were able to replicate the newspaper results. As wili be seen below, when we used the database we received from the state, the newspaper results were only partly replicated, We are not sure why the newspaper provided database and the stme provided database were different. The newspaper database only contained school level results while the state database had student level results which we shen aggregated to the school level, Two Additional Analyses to Test Validity of Newspaper Results The newspaper results were based on all students in each ofthe two years. Thus, the results did not control for possible changes in student body composition. To control for possibte changes, we completed two analyses. First, we did analyses on longitudinal data. The analyses are based on only the students in a grade at 2009 that were in the same school in 2008 and had test scores for 2008. For example, the graders in 2009 were fin" graders at the same school in 2008. As yet a third analysis, we regressed 2009 on 2008 data at the school level, just as did the newspaper, but we added as control variables l2 difference variables contrasting the composition ofthe group of students in 2009 to the composition in 2008. These variables were differences in: enrollment, female, white, black, Hispanic, Asian, American Indian, multiracial, economically disadvantaged, disability, LEP, migrant. We call these regression analyses "student demographic." Results of all Three Analyses Table suinniarizes the results offour regression analyses: newspaper, newspaper analysis on state supplied database, Eongitudinal, and student demographic. The rows in the table indicate Atlanta schools with grades 3, 4, and 5 that were identitled as an outlier by one or more ofthe three analyses, Arr indicates that the school residual was four standard errors of estimate or more above (or below) the school's predicted level of achievement based on 2008 data (a very high standard for statistical significance but reasonable given the number ofstutistical analyses done). The columns 0l` the table divide the analyses by grade level and within each grade level by test and each test, by the three analyses. The set of schools in the table are for those with positive residuals and the last set of schools are forthe two schools with negative residuals. As can be seen in Table l, there were 22 instances of unusually lardge positive or negative residuals identified by the original newspaper analysis, seven at 3" grade, four at ell" grade, and eleven at grade. Fourteen of the 22 newspaper identified large residuals 2 AEF EL 014611 751 were replicated and 8 were not. The newspaper style regression on the state supplied data did not identify any new instances of unusually large residuals. Of those 22. eight were found in both the longitudinal analysis and the student demographic analysis; live were tbund in the demographic analysis only and one was found in the longitudinal only. Eight unusually large residuals tbund in the newspaper analyses were not lound in either the longitudinal nor the demographic, Three unusually large residuals found in the Eoiigitudinal analyses were not found in either the newspaper nor deniographic analyses; the demographic analysis did not identify any unusually large residuals unique to that analysis. Thus, ofthe newspaper identified large residuals, more than half' held up in part or in whole in our additional two analyses; 14 ofthe 22 were found in at least one of the two additional analyses with better controls. 'l`he eight fully supported newspaper results are across schools, In short, no one school was identified as having unusuaiiy large positive residuals ac-ross all grade levels and tested subjects. 'l`o the contrary, the unusually large positive residuals that were fully supported were unique to a grade level and tested subject at a specitic school. Blalock was the sole exception with two. One ofthe two partially supported results was in yet a ninth school. Large residuals were not systemic. Table 2 reports the multiple correlation squared for each regression equation and identities the signiicant predictors in the regression equation. All ofthe R25 are substantial as one might expect when prior achievement is used to predict subsequent achievement. There is no real pattern to these Rzs. Roughly from half to three lburths ol` the variance is accounted for by the predictors. Perhaps surprising, the longitudinal analyses did not yield the highest Res, longitudinal analyses used the same students in 2008 to predict their performance in 2009. In all cases, the 2008 achievement level was a signilicant predictor of achievement level in 2009. ln the student demographic analyses, there were as many as tive predictors in addition to prior acliievement. Student disability and economically disadvantaged were the two most common signiticant predictors, with gender and enrotlrnent next. The general conclusions lions these results are first, the Was a good predictor of performance in 2009 and second, the student demographic analysis, using changes in student body composition as predictors, worked in the sense that several of these variables were signiticant predictors and so, helped to control ibr shifts in demographics. As has been described, the longitudinal analysis was based on only the students that were 'tested in the sarne school, both in 2009 and 2008. Table 3 gives the percentages of students tested in 2009 that were tested in the same school in 2008 for each ofthe schools as having an unusually large residual, As seen in'1'able 3, statewide stability rate school-wide across grades 3, 4, and 5 was 74.63% and the Atlanta retention rate was 67.43, From grade 2 to grade 3, analogous percentages are 70. l7 and 66,51 and for gade 3 to grade 4, the percentages are 74.78 statewide and 66.28 for Atlanta and grades 4 to 5, 78,42 and 68.78. Clearly, year to year stability for students attending the same school is greater inthe state than it is in Atlanta. Table 3 gives the results for each individual school. Blalock had the lowest student stability rate, only 39.34% for grade 3 to grade 4, 45.10% for grade 4to grade 5, and 41.38% tor grade 2 to grade 3. One might 3 EL M4612 752 imagine with the relatively low stability for Blalock, newspaper results might be less likely to be replicated, but that is not fully borne out in the results, Blalock was identified by the newsp apcr as having an unusually large positive residual tor grade 3 English Language A.rts and Mathematics, grade 4 English Langiage Arts, grade 5 English Language Arts and Mathematics, Ofthose tive large positive residuals tor the newspaper analysis, two were fully supported, and one partially supported by the student demographic analysis. No other patterns emerge trom Table 3. Size ofResiduals When interpreting the results in Table l, a good question to ask is how large were these unusually large residuals? ln short, while they were unusually large, in terms ofheing more than four standard errors from what was predicted, were they also large in terms of the scale score metric? ln Georgia, scale scores are not vertically equated, making it impossible to compare a score in one grade level to a score in another grade level. Scale scores arc, however, equated fiom one year to the next so that comparisons over time within grade and subject are possible. They are structured to range from 650 to 900 or above. For grades 3, 4, and 5, 650 is the lowest possible score for each of the Three tests, The highest possible score is 920 for Reading, 930 for English Language Arts, and 990 for Mathematics, indicating a greater possible range of scores for Mathematics than for English Language Arts and for English Language Arts than for Reading. For the data in the analysis tile, standard deviations for English Language Arts and Reading were about 11.5, regardless of grade level, with the exception of grade 5 Reading where the standard deviation was lO.i. The mathematics standard deviations were larger, ranging from 18.6 for gade 3 to l9.7 for grade 4 and grade 5. In Table 4, each of the residuals identified by one or another ofthe three regression analyses that exceeded four standard errors are reported in hold. The largest residuals are for grade Mathematics. West Manor residuals, for which the newspaper tinding was supported, are 74.72 for the newspaper analysis, 75.42 for the longitudinal analysis, and 68.72 for the student demographic analysis. These results are 3-4 standard deviations ill size, which would be seen as large by any standard. or example, in education research, when the size of the efect of an intervention is being described, it is olten described in terrns ol' nurnb ers of standard deviations. An intervention that has an effect size of .5 standard deviations is seen to be a strong intervention. For the supported results, Table 4 indicates which analysis found the largest residual. The pattern was for the longitudinal analysis to yield the largest residual. For example, in Perkerson and Usher Elementary Schools Su' grade Reading, the longitudinal analyses got by far the largest positive residual in comparison to the other two analyses. We believe that the best control for shifts in student body composition from is represented in the longitudinal analyses, yet ollcn those were also the largest residuals. Table 5 reports the average residuals for Atlanta public schools by grade, tested subject, and each ofthe three analyses. First, statewide, the residuals have to average to zero; that is a statistical lhot, Each regression predicts values and the average predicted values equal the average actual values, For some schools, the prediction is lower than the 4 Alli' EL 014013 753 achievement level obtained; for others, it" higher. In 'lable 5, it is clear that the average residuals forthe Atlanta public schools do not differ much from zero. In short, across ai] schools in Atlanta, they are no more likely on average to have st large positive (or a large negative) residual than schools elsewhere in the state. Second, while there are differences across the three analyses, none are striking. All are less than live scale score points. Nevertheless the standard deviations of residuals are systematically larger for Atlanta schools than forthe state, with a tendency for the geatest variance in Atlanta to be for the student demographic residuals. The larger variance for Atlanta than for the state is consistent with the greater number ol` large positive or negative residuals for Atlanta. In Tabte 4, we can also investigate the possibility that the unusually large positive or negative residuals identilied, each of which exceeded the four standard errors ofcstimate criterion for size, might hide nearly as large positive or negative residuals that just missed the four standard errors criterion. Not surprisingly, there are additional largc positive and negative residuals, but not as many as might be expected. Eor grade Mathematics, Perlrerson has systematic-ally large positive residuals that didn't incct the four standard errors criterion. The residuals for Usher in Sd" grade Mathematics are large and positive across all three analyses despite not being flagged in the newspaper analyses. In English Language Arts at the grade level, all three regression analyses showed fairly large positive residuals for Benteen Elementary School, but only two of them matched the four standard errors or more criterion. In grade for Reading, the negative residual for Toomer comes close to being fully supported as there was a large negative residual the longitudinal analysis as well that again didn't meet the four standard errors criterion, For English Language Arts, the Blalock and Peyton Forest newspaper identitied schools could bc considered fully supported, as both the longitudinal and student demographic analyses estimated large positive residuals as well. For 3"l grade English Language Arts, the Blalock large positive residual identined by the newspaper could be considered supported cy the longitudinal analysis even though the residual didn't quite nncet the four standard errors criterion. Using a scale score residual of 15.00 or larger as a criterion, 41 ofthe EOS school by subject by grade level combinations had large residuals across ali three analyses. Still, not one ofthe l2 schools was tiagged across all grades and subjects. We looked at the report "2009 CRCT Analysis by School Report" on the Governors Office of Student Achievement website. In that report, they list "percent of classes flagged in the erasure analysis." Eight of the i2 schools flagged bythe newspaper analysis have 2/3 or more of their classes ilagged in the erasure analysis. The erasure analysis was done on the spring 2009 data just the sarne as the residual regression analyses. In order to dotmrninc the impact of the outliers on district-level pcricrrnancc, wc compared the 2009 residuals for ali students to the re-siduais when the students in grade 5 Alil* LL 014614 754 levels and tested subjects flagged as having unusually large positive outliers were deleted from the sample. Because our longitudinal analyses are the most convincing we identified large positive outliers based solely on longitudinal results. There were no outliers identified in grade 4 English Language Arts, nor Readinlg. Forty three students were eliminated for the reduced sample because of outliers for 4' grade Matliematics. For 51" grade, 63 students from English Language Arts, 132 from Readin g, and 105 from Mathematics were eliminated. Using both the fuii and reduced databases, we did district level regressions for the state, predicting '09 li'orn and calculating district level residuals. In the case of4"" grade mathematics, the Atlanta residual went from 1.01 to -.07 after deletion. For grade 5 English Language Arts, the Atlanta residual went from 3.00 to 2.5l; for Reading, the residuai went from 2.40 to 0.93 and in Math, from 3.03 to 6.58. District residuals decreased no more than 2.5 se-ale score points. In contrast, large flagged positive residuals as seen in Table 4 ranged irom a low of 26,3 to a high of 75.4 scale score points. We conclude that even if' the large positive outliers identified in the longitudinal analyses were due to cheating, and we're not saying that they were, eliminating those data from the sample had only a minimal effect upon district level performance. The residuals reported for the district can be thought of as an estimate of how much better (or worse) Atlanta' in 2009 was fiom what was predicted lioni 2003 performance. If Atlanta were getting better over time, these residuals would be positive if Atlanta were getting worse over time, these residuals would he getting negative. A look at Table 4 indicates that a. large majority of the residuals for longitudinal analyses for Atlanta calculated at the school level are positive. The district level residuals are largely positive though not statistically signiticantly greater than zero. Whether or not outliers are deleted, district level peribrrnance in 2009 was but not significantly better than predicted from 2008. Summary Over the course of the last several months, several analyses have been conducted of student achievement in Georgia public schools with an toward identifying possible instances ofelicating. One set of those analyses focused on changes between resuits in 2007/8 and 2008r`9 for grades 3, 4, and 5 on each ofthe tested subjects, English Language Arts, Reading, and Mathematics. The analyses reported by Heather Vogell inthe Atlantic Journal Constitution used a prior grade in 2008 to predict the following grades in 2009 for all ol` the students inthe school in those two years and those grade levels for each ofthe schools in the state. The newspaper analysis identified 12 schoeis in Atlanta. for which there were unusually large residuals. The analyses reported here investigated the validity ofthe newspaper results. The anaiys es explored the validity of those newspaper results by conducting anaiyses based just on students who stayed in the seine school for testing in 2008 and then again testing in 2009 at the next grade level. These are called the longitudinal results. We also added 6 GOSANAPS AEP EL 014615 755 shifts in student demographics as control variables to the newspaper analyses and these were called the student demographic anajtyses. Both the longitudinal analyses and the student demographic analyses were meant to provide controls not provided in the newspaper analyses for shifts from one year to the next in student demographics that might have accounted for the unusually large positive and occasionally negative residuals the newspaper identified and that might 'ce an indicator of possible cheating. Newspaper Endings not found also in the two additional analyses with better controls would be suspect. First, the size ofthe positive and occasional negative residuals were often large, as many as 3 or 4 standard deviations in size. Second, many of the newspaper results were supported both the longitudinal and student demographic analyses, Not all newspaper results were supported and as might be expected, the additional analyses identified additional subjects and grade levels in the already identified schools that might also be outliers having unusually large or occasionally unusually large negative residuals. No additional Atlanta schools were identined by these two additional analyses. To determine the effect of identified unusually large positive outliers on district level performance, we did district level regressions and calculated district level residuals for each ofthe tested suoiects in grades 3, 4, and 5. The impact on district level residuals of deleting outliers was minimal, indicating that overall district level performance in Atlanta improved from 2008 to 2009 with or without outliers. Our two adj acent year analyses do not address the question ofthe size and direction of district change in achievement over a longer period of years. We had access to only school level results forthe more recent erasure analyses published by the governors othcc of student achievement. Eight ofthe l2 newspaper flagged schools had 2/3 or more of their classes flagged in the erasure analysis. In summary, controlling for shifts in student demographics from 200810 ZGU9 through the longitudinal analyses and the student demographic analyses did not in large part negate the newspaper findings. Schools with unusually large residuals indicating that the school did much better or much worse than would have been expected 'eased on prior achievement found in one analysis tended to be found in the other analyses, as well as in the erasure artalyses. 'lliese results do not prove that cheating occurred, but they do point to student achievement gains and losses that are highly unusual and for which cheating could be one explanation. Nevertheless, unusually large residuals were not systemic across grade levels and tested subjects in a school, suggesting that the unusually large residuals are localized to specidc grades and subjects. Further,tl1e average residuals for Atlanta schools were comparahie to the average residuals for schools statewide. To the extent that there were unusually large positive residuals, these were offset by unusually large negative residuals in Atlanta schools, The standard deviations of residuals were, however, larger lor Atlanta schools than for schools statewide. In short, there were more large positive and negative residuals for Atlanta schools than was typical for schools in 7 AEF EL 014616 756 the rest ofthe- stale. Finally, it is also time that if ?1 school cheater! in 2008 Lo get student achievement performance and then cheated again in 2009 inthe same Way, neither the newspepcr analysis, nor our longitudinal and student demographic analyses would flag this school as an outlier with an unusually large residual, -i 757 GosAkAPs_AiaF_ni._n1461? Table 1' Ra ression Ana' sas GradP 3 (`rade4 Grade 5 Schuolhlame Ei 5 Ei Usher Elem Elemenia on Vouesl Elemertary Blalock Elemenian/ Buniuun Elurnm tu Capitol i Elementary Dunbar Elememary i 1 an F. L. SXAEFIIOH E?ernentary Betiiunn' Tuomer Elementary' 5 finale: Gulumn A Newspaper Analyses; Column Newspaper Analysis on The data sei received forihe slate and on which analyses under Column and are based; Column Column Studnm Demographics Analyses Inuicales Isle residuals were negative (school performed significantly worse than predicted) Readi mash RSHQEQ Maui Perkersnn Elemenfyy 758 Tebll: 2: niliuarri Predictors Grade Subject Anal sis R2 Signiticanl Predin?ors newspaper 0.?2 ela DS replinaliun 0.52 elir 05 lunniludiraal 0.43 ela EIS school derzs 0.63 ela 08 econxlisadvanlagelil disability newspaper' read UU Grade R01 replication 0.68 read GS 3 longitudinal 0.49 read C8 scrrool dem 0.69 read C8 disability newspaper 0.71 mari: 06 Mm replicalicn one mam as 0.n0 rnalh 05 school dem 0.62 main D6 disability LEP newspaper 0.72 ela E8 ELA replication 0 4? ela [18 lungiiudina? 0.56 Sia E8 school dem 0,56 els G8 Female multiracial ecomdisadvantaged newspaper 0.70 read OB Grade read fepllcatlorr 0.61 reae UB 4 longitudinal 0.50 mad 08 school dem 0.54 reari 08 female hispanic mulriraciil newspaper 0.59 rru=\ll| Dill main replication 0.59 math U8 lengiludinal 0.54 math D8 school dem 0.64 rnarh U8 mmlranlal newspaper 0.73 ela OB ELA replication 0.71 els OB 0.53 els OB school dem 0.72 ela OB erzrullmerrt disability newspaper Off? reed 08 Grade replieulinn 0.12 mea ua 5 Iuragiludirial 0.59 read us School dem ORS read U8 eocndisadvanlaged disability rruwspupcl' 0.02 U5 0.63 malh U5 lungriuarnar ees main ua rlern rnulh UB 759 disebllily Tabie 3: Percent of Students tested in 2009 that were tested is1 the same school in 2005 Ecllcol 09% GFZO8 to GFS IGGF4 Gl40510Gl5 School Name Ox-erlag O9 Overlap G9 Ovellag 175 Overlag STATE RETENTION RATE APS AVERAGE RATE SCHOOL RETENTION Perkerson Elementary School Usher Elementary' School Venelian West Manor Elementary School Payton Forest Elementary School Blalock Elementary School Bonlcen School Capitol View Elementary School Du nhar Elementary School F. L. Stanton School Bethune Tonmer 14,63 10.11 71.711 713.42 56.51 ee.2s sais :mo 52.17 64.44 sv wb 71.50 moo ee.1e 77.94 sn.s1 sane e1.sz 61,70 moe mme mee mae some v0.91 ea.s4 s1.ae 41,75 41,58 39.34 45,10 55.10 asm emo sue eoev ss.41 ewes 74,00 51,96 41.oe 51,35 51,115 552| 5 58,75 55.13 72.40 15.45 va.-sz 70.42 57.59 s1,es__ $1.54 1 1 sr/rrs Avsruss RAW RESDIURLS APE AVERAGE m1L1uA1_s WDIVBUAL SCHOOL Forms: asamcx Elementary Bnnfann Eiemenmafy L. SKHNEIR - ELA L- 0.1 Rsswuns ma 111.2 .2'2_s -10.5 2s.u 35,4 -115 .1.m 2.5 12.4 15.11 31.7 4.1 0.5 2.4 2:11. -5_1 -11.5 -1.7 17.5 .1_1 -5.11 -1.1 -0.11 1 Read .11 0 111 ua 1_0 :La 1.5 25.5 14.2 24.11 13,5 22.11 311.5 22.5 13.5 17.5 15.1 14.5 15.6 10.3 BR 9.1 15,5 15.5 15.7 17.4 40.3 10.1 3.5 5.3 25.9 11.2 :ss 9.2 -1,3 9.5 11.8 6,3 111] 21.7 27.11 23./1 206 121.5 1137 127 -17.1 -11.11 -YE4 -193 49,6 -1-5.l -2.5 -15,7 Grxdeii Grideli Rea.: mam ELA Read Math 5.5.1 2.2 .1.4 .za -1.11 02 1_7 -11.1 0_1 o_a on -0.5 -0_5 -as 11 zu 1o_s 19.7 19.7 1s.a 12.5 7.2 12.1 ~1:1.a .4.e >>17.0 531 21.1 21.1 .Q_s .err na 13.11 4_1-1 GT 3.3 -12.7 -14.5 42.5 2_0 4_9 3.7 5_5 9_5 5_5 12B 13.2 13.2 13.2 QA 30.11 27.1 27.2 24.9 15.4 15.5 -7_7 2_7 -4_3 3_2 24.5 10.0 -13.5 0,6 40,9 11.6 .112 55.5 21.2 15.3 /12.0 20.5 5_4 17 -1_5 7_7 31 11? 11.4 711 16.3 111.5 5,5 -as os 4.5 -155 -as -11.11 211.11 30.2 28.7 1e_\ 27.9 15.3 sas sm an.: 15.5 no as.: 22.4 19.2 211.7 4n.n ao: :ees z4.s 19.2 24.3 12.5 12,1 111.2 2.1 5.9 11.5 411.5 411.5 -5.11 s-1.2 27.7 211.7 25.5 uw 19.2 1u.z -11.9 -u.u -1u.2 5.7 u.u 7.9 21.1 511.1 23.7 53.0 25.2 31.0 25.3 201 25.3 -15.7 -15.3 -EYE -20.3 2_2 10,5 -1.3 -|15 2.7 .119 112 .1a.:a .4_7 .1.5 .1_u 22:1 20.1 17.2 .1o_s 0.1 37.5 391 .au 11.15 7.4 211.4 251: 23.5 '15 23.4 154 -13 D.U 3.21 -0_3 ?2 -12.0 -7_7 20.1 191 -1_2 0_3 41.1 4.1) -ILE -151.2 -4_2 13.9 11.5 7.1 5_8 1_2 BJ R1 5_4 7_8 -15.0 -15.4 -4.1 41.1 1_9 -2,5 -23,5 f2E.4 -29,7 -15.4 -4,5 43.9 29.3 15.2 23.1 21.1 -54.11 411.11 lne res1a.=.= were .me mar. pf=.1iaeu> 1301126 dll! are were Herriifird 5 uLiiHB 761 Mall! E- 0 1_0 1 4 23.3 2115 7517 45.2 3.8 zs 74-7 75.4 12 5 14.6 72.11 67.1 7_3 15.3 ,as 5.5 9_5 14.8 19.1 15.4 -af. -25.4 -52 -4.5 I2 0 47.5 18.2 o_o ss.. 5.9 85.1 41.2 -3.E 12.1 151 41.5 ?able 5: Average Residuals and S1andard Deviations forthe Allania Schools Grade 3 ELA Ruud Malh newspaper Iongiludinal schuci dem newsilapsr school dem newspaper lurruitudinal school dem Grade 4 Grade 5 ELA Read Malh ELA Read Math newspaper rungnudrnas school dem newspaper longliudlrral schooi dem longitudinal Qchuol dem rongnuuinar dem newspaper lorrgiludirral schuul dum newspaper lungiiudinal Standardized Reslduals Unstandardlzed Residuals Average APS Stale Average APS Slafe APS Residual Residual APS Resi-dual Residual Residual SD SD RusiduaF SD an >>0.25 1.737 1.003 -1.51 $0 841 6.193 0.01 1.102 1 D00 0.UE 11.523 10.450 1.112 1.417 0 995 -0.94 11.161 7.531 0.51 1.615 1.003 2.90 10.474 5.714 0.35 1.319 1.000 3.07 11.500 EI 51 1,543 0.995 3.16 9.548 6.153 -IJ 14 1.492 1.004 -1.35 15.051 10.011 -fl 18 1.048 1.000 -2.62 15.505 14.866 -0.08 1.252 0.995 ~1,01 15.141 12.034 |104 1.720 1.003 0.20 10.450 6.050 0.20 1.320 1.000 1.69 11.091 501-I -0.45 2.785 0.995 -4.44 27.562 9.351 -0.05 1.804 002 -0.40 9.794 5.388 0.03 1.183 1.000 0.27 12.059 10,109 0.03 1.274 0.995 0.29 11.120 5.580 -0.00 1.455 1.004 14.308 9.741 -0 D7 1,385 1.000 -0.79 15.419 11.107 -U26 2.243 0.995 -3.20 28.469 12.627 U19 2.150 1.005 1.10 12.745 5.802 EI 35 1.719 1.000 2.83 14.003 8.143 . 0.13 1.950 0.995 0.19 12.148 6.199 u.1e ilob?a o.au 15.362 4.516 0.16 1.835 1.000 1.01 11.504 6.257 0.05 0.27 13.714 4.900__ 0.09 1.608 1.005 1.04 39.636 12.171 0.13 1.7% 1.000 1.?0 59.001 11.132 -0.04 1.601 |1995 -0.47 17.79-I 4,9313 dem 762 I3 GEURGM 5EQUEST Dr, Beverly Hail Aikiitil. 'Demi :gi Ure Agn, .Secifon 50-18j-7_0 et ali, The* Ailhnta is iequestllig pubiigdocamenis with oi tliosepollieims syecificiilly exempted by slate law: -If you believe muy feq?iestenl material is exempt, pleasepmvide the citaiion from the law' that -As you know, tlieileorgia Open Regards Act search and reerieval fees after the hopr or a ma-ximum standard charge of 2-5 cents per page, or in the maintained on a- computer, the actual cosmf the disk or tape onto which the Please notify us of the cost, of reuieval and copying in adwim QE: ofilling the request. 'lihmileyou fer your I cajn bc contacted at (4lO4?526-7l 13 or via email at Alan aim he qoutzicred at QI zgudd@ajc,com. 'P-lease do not he?ifaiki Eo e?sttact either-ef me with questions. Qixxcereljl, Hea1:her"Vogell` faelploliur -Mari Jiidd Kepomer 763 Ka Rebecc? From: Pitts, Sharron Sent: Thursday, July 22, 2010 11:08 AM To: Kaye, Rebecca Subject: RE: Open Records Request Gates>>'GE reports, video, Dr. Hall honoranum records Rebecca, I with Dr. Hall and Dr. Augustine yesterday. Neither of them have the Porter report. Dr, Augustine will have to look to see if she received any material frem Westlicl when she returns on Monday. From: Yeager, Su -y Sent: Thursday, July 22, 2010 9:01 AM To: Kaye, Rebecca; Pitts, Sharron Subject: FW: Open Records Request: reports, video, Dr. Hall honorarlum records See below for the link to the video. Forwarded Message From: "King, Scott" <@ng@atianta.k12.ga.us> Date: Thu, 22 jul 2010 08:46:06 -0400 To: Suzanne Yeager Subject: RE: Open Records Request: Gates/GE reports, video, Dr. Hail honorarium records Yes, lt has been amhlved. maln.hl1n| From: Yeager, Su Sent: Wednesday, July 21, 2010 6:31 PM To: King, Scott Subject: Fw: Open Records Request GatesfGE reports, video, Dr. Hall honorarlum records IMPOHZHIICBI High Is this video in the media gallery? Sent from my Verizon Wireless BlackBerry From: "Kaye, Rebecca" Date: Wed, 21 Jul 2010 18:15:00 -0400 To: Augushhe, Pitts, Yeager, 2,ga.us> 1 764 00110 Ka e, Rebecca From: Kaye, Rebecca sem; Thursday, July 22-, 2010 3:55 PM 'Vogell, Heather (CNI - cg; Aclaszewski, Maciej; Bromeey, Keith RE: ORA l-ieathe r, As soon as sent this got information from AEF that the Porter report is included ln the Blue Ribbon investigative materials, so it will not ?e released until the repeat is released on August 2. .lust wanted don't have Etl (C) . Let me know what else can du to assist. l"l| follow up with you on Westlicl on i\/Tonday. -Rebecca From: Kaye, Rebecca Sent: Thursday, July 22, 2010 3:39 PM To: 'Vogell, l-leather (CNI - Cc: Aclaszewski, Maciej; Bremen/, Keith Subject: RE: ORA requests Heather, - A copy of the Porter report does not exist in the district. The WestEd repost will require additional research time. I do not beiieve we have received it either, but I :reed to verify with an employee who has been on leave and will return tu the oflice on l\/lonclay. will gel back to you on Monday with the information about the WestEd report. When I hear anything about the Porter report, will let you know. Hope yr3u're still feeling well with this heat! -rdl< Rebecca D. Kaye Director, Policy Development and Governmental Relations Atlanta Public Schools 130 Trinity Ave., SW Atlanta, GA 30303 404.802.2897 404.502.1867 (Fal:] Link to: Qalicies and :egg Link to: Proposed policies available for public curnrnent 1 765 DU113 The Atlanta Journal-Coristitution GEORGIA OPEN RECORDS ACT REQUEST Thursday, December 6, 2Ol0 Dr. Beverly l-lall Superintendent Atlanta Public Schools Dear Superintendent Qall, Pursuant to the Georgia Open Records Act, Section 50-18-70 et al., The Atlanta .louriialdlonstitution is requesting to review the following public documents with the exception of those portions specifically exempted by state law: Copies of all materials, including but not Eimited to documents and e-mails, provided to the special investigators via CD. If you believe any of the requested material is exempt, please provide the citation from the law that permits such exemption. As you know, the Georgia Open Records Act allows reasonable Search and retrieval tees alter the first quarter hour er a maximum standard charge of 25 cents per page, or in Lhe case of recoids on a computer, the actual co st of the disk or tape onto which the infoimation is transferred. Please notity us of the cost of retrieval and copying in advance of filling the request. Thank you tor your assistance. can be contacted at l3 or via email at Please do not hesitate to contact either of us with questions. Sincerely, Heather Vogell Reporter 766 Ka e, Rebecca From: Kaye, Rebecca Sent; Thursday, July 22, 2010 3:55 PM Tp; 'Vogell, Heather - Cc: Adaszewski, Maciej; Bromery, Keith Subject: RE: ORA requests Heather, As soon as I sent this got information from AEP ti-lat the Porter report is included in the Blue Ribbon imegrigative materials, so it will not be released until the report is released on August 2. .lust wanted to Eli you in gn why vve don'I have itl C9 Let me lcnovv what eisel can dc to assist. |'ll follow up with you on WeS'tEc| on Monday. -Rebecca From: Kaye, Rebecca Sent: Thursday, July 22, 2010 3:39 PM To: 'Vogell, Heather (CNI - Cr::Ada1ewski, Maciej; Bromery, Keith Subject: RE: ORA requests Heather, A copy ofthe Porter report does not exist in the district. The Vl>>'estEd report will require additional research time. do not believe we have receit-'ed it either, but I need to verily with an employee who has been on leave and will return to the office on Monday. I will get l:|acl< to you on the information about the WestEd report. When I hear anything about the Porter report, Iwill let you know. Hope y6u're still feeling well with this heat! ~rdl< Rebecca D. Kaye Director, Policy Development and Relations Atlanta Public Schools 130 Trinity Ave., SW Atianlra, GA 30303 404.802.1807 gl Date: MOH, Mal' 2019 17123257 -0500 To: Subject: RE: Atlanta report Dear ali, Sorry forthe delay, but attached is the next installment on our Atlanta analyses. The new stuff ls highlighted so that you-can rind it easily. What you will see is we did district-ievel regressions of 2003 on 2008 student. achievement statewide to-get residuals for 2009. You can think of these residuals as estimating the extent to which a district is getting better or worse from 2008 to 2009 in terms of student achievement We did the analyses once on all students and anotherhrne with the students deleted who were flagged as having unusually large school residuals. The text shows how many students were deleted from Atlanta. The purpose of these dislrict regressions was to see lf the large positive residuals lnliuenced the results at the district level. The answer is not much. 'l`he residuals for Atlanta grades 4 and Sacross the three tested subjects are largely positive, with one case ofa near zerc residual. At due same time, these residuals are notslatistically QFBGUSF than ZSIO- In Shaft, there was not much changein student achievement from 2008 to 20_D9'whether the unusually large positive residuals are included or excluded. Long story short, district level, if the unusually large positlve residuals were clue to cheating, they still aftect the district-level results _by much, Andy - - 769 UGUST ?10017 APS-KA - Identifying Atlanta,Eleinentary Schools that Had Unusual1y`LmgeGsins Student Scores Eoin fue Year 2007/8 to - Jennifer University of ln stelces student achievement testing, the validity of results is Validity can be compromised in any one offa number of wuys,.onc of which is through i_ cheating. In December of 2009, the Atlantic Joumal - Constitution raised the possibility of cheating in Georgia -schools by noting "improbably gains_at_some schools on tests taken first in spring and _then in summer" (June ll, 2009). Apparently, that _prompted the govemofs Office of Student Achievemciit for the state of Georgia to conduct erasure analyses that identified four schools in the state where a large number of student wrong answers had been erased and filled in with the correct answers "causing passing rates onthe state's criterion-referenced competency tests to spike" (June 11, 2009). In an October 18, 2009 issue of the Atlantic Journal Constitution, rcportcr' Heather Vogell, working with' data analyst John Perry, reported onthe results dom regression analyses using spring 20()7fS results to_predict.sp1ing 2008/9 results for grades 3, 4, and 5 for each ofthe tests in English Language Arts, Reading, and Mathematiesfor the elementary schools in Georgia. Statewide, 19 schoolswere Zidendlied with - "e2ttraordina.ry gainsor drops in scoresbetween spiing last year and this year. A dozen weve in Atlanta" (October .1 9, 2009). Vogell wondered whether cheating might have occurred irrthese schools, which did substantially better or Worse than was predicted from the schools' levels tlicspring before. -Overview is Plan I The Atlanta Education Fund contacted the authors of this xeport to ask if we would do analyses to determine-the f's.cc'uracy, the validity ofthe AJC's findings and identify alternative explanations for what is found." Our analyses were to he focused on just the 2007H8 and 200319 years in just the grades 3, 4, and 5, as analyzed by thenewspaper.. We asked' about 'analyses that extended beyond grades 3, 4, and 5, the focus ofthe newspaper analyses, to include grades 55 through 8 on which there is annual testing. We were advised to stay focused on grades 3, 4, and 5 . We asked the state for teacher-level data so we could look at not only variance between schools, but also variance 'mnong teachers within schools, but the state could not supply. We also asked for item-levei responses so we _coul_d do some erasureanalyses, but again the state could not supply. We asked for grade 2 student level scores but did not receive them, so the 3" grade analyses could not be done. At iirst, we thought that students were not tested in 2"d grade as they typically are not in other states. That must not be the ease, however, as the newspaper analyses used 2" grade. 1 770 APS-KAUGUST On December 5, 2009, we agreed to' undertake the requested analyses. We irnmediately requested the necessary from Melissa Fincher ofthe Georgia.Depa1't'fnent of i Education; Data were obtained in a iilesuitable tor analysis 'on January 28, 2009. Replication of Newspaper Results Our first %Uf&lyS - Sougl1t,to_ replietile the of the newspaper analyses. The - newspaper regressed 2008/9 school-level acliievenaent rneans on 2007/8 'school level achievement means for eachof the three grades and each .of the three tests: English Language Arts, Reading, and Mathematics. The newspaper calculated residuals for each school in thestate with grades 31, 4, 'and -5 where aresidual is .defined as t11e;scl1ool's actual mean level performance in comparison to its predicted mean level performance for spring 2009 based on achievement in 2008 for the grade prior: Schools with residuals larger than 4 standard errors ofes1:imate:Were identified as outliers, Using the same analysis lilc as used by the newspaper, wc were able to -replicate the iiewspuper results. Two Add@1z1fxnal?@_ es to lest Validity 0fNewsQ?lp_ er Results The newspaper results. were based on all students in each of the two years. Thus, the -results did not control for possible changes in student body composition. To control for possible changes, we completed two analyses, First, we did analyses on longitudinal data. The longitudinal analyses wcreonly possiljlefor ggndes-four and Evo becsusowe were notprovided grade test scores. The analyses are based on only the students in a grade nt2009 that were in the same school in 2008 and had test sooiesfoi' 2008.2 For example, the Su' graders in 2009 were 4m graders atfthe same sehooi in 2008; As yet a third analysis, we regressed 2009 on 2008 data at the school level, just as the . newspaper, but we added as control variables I2 diference variables contrasting the composition ofthe group of students in 2009 to the composition in 2008. variables were differences in: enrollment, female, white, black,_Hispanic, Asian, American Indian, multiracial, economically disadvantaged, disability, LEP, \`Ve call these regression analyses "student demographic." As stated previously, there was some confusion about the availability or not of grade 2 test scores. In one analysis Ele, we had the data provided by the state that we requested, which did not have grade 2. In another analysis file, we had the data given to us by the newspaper, which is what they analyzed, which in retrospectwe saw had gradei. we had earlier thought that the newspaper ar1alyses.l1ad regressed grade 3 in_20_09 on grade 3 in 2008. lnstead, theylaadregressed grade 3 in 2009 on grade 2, 2008. So, in all of our results, the newspaper results are based on the prior. year grade as a control. We could not replicate those grade 3 results with either of our longitudinal nor student demographic analyses because we didn't have grade 2 in the state providedsnalysis tile. We believe we could get grade.2 dataand do those .analyses if that is desired, In what tbllows in; results, we do report on a-student demographic analysis for grade 3,-but it grade 3 in 2008 to predictgrade 3 in 2009, controlling for shifts in demographics. For grades 4: and 5, the student demographic analyses use the prior grade in 2008 to predict the subsequent gradejn 2009. .V ali`Tl'iree Ax1sl}?es' - 2 . 771 APS-KAUGUST DD019 Table 1 the results of the three regression-analyses: newspaper, longitudinal, and student demographic. The rows in the table indicate Atlanta schools with grades.3, 4; and 5 that were identified es an outlier by one or more ofthe three analyses, An iH?1iCaf?S that ilw SC11QQ1 was -four standard errors. of estimate or more above (or below) the school's predicted level of achievement based on 2008 data .fa very-high for statistical .signidcaoce but reasonable g,iven_,the` number of statistical analyses done), The columns ofthe table divide the analyses by g.IHd?1evel_ and within each grade levelhy test and within each test, by the three analyses. The first. set of schools in table are for those with positive residuals and the last set of schools are forthe two schools with negatiye-residuals. As can be seen in Table l. there were 22 instances ofunusually large positive or negative residuals identified by the' newspaper analysis, seven at 3" grade, four at 4th grade, and eleven at Sm grade. .Of those 22, oigl1t'Wore_fou11d in both the .longitudinal analysis and the student demographic analysis. 'Two were found in either the longitudinal or student Thus, of the 4?hm1d Sn* residuals,_they often held up in part or in whole in our additional two analyses, The 4'hg .t1l'lCl..5th grade analyses are-better the grade analyses because they use as tl achievement in 200_8`in the prior' grade. The' eight fully supported newspaper results are across seven different schools. In short, no- one school was identified as having unusually large positive residuals across all grade levels and tested subjects. To the contrary, the unusually large positive residuals that were fully supported were unique to a specific grade level and tested subject at a specific school, Bladock was the sole exception wim two. One of the two partially supported results wus in yet ninth school. Large residuals were not systemic. Table 2 reports the squared for each regression equation and identifies the signicant predictors in the regression equation. All of the_R2s are substantial as one might expect when plior achievement is used to predict subsequent achievement. There is no real pattern to these Rzs, other than that in most eases, roughly two thirds ofthe vatianoeis accounted' for hy the predictors. Perhaps surprising, the longitudinal analyses did not yield the highest Rzs, yet the longitudinal the -same students in 2008 -to predict their perfonnance in 2009. In all eases, the 2.008 achievement level was predictor of the achievement level in 2009. In the 'student demographic analyses, there were as many as ive significant predictors in addition to prior achievement. Student disability and economically -disadvantaged were the two most common significant predictors, with gender and enrollment next. The general conclusions from these results are first, the information was good preclietorof performance in 2009 and second, the student demograpiaic analysis, using 'changes in student body composition as predictors, Worked in thesense that ,several of these variables were significant predictors and so, helped to control for shifts in demographics. As has been desciibed, the longitudinal analysis was based on students that were tested in the same school, both in 2009 and 2008. Table 3 gives the percentagesof students tested in,2009 that were tested inthe same school in 2008 for each ofthe schools 3 772 APS-KAUGUST 00020 - ~iden;n?d as having an unusuuiy large residual. as sea; in Table 3, statewide stability rate school-Wide across grades 4 and 5 was ?6.59% and the Atlanta retention rate was From grade 3' to grade 4, the analogouspercentages are 7458 and 66.25 and for evade 4 tesrade 5_.fl1? percentages are 78.42 statewide and 68.7s_ for Ariana. Clearly, year to year stability for students attending, the same school is greater in 131e state than it is in Atlanta, Table .3 gives theresuits for each individual school and thecolumns for grade' 3 to grade 4 and' grade 4 to grade 5 me rhost relevant to interpreting the longitudinal analyses. in both cases, Blalock_I'fad the lowest student stability rate, only 39,3 4% for grade 3 to grade 4 and 45.10% for grade 4 to grade 5. Onemight irnaginewith .the relatively low stability for Blalock, newspaper results might be less likely tube replicated, but that is not tiglly borne out in the results. Blalock was identified by the newspaper ns having an unusually large-positive residual for grade 3 English Language Arts and Mathematics, grade 4 English Language Arts, grade 5 English Language Arts and Mathematics. Of thosedive large positive residuals for the neivspaper analysis, two. were fully supported, and two were in 3'd grade. No other patterns emerge from Table 3. Size of Residuals - 1 -When interpreting the results in Table . 1, a good question to ask is how large were these unusually In short, while they were unusually large, in terms of being more than_four.standard what was predicted, were they also large in of - the scale score metric? In Georgia, scale scores are not vertically equated, making it impossible in compare a score in cinc grade level to si-score inanother grade level Scale scores are, however, equated irom one year to the next so that comparisons over tirne within grade and subject are possible. They are structured to range from 650 to 900 or above, For grades 3, 4, and 5, 650 is the lowest possible score for each ofthe three tests, The highest possible score is 920-for-Reading, 930 for English Language Arts, and 990 for Mathernatics, indicating a greater range of 'scores for Mathematics than for English Language Arts and for English Language Arts thanfor Reading. Foivtlie data in the state-provided analysis tile, standard deviations for English Language Arts and Reading were about 11.5, regardless or' grade level, with the exception of grade 5 Reading where the standard deviation was 1061. The mathematics standard deviations were larger, ranging fl`01I1 18.6 for grade 3 to 19.7 for grade 4 and grade 5. In Tabie 4, each ofthe residuals identined by one or another of the three regression analyses that exceeded four standard errors are reported in bold. The largest residuals are for 56" grade Mathematics, West Manor residuals, for which the newspaper Ending was fully supported, are 74.72 forthe newspaper analysis, 7 5.42 forthelongitudinal analysis, and 63.72 for the student demographic analysis. These results are 3-4 standarddeviations in "size, which would be seen as large by any standard, For example, in education - research, Ween the size ofthe effect of an intervention is being described, itis often described in terms of numbers of standard deviations. that has an effect size of .5 standard deviations is seen to be a si:rong_inte1vention_ - rted results Table 4 indicates which analysis found the largestbresi idal - - Forthe uppo The pattern was for the longitudinal analysis to yield tlwliargest residual. - Eerkerson and Usiner Elernen tary Schools 51? grade Reading, the longitudinal analyses got 4 773 u`no21 by-far the largest positive residual in comparison to the _otlrertvvo analyses. Webelieve that the best control for shilis in student body composition from is represented in the longitudinal analyses, yet often these were also the largest residuals. Table 5 reports the average residuals for Atlanta public schools by grade, tested suhjject, and each ofthe three analyses. First, statewide, the _residuals have "fo averagete zero that is a statistical fact. Each regression predicts values and the average predicted values equal the average. actual values. Forsome schools, thepredictign is lovyer than the :achievement level obtained; tbrothers, it's higher. In Table 5, it is' clear that the average residuals for the Atlanta public schools do not differ much frioin zero.-_ In short, across all, schools in Atlanta, they are no more likely on average to have a larjgepositive (or a large negative) residual than schools elsewherein the state. - Second; while-there are - differences across the three analyses, none are All are less than five scale score points. Nevertheless the standard deviations ofrcsiduals are systematically larger for Atlanta schools than for the state, atendency for the greatest variance in Atlanta to be for the student -dernographic residuals. The laicr variance _t`or'Atlanta than for the state is consistent with the 'greater numlzperrof large positive or negative residuals for Atlanta. - - - ln Table 4, we can also investigate the possibility that the unusually large positive or residuals each of which exceeded the four standard errors of estimate criterion for size, might hide nearly as large positive or negative residuals that just missed the four standard errors criterion. Not surprisingly, there are additional large positive and negative residuals, but not as many as rnight he expected. For Sn' grade Mathematics, Perkerson has systematically large positive residuals that >>didn't meet the four standard errors criterion. The residuals for Usher in Sm grade Matlieintliics large and positive across all three analyses despite not being flagged in the newspaper analyses, In English Language Arts at the sf* grade level, all three regression analyses showed fairly large positive residuals for Benteen Elementary School, but only two of them matched the four standard errors or more criterion. . In 40' grade for Reading, the- negative residual for Toorner comes close to being fully supported as there was. a large. negative residual for the longitudinal analysis as well that again didnit meet- the tour standard errors criterion. For English Language Arts, the Blalock and"Peyton Forest newspaper identified schools could 'oe considered fully supported, as both the longitudinal and student demographic analyses estimated large' positive residuals as well. For 3"i.grade English Language Arts, the Blalock large positive residual identified by thenewspaper-could be considered supported by the Student demographic analysis even though the residual didn't quite meet the _four stan dard errors criterion.. We looked at the report CRCT'Analysis by School Report" on the Governor's Office of Student_Achievement website. ln that report, they list the "percent of classes flagged in the erasure analysis? Eight -ofthe i2 schools flagged by the newspaper fc, analysis have 2/3 or more of their classes flagged in the erasure Et- I 5 774 224? APS-KAUGUST 000 analifsis was done on the spring_2009 data just the same as 'ri1e.resifiue1 :vie - - -- A - Over the course of tl1e:!ast several nlontlm, several analyses havcbof:n_cor1_ductod of -- Studezit achievement in Georgia publib schools with an toward identifyingpossible instances ofoheating. One set ofthose analyses focused on changes between zooms and zoosfs for gradee 3, 4,-and 5 on emu ofthe Language- -Pmiw, Reading, Heather Vog?ll in the - Journal --'Constitution used a prior gyaiie in 2008- to predict ihe followinggrades 6 775 23 APS-KAUGUST DUO in 2009 for all ofthe students in the school in those two years those for I each ofthe schools in the state. The newspaper analysis identihed 12 sehools in Atlanta for which there were unusually large residuals. Th? analgisesrepcrted here investigated the validity of -the newspaper results# The analyses explored .the Validity oi`_tl1Qsenewspaper results by conducting apalyses based just on students who Stayed in me same school for testing @2008 and then asain testing in 2009 at the next grade level. These are called the longitudinal results. we also' added shifts in student deinograplznics as control variables to the neivspaper analyses and these were called the student deihographic analyses. .Both the-longitudinal analyses andthe 'student demographic' analyses were meant to provide controls not provided in the anal :ees -for Shiite from one year to the 11cxti11_studentdemngraphics that ruigit have accounted for the unusually large positive and residuals the newspaper identified and that might be an _indicator of possible cheating, Newspaper findings not found also in the two additional analyses would he suspect, having not had adequate oontrols. - First, the size ofthepositive and occasional -negative residuals were often as 3 or 4 standard _deviations in size. Second, many ofthenewspaper results were supported with -huili the longitudinal -and student 'demographic -anal_yses,- Not all newspaper results were supported and as might be analyses _identified additional snbjeets and grade levels in the already identified schools fhat.rnighi' also be outliers having unusually large or occasionally' unusually large negativeresiduals. No additional Atlanta schools in the newspaper dataset-were identified by these two additional analyses. - levels lf? PM Qi dtliiliifi - - 'We weren't given the morereoent erasure analyses published by the governofs ofhee of. student acl1ieve1:nent at the grade and tested sjabj ect level, only as the school level. Eight ofthe l`2 newspaper flagged schools had 2/3 or more of=their classes flagged in the erasure analysis, In surnmary, controlling for shifts in student demographics from 204118 to 2009 through the longitudinal analyses andthe student demographic analyses did not inlarge part negate/the newspaper findings. Schools with unusually large residuals in_dica,ting that the school did rnuch better or inuch worse than would have been _expeetedibased onqprioifl acliievemerlt found in- one analysis tended to he found in the other analysesz afiavell aa the erasure analyses. 776 i UG UST 130024 APS-KA These results do not prove that cheating occurred, but theydo point to student achievement gains and losses that are highly unusual and 'for which cheating could be one explanation. Nevertheless, unusually large residuals werenot systemic across grade levels and tested subjects in a school, suggesting that the unusually largeresiduals are localized to specific grades and _subjects for atleast 10 ofthe 12 schools, Further, the average residuals for Atlanta schools were comparable to 'dieaverage residuals for _schools statewide. 'Ifo the extent that there were unusually large positixie residuals, these were offset by unusually large negative .residuals in Atlanta schools. The standard deviations of residuals Weie,-however, larger for Atlanta schools .than for schools statewide, In short, there wene rnorelarge positive and negative residua1s>>forAtlanta schools than was typical for schools in the-rest ofthe state. -Finally; it is also school cheated in 2008 to get better smdent achievement per-fonnance and then cheated again in 2009 in the same way; neither the newspeper analysis, hor our longitudinal iaud student demographic analyses Would' flag this school as an outlier with _au unusually large residual. 5 777 APS-KAUG UST 00025 0 'gsm . 3`?:Egg i 0 ?85n Firm 1_5 ?saw nasI-fi I-G 3 - - Lucy ef'mg"UUFindwc/am no Q, mu mga ot.- 1 ei? gf 2 JE I-_Qnru -UGUST GD026 APS gag Um_mEw>ummE_Egm_ no _gd Wm __amp_ _=nNm__u WD Sd Emgoorum QJW4 I E__mm_O_w ms md no ug__ Em_u_g?Um__ we mg _0_wEmgicmcmutuem "Egg mo gg EMO EQEGEUW immwx wo NE Ng ju NEO Mummgm 9 Eg IE "Em _cog Wm; _gi _"Emi I mga 3_3 gg wg; v_g_m_m_ _wmg 3_2 aww __gr_uw gm ggi aim _Eg E__mwEwm_ mga m_Em_zO__Fzm_E~_ maj NEB Ei gg ESM _ggml _io Em ?_mQiw mo Smow? Eg QQ g$g? if NOON GENW mr# UUMWBV GBE _ma meow; "mummy ECQUEM nm GENE Schools EU - ier IdenU11 Residuals furthe 12 Naw Table -11cn11-111: 1|-1.1.1 qt F3 3 11: 51' _ug 3331452-1 1r-_ .. . 1 1: +152 11.vnu 411;-Bolded ihat we med sig nhrias ofnla SS Caf? B2 lndi -Nd( (U 3 nu if as rfor G'led' 5 cqooi ps 19 3 .. aiiye- (5 .5 155' cz 3 . 1 UGUST UD029 -KA APS ogg 2 33 Ear gg E3 aol _emgoorom ENE mwvaw mod wg_?H mg; mg awkE?mgmn mm; m:_Nv Ed _""gm mid* mg; 83 who 6% gm: gm? _Wmm_o__ HQNN ww? gig? mg? gn: ?2 EQ mg: _Ebel _gi mg; _gdl Omg CNW: mg _gd EE waz: mam? EO Doo; ww; as gg wad NDS _wnvq an *Eg gm_O__ Mg; mg; __jm 32 EQ: mg gn: nog gg om_N_2_ Og gn: NONE gd SN: Vgm? wN_m_ _mag ww? _Emu__Co:Dm m%wWm3__ gi gg _w _mga _wud NE NE EXE moe Ed _og Omog _o3_ Hw_i_WmN_ _mjummwm mg Emp" mn; wma mmEm>< _uwivc Q5 EM U_~mu_m_mqw Em m_m__UwmN_ 9 EE<__mmVm__n? IONES DAY |420 STREET. - SUITF H00 GEDRGM. 30309 TELEPHONE: 484353) 3939 FACSIMILE: 4045318330 Direci Number: (404) fill l<8 502 rhdeauc@JoncsDay,com JPG030U4 June S, ZUEE 225169-600061 Michael J. Bowers, lisq_ Richard L. Hyde lialeh LLP 30 Ivan Allen Jr. BEvd,, Suite TOO Atlautii, GA 30308 Robert E. Wilson, lisq. Wilson, Monon Downs, LLC 125 Claircm-:ant Ave.. Suite 420 Decatur, GA 30030 Re: Entervlew of Dr. Beverly L. Hall Dear Mike, Bob and Richard: During yoor interview ol`D1'. llall on May 201 l, you asked about several matters which we we would look into get back ro you on Dr. i-Iaill's huhaili `l'hu first ofthose matters relates to your questions to Dr. Hall inquiring whether APS stat? Eawyers lied clone worli liar her personally or for family nieinhers. In responding to the question, Dr_ Hall answered generally but she then discussed an episode which you acknowledged recognizing. involving Mr. Felipe J, Farley. As We understand the matter, ul some point. Dr. Hall asked (ienerztl Counsel, Ms. whether a Stal? lawyer might answer a question that Dr. Hall was relaying on behall`ol`ltci' luisltand. details of this request are now sketchy, but Mia. Nlaicyck considered the matter de rarirririzis and told Dr. Hall that she had no objection to the request. Dr. Hall lhllowed up by asking, wltatevet' the question was, of Mr. Farley. Shortly tiicreoller, Mr. liarley gave Dr. Hail several pages which appeared to have been copied from some source, in response to Dr. l~lall`s question. Our u11derstanding is that the 'matter ended there and that notlting more was asked of Mr. ?'nrley. We believe these events occurred sometime in early 2007, prior to Mr. Fs1_rley' termination from APS. On July 7, 2009, more than two years after this innocuous request, Mr. lfarley sent Dr. l-lali a document which he eliaracteiived as an "invoice for personal legal Work." ln subsequent emails and correspondence from Ms. lviazyck and other outside counsel to Mr. Farley, even as late as December. 2010, Mr. Farley was asked to provide the basis for his invoice and to explain the nature ol' his representation warimitiugr a $2,300.00 lee request. l\/lr. Farley uzums nl ii irvm AND 0| 1 A in mimi init xrunr Hom" Kari: HciusToN vm moon io rf wut nu: mLxi