Case Document 421-112 Filed 06/15/18 Page 1 of 10 EXHIBIT 112 Case 1:14-cv-14176-ADB Document 421-112 Filed 06/15/18 Page 2 of 10 From: Bever, Erica Jane [erica_bever@harvard.edu ] Sent: Wednesday, May 01, 2013 5:56:26 PM To: Fitzsimmons, William CC: Pacholok, Olesia; Driver-Linn, Erin; Hansen, Mark Francis Subject: Admissions memo Attachments: LowIncomeAdmissionMemo_FINAL_20130501.pdf Dear Fitz, Attached is a memo describing our recent analysis of low income admissions. In the memo we describe our approach and results. At your suggestion, we reviewed a small sample of literature to put this in context and realized our approach was consistent with what others have done. We'd appreciate any comments or suggestions you have. We thought, based on our conversation last week, that it would also make sense to share this with Jeff Neal and Christine Heenan, Nina Collins, and Sally Donahue. Does that make sense? Are there others you would like to include in this conversation? Let us know if you have any questions! Best, Erica Erica Bever Assistant Director, Office of Institutional Research Harvard University Holyoke Center Suite 780 1350 Massachusetts Avenue Cambridge, MA 02138 617-495-2718 CONFIDENTIAL HARV00023547 Case 1:14-cv-14176-ADB Document 421-112 Filed 06/15/18 Page 3 of 10 CONFIDENTIAL DRAFT To: Bill Fitzsimmons From: Erica Bever, Erin Driver-Linn, Mark Hansen Re: Date: Harvard College Admissions and Low Income Students May 1, 2013 As you have discussed with us, there may be value in responding to recent press about the rate of admission for low income students at elite institutions and in particular for Harvard College. Critics like Bill Bowen have suggested for years that need-blind admissions policies prohibit Harvard and others from using important information to evaluate the application of a low income student. In Equity and Excellence in American Higher Education, Bowen, Kurzweil, and Tobin note that,"We see that there was no perceptible difference in the chances of being admitted, at any given SAT level, for students from the two low-SES categories and for all other(non-minority) students"(Bowen, Kurzweil, & Tobin, 2005). However, the reality in admissions may be more complex than need-blind policies suggest as noted in Caroline Hoxby and Chris Avery's recent study:"many admissions officers say that they use students' essays, teachers' letters, parents' education, attendance at an a 'under-resourced' high school, and similar indicators to identify, provide favorable terms of admission to, and strongly recruit students who they believe to be economically disadvantaged"(Hoxby & Avery, 2012). At your request, we undertook an analysis to determine if the chance of admission is any different for low income students, holding all other admissions characteristics constant. Below, we briefly describe the data used for our analysis and its limitations, our approach, and our findings. At the conclusion, we outline some issues we believe are important to consider prior to public dissemination of this analysis. Data Sources and Limitations Applicant data was provided to the Office of Institutional Research by the Office of Admission. Data on income comes from the CSS profile section of the financial aid application and was supplied to the Office of Institutional Research by the Financial Aid Office for the classes of 2009 to 2016. Of the 192,359 students who applied for admission for those classes,49% also submitted the CSS profile portion of the financial aid application. We do not have income data for students who did not apply for aid. Analysis Approach and Results Similar to the analyses conducted by Bowen et. al. in Equity and Excellence in American Higher Education, we first examine the admit rate of low-income applicants (defined as applicants with family incomes less than or equal to $60,000) by a measure of academic qualification (such as SAT score) to see if there is any evidence of a preference for low-income applicants. If groups of applicants with similar academic qualifications, but different incomes, are admitted at different rates, this might suggest the presence of a "tip" for low-income applicants. Exhibit 1 illustrates the relationship between income and SAT I score. Fewer than 20% of applicants in the lowest income group (Less than $10K) have SAT I scores above 750, while almost 30% have scores Highly Confidential - Attorneys Eyes Only HARV00023548 Case 1:14-cv-14176-ADB Document 421-112 Filed 06/15/18 Page 4 of 10 CONFIDENTIAL DRAFT below 600, where the admission rates are below 1%, without controlling for additional factors. As incomes increase, the proportion of students with SAT I scores above 750 increases, while the proportion with scores below 600 decreases. Based on a preference for high SAT scores in the admission process (applicants with SAT I scores lower than 600 have a very low chance of admission), we would expect that applicants from low-income families would be admitted at a lower rate. However, for all SAT I scores greater than 600, we see that applicants from families with incomes less than or equal to $60,000 are admitted at a higher rate than applicants with similar SAT scores from families with higher incomes (Exhibit 2). The differences noted above could be related to other factors important in the admissions process. In order to control for those potential issues, we implement a logistic regression model to predict the probability of admission, controlling for demographic characteristics and a variety of metrics used to asses qualification for admission. Demographic characteristics include gender and race/ethnicity. Qualifications used in admission include academic index, academic rating, extracurricular rating, personal rating, athletic rating, and legacy status. This approach has several limitations; we picked a small set of variables that would factor in admissions decisions. The selection of a wider set of variables might result in a better fitting model, one that accounts for more of the variation in individual applicants and their potentially unique contributions to the entering class. For example, the model does not capture exceptional talent in art or music explicitly (although ratings may capture some aspect of these attributes). In addition, our model is limited to main effects, not examining interactions between variables. Our analysis should not be considered exhaustive. In spite of these limitations, the logistic regression model results are consistent with the descriptive analysis described above and shown in Exhibits 1 and 2. Exhibit 3 illustrates the difference between the predicted admission rate and actual admission rate for students at each income level. The predicted rate controls for demographics, legacy status, athletic skills, ratings, and measures of academic qualifications. Given what we know about the relationship between income and SAT scores and the extracurricular opportunities available to low income applicants, we would expect low income applicants to be admitted at lower rates than their peers (this is reflected in predicted admit rates). However, we find actual admission rates indicate that applicants with incomes below $120K are admitted at higher rates than we expected. To get a sense of the size of the admissions advantage conferred to low-income applicants relative to other groups of applicants, the so-called "thumb on the scale," we include low-income status in a second logistic regression model. The table below is sorted based on the effect size of each of the variables included in the model. The variables with the largest effects on the probability of admission are athletic rating, personal rating, and legacy status. Compared to athletes and legacies, the size of the advantage for low income students is relatively small. Table: Logistic Regression Predicting Admission from Classes 2009 through 2016 Variable Highly Confidential - Attorneys Eyes Only Coefficient P-value HARV00023549 Case 1:14-cv-14176-ADB Document 421-112 Filed 06/15/18 Page 5 of 10 CONFIDENTIAL DRAFT Estimate Athletic rating of 1 Personal Rating 1 or 2 Legacy African American Native American Extracurricular 1 or 2 Academic 1 or 2 Standardized Academic Index Hispanic CSS self-reported income less than or equal to $60K International Asian Constant Unknown/Other Female 6.33 2.41 2.40 2.37 1.73 1.58 1.31 1.29 1.27 0.98 0.24 -0.37 -6.23 -0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.41 0.87 N = 192,359; Pseduo R2 = 0.45 The relative sizes of the admissions advantage conferred on different groups can be seen by looking at the differences in actual admit rates as well. In Exhibit 4, we limit our analysis to students with high academic ratings(1 or 2) and examine the differences between athletes and non-athletes, legacy students and others, Asian students and all other students, and low income students and all other students. An athlete that is also an academic 1 or 2 has an admit rate of 83% compared against 16% for non-athletes with an academic 1 or 2. Fifty-five percent of legacies who are academic is and 2s are admitted compared with 15% of all other academic 1 and 2s. Asian applicants with an academic 1 or 2 are admitted 12% of the time compared against an admit rate of 18% for non-Asian applicants. By comparison, low income applicants with an academic 1 or 2 have an admit rate of 24% compared against 15% for all other applicants. Issues to consider before sharing these results publicly We imagine that sharing any analysis of admission weights will draw attention to the variety of factors that compete with one another in the admissions decision. To state the obvious, with only -2,200 spaces for admitted students per year, implicit tradeoffs are made between athletes and non-athletes, legacy admits and those without affiliation, low income and other students. We know that many are interested in the analysis of the relative tradeoffs. While we find that low income students clearly receive a "tip" in the admissions process, our descriptive analysis and regression models also shows that the tip for legacies and athletes is larger and that there are demographic groups that have negative effects. Highly Confidential - Attorneys Eyes Only HARV00023550 Case 1:14-cv-14176-ADB Document 421-112 Filed 06/15/18 Page 6 of 10 CONFIDENTIAL DRAFT Works Cited Bowen, W. G., Kurzweil, M. A., & Tobin, E. M.(2005). Equity and Excellence in American Higher Education. Charlottesville and London: University of Virginia Press. Hoxby, C. M., & Avery, C.(2012, December). The Missing "One-Offs": The Hidden Supply of HighAchieving, Low Income Students. NBER Working Paper Series. Highly Confidential - Attorneys Eyes Only HARV00023551 1:14-cv-14176-ADB Document Filedby 06/15/18 Page 7Classes of 10 Exhibit 1: DistributionCase of Applicant Average SAT421-112 I Scores Income, of 2009-2016 CONFIDENTIAL DRAFT • Income and SAT scores are positively related. 100% 90% 80% 70% 60% 50% 40% - •750+ 30% - •600-750 20% - •Less than 600 10% I I t I le 0 0 0 0 0 0 0 0 a—I CV Cfl Sr Lfl o r•-• co -to. 0000000 m Highly Confidential - Attorneys Eyes Only I I 0 I 0 0 0 0 0 0 0 0 0 0 0 0 rO St in to r-• co cri 0 a—I CV CV CV CV 0 030 0 0 0 0 0 0 0 0 0 VT CI 0 a—I (NCO Sr in t..0 h CO a—I a—I tr} 4.4 tr} + 0 0 0 m in 0 CV CV (-4 In esi 0 0 0 Oi 0 t•Nr (NJ in. in. tn. 0 0 0 CV CO St rsi No Aid Application 0% HARV00023552 1 I 1:14-cv-14176-ADB Document 421-112 Page 8 of 10 Exhibit 2: Admit RatesCase by Income and SAT Score, ClassFiled of 06/15/18 2009-2016 CONFIDENTIAL DRAFT • Using SAT as a proxy for admissions qualifications, we see at every score level, lower income students have higher admit rates. 40% - Less than or equal to $60K 35% 30% Greater than $60K* 25% 20% 15% 10% 5% 0% 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 Notes: The analysis above uses the average of the maximum math, writing, and reading scores a student received. Average SAT I Scores less than 600 are excluded from the exhibit above asgtinpateiroglfiiitepkatatpglefeksomifh SAT I scores less than 600 have an admit rate of less than 1%. HARV00023553 * Category includes those with no (missing) self-reported income 2 I Case 1:14-cv-14176-ADB Document Filed 06/15/18 of Page 9 of 10 Exhibit 3: Predicted and Actual Admit Rates by 421-112 Income, Classes 2009-2016 3 CONFIDENTIAL DRAFT Predicted and Actual Admit Rates by Income Band 14% 12% 11% 11% 10% 10% 9% 10% 8% 13% 6% 11% 10% 9% 4% 8% 8% 6% 2% 0% $0-40K $40-80K •Predicted Admit Rate $80K-120K $120K-160K $160K-$200K 0 Difference Between Predicted and Actual Admit Rates $200K+ unknown •Actual Admit Rate • Predicted admit rates by income are based on logistic regression models that control for academic index, academic rating, athlete, legacy, extracurricular rating, personal rating, ethnicity, and gender. • Low income students are admitted at higher rates than predicted. Higher income students are admitted at a lower rate. • Admit model has a pseudo R-squared of 0.44 Highly Confidential - Attorneys Eyes Only HARV00023554 Document 421-112 Filed 06/15/18 Page 10 of 10 Exhibit 4: Admit RatesCase by1:14-cv-14176-ADB Selected Characteristics, Classes of 2009-2016 CONFIDENTIAL DRAFT • Among top academic achievers (academic rating = 1 or 2), those who are athletes or legacies have much higher rates of admission. • Low income achievers also have higher rates of admission. • Asian high achievers have lower rates of admission. Average Admit Ratesfor Top Academic Achievers(Academic Rating 1 or 2)by Selected Demographic Characteristics 90% — 80% — - - - 70% — - - - 60% — - - - - 50% — - - - 40% — - - - 30% — - - Admit Rate for All Academic 1 & 2 = 16% 20% — - - - - 0% # in group # not in group Athlete Legacy Low Income Asian 501 2,636 7,065 24,692 74,755 72,620 68,191 50,564 •Admit rate for group Highly Confidential - Attorneys Eyes Only Amit Rate for those not in group HARV00023555 4 I