Report to the Admissions and Education Committee and the Committee of Bar Examiners
Regarding Public Comments on the Standard Setting Study
August 31, 2017

I.

Executive Summary

In February 2017, the Supreme Court of California (Court) called for the State Bar of California
(Bar) to undertake a “thorough and expedited study” of the pass rate on the California State Bar
Exam (CBX). The Court directed the Bar to ensure that the study includes:
“(1) identification and exploration of all issues affecting California bar pass rates; (2) a
meaningful analysis of the current pass rate and information sufficient to determine
whether protection of potential clients and the public is served by maintaining the
current cut score; and (3) participation of experts and stakeholders in the process,
including psychometricians, law student representatives and law school faculty or
deans.”
To accomplish this, the Bar organized the work into four interconnected studies – a Standard
Setting Study, Content Validation Study, and two studies focused on student performance on
the bar exam – and contracted with nationally recognized experts in the fields of professional
certification, testing, and psychometric evaluation. For the Standard Setting Study, Chad
Buckendahl, Ph.D., conducted the research with additional consultants contracted to observe
and review his work. Those reviewers included Mary Pitoniak, Ph.D., a nationally recognized
expert in standard setting, and Tracy Montez, Ph.D., Chief of Programs and Policy Review at the
California Department of Consumer Affairs, which includes the Office of Professional
Examination Services.
In addition to the external consultation provided by these subject-matter experts, the Standard
Setting Study was conducted with extensive stakeholder engagement including: bi-weekly
conference calls with deans of California’s law schools (ABA and California accredited and nonaccredited); and bi-weekly conference calls with a working group comprised of representatives
of the Admissions and Education (A&E) Committee of the Bar’s Board of Trustees, the
Committee of Bar Examiners (CBE) and Supreme Court staff.
Dr. Buckendahl’s report, Conducting a Standard Setting Study for the California Bar Exam
(Standard Setting Study) was submitted to the A&E Committee and the CBE on July 31, 2017.
That report was accompanied by a staff agenda item recommending the release of the
Standard Setting Study for public comment and presenting two options: maintaining the
1

 current bar exam cut score of 1440, 1 or reducing the cut score to 1414 on an interim basis to be
used exclusively for the July 2017 administration of the CBX. The A&E Committee and the CBE
accepted the report and released it for public comment immediately following the meeting.
In the month since releasing the report for public comment the Bar has solicited additional
input from stakeholders, holding two full-day sessions – one day each in northern and southern
California – to receive public comments, and distributing an online survey to licensed attorneys
and applicants for admission. In response to these calls for public input, the Bar received,
compiled, and summarizes in this report:
•
•

•

Over 5,000 public comments on the Standard Setting Study;
Over 34,000 survey responses from licensed attorneys in California, approximately 15
percent of the population of active and inactive attorneys to whom the survey was
distributed; and
Over 4,000 survey responses from applicants who took the July 2017 bar exam,
approximately 45 percent of the applicants to whom the survey was distributed.

These extraordinary measures have been undertaken to ensure that the process of evaluating
the bar exam pass line is analytically rigorous, inclusive, and transparent.
In addition to summarizing and responding to survey results and public comments, this report
provides simulations of the impact of different cut scores on pass rates as applied to 2008 and
2016 exam data; discusses a number of policy issues related to the impact of the cut score, and;
recommends that the A&E Committee and the CBE consider a third option regarding the cut
score on the bar exam: a total scaled score of 1390 which remains within the confidence
interval of two standard errors below the median score determined through the standard
setting exercise. Thus, the options presented for consideration following the period of public
comment are:
1. No change to the current CBX cut score; or
2. Adopt an interim cut score of 1414; or
3. Adopt an interim cut score of 1390.

1

Unless otherwise noted, this report will refer to total scaled scores when discussing the cut score. Throughout the
report the terms “cut score” and “pass line” are used interchangeably.

2

 II.

Survey Responses and Public Comments

Surveys of Attorneys and Applicants on the Cut Score
An online survey was distributed to all licensed California attorneys, including both active and
inactive members, to solicit their views about the proposed recommendations for the cut score.
The survey was open for nine days from August 10 through 18 during which time nearly thirtyfive thousand (34,295) attorneys responded (a response rate of about 15 percent). A slightly
modified version of the survey was also distributed to bar applicants who took the July 2017
CBX. It was open for eight days from August 11 through 18. Out of a total of more than nine
thousand applicants (9,175), more than four thousand (4,188) responded, (a significantly higher
response rate than that of attorneys, at 46 percent).
Key questions for both surveys included options on the bar exam cut score resulting from the
July 31 meeting of CBE and the A&E Committee:
•
•
•
•

Keep the current cut score of 1440;
Lower the cut score to 1414;
Lower the cut score further below the recommended option of 1414; or
Other.

Respondents were also asked to rate the importance, on a 10-point scale, of various factors
that might be relevant in consideration of setting an appropriate cut score. These factors
included:
•
•
•
•
•
•
•

Increasing diversity of attorneys from different backgrounds;
Increasing access to legal services for underserved populations;
The fact that the cut score in California is the second highest in the nation;
Maintaining the integrity of the profession;
Protecting the interest of the public from potentially unqualified attorneys;
Declining bar exam pass rates in California; and
The burden of student loan debt for law school graduates unable to find gainful
employment after failing the bar exam.

While not an exhaustive listing of policy issues relevant to the bar exam cut score, closed-ended
responses allow for statistical reliability in assessing the importance that respondents assign to
different factors. The collection of additional information related to respondents’ backgrounds
also provides context to better understand their views and the source of their concerns.
As shown in Table 1, the views of licensed attorneys and applicants on the cut score are sharply
divergent. About 80 percent of attorney respondents are opposed to lowering the cut score,
3

 whereas only 2 percent of bar applicants hold the same view. For those attorneys who favor
lowering the cut score, approximately 12 percent support the option of lowering it to 1414, and
four percent lowering it further, with the remaining five percent in favor of a variety of other
options, including raising the cut score, implementing the Uniform Bar Exam, eliminating the
bar exam entirely, undecided or no opinion.
More than 90 percent of applicant respondents support lowering the cut score and over half
(57 percent of all applicant respondents) favor lowering it below 1414.
Table 1. Survey of Attorneys and Bar Applicants on Bar Exam Cut Score – Position in Favor of
Cut Score Options and Relative Importance of Various Factors
Average Importance Rating on 10-Point Scale
(1 = Not at all important - 10 = Very important)
High
Protecting
Increasing California Integrity of
Public Declining Student
Access
Cut Score Profession Interest Pass Rate
Loan

% of Total
Response

Increasing
Diversity

Attorney (N = 34,295)
Keep the current cut score
Lower it to 1414
Lower it further
Other
Total

79.8
11.6
4.2
4.4
100.0

4.9
7.5
7.7
5.3
5.3

5.8
8.0
8.3
6.2
6.2

3.5
6.8
7.7
3.8
4.1

9.6
8.2
7.1
8.9
9.3

9.6
8.1
6.9
9.0
9.3

2.4
6.4
7.7
3.3
3.1

3.8
7.0
7.9
4.6
4.4

Applicant (N = 4,188)
Keep the current cut score
Lower it to 1414
Lower it further
Other
Total

2.3
36.4
56.7
4.6
100.0

6.4
7.9
8.4
7.5
8.1

7.4
8.6
8.9
8.4
8.8

4.1
7.9
8.1
7.3
7.9

9.6
8.1
7.6
7.8
7.9

9.5
7.8
7.2
7.7
7.5

4.1
8.2
8.8
8.5
8.4

5.0
8.5
9.0
8.6
8.7

Option

Importance ratings vary significantly between attorney and applicant respondents who favor
lowering the cut score. As shown in Table 1, public protection and integrity of the profession
were the top factors for attorneys regardless of their position on the cut score. For applicant
respondents in support of lowering the cut score, these are rated as the lowest two factors.
Instead, student loan debt and access are considered the most important factors, as well as the
recent decline of bar passage rates.
Additional findings from the surveys include the following:
Attorney Employment Type: Attorney respondents employed in government, academia and
nonprofit organizations are more likely to support lowering the current cut score. Other things
being equal, those in solo practice are least likely to support lowering the cut score.
Gender: Female respondents, both attorneys and applicants, are more likely to express support
for lowering the cut score, with a larger gender difference among attorney versus applicant
respondents.
4

 Race/Ethnicity: Among applicants, African Americans are more likely to support lowering the
cut score (at a marginally statistically significant level) than other applicants. Other than that,
there is no clear racial/ethnic distinction in applicant responses. When attorneys of color are
compared to white attorneys as a whole, attorneys of color are more likely to favor the status
quo. Broken into subgroups, the data shows a stark contrast between Asian and African
American attorneys, with the former significantly more likely to support status quo whereas the
latter support lowering the cut score; there is no discernable difference between Latino and
white attorney respondents.
School Type and Bar Exam Repeaters: Attorneys who graduated from non-ABA schools or took
the bar exam more than once before passing were more likely to support lowering the cut
score. Responses from applicants show no difference with respect to school types. Applicants
who took the exam once or twice revealed no difference either; repeat applicants with three or
more attempts, however, are more likely to be in favor of lowering the cut score. Repeat
applicants as a whole are also more likely to show preference for lowering the cut score further
below 1414.
In summary, the attorney and applicant surveys revealed a sharp divergence between attorneys
and applicants in their assessment of the desirability of lowering the cut score. In addition, the
profile of respondents shows that those who favor lowering the cut score are more likely to be
women; African Americans; attorneys employed in government, academia, or nonprofit
organizations; and those who had taken the bar exam more than once. Additional analysis also
shows geographic variations, with both attorney and applicant respondents from Bay Area
counties showing a significantly higher propensity in favor of lowering the cut score.
Public Comments and Hearings
After the joint meeting of the CBE and the A&E Committee, in which the Standard Setting Study
results were discussed and staff recommendations were approved, the Standard Setting Study
and corresponding options for consideration were released for public comment on August 1.
The public was able to submit comments through an online public comment form or via
dedicated email address. At the close of the public comment period on August 25, a total of
5,248 comments were submitted by online comment form, an additional 200 by email, and a
few dozen by US mail. During two days of public hearings on August 14 in Los Angeles and
August 15 in San Francisco, 25 individuals spoke on the topic.
The comments came from a variety of sources, including the general public, attorneys, bar
applicants, law professors, judges, researchers and institutions, from both California and across
the nation. Representatives of California law schools – ABA, California accredited, and nonaccredited – also submitted letters expressing their views on the matter. For those who
5

 submitted comments via the online comment form, about 35 percent selected one of the
options presented without providing additional narrative comments.
The comments touched on a variety of issues related to the Standard Setting Study, the bar
exam, and the legal profession in general, and varied from brief statements of a few words to
lengthy papers involving detailed research and analysis.
It is important to note that the comments are not a representative sample and cannot be
generalized to any specific population. The online public comment form had no mechanism to
prevent duplicate submissions. The number and source of comments submitted was also
affected by the attorney and applicant surveys that were distributed on August 10 and 11, as
shown in the chart below.
Before August 10, online comments were submitted at an average rate of 100 per day and the
proportion of responses in support of lowering the cut score outweighed those in favor of the
status quo by three to one. During the first two days after the attorney and applicant surveys
were distributed, more than 1,800 online comments were submitted, which also shifted the
aggregate views with respect to cut score options. Table 2 shows that, by the end of the public
comment period, 53 percent of online comments were in favor of the status quo, compared to
40 percent opposed.
Table 2. Selection of Cut Score Options from Online Comment Box
Option
Keep the same*
Lower cut score**
Other***
Total

Freq.
2,789
2,073
386
5,248

Percent
53.1
39.5
7.4
100.0

* Option 1 - keep current cut score of 1440 or if the option is modified.
** Option 2 - lower cut score to 1414 or if the option is modified.
*** Agreeing or disagreeing with both options.

In Appendix A the public comments are organized based on the method by which they were
submitted. Online comments are grouped by the three response types noted above; individual
comments within each category are then arranged in chronological order. Email submissions
are the second type, followed by all others. 2

2

Due to its size – over 1,500 pages – Appendix A is posted on its own link on the Bar’s web site.

6

 Major Themes from Public Comments
Due to the fact that the public comment period closed on August 25, the summary of major
themes provided here is mostly preliminary; lengthy comments with carefully crafted
arguments have however been reviewed in detail.
To a considerable degree the seven factors presented in the attorney and applicant surveys
appear frequently in various forms as major themes in the comments. For purposes of
organization, key issues discussed in the comments can be categorized as follows:
•
•
•
•

•

The cut score and its relationship to public protection and integrity of the profession;
The bar exam and what it is designed to measure with respect to attorney competence
and skills;
Economic issues related to the supply and demand for attorney services, touching on
student loan debt, with implications for access and diversity as well;
Critique of the Standard Setting Study methodology, submitted primarily by law schools
and academics in the form of letters, research papers, or public hearing presentations;
and
Reference to other states’ cut scores, considering California as an unjustified outlier and
citing benefits associated with lower cut scores.

Public Protection and Integrity of the Profession
While each of the broad categories outlined above reflects a wide diversity of views and
individual life experiences, arguments in support of a particular position on the cut score often
reflect the same concepts. Members of the public who referred to public protection and the
integrity of the profession as important functions of the bar exam generally took the position
that the cut score should not be lowered (with not a small number expressing support for
raising it). These opinions are often expressed with an unmistakable pride that the
commentators associate with the legal profession, as exemplified by the quotes below.
“…that California's cut score is the second highest in the nation is something to be proud
of, not something to be concerned about. I have interacted with attorneys from many
other states using lower cut scores. I may be biased, but I believe that in general,
California attorneys are better educated and more competent than attorneys
elsewhere.”
“I started community college late and was placed in remedial English and algebra....I
scored a 148 on the LSAT. If I listened to arguments about how testing formats and cut
scores would prevent me from achieving my dreams, I would have quit before I even
tried because my credentials were not those of the "people who become lawyers." I'm
7

 glad I didn't listen and believed in myself. I worked hard to be where I am, and that is a
source of pride.”
Those who take this view are prone to attribute the declining bar pass rate to the declining
quality of law school students or to law schools passing unqualified students. In contrast, those
who support lowering the cut score emphasize the need to consider other factors and question
the relationship between the current cut score and the mission of protecting the public.
“The State Bar needs more cut score studies in order to determine with reliable empirical
evidence what the proper cut score is to balance consumer safety, access to justice, and
diversity in the bar.”
“This high bar score does not at all reflect well on the integrity of the profession, rather it
goes against the progressive stance and nature of what it means to be CALIFORNIAN.
We are a state that prides itself in its forward thinking views, diversity, culture, and
fairness.”
Bar Exam and Measurement of Attorney Competence
“In my 15 years of practicing law, I have observed too many attorneys who lack minimal
competence. It is unclear to me how lowering the cut score to allow people to practice
law, who otherwise have not been able to pass, will HELP society.”
“…the bar exam does a poor job determining who is actually fit to practice law. If the
exam were more relevant and better tailored to determine minimal competence, then
the set cut score would be much more important and should not be lowered below what
California thinks is a competent threshold. However, the bar exam as it is now does not
test true competence to practice law.”
The two comments above represent opposing views regarding the CBX as a tool for measuring
minimum competence. Those who are opposed to lowering the cut score tend to share
observations of what they consider to be less than minimally competent attorneys in practice,
in particular the purported decline of the writing skills of younger attorneys. Recent changes to
the bar exam format - shortened from three to two days, Performance Test reduced in number
from two to one, and the weight of the overall written section lowered from 65 to 50 percent –
are also mentioned as evidence of the bar exam standard already being lowered.
In contrast, those in support of lowering the cut score raise questions as to the validity of the
bar exam in its various aspects. At the most basic level, many point to the mere fact of the
declining pass rates in recent years as an indication that something must be wrong with the
exam. Going further, some of the comments criticize the CBX’s heavy emphasis on rote
8

 memorization of a broad range of subject matters, a modality that is considered to be more a
reflection of applicant test-taking abilities than the requisite skills of competent attorneys. The
grading process of essay questions is also considered by many as flawed with testimonials from
individuals who failed the bar exam attributing that failure to the “arbitrary” and “subjective”
grading process that left them just a few points short of achieving the cut score level.
Independent of the role played by the CBX in testing competence, comments from those
opposed to lowering the cut score often expressed the view that those who failed the exam
lacked the discipline, grit, or work ethic to succeed in passing the bar and by implication, to be a
competent attorney. These comments focused on personal traits or character rather than skills
or knowledge acquired through learning and training. These comments – as shown in the
examples quoted below – reveal deeply embedded cultural values and often suggest intergenerational attitudinal differences, as one comment started by stating “This is typical Millenial
[sic] Stuff!”.
“I passed because of sheer grit and directed effort. I graduated as a special student from
a California accredited law school while being the sole supporter of a family of 5.”
“To me, the declining pass rate seems to be more of a millennial problem (I say this as a
millennial). Students don't want to work hard for anything, they expect it to be given to
them. If they're not willing to put in the work, they should not be admitted as lawyers.”
In addition to illustrating a generational shift in values, at a personal, psychological level the
interpretation of competence as a reflection of personal traits also suggests a common
cognitive bias at work on both sides of the debate. “Attribution error” refers to the tendency to
explain positive outcomes that we experience to our character and ability (I tried hard and
passed the bar) and negative outcomes to external, uncontrollable events (I didn’t pass because
the exam was problematic).
Economics – Supply and Demand of Legal Services, Access and Diversity, Student Loan Debt
Each of the seemingly disparate topics summarized in this section deserves its own treatment;
the synthesis presented here does not pretend to do justice to the complexity of the
interconnected issues. There is, nonetheless, a common thread of economic or market-related
concerns raised when issues of access, diversity, and student loan debt are discussed in
connection with bar pass rates and cut score. This is also evident in attorney and applicant
surveys, as discussed above, in which the ratings on access, diversity, and student debt are
closely associated with one another.
Many who provided comments in favor of keeping the current cut score asserted that there is
no shortage of attorneys in California as a primary argument against lowering the cut score. In
9

 fact, references to the job market for attorneys appeared in nearly a quarter of all online
comments taking a position in support of keeping the current cut score.
“To be clear we DO NOT have a shortage of lawyers in this state but we do continue to
have a shortage of high-quality legal jobs.”
“Consider: there is, most assuredly, no shortage of attorneys in the state -- judges will
tell you that there is a shortage of *effective* attorneys in their courtrooms. Lowering
standards will not address this problem, but merely make it worse.”
For those in favor of lowering the cut score, economic issues – whether employment, job
market, or student loan debt – are often presented in narratives involving difficult choices and
personal sacrifices, lost opportunity for gainful employment and service to the community,
crushing financial burden from student loan debt and mental agony in dealing with all of these
challenges. The emotional intensity appears to be captured in applicant survey results showing
that access and student loan debt are rated at the top as most important factors for applicants
in favor of lowering the cut score.
“…it has been an extreme financial burden as I have spent over $20,000 dollars in bar
fees, accommodations, and resources to prepare for the bar. Moreover, my employment
opportunities and earning potential are on standby until I pass the bar.”
“During my final semester of law school, I interviewed for legal jobs across the country.
When time came to register for a bar exam prep course, I was faced with a quandary: do
I bet on the California bar exam, in hopes that a California employer says yes to me? Or
do I study for a bar exam with greater reciprocity, such as the New York State bar exam,
to allow me to start practicing immediately in whatever state I may land a job?
Ultimately, I chose to take the California state bar exam, which cost me several
promising prospects.”
Reference to Common Cut Score Ranges in Other States
Many public comments point to other states’ cut scores as evidence that California’s pass line is
unjustified and extraordinarily high. These comments often cite the benefits associated with a
lower cut score, primarily by increasing diversity of the profession and access to legal services.
As to the specific cut score below the current level, two stood out as preferred by the vast
majority: 1390 (rounded from 1390, or 139 based on a 200-point scale comparable to other
states) and 1350, the most common cut score in other states.
Preference for 1390
Two main arguments are often presented as justifying a cut score of 1390:
10

 1. it is within the confidence level of the cut score range from the Standard Setting Study;
2. it is consistent with the score range from 1390 to 1439 for written exams that are
currently eligible for second read (some comments assert that papers within this range
are often comparable in their quality, aside from grader subjectivity, calling into
question the reliability of differentiating performance within this range).
It is true that from the Standard Setting Study result, 1390 lies at the lower end of the
confidence interval two standard errors away from the median. It should be pointed out,
however, the further a score point within the confidence interval deviates away from the
center (1440), the less confidence one would attach to the score due to the bell-shaped
distribution. Following the same reasoning, one would have similar confidence in preferring a
cut score to the right of the center at a higher cut score level. However, while statistical
evidence presents a framework to evaluate the degree of certainty in selecting a particular cut
score level, the decision must ultimately be informed by considering all of the policy
implications.
It is not true, though, that the exams currently eligible for a second read are indistinguishable
from those that pass. Historically the proportion of applicants who passed the bar after second
read has remained fairly constant around 15 percent of the total second-read papers. The
closer a first-read score is to the pass line, the greater the likelihood that it would pass after the
second read. For example, for applicants whose scores fell between 1430 and 1439 on the 2016
July exam, approximately 35 percent passed after second read. That proportion falls by almost
half for each 10-point decline within the second-read zone. For those at the bottom end of the
zone, between 1390 and 1400, less than two percent passed after the second read.
Preference for 1330 or 1350
Those who advocate lowering the cut score below 1390 often point to the fact that California
has the second highest cut score in the nation. Even lowering it to 1390 would leave California
with one of the highest cut scores in the country, particularly after Oregon and Nevada, two
states at the higher end, recently decided to lower their cut scores. These comments also tend
to point out that in states with lower cut scores, including New York with a cut score of 1350,
there is no evidence that the competence of attorneys is compromised or that more
disciplinary actions are associated with lower cut scores. As the range between 1330 and 1350
is most common in other states, a significant number of proponents for lowering the cut score
in California advocate this range by appealing to notions of the “wisdom of the crowd” or
“crowd sourcing” as validity evidence regarding attorneys’ minimum competence.

11

 Methodological Critiques of the Standard Setting Study
A third category of public comment stands in contrast to survey results from attorneys and the
majority of public comment narratives. This third category, primarily from law school
representatives and other academics, is more technical in orientation and tends to focus on
specific methodological and procedural issues. The assertions contained within these
comments call into question the very validity of the Standard Setting Study. As such, these
critiques are addressed systematically below.
1. Panelist Composition
One of the key elements of the Analytical Judgment Method (AJM) used for this study and for
standard-setting studies in general, is an emphasis on the transparency and inclusiveness of the
process. From the initial stage of recruiting panel members to the participation in the study, the
State Bar reached out to various stakeholders to solicit nominations. The final selection of
panelists made by the Supreme Court was drawn from almost 40 nominees, and consisted of
practitioners from a broad representation of backgrounds in terms of practice area, experience,
demographics, and geographic distribution.
Broad stakeholder participation in the panelist nomination process is an essential element of
the process for ensuring procedural validity of the AJM. In this instance, such participation was
sought from all three California law school types. Although Dr. Tracy Montez pointed out that
educators are generally not recommended to be part of the panel, two educators were selected
to serve on the panel. This decision was made to balance the desire for inclusivity with the need
for analytic rigor and procedural fidelity. Given the size of the 20-member panel, which typically
ranges from 15 to 30 depending on the particular setting and specific study design, coupled
with the large number of exam papers reviewed, we believe that the educators made
important contributions to the study without skewing the final results.
2. Minimum Competence Definition, Grading Rubric, and Panel Dynamics
Two other components of the method – a statement of minimum competence or Performance
Level Descriptor (PLD), and grading rubrics for exam papers – also generated feedback from
law schools. Two early versions of the PLD, excerpted and modified from bar exam grading
guidelines, were first presented for discussion with law school deans at the Law School Council
meeting in April 2017. The topic was on the agenda several times thereafter during the biweekly conference calls with law school deans that the Bar has consistently held since April.
The final version of the PLD reflected the combined input of law schools, Supreme Court staff,
and the study psychometrician, Dr. Chad Buckendahl, whose focus was on ensuring that the
language used in the statement complied with best practice in standard-setting procedures.
12

 With the PLD serving as the criterion to evaluate the performance of exam papers, grading
guidelines or rubrics are intended to provide more concrete information to panelists as a way
of focusing on specific elements or traits of the exam. Used with the more general definition of
PLD, the purpose is to ensure that the papers are evaluated on the basis of consensus definition
as criteria-referenced activity rather than norm-referenced activity that relies on the relative
performance of a set of papers being evaluated.
Criticisms directed to these two components of the study raise concerns ranging from the short
period of time devoted to the development of PLD, the lack of detail in the definition, and the
absence of fully-detailed grading rubrics for each exam question presented at the workshop.
Related issues are raised regarding the amount of time spent with the panelists at the
workshop to discuss and refine the PLD, or what was perceived as the excessively influential
role of one particular member of the panel, who had extensive experience grading bar exam
papers, during the discussions.
Many of the procedural issues flagged in these critiques are mentioned by one of the outside
reviewers, Dr. Mary Pitoniak. In her detailed overview and critique of the process, Dr. Pitoniak
took exception with a number of these same issues – she noted that more time could have
been spent defining the PLD and also noted the apparently outsized influence of one panel
member.
The comments on the PLD represent preferences rather than substantive flaws in the method.
The PLD was directly related to the judgments that panelists were asked to make on the
examination, specifically the written portion. As a policy statement, the language is intended to
provide a profile of what a minimally competent candidate knows and is able to do. This
purpose was accomplished through the initial draft and discussion among the panelists. The
appropriate question for the panelists was whether the samples they reviewed were
characteristic of the performance of minimal competency as guided by the PLD, and this was
accomplished during the workshop.
The original research design included individual rubrics for the evaluation of each question. The
decision to use a generalized rubric instead of question-specific rubrics was based on concerns
expressed by law school deans that rubrics at the item level would bias the panelists. Once
again, however, this modification in the method represents a preference rather than a flaw in
the method and balances the input of stakeholders without threatening the validity of the
research.
As a follow up to the Standard Setting workshop, Bar staff conducted interviews with panelists
to get their feedback about the workshop process. One of the questions specifically addressed
whether they felt that they had the opportunity to fully participate in the discussions, and
13

 whether any individual members had more influence than others in steering the direction of
the discussions. Of the 15 panelists interviewed, all but two members indicated their awareness
that one of the members seemed more vocal due to his in-depth knowledge of bar exam
grading.
As to whether their views had been influenced by this panelist or whether this panelist affected
the dynamic of the group discussions in an undesirable manner, the common response was to
dismiss his influence. The common sentiment expressed in these interviews was that the
panelists are all independent-thinking and successful attorneys, and that diverse views are
helpful; when people make a forceful argument, they still have to convince others. This reaction
is consistent with the literature on discourse analysis which emphasizes the role of social status
differentials in a group setting.
In her summary of the entire standard setting process, Dr. Pitoniak concludes that:
“in my opinion there were no fatal flaws. The panel-recommended passing score, and
the recommendations for adjusting it made by Dr. Buckendahl, represent credible
information for the Supreme Court to consider when they make their policy decision.”
3. Measurement Issues
In addition to the various procedural issues noted above, criticism of the study also focuses on
the technical issue of the measure of central tendency employed in the study. This critique
suggests that the cut score results derived from each of the panel members were too widely
dispersed and that the distribution pattern lacks the characteristics of a typical normal
distribution as the basis to calculate an overall mean or median cut score value.
The most detailed critiques focused on this aspect of the study came primarily from two
sources: Professor Deborah Merritt of Ohio State University, Moritz College of Law, and Dr.
Benjamin Nyblade, director of Empirical Research Group at UCLA School of Law. Professor
Merritt and Dr. Nyblade point to some of the same procedural issues discussed above but also
assert that the statistical patterns presented in their critiques demonstrate a “fatal flaw” in the
study results.
To summarize the response in advance of providing the detail that supports it: the analyses and
conclusions of Professor Merritt and Dr. Nyblade are correct in terms of the dispersion of the
data that they reviewed – but they were looking at the wrong data. Instead of reviewing the
entire range of panelist responses – 320 data points were used to derive the estimated mean
and median – these two critiques looked at only the final 20 data points that were calculated
from the larger sample.

14

 •
•
•

In the first step of the Standard Setting Study, the 20 panelists evaluated 30 essays for
each of four questions – a total of 120 papers per panelist for a total of 2,400 essays;
Panelests then selected four essays for each question – two marginally above
competent, and two marginally below;
Thus, the four different questions multiplied by 20 panelists and then multiplied by the
four essays selected yielded 320 data points – 160 on each side of the border of
minimum competence.

In Dr. Nyblade’s public comment, the presentation of these data points is compressed into a
graph that sets the minimum and maximum values equal to the range of these 20 values – from
380 to 460. This gives the appearance of cutoff scores that are, in Dr. Nyblade’s words, “all over
the map.” When looking at the entire process, however, including each step of selection, and all
of the data points used to calculate the 20 average values, the problem appears to be more
akin to looking at the inset of a map and noting how broadly dispersed the landmarks are within
it, without recognizing that this inset is on a different scale from the larger map under
consideration.
To reiterate how the final cut score recommendations were derived, it is important to note the
multiple steps that the panelists followed in evaluating the papers, the meaning of the data
derived from each step, and its interpretation in proper context. Before the panelists started to
formally evaluate the papers, they were first given ten papers as an exercise, shortly after the
conclusion of the PLD discussions, to apply the consensus concept of minimum competence to
exam papers as an independent judgment activity. With further discussion after this first round
of exercise activity, panelists began to review 30 papers for the first question. The 30 papers
were first classified into three performance levels: below, meeting, and exceeding minimum
competence. Analogous to a calibration session to further refine the PLD, the panelists honed in
on “borderline” cases by selecting two cases from the below-competence group and two from
the meeting-competence group; the exceeding-competence papers were not included in this
step.
Following the steps described above, each panel member classified 120 papers, for a total of
2,400 papers for the entire panel. The second part of the process produced rating results for
320 papers (4 papers from each question, 4 questions from each panelist, yielding 320 (4 × 4 ×
20). These 320 data points provide the basis for calculating the cut score recommendations
reflected in the study results.
The three graphs below show the distributional patterns of these 320 data points from
different angles. Figure 1 arranges the 320 papers into two groups: best of non-competent and
worst of competent papers. It shows where along the score ranges (vertical axis) the papers are
clustered, range of the middle 50 percent of the papers (rectangle boxes at the center of the
15

 leaves), and the median value within each group (hollow dots at the center). Scores for the first
group of the best of non-competent papers are clustered around 50 points (mode of the
distribution), with median value located at approximately 60 score points. For the second group
to the right, the scores are clustered near the center of the leaf around 60, with median value
located at approximately 65. Each group consists of scores from 160 papers.
Figure 1. Score Distribution Patterns of Borderline Papers

When the 320 papers are grouped by panel members, Figure 2 shows the average scores for
the two groups of borderline papers for each panel member. Red dots represent the worst of
competent papers and hollow squares represent the best of non-competent papers. The
average scores for the worst of competent papers (red dots) are expected to be higher than
those for the best of non-competent papers. That is, red dots are expected to be located to the
right of hollow squares. The distance between the two data points for each panelist could serve
as an indication of the degree of precision in identifying borderline cases at the margin. The
mid-point between the red dots and hollow squares for each panelist represent the cut score at
the level of exam questions.
It should be noted that, with the exception of two panelists whose mid-point falls around 50,
most mid-points fall slightly above (to the right on the horizontal axis) 60. At this level of
analysis, there are 16 data points for each panel member, again for a total of 320.
16

 Figure 2. Average Scores of Borderline Cases, by Panel Member

To review the convergence of the score points, Figure 3 shows the average value for each of the
two groups of borderline cases, as well as the entire borderline cases. The red dots represent
the average value, with the length of the vertical line through the dot representing margin of
error around the mean value. The average score for all borderline papers is derived as 61 score
points, with lower bound in the confidence band at 59.98 and upper bound at 62.15. Taking 61
as the overall average for borderline papers and aggregating to a full exam consistent of 5
essays and 1 Performance Test, the total score of 427 (61 X 7) approximates the mean cut score
of 428 reported in the final report of the study.

17

 Figure 3. Overall Score Point Averages of Borderline Cases

The analysis at each level of aggregation demonstrates the convergence of the scores derived
from the independent judgment of the 20 panelists, despite a small number of outlier cases. It
also provides indirect confirmation of the consensus among panelists in their discussion of PLD,
and the consistent application of grading guidelines to the performance of the papers.
Instead of looking at the entire sample of 320 data points that reflect the result of a two-step,
calibrated evaluation process, critics of the study chose to focus on 20 data points to assert the
absence of a valid distribution of the data, thus calling into question the validity of the entire
study. Admittedly the multiple steps in the standard-setting process might have caused
confusion regarding the proper unit of analysis. Regardless of the reason for the confusion, this
explanation should suffice to minimize any misunderstanding regarding these concerns about
the methodology.

III.

Policy Issues Related to the Pass Line

Though the CBX is an essential component of admission to the practice of law in California, no
exam can perfectly predict the performance of applicants, safeguard the public with certainty,
or ensure an adequate supply of attorneys to meet the demand for legal services. While the
admissions process addresses some of these factors – through the moral character evaluation

18

 and Multistate Professional Responsibility Examination, for example, others must be addressed
as matters of public policy.
The following section of the report looks at the potential impact of changing the cut score for
the CBX looking at the differential impact of the cut score on various applicant groups as well as
questions of access to justice and the diversity of the pool of attorneys in California. Based on
the public comments calling for a cut score below 1414, the fact that a cut score of 1390 would
still be within two standard errors of the median, and concerns that have been expressed
regarding access and diversity, we include in this evaluation an analysis of the simulated impact
of cut scores of 1414 and 1390.
The Simulated Impact of Changing the Cut Score in California
The implications of selecting a modified pass line of 1414 or 1390 are shown in Table 3 and
summarized below. This discussion describes what the impact would have been if the pass line
had been altered and applied to the 2008 and 2016 administrations of the CBX. 3 Because a
decrease in the cut score will necessarily increase the number of applicants who passed the
exam on the two samples to which we apply this simulation, the issues examined here relate to
the magnitude of the changes at different scores and the differential impact of the changes on
different sub-populations of the total applicant pool.
•

•

•

Simulating the impact of a cut score of 1414, the total percentage of applicants who
passed the exam would have increased from 61.9 percent to 65.5 percent in 2008 and
would have increased from 43.3 percent to 46.8 percent in 2016;
If the cut score had been 1390 for those two administrations of the exam, the total
percentage of applicants who passed the exam would have been 69.9 percent in 2008
and 52.1 percent in 2016;
Simulating the different pass points for men and women taking the exam:
o A 1414 pass line would have resulted in an increase from 60.9 percent to 64.3 for
men taking the exam in 2008 and an increase from 44 percent to 47.4 percent of
men taking the exam in 2016;
o Applied to women, a 1414 pass line would have increased the percentage
passing from 63 percent in 2008 to 66.9 percent and would have increased the
percentage passing from 42.5 percent to 46.1 percent in 2016;
o A pass line of 1390 would have increased the percentage of men passing the
exam to 68.9 percent in 2008 and to 53.1 percent in 2016;

3

Although the discussion here looks only at the simulated impacts of cut scores of 1414 and 1390, in response to a
public records request, Appendix B has been added to this report showing the simulated impact of cut scores of
1330, 1350, 1390 and 1414.

19

 •

o For women, a 1390 pass line would have increased the percentage passing the
exam to 71 percent in 2008 and to 51.2 percent in 2016.
Before presenting simulation data outlining the impact that different pass points would
have by various racial and ethnic categories it is important to make a brief comment on
the math involved. Groups with lower pass rates at the current pass line will almost
necessarily show greater percentage increases in pass rates as the pass line is lowered
than groups with higher pass rates at the current pass line. As a result, African
Americans and Latinos show the largest percentage increase in pass rates despite larger
raw numbers of whites and Asians who would pass at a lower pass line.
o Thus at a pass line of 1414:
 38.1 percent of African Americans taking the exam in 2008 and 23.1
percent taking the exam in 2016 would have passed, increases of 10.4
and 12.5 percent respectively;
 53.2 percent of Latinos taking the exam in 2008 and 37.5 percent taking
the exam in 2016 would have passed, increases of 8.8 and 10.6 percent
respectively;
 59.9 percent of Asians taking the exam in 2008 and 40.5 percent taking
the exam in 2016 would have passed, increases of 6.4 and 8.7 percent
respectively;
 71. 6 percent of whites taking the exam in 2008 and 54.9 percent taking
the exam in 2016 would have passed, increases of 5.2 and 7.2 percent
respectively.
o In addition to looking at the simulated changes within groups at different pass
lines, we can also look at the changes in the relationships among groups, or the
disparities in pass rates among different groups. Looking at the 2008
administration of the CBX:
 At the current pass line of 1440, the pass rate of whites is 97.1 percent
higher than that of African Americans, a gap of 33.5 percentage points. At
a simulated pass line of 1414, the gap in the pass rates between whites
and African Americans declines to 87.9 percent, and at 1390, the gap in
the pass rate between whites and African Americans declines to 65.6
percent, a difference of 27.9 percentage points;

20

 Table 3. Simulated Impact of Pass Rates at Different Cut Scores
2008 CBX
Cut Score

2016 CBX

1390

1414

1440

1414

1440

# Passing
% Passing
% Increase*

6,017
69.9%
12.9%

5,642
65.5%
5.9%

5,329
61.9%

4,010 3,598
52.1% 46.8%
20.3% 8.0%

3,332
43.3%

First Time

# Passing
% Passing
% Increase*

5,078
81.4%
8.5%

4,870
78.0%
4.0%

4,682
75.0%

3,317 3,066
64.5% 59.6%
14.5% 5.9%

2,896
56.3%

Repeat

# Passing
% Passing
% Increase*

939
39.7%
45.1%

772
32.6%
19.3%

647
27.3%

693
532
27.2% 20.9%
58.9% 22.0%

436
17.1%

Male

# Passing
% Passing
% Increase*

3,121
68.9%
13.2%

2,911
64.3%
5.6%

2,756
60.9%

1,970 1,760
53.1% 47.4%
20.5% 7.6%

1,635
44.0%

Female

# Passing
% Passing
% Increase*

2,890
71.0%
12.5%

2,726
66.9%
6.2%

2,568
63.0%

2,005 1,805
51.2% 46.1%
20.4% 8.4%

1,665
42.5%

Asian

# Passing
% Passing
% Increase*

1,205
64.8%
15.2%

1,113
59.9%
6.4%

1,046
56.3%

835
735
46.1% 40.5%
23.5% 8.7%

676
37.3%

Black

# Passing
% Passing
% Increase*

215
45.3%
31.1%

181
38.1%
10.4%

164
34.5%

146
117
28.9% 23.1%
40.4% 12.5%

104
20.6%

Hispanic

# Passing
% Passing
% Increase*

471
58.0%
18.6%

432
53.2%
8.8%

397
48.9%

478
419
42.8% 37.5%
26.1% 10.6%

379
33.9%

White

# Passing
% Passing
% Increase*

3,765
75.5%
11.0%

3,570
71.6%
5.2%

3,392
68.0%

2,369 2,165
60.1% 54.9%
17.3% 7.2%

2,019
51.2%

Other

# Passing
% Passing
% Increase*

71
57.3%
18.3%

67
54.0%
11.7%

60
48.4%

66
56
44.6% 37.8%
26.9% 7.7%

52
35.1%

ABA

# Passing
% Passing
% Increase*

3,767
82.3%
10.3%

3,571
78.0%
4.6%

3,415
74.6%

2,629 2,387
63.8% 57.9%
17.8% 7.0%

2,231
54.2%

CA Accredited

# Passing
% Passing
% Increase*

265
35.6%
35.2%

225
30.2%
14.8%

196
26.3%

169
131
21.9% 17.0%
69.0% 31.0%

100
13.0%

Registered

# Passing
% Passing
% Increase*

107
33.5%
40.8%

88
27.6%
15.8%

76
23.8%

44
38
16.2% 14.0%
25.7% 8.6%

35
12.9%

Out of State

# Passing
% Passing
% Increase*

1,369
73.2%
10.2%

1,307
69.9%
5.2%

1,242
66.4%

801
730
56.5% 51.5%
16.9% 6.6%

685
48.3%

Total

1390

* Percent increase of the number of applicants that would have passed under each simulated cut score level
relative to the number of passing applicants under the current cut score of 1440.

21

 

Looking at the 2016 administration of the CBX, the gaps are much larger
between whites and African Americans but the diminution of the
disparity is also significant at the simulated pass lines. At the 1440 pass
line, whites passed the bar exam at a rate one-and-a-half times greater
than that of African Americans (149 percent). Simulating the impact of
lower cut scores, that gap would have diminished to 138 percent at 1414
and to 108 percent at 1390.

Issues of Diversity and Access
Lowering the cut score on the bar exam and, as a result, increasing the number of attorneys in
California would not, by itself, increase the availability of attorney services for those who most
need them. Access to legal services depends on where attorneys choose to practice, the type of
law they choose to practice, the cost of legal services, and other factors beyond simply
increasing the pool of attorneys who practice law in California.
That said, data on the types of law practiced by attorneys of different backgrounds suggests
that attorneys of color and women tend to choose practice areas of public-interest and nonprofit law more often than white men. In a recent survey conducted by the State Bar of
California, just under four percent of white respondents indicated that they worked in the nonprofit sector. In contrast, over eight percent of Latino attorneys, six percent of African American
attorneys, and 6.3 percent of Asian attorneys indicated that they worked in the non-profit
sector.
These findings align with research conducted by Wendy Espeland and Michael Saunder who
found that law school graduates of color are more likely to start their law careers in
government and public interest law than their white counterparts. 4 As a consequence, issues of
diversity appear to be linked to issues of access.
The need for attorneys outside of the private sector has been documented by the Legal Services
Corporation (LSC) in a report describing the negative impacts that the justice gap has on varying
communities. In their work, LSC defines the justice gap as the “difference between the civil
legal needs of low-income Americans and the resources available to meet those needs.”
Specifically, the lack of legal services in civil matters has the greatest impact on seniors, rural
residents, veterans, persons with disabilities, parents of children under 18 and survivors of
domestic violence or sexual assault.
Surveys of 2,000 adults living at or below 125 percent of the Federal Poverty Level reveal that
while 71 percent of these households had at least one civil legal problem in the past year, only
4

See “Rankings and Diversity,” Review of Law and Social Justice, 2009, 18, 587-608.

22

 20 percent of low-income Americans sought professional legal help for these legal problems.
Low-income Americans are the least likely to understand the complexity of the legal system or
trust it as an institution to which they can turn for assistance. When confronted with a civil legal
problem, many low-income Americans may not even realize that the problem has a potential
legal remedy.
Diversity in the legal field is also important to the extent that public perception of the law instils
– or undermines – confidence in the legal system. New York State Court of Appeals Judge Jenny
Rivera asserts that there are three goals of diversity in law: establishing a profession that
represents the broad diversity of the population it serves; providing proof that the legal system
does not have barriers based on race, ethnicity, gender; increasing public confidence in the
administration of justice; and to promulgate the belief that the system is fair. 5
Diversity advocates argue that increased public trust and confidence in the legal system also
contributes to increased compliance with the law. The Strategic Plan of California’s Judicial
Branch includes Access, Fairness, and Diversity as the first of its seven Strategic Goals6 and
diversity considerations fall squarely within the State Bar’s newly adopted mission statement:
“The State Bar of California’s mission is to protect the public and includes the primary
functions of licensing, regulation and discipline of attorneys; the advancement of the
ethical and competent practice of law; and support of efforts for greater access to, and
inclusion in, the legal system.”
Despite the lofty judicial branch diversity goals that have been articulated for a number of
years, California’s attorney population remains disproportionately white and male.
Comparing the demographic make-up of California with the findings of a recent survey of
licensed attorneys in California shows that while Latinos make up 35.4 percent of the
population over the age of 18, they represent less than five percent of California’s licensed
attorneys. African Americans make up 5.9 percent of the state’s population over 18 years of age
but account for less than two percent of licensed attorneys while Asians comprise 13.8 percent
of the population over 18 but just under six percent of licensed attorneys.
While the root causes of disproportionate rates of passage are beyond the scope of this report,
it is clear that the bar exam exerts a disparate impact upon applicants of color. This impact,
when combined with disproportionately lower numbers of people of color in the pipeline to
higher education and law school, has resulted in a pool of licensed attorneys in California that
does not reflect the population that it needs to represent.
5
6

Rivera, Jenny. (2016) Diversity and the Law. Hofstra Law Review, 1271-1286.
Judicial Council of California website (2017). Retrieved from: http://www.courts.ca.gov/3045.htm.

23

 As Table 3 on page 21 shows, reductions in the CBX cut score would not eliminate disparities in
the pass rate among different groups of applicants. Reductions in the cut score would,
however, reduce those disparities and, if the patterns of career choice hold for new attorneys,
might also improve access by licensing more attorneys with a propensity to work in non-profit
and government.

IV.

Staff Recommendation

In the report submitted to the Committee of Bar Examiners at its meeting on July 31, staff
presented the Standard Setting Study report and, based on the study findings, recommended
that the Committee consider two options: 1) maintain the current cut score of 1440, or 2)
reduce the cut score to 1414 on an interim basis. After reviewing public comments submitted
during the public comment period, staff now recommended that the Committee consider a
third option: reducing the cut score to 1390.
Staff presents this third option for three reasons. First, in response to public comments urging
further reduction to 1390; second, because 1390 falls within the confidence interval from the
study; and third, in light of the analysis showing the positive impact on diversity and access that
a lower cut score may produce.
With these three options, the Committee would consider making a recommendation to the
Board for review during its September 6-7, 2017 meeting. The recommendation from the Board
would then be forwarded to the California Supreme Court.

24

 Appendix B
Simulated Cut Scores for July 2008 GBX
1330
1350
1390
1414

1440

Simulated Cut Scores for July 2016 GBX
1330
1350
1390
1414
1440

# Passing
% Passing
% Increase*

7,242
84.1%
35.9%

6,920
80.4%
29.9%

6,017
69.9%
12.9%

5,642
65.5%
5.9%

5,329
61.9%

5,451
70.9%
63.6%

5,053
65.7%
51.7%

4,010
52.1%
20.3%

3,598
46.8%
8.0%

3,332
43.3%

# Passing
% Passing
% Increase*

3,795
83.8%
37.7%

3,617
79.9%
31.2%

3,121
68.9%
13.2%

2,911
64.3%
5.6%

2,756
60.9%

2,679
72.2%
63.9%

2,484
66.9%
51.9%

1,970
53.1%
20.5%

1,760
47.4%
7.6%

1,635
44.0%

# Passing
% Passing
% Increase*

3,441
84.5%
34.0%

3,297
80.9%
28.4%

2,890
71.0%
12.5%

2,726
66.9%
6.2%

2,568
63.0%

2,722
69.5%
63.5%

2,525
64.5%
51.7%

2,005
51.2%
20.4%

1,805
46.1%
8.4%

1,665
42.5%

# Passing
% Passing
% Increase*

1,520
81.8%
45.3%

1,435
77.2%
37.2%

1,205
64.8%
15.2%

1,113
59.9%
6.4%

1,046
56.3%

1,161
64.0%
71.7%

1,066
58.8%
57.7%

835
46.1%
23.5%

735
40.5%
8.7%

676
37.3%

Black

# Passing
% Passing
% Increase*

314
66.1%
91.5%

287
60.4%
75.0%

215
45.3%
31.1%

181
38.1%
10.4%

164
34.5%

252
49.8%
142.3%

222
43.9%
113.5%

146
28.9%
40.4%

117
23.1%
12.5%

104
20.6%

Hispanic

# Passing
% Passing
% Increase*

621
76.5%
56.4%

591
72.8%
48.9%

471
58.0%
18.6%

432
53.2%
8.8%

397
48.9%

734
65.7%
93.7%

663
59.3%
74.9%

478
42.8%
26.1%

419
37.5%
10.6%

379
33.9%

White

# Passing
% Passing
% Increase*

4,368
87.6%
28.8%

4,200
84.3%
23.8%

3,765
75.5%
11.0%

3,570
71.6%
5.2%

3,392
68.0%

3,063
77.7%
51.7%

2,874
72.9%
42.3%

2,369
60.1%
17.3%

2,165
54.9%
7.2%

2,019
51.2%

Other

# Passing
% Passing
% Increase*

98
79.0%
63.3%

91
73.4%
51.7%

71
57.3%
18.3%

67
54.0%
11.7%

60
48.4%

100
67.6%
92.3%

93
62.8%
78.8%

66
44.6%
26.9%

56
37.8%
7.7%

52
35.1%

Total

Gender
Male

Female

Race/Ethnicity
Asian

B-1

 Appendix B
Simulated Cut Scores for July 2008 GBX
1330
1350
1390
1414
First Time or Repeat
First Time

1440

Simulated Cut Scores for July 2016 GBX
1330
1350
1390
1414
1440

# Passing
% Passing
% Increase*

5,657
90.6%
20.8%

5,521
88.4%
17.9%

5,078
81.4%
8.5%

4,870
78.0%
4.0%

4,682
75.0%

4,089
79.5%
41.2%

3,881
75.4%
34.0%

3,317
64.5%
14.5%

3,066
59.6%
5.9%

2,896
56.3%

# Passing
% Passing
% Increase*

1,585
67.0%
145.0%

1,399
59.1%
116.2%

939
39.7%
45.1%

772
32.6%
19.3%

647
27.3%

1,362
53.5%
212.4%

1,172
46.0%
168.8%

693
27.2%
58.9%

532
20.9%
22.0%

436
17.1%

# Passing
% Passing
% Increase*

4,240
92.6%
24.2%

4,119
90.0%
20.6%

3,767
82.3%
10.3%

3,571
78.0%
4.6%

3,415
74.6%

3,397
82.5%
52.3%

3,196
77.6%
43.3%

2,629
63.8%
17.8%

2,387
57.9%
7.0%

2,231
54.2%

CA Accredited

# Passing
% Passing
% Increase*

458
61.5%
133.7%

408
54.8%
108.2%

265
35.6%
35.2%

225
30.2%
14.8%

196
26.3%

356
46.2%
256.0%

294
38.1%
194.0%

169
21.9%
69.0%

131
17.0%
31.0%

100
13.0%

Registered

# Passing
% Passing
% Increase*

194
60.8%
155.3%

165
51.7%
117.1%

107
33.5%
40.8%

88
27.6%
15.8%

76
23.8%

111
41.0%
217.1%

95
35.1%
171.4%

44
16.2%
25.7%

38
14.0%
8.6%

35
12.9%

Out of State

# Passing
% Passing
% Increase*

1,629
87.1%
31.2%

1,556
83.2%
25.3%

1,369
73.2%
10.2%

1,307
69.9%
5.2%

1,242
66.4%

1,033
72.9%
50.8%

975
68.8%
42.3%

801
56.5%
16.9%

730
51.5%
6.6%

685
48.3%

Repeat

School Type
ABA

* Percent increase of the number of applicants that would have passed under each simulated cut score level relative to the number of passing applicants under the
curent cut score of 1440.

B-2