Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 1 of 30

Early Intervention
System Audit Report
April 2019

1

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 2 of 30

Introduction
Under Paragraph 223 of the Consent Decree, the Court retains jurisdiction over this
matter “until such time as the City has achieved full and effective compliance and
maintained such compliance for no less than two years.” On January 10, 2018, the Court
entered an order finding the Department to be in “full and effective compliance” as of
the date of the Order, thus commencing at that point the two-year “sustainment period.”
Dkt. #439. The Court further ordered the parties and the monitor to “meet, confer, and
prepare a plan for discharging their obligations under the Consent Decree” during this
two-year period.
On March 13, 2018, the Court entered an order approving the Sustainment Period Plan
developed pursuant to the Court’s January 10th order. This plan, and an attached matrix
of deadlines, became the governing documents for this Sustainment Period.
As relates to SPD’s obligations under the Consent Decree, the matrix contains deadlines
for SPD self-reporting or assessment of specific topical areas (“Commitments”) of the
Consent Decree. These assessments, implicated paragraphs of the Consent Decree, and
associated deadlines are provided in the chart that follows:

Sustainment
Matrix
Deliverable
Nos.
6
114
120
134
142
99
111
172
180
25
33
42
50

Systemic
Report/Audit/Assessment

Filing Deadline

Community Engagement
Stops and Detentions – Outcome
Report
Stops and Detentions – Audit
Report
Crisis Intervention – Outcome
Report
Supervision

May 31, 2018
June 30, 2018
May 31, 2019
January 31, 2019
October 31, 2019
October 31, 2018
October 31, 2019
October 31, 2018
November 30, 2019
October 31, 2018
October 31, 2019
October 31, 2018
July 31, 2019

Force Reporting, Review, and
Investigation – Type I
Force Reporting, Review, and
Investigation – Type II
2

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 3 of 30

Sustainment
Matrix
Deliverable
Nos.
105
56
62
79
71
194
202
156

164

Systemic
Report/Audit/Assessment

Crisis Intervention/Use of Force
Use of Force – Outcome Report
Force Review Board
Use of Force – Comprehensive
Early Intervention

Filing Deadline

December 15, 2018
January 31, 2019
January 10, 2020
July 31, 2019
October 31, 2019
March 15, 2019
December 1, 2019
April 30, 2019

Bias-Free Policing – Disparity
Review (Use of Force and
Search/Seizure)
Bias Free Policing – Disparity
December 31, 2019
Review (Use of Force/Search and
Seizure/other activity)

This report covers SPD’s obligations with respect to Sustainment Matrix Deliverable No.
194. Following the agreed methodology attached as Appendix A to this report, this report
demonstrates SPD’s sustained compliance with those commitments of the Consent
Decree (paragraphs 157-163) that relate to Early Intervention.

I.

EIS – Consent Decree Requirements

The Consent Decree between the City of Seattle and the Department of Justice contains
seven paragraphs relating to EIS, reproduced here:
157. “The City’s EIS system will continue to be used for risk management
purposes and not for disciplinary purposes. SPD will monitor the EIS to
ensure it is meeting its objective of providing SPD with notice before
behaviors become problematic.
158. SPD will review and adjust, where appropriate, the threshold levels for each
of the current EIS indicator criteria, and the EIS indicators. The Monitor will
review and approve the revised EIS threshold levels and indicators.
159. SPD will revise its EIS policy to include a mechanism for review of an officer
whose activity has already triggered a threshold for one of the EIS indicator
criteria, so that the threshold level is lower if EIS is triggered again, where
3

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 4 of 30

appropriate. For example, if an officer has participated in a certain number
of uses of force in a six-month period, SPD will design a protocol for lowering
the threshold for subsequent review.
160. SPD will collect and maintain information related to supervisor, precinct,
squad and unit trends, consistent with the provisions of this section.
161. SPD will collect, maintain, and retrieve information related to the following
precinct-level activity:
a) uses of force;
b) OPA complaints and their dispositions;
c) number of individual officers who have triggered EIS reviews; and
d) supervisor reviews with officers.
162. Supervisors should periodically review EIS activity of officers in their chain
of command.
163. SPD will revise its EIS policy and procedure, as necessary, so that
interventions assist officers in avoiding potentially troubling behavior.
Specifically, SPD policies and procedures will ensure that (1) the
intervention strategy is implemented in a timely manner; (2) data regarding
the implementation of the intervention is tracked in EIS; and (3) if
necessary, the employee’s supervisor reviews the progress of the
intervention strategy.”
Consent Decree ¶¶ 157-163.
SPD’s compliance with these provisions is discussed below.

II.

SPD’s EIS Policy

To satisfy these requirements of the Consent Decree, SPD implemented a new EIS
policy in mid 2015. See Monitor’s Third Semiannual Report (dkt. 123) at 76-89. The
policy was approved by the U.S. Department of Justice, the Monitor and the Court. Id.
SPD’s EIS policy relies on triggers, thresholds, and alerts. The following definitions are
instructive.

4

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 5 of 30

Trigger: This is the basic building block of EIS. The system monitors the
occurrence and frequency of triggers for each officer. Triggers are either
individual actions by an officer (e.g., an officer uses force) or an event related to
an officer’s actions (e.g., an OPA complaint or lawsuit is filed against an officer).
SPD’s EIS policy refers to these actions or events as “triggers,” “indicators,” and
“indicator criteria.”
Alert: Alerts occur when an employee reaches a predetermined number of
triggers within a set time period. The EIS module of IAPro (one of SPD’s case
management systems) generates the alert, which is received electronically by the
EIS Coordinator. The Alert contains a listing of the triggers that caused the Alert
and details about the employee, such as name and assignment.
Threshold: SPD’s EIS is threshold based. The threshold is the number of triggers
and the time period that cause an Alert. For example, the threshold for
preventable vehicle collisions is 2 collisions within 6 months.
Early Intervention Assessment: The EIS Coordinator receives an Alert and then
assigns an Assessment that must be completed by the chain of command. The
Assessment document makes a recommendation as to whether the officer
should be placed on a mentoring plan. As explained in Section III.C, not all Alerts
lead to an Assessment.
Early Intervention Mentoring Plan: When a chain of command determines that
the reviewed triggers represent a need for performance improvement. A
mentoring plan will be created when, in the judgement of the chain, the
performance could be mitigated with a defined curriculum. The plan will be
written by the employee’s direct supervisor and receive command approval to
the deputy chief level. In addition, the chain of command can initiate a
mentoring plan at any time based on their own observed concerns, even if an
employee has not alerted.
The Performance Review Committee (PRC): The PRC includes both members of
Command Staff and Department representatives across multiple bureaus,
including the Chief’s designee (Deputy Chief), the Executive Director of Human
Resources, the Executive Director of Legal Affairs, the Early Intervention
Coordinator, and representatives from Patrol Operations, the Audit, Policy and
5

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 6 of 30

Research Section, and the Education and Training Section. The PRC meets
monthly and, per policy and by practice, is responsible for:
• Ensuring Department-wide consistency and uniformity in the
implementation of the EIS, including timeliness of Assessments
consistent with policy requirements;
• Determining whether Assessments and proposed Early Intervention
Mentoring Plans are adequate to address concerns about risk
factors, demonstrated indicators of stress, or training opportunities,
and to provide the employee with any necessary tools and support.
EIS Coordinator: a supervisor-level position housed in Human Resources,
responsible for tracking and routing EIS Alerts, monitoring timelines associated
with EIS review deadlines, and tracking the dispositions of EIS Assessments
(also available in DAP).
The Consent Decree requires SPD to monitor, review, and adjust, where appropriate,
the EIS triggers and thresholds. ¶¶ 157-58, 163. On January 1, 2019, the triggers and
thresholds were revised, with approval from the U.S. Department of Justice, the
Monitor, and the Court. Dkts. 502 & 510. The rationale and goals for these changes,
and other policy changes intended to improve the EIS program, are explained at length
below.
During the Study Period—January 1, 2017 to June 30, 2018—the triggers and
thresholds were as follows:
Trigger

Threshold Levels

A. Chain-of-command recommendations /
discretionary assessments

Each will be reviewed

B. Use-of-force (Type I)

Reaching the top 1% of officers who have used
force investigated at Type I within 6 months

C. Use of-force (Type II)

Reaching the top 5% of officers who have used
force investigated at Type II within 6 months

D. Use-of-force (Type III)

Reaching the top 5% of officers who have used
force investigated at Type III within 6 months

E. Vehicle collisions

2 Department vehicle collisions within 12
months
6

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 7 of 30

F. Receipt of OPA complaints / Supervisory
Action

3 complaints within 12 months

G. Receipt of OPA complaints / Frontline
Investigation

3 complaints within 12 months

H. Receipt of OPA complaints / OPA
Investigation

3 complaints within 12 months

I. Receipt of EEO complaints

2 complaints within 12 months

J. Named in police actions claims or Police action
lawsuit against the City

2 within 24 months

K. Vehicle pursuits

2 within 6 months

L. Unexcused failure to appear in mandatory
training

1 within 12 months

M. K9 apprehension-bite ratio

More than 15% K-9 apprehension bite ratio in a
12 month period

N. Officer-involved shooting

Single incident threshold

Additionally, an Early Intervention Assessment will be conducted at the aggregate level
when an employee has a total of six (6) indicators between A – M during any six-month
window when the employee is not on a mentoring plan.

III. Objectives and Methodology
This report covers a study period of January 1, 2017, to June 30, 2018. Data for this report
are derived from three sources. Data on supervisor reviews and individuals who have
triggered EIS reviews is contained in records maintained by the EIS Coordinator. Data
regarding precinct and squad trends with respect to use of force and closed OPA
complaints/dispositions are sourced from the Data Analytics Platform (DAP, as described
in detail in prior reports).

7

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 8 of 30

Data is also drawn from IAPro, the Department’s database solution for records and case
management relating to activities that may result in EIS triggers. IAPro provides the
technical administration of EIS (e.g., it generates alerts and tracks workflow); the DAP
collects EIS alert and workflow information from IAPro and renders it for analysis. While
some additional data may be joined to provide additional analytical context for the EIS
“incident,”1 the record of the alert is discrete and generated programmatically, resulting
in a high-quality / high-confidence record that is not subject to functional error handling
rules.2 Of note, when the EIS code was delivered to the Department by Accenture, the
contracted vendor of the DAP in Fall 2016, the code was accepted only after passing exit
criteria3 for technical testing and User Acceptance Testing (UAT).

A. Objective 1 – Assess Whether EIS Is Accomplishing Its Goals (Paragraphs
157-159; 163)
SPD is committed to continuously evaluating the effectiveness of EIS and making
evidence-based improvements as warranted.
The EIS program is successful at developing supervisory skills and providing effective
coaching and mentoring to officers. Each Early Intervention Assessment that is completed
requires the officer and his or her supervisor to meet in person to discuss the incidents
that caused the alert. These assessments, alone, document a large volume of active,
engaged supervision that otherwise would not have occurred. Mentoring plans also
require supervisors to problem solve and engage with their employees on an ongoing
basis, often over a period of several months. Another feature of the program is that the
EIS Coordinator serves as an important resource to supervisors. In addition to reviewing
and providing feedback on supervisors’ written assessments and mentoring plans, the EIS

1

IAPro captures threshold exceedances through the creation of new “incidents” and adds every new EIS
alert to the incidents table in the IAPro database.
2

The DAP maintains rules for functional error handling intended to protect the analysis from “orphan” or
partial records. For data concept areas such as UoF, where multiple complete elements are required for
a valid count (i.e. unique incident, officer and community member), a functional error may be applied,
flagging the record for correction. With regard to EIS, data convergence occurs entirely within the IApro
system (a trigger cannot occur without a complete record of a UoF, for example), functional error handling
is therefore not applicable.
3

Technical testing involved source-to-target testing at every layer of the DAP system. UAT acceptance
criteria required the delivered product (code and semantic layer) function analytically, sufficient to
respond to questions necessary to demonstrate compliance.
8

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 9 of 30

Coordinator also maintains a repository of reference materials including training videos,
books, and sample mentoring plans.
Despite these successes, SPD has identified shortcomings in the EIS program. In an
extensive effort to address these issues, over the past several months, SPD implemented
numerous, substantial policy changes which were approved by DOJ, the Monitoring
Team, and the Court.
1. Has SPD monitored EIS to determine whether EIS is meeting its risk
management objectives? (Paragraphs 157 and 163)
During the study period, in order to evaluate the effectiveness of SPD’s EIS in achieving
its risk management objectives, SPD partnered with a team of researchers at Washington
State University to compare overall officer performance across a series of behaviors,
based on interval-level metrics, between a group of officers who received an EIS alert and
a group of officers who did not. A copy of the report, detailing the full methodology and
results, is attached as Appendix B. Using a blinded scoring design (where study personnel
assigned to scoring officer performance were not aware of which reports belonged to the
treatment group and which belonged to the control group), researchers examined a set
of 500 randomly selected general offense reports generated from incidents that resulted
in EIS alerts between 2015-2017 and a set of 500 randomly selected reports from this
same time period that did not result in an EIS alert.
Researchers evaluated the incident reports and used a blind-scoring design to calculate a
performance score for each officer. The performance scores were based on three sets of
metrics: use of force, tactical social interaction, and crisis intervention. For example, a
scorer would determine, if applicable to the situation, whether the officer actively
listened to the person in crisis or whether the officer demonstrated proficiency with lesslethal force options. The performance scoring is explained at pages 4-12 of the report.
As of the time of this filing, WSU’s report has been submitted to an academic journal
for publication; throughout this process it will be peer reviewed.
The report concluded that the EIS triggers in place at the time did not adequately
account for the individual officers’ exposure rates to high-risk encounters and situations
and, as a result, the triggers focused too much on officers who are more proactive and
who are assigned to work in areas with a higher volume of calls for service.
Additional salient findings include:

9

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 10 of 30

• No significant difference between EIS-flagged and control officer total
performance score.
• The only significant difference in performance scores was that EIS flagged officers
had better performance scores on the “observe/assess” level during crisis
encounters. The “observe / assess” category relates to officers’ ability to accurately
read people during interactions.
• EIS-flagged officers were significantly more likely to report using (in policy) force
than other officers (17.5% vs. 11.7%).
The second unintended consequence of the EIS program that SPD identified is anxiety
and negative perceptions on the part of officers. To both complement the empirical
study with WSU and to inform work SPD is partnering in at the national level, SPD also
sought feedback, in focus groups, from individuals directly engaged with the system –
including members of the Performance Review Committee, patrol officers, and
supervisors and commanders. The following themes were consistently expressed.
• Officers and supervisors believe that EIS does not reliably identify officers at risk of
future problematic performance.
• Officers experience anxiety over the prospect of being more likely to receive a
higher number of EIS alerts because of their patrol assignment.
• Because EIS review is triggered by incidents that are already subject to intense
review and investigation by the chain of command and/or by OPA (e.g. use of force
reports, OPA complaints), EIS is perceived by officers and supervisors alike as
redundant. To be sure, it is important to keep the review processes that can lead
to discipline separate from the EIS program, because the goals of the EIS program
are best accomplished through coaching, mentoring, and support—not discipline.
Nonetheless, supervisors and officers experience the dual review as repetitive.
• EIS review of Type I uses of force, in particular, is seen as an unnecessary drain on
supervisory resources. The data show the reason for this perception. During the
study period, Type I force accounted for just shy of 40% of all alerts. Yet, in 2017,
there were no sustained allegations of out of policy use of Type I force.

10

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 11 of 30

To address the issues identified by the WSU study, SPD implemented substantial
changes to its policies, as described in section 2 below.
2. Has SPD reviewed and adjusted EIS indicator criteria and thresholds as
appropriate? (Paragraphs 157-158 and 163)
Throughout the second half of 2018, SPD undertook a major effort to reform the EIS
program and related policies. This work was undertaken in consultation with the
Monitor and DOJ, and based on evidence and feedback from numerous sources. The
policy revisions are informed by all of the following: (1) a review of EIS data over the
past two years, both internally and as provided in the WSU study; (2) feedback from
the Performance Review Committee; (3) focus group sessions with officer, sergeants,
lieutenants, and captains; and (4) new analytic tools for aggregating and analyzing
force at increasingly granular and comparative levels.
The first set of policy changes involved revising the definition of Type I Force to clarify that
discomfort caused by wearing (properly applied) handcuffs is not reportable as force. SPD
had determined that such reports of discomfort or minor pain—i.e., the level of discomfort
that is inherent to wearing handcuffs—made up an increasingly high percentage of
reported use-of-force incidents (approximately 44% of Type I force in 2017 and two-thirds
of Type I force in 2018).4 The policy now distinguishes between pain resulting from the
use of force involving handcuffs (e.g., improper application of handcuffs) versus pain that
can be attributed solely to the inherent discomfort of wearing handcuffs. SPD’s revised
Use of Force policy defines “handcuff discomfort” as “discomfort or minor pain caused
solely by wearing handcuffs properly applied.”
Accompanying that change is a revision which provides that such complaints must be
reported and screened by a supervisor. These revisions are intended to ensure that all
complaints of pain from handcuffs will be investigated as either a Type I use of force
(application pain) or as Handcuff Discomfort (wearing handcuffs) and properly tracked by
the Department. These changes were discussed at length with DOJ and the Monitor and
approved by the Court. Dkt. 477.
In addition, the Department recently tested and deployed new handcuffs which are
beveled and made of lighter-weight aluminum, and which it anticipates will be equally
effective and cause less discomfort. These policy and equipment changes are expected to
reduce the amount of time and effort spent by the chain of command on instances where
handcuffs, properly applied, cause unavoidable discomfort.

4

See SPD’s 2017 and 2018 Use of Force Outcome Reports, previously filed with the
Court. Dkts 443 at 47 and 524-1 at 5.
11

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 12 of 30

Second, SPD worked, consistent with Paragraphs 158 and 163 of the Consent Decree,
to adjust and revise EIS thresholds and indicators. As with the use of force policy
revisions, SPD engaged in multiple rounds of discussions with the Monitor and the
Department of Justice in developing the proposed revisions, and—with their
acceptance—received approval from the Court on December 17, 2018 (Dkt. No. 510).
SPD continues to be in compliance with this requirement.
The revised EIS policy modified the description of the program’s purpose, triggers, and
risks to better align the policy with the philosophy of the program which is not intended
to be punitive or disciplinary. For example, instead of describing the EIS program solely as
a “strategy to address at-risk behavior,” the revised policy identifies a more holistic set of
goals, including to support employee wellness and professional growth and to address
underlying factors that can lead to negative performance issues (including incident-based
stress or training needs). The revised policy also clarifies that the purpose of a mentoring
plan under EIS is to support the employee in achieving professional goals.
In addition, the EIS policy revisions adjusted the trigger levels. To respond to the analysis
and conclusions summarized above, the triggers for use of force are adjusted so that they
are still based on percentages, but now are also subject to a baseline number of uses of
force at or beneath which there will be no alert. For Type I force, the trigger is still the top
1% of officers, with a baseline of five incidents. Similarly, for Type II the percentage stays
at 5%, with a baseline of three incidents. The threshold for triggering review is raised from
3 OPA complaints to 4 OPA complaints. The adjustments are intended to exclude from
review the officers who fall within the top percentages for use of force, but still have used
force very rarely (to account for the fact that Department-wide the rate of force being
used is extremely low).
Four of the trigger categories were eliminated, because they are duplicative of other policy
provisions or review processes: (1) the unexcused failure to appear at training category
was eliminated, because supervisors are already required elsewhere in policy to discuss
any and all missed trainings with officers; (2) the K9 apprehension bite category was
eliminated, because these incidents already trigger review under the Type II use of force
category; (3) the vehicle pursuit category was eliminated, because it is already extensively
reviewed by the chain of command pursuant to SPD’s vehicle eluding/pursuits policy; and
(4) similarly, the officer involved shooting (OIS) category was eliminated as an EIS trigger5
because the Department has adopted a far more extensive process for OIS incidents—
after every OIS incident the Department places the involved officer on mandatory leave,
Type III force is still a trigger, and under the current threshold any officer who exceeds
one use of Type III force (including more than one OIS) within six months will be subject
to an EIS alert.
5

12

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 13 of 30

the specially trained Force Investigation Team conducts an investigation, and an in-depth
inquiry is undertaken by the Force Review Board.
The Department expects that these changes will more closely focus the EIS program on
activities that are relevant to risk and achieve better alignment with the program’s goals.
3. Does the EIS policy contain a mechanism by which the threshold for
review is lower after an officer has already alerted? (Paragraphs 159
and 163)
SPD continues to be in compliance with paragraphs 159 and 163. The EIS policy as
approved by the Court on December 17, 2018, retains the language of the Department’s
prior (Court approved) policy with respect to this requirement and states as follows:

This policy is implemented by the EIS Coordinator. By design, once an officer has alerted
(and therefore is already being assessed or on a mentoring plan), then the EIS Coordinator
receives electronic notification for every new trigger that occurs for that officer.6 (As
noted above, policy uses the word “trigger” to refer to the occurrence of an incident or
action that is tracked by EIS, e.g., a use of force or the filing of an OPA complaint.) The EIS
Coordinator reads each new alert and reviews the employees’ record to ensure that the
alert is not repeating previous triggers (e.g., one new of pointing a firearm, when the
officer is already being assessed or on a mentoring plan based on similar Type I uses of
force). If the new alert is based on three or more new triggers, the officer’s supervisor is
required to complete an updated assessment (using a form titled Subsequent Alert
6

This occurs by design, because the EIS alert system does not reset or “zero out” after an officer alerts
until the officer is outside the measuring period (e.g., if an officer alerts for six Type I uses of force within
six months, then each new Type I force will cause a new alert until the system “resets” six months after
the first incident). The purpose is to ensure close and continuous monitoring of the officer’s activities.
13

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 14 of 30

Update). However, if an officer who is already on a mentoring plan receives even one new
trigger, then the officer’s lieutenant or manager is required to perform an additional
review.
The purpose of an update assessment is to inform a supervisor that an employee the
supervisor has recently completed an Assessment on has at least three new unreviewed triggers. The supervisor is required to review the new triggers and assess
whether or not the new information alters his or her assessment of the employee’s
performance when analyzed in conjunction with the previous history.
To illustrate, assume an officer receives an “aggregate” alert based on six triggers: e.g.,
two instances of pointing a firearm, two complaints of handcuffing discomfort, and two
OPA complaints.7 An Assessment is assigned to the officer’s supervisor. Within two
months of the alert, the officer has three additional Type I uses of force. The EIS
Coordinator would assign the supervisor to complete an update assessment (if the
officer had not already alerted, it would take six Type I uses of force to cause an
assessment).
SPD accordingly continues to be in compliance with this requirement of the Consent
Decree.

B. Objective 2 – Assess Whether SPD Is Tracking EIS-Related Outcomes
(Paragraphs 160 & 161)
In its Phase I assessment of the EIS program, the Monitor found SPD to be in compliance
with all EIS requirements in the Consent Decree, including data collection and tracking.
See Monitor’s Eighth Systemic Assessment, Dkt. 374 (March 23, 2017). SPD’s data
collection processes have continued in the same manner since that time. Below the
specific mechanisms employed to collect and track EIS-related data are described.
• Has the number of individual officers who have triggered EIS reviews
been completely and accurately collected, maintained, and retrieved
at the precinct level?
• Has the number of supervisor reviews of officers based on EIS triggers
been completely and accurately collected, maintained, and retrieved?

7

During the study period, EIS policy, section 3.070-POL-2, provided that “an Early Intervention Assessment
will be conducted at the aggregate level when an employee has a total of six (6) [triggers] between A – M
during any six-month window when the employee is not on a mentoring plan.” The revised policy which
took effect on January 1, 2019, increased that number to ten triggers within six months.
14

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 15 of 30

SPD has maintained compliance with these requirements of paragraphs 160 and 161.
The required data on individual officers and supervisor reviews is collected and
maintained by the EIS Coordinator and recorded in IaPro. During the study period 317
EIS Assessments (referred to in the Consent Decree as “supervisor reviews”) were
completed.
Over the study period, 344 unique officers received at least one alert. Approximately
75 percent of sworn officers did not receive any alerts. More than three-quarters
(75.87%) of the officers who received at least one alert received fewer than five alerts
during the period; two officers received more than 30 alerts. See Figure 1. The total
number of unique officers who received an alert is greater than the number of
completed assessments (317), because not all of the alerts that occurred during the
study period had time to work their way through the system to result in a completed
Assessment. Ultimately, each of these 344 officers will receive an Assessment and
some will be both placed on a mentoring plan.
Figure 1: EIS Alerts Per Officer

• Has use of force information been completely and accurately collected,
maintained, and retrieved at the precinct level?

15

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 16 of 30

The Department has been collecting, maintaining, and reviewing information relating to
use of force at the precinct level for many years. The Monitor recognized these practices
in Phase I of the Consent Decree. See Dkts. 231 & 360. More recently, the Department
has demonstrated its gathering and tracking of such information through its annual Use
of Force Outcome Reports, which contain a comprehensive overview of all use of force
for a given year, see Dkt. 524-1, through its Use of Force Reporting and Investigation Audit
Report (dkt. 497-1), and through its Use of Force dashboard, a version of which is publicfacing8.
• Has information on closed OPA complaints and their dispositions been
completely and accurately collected, maintained, and retrieved at the
precinct level?
As discussed in the Monitor’s Fourth Systemic Assessment (Dkt. 259-1 at 18), OPA began
using IAPro as its day-to-day case management system in 2014. OPA issues annual, public
reports on this data and its other activities. OPA reports are available at
https://www.seattle.gov/opa/opa-reports
Through the EIS module of IAPro, OPA complaints that lead to EIS alerts are tracked and
reviewed by the chain of command and the PRC for the purpose of making assessments
and mentoring plans. Because coaching and mentoring are most effective when they are
conducted close in time to the underlying incident, the EIS process is not delayed to await
the outcome of OPA’s review. Therefore, an officer’s assessment or mentoring plan is
frequently completed before OPA has finished investigating or reviewing the underlying
complaints.
Separate and aside from the EIS program, OPA findings are used in other ways by SPD.
After a recommended finding of sustained has been issued by the OPA Director on an
allegation of misconduct, the Chief of Police decides what discipline to impose on the
named employee. If the Chief decides not to follow one or more of the OPA Director’s
recommended findings, the Chief must provide a written statement of the reasons for
the decision within 30 days of making that decision.
OPA training recommendations are always followed by SPD. Sometimes the disciplinary
action recommendation includes a training requirement, which SPD requires the officer
to complete regardless of the ultimate disciplinary resolution. In addition, for not

8

https://www.seattle.gov/police/information-and-data/use-of-force-data/use-of-force-dashboard)
16

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 17 of 30

sustained complaints, OPA may recommend that an officer undergo additional training.
Officers must complete the recommended training in that situation as well.
• Has information related to supervisor, precinct, squad and unit trends
with respect to EIS been completely and accurately collected,
maintained, and retrieved?
In the following section, the Department reports on data relating to EIS tracking and alerts
as required under Paragraphs 160 and 161 of the Consent Decree.
During the study period (January 1, 2017 to June 30, 2018), a total of 1,316 EIS Alerts
(“alerts”) were generated. A breakdown of these alerts, by trigger, is presented in Figure
2.
Figure 2: EIS Alerts by Trigger9
500

455
375

400
300
200
100

125
44

30

34

40

164

49

0

Over the study period, 23% of alerts led to an Assessment. That is because many of the
alerts are combined into one assessment, since the EIS system does not reset after an
officer has alerted (as explained above in footnote 5). In addition, officers can receive
9

Aggregate alerts are represented on Figure 2 based on the final trigger that generated the alert. That
is because EIS “aggregate” alerts are based on multiple different types of triggers. During the study
period, six different triggers of any type within six months would generate an “aggregate” alert. As of
January 1, 2019, that number has been increased to ten within six months.

17

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 18 of 30

multiple alerts for the same underlying incident. To illustrate, assume an officer uses
Type I force. That single use of force causes the officer to receive three alerts: one alert
because it is the officer’s sixth10 Type I use of force in six months; one alert because it is
the officer’s sixth trigger (of any type) in six months; and one alert because the subject
of force made an OPA complaint. In this instance, the officer’s supervisor is assigned to
complete one Assessment—not three Assessments. Finally, some alerts are exact
duplicates of prior alerts (largely due to how the EIS module handles reclassifications of
incidents.) The EIS Coordinator reviews each and every new alert and the underlying
triggers and ensures that multiple EIS Assessments or mentoring plans are not
completed for the same officer based on the same underlying triggers.
The two incident types that caused the largest portion of alerts are “Level 1 – Use of
Force” and “OPA – Investigation.” “Level 1 – Use of Force” refers to Type I (least serious)
force. During the study period, Type I force included, for example, the pointing of a
firearm, discomfort caused by handcuffing, and other force that causes transitory pain or
a complaint of transitory pain.11 Seattle Police Manual § 8.050. As shown in Figure 2, Type
I uses of force caused 455 EIS alerts.
“OPA – Investigation” refers to complaints received by OPA that have been classified for
investigation (i.e., OPA has determined that at least an initial investigation is warranted).
OPA complaints classified for investigation caused 375 EIS alerts.
The vast majority (86.3%) of alerts were generated by officers assigned to the Operations
Bureau, with West Precinct officers represented most frequently (31.33%), followed by
North Precinct officers (26.37%). See Table 1.

10

This is the current threshold for Type I force. For most of the study period the threshold was lower.

11

As explained above, minor discomfort caused by properly applied handcuffs is no longer classified as
Type I force.
18

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 19 of 30

Table 1: Distribution of EIS Alerts by Organizational Assignment

The distribution of alerts by organization assignment closely mirrors calls for service,
crime, force, Terry stops and crisis activity, suggesting no obvious pattern. In other
words, the highest volume of EIS alerts occurred in West Precinct and North Precinct,
which are the precincts with by far the highest volume of 911 calls, crime rates, and thus
policy activity.
By rank/position of employment, Police Officers accounted for nearly 80% (79.41%) of all
alerts generated during the study period. Twenty (1.48%) alerts were generated for
civilian employees, 18 of which were generated for Parking Enforcement Officers. See
Table 2.

19

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 20 of 30

Table 2: Distribution of EIS Alerts by Rank/Employee Position

Descriptive analysis suggests no obvious pattern at the Department-wide level, for EIS
alert generation. Distribution of alerts by assignment, across age and experience closely
mirror the demographics of the Operational Bureau, suggesting the most frequently
observed alert activity is the predictable result of police service (complaints, force, vehicle
operations).

C. Objective 3 – Assess How EIS Is Being Implemented (Paragraphs 162
& 163)
• Are supervisors periodically and appropriately reviewing EIS activity of officers
in their chain of command?
Aside from mentoring plans, EIS Assessments are the primary way that supervisors review
the EIS activity of their employees. Sergeants are also required to monitor their
20

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 21 of 30

employees’ EIS activity on electronic dashboards weekly, and in this way they learn when
an officer is approaching EIS thresholds. However, Assessments make up the vast majority
of all work produced by the EIS program. Each Assessment requires an in-person meeting
between the employee and their supervisor in which the supervisor reviews each of the
triggering events with the employee. This type of supervisor-employee engagement
provides the opportunity for a two-way conversation about the employee’s work
performance, well-being, and more. Supervisor-employee engagement is an important
component for supervisors to guide, counsel, and mentor their employees. An
Assessment may or may not result in a mentoring plan however the supervisor-employee
engagement is central to employee management.
Once the Assessment is completed by the employee’s supervisor, it is routed through the
chain of command. The chain of command – from the supervisor through the
commanders – must review it and make a determination on the following basic question.
Do the events indicate that an improvement in performance could be achieved with a
mentoring plan?
The majority of Assessments come back with the “No action box checked,” meaning that
no formal mentoring plan is required, which is expected in a threshold-based system.
However, checking the no action box does not mean that nothing was done to address
issues and or coach and mentor the employee. In fact, a tremendous amount of good
leadership can be found documented on these forms. To conduct a proper and accurate
assessment, a supervisor must review the cases, speak with the employee about the
triggers, discuss policy, document employee understanding and provide justification for
decisions made. If the supervisor and chain of command does not properly perform and
document an Assessment, the PRC (the executive committee which reviews and approves
all assessments) will return the Assessment for further work. This system fosters careful
attentive leadership skills in our supervisors.
All of these Assessments are reviewed through the chain of command and all must meet
PRC approval. PRC review yields a fairly consistent return rate of about 10 percent for
Assessments that are not approved. Typical reasons for Assessment returns are for minor
corrections on format, or inadequate review of triggers. In addition, the chain of
command and the PRC review the supervisor’s determination as to whether or not a
mentoring plan is warranted. If the supervisor determines there should not be a
mentoring plan and the chain of command or the PRC disagrees, then the supervisor must
write a mentoring plan. The supervisor must then return the Assessment or mentoring
plan within a specified time period. It will then be reviewed again by the chain of

21

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 22 of 30

command and the PRC and only proceed if the requested corrections were made in a
satisfactory way.
Out of a random sample of 15 EIS cases that involved mentoring plans, in 6 of the cases
the chain of command determined that no mentoring plan was warranted, but the PRC
required that a mentoring plan be completed. In one case, the EIS Coordinator returned
a mentoring plan for the sergeant to make substantive improvements. In two cases, the
chain of command returned one or more documents for the sergeant to make substantive
improvements.
Throughout the study period, the EIS Coordinator has observed a steady increase in
acceptance and recognition of the goals and worthwhileness of mentoring plans by the
chain of command, demonstrating a positive development in supervision and leadership.
• Do supervisors review the progress of assigned intervention strategies as
appropriate?
• Are EIS intervention strategies being implemented in a timely manner?

SPD policy and the practices of the PRC ensure that EIS intervention strategies are
implemented in a timely manner. Once an EIS Assessment is assigned to a supervisor, the
supervisor is required to complete the Assessment within fourteen days; the lieutenant
and captain have 3 and 7 days respectively for their review. If a mentoring plan is
warranted, then the supervisor is assigned to complete it within a specified amount of
time (ranging from 10-24 days depending on complexity). Each mentoring plan contains
a timeline within which the supervisor and chain of command must complete the
intervention strategy; the most common mentoring plan timeline is 30 days and the
longest is 90 days. The EIS coordinator reviews and monitors all in-progress mentoring
plans regularly to ensure that they are closed out in a timely manner.
Out of a random sample of 15 mentoring plans from the study period (each for a different
officer), the tasks specified in eight of them had been completed by the officer and
supervisor within four months. Three additional mentoring plans took between four and
five months to complete. One mentoring plan, created to address OPA complaints
involving allegations of bias and excessive force, took 181 days to complete. These
timeframes were all appropriate, given the subject matter being addressed. For example,
a mentoring plan to address routine driving issues can be completed relatively quickly, as
it typically involves taking a driving course that is offered regularly. Interventions assigned
to address more complex or weighty topics appropriately take longer and involve

22

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 23 of 30

substantially more interactions over a longer period of time between the officer and his
or her supervisor.
Although three of the fifteen plans were not completed, there were legitimate reasons
for each. In one, the officer went on an extended leave and the mentoring plan was placed
on hold. For the other two, the officers did not complete the interventions and the EIS
cases were ultimately closed out because the chain of command determined that the
disciplinary process would be the more appropriate approach to correct the identified
issue.
For the 317 assessments that were assigned and completed during the study period, more
than one third were completed with one week of the initial EIS alert. All but 14 were
completed within two months.
Table 2: Distribution of EIS Alerts by Rank/Employee

EI Assessments - Days to Complete
111

76

50
39
25
16
0-7 Days

8-14 Days

15-21 Days

22-30 Day

31-60 Days

60+ Days

As with the underlying Assessments, mentoring plans provide an opportunity for
continued supervisor-employee engagement. The mentoring plan process requires that
the supervisor meet in person with the officer and, where appropriate, the supervisor
must participate in the assigned intervention. For example, the recommended action step
for an Assessment related to receipt of an OPA Complaint is “[r]ide along with employee
for a minimum duration of one hour per shift, twice per work week.” The EIS AssessmentMentoring Plan Form contains other suggested intervention strategies and is attached to
23

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 24 of 30

this report as Appendix C. However, there is no formula that a supervisor can apply when
determining an appropriate intervention; the decision must be based on the supervisor’s
interactions with and knowledge of the welfare and status of their employees. In addition,
the EIS Coordinator maintains a large repository of high-quality Assessments and
mentoring plans that have been anonymized to provide instructive examples.
• Is data regarding interventions being tracked in EIS?
For every EIS Assessment or mentoring plan, a “final disposition form” is entered into
IAPro. Using this form, the Department records the specific interventions assigned, if any,
and the dates on which the assessments or interventions are completed.
In sum, the Department has continued to sustain compliance with each of Paragraphs 157
through 163 of the Consent Decree.

FUTURE STEPS
The findings of this audit reveal significant challenges and critical areas for improvement
in the EIS program. More broadly, as the social science around early intervention theory
has evolved substantially over the past several years, SPD must ensure that its system
evolves to keep pace. Committed to doing so, the Department describes below the work
that it is doing in this regard, independently and in collaboration with partners, nationally.
In the near-term, the Department’s next EIS Audit, scheduled to be completed in
December 2019, will analyze the effects on the EIS program of the changes to the
definition of Type I force and adjustments to EIS triggers and thresholds.
The Department is also taking a leadership role nationally by exploring ways to leverage
multi-disciplinary social science research to improve upon the EIS model. In concept, Early
Intervention Systems are designed to leverage data-based metrics to identify officers at
risk of future performance issues, providing police supervisors an opportunity to
intervene through mentoring, counseling, additional training, or other non-disciplinary
measures at a point before the officer is involved in a significant adverse event. Such
systems are intended to promote chain of command accountability for employee
performance while also providing supervisors a foundation to foster and recognize the
success and professional growth of their employees.
In recent years, the model of tying EIS review to the occurrence of policing activities, such
as the use of force, has been criticized in academic literature as ineffective, often
counterproductive, and perceptually, as punitive and stigmatizing. See, e.g., Worden et
24

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 25 of 30

al. (2013) Intervention with Problem Officers: An Outcome Evaluation of an EIS
Intervention. Crim. Justice and Behavior, Vol. 40, No. 4 409-437;
http://www.latimes.com/local/la-me-lapd-problem-officers-20140826-story.html.
SPD fully endorses the importance of developing an effective early intervention model,
from both an officer safety/well-being and Department risk management perspective.
Accordingly, while appreciating that designing an accurate model for attempting to
predict how any one individual will behave in dynamic, unpredictable, and often
dangerous situations is an inherently complex and iterative task, SPD is actively engaged
in efforts on multiple fronts to further advance the social science underlying this goal. At
the national level, SPD is a core contributing member of a multi-disciplinary advisory
committee, comprising representatives of law enforcement, medical and mental health
fields, and research institutions from around the country, working in collaboration with
the Chicago Police Department, the Los Angeles Police Department, and the University of
Chicago Crime Lab to design a next-generation EIS. At the organization level, continuing
its work with research partners from Washington State University, SPD is exploring how
it might best leverage the increasing sophistication of its data and analytic tools to allow
for more validated identification of relevant variables to an EIS, both psychological and
situational.
Paramount in both efforts is a shift of focus away from the outcome of an incident
(outcomes that, again, are subject through separate processes for timely and rigorous
critical review) and instead towards the nature of incidents to which officers are called to
respond. The premise for this shift is grounded in the impacts – physiologically,
psychologically, and behaviorally – on cumulative incident stress and vicarious trauma on
first responders. In simplistic form, the Department’s longer-term vision for EIS is built
on the following premises:
1. By nature of their occupation, law enforcement officers are regularly exposed to
incident stress likely to manifest, acutely or over time, in secondary traumatic
stress/vicarious traumatization.12

12

“Vicarious trauma” has been explained as
[A]n occupational challenge for people working and volunteering in the fields of victim services,
law enforcement, emergency medical services, fire services, and other allied professions, due to
their continuous exposure to victims of trauma and violence. This work-related trauma exposure
can occur from such experiences as listening to individual clients recount their victimization;
looking at videos of exploited children; reviewing case files; hearing about or responding to the
25

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 26 of 30

2. Repeated stressful and/or traumatic events can chronically elevate the body’s stress
response, leading to poorer physiological and psychological functioning, up to and
including post-traumatic stress disorder (PTSD).13
3. Recognizing, mitigating, and ideally preventing vicarious trauma and its sequelae is
both a critical officer safety issue14 and an organizational risk management priority.15
4. If the goal of an EIS is to identify potentially problematic performance indicators
before they culminate in an adverse (out of policy, or worse) event, then such a system
must be:

aftermath of violence and other traumatic events day after day; and responding to mass violence
incidents that have resulted in numerous injuries and deaths.
Vicarious trauma overlaps with concepts of secondary traumatic stress (the “natural consequent
behaviors and emotions that often result from knowing about a traumatizing event experienced by
another and the stress resulting from helping, or wanting to help, a traumatized or suffering person[,]
with symptoms [that] can mimic those of posttraumatic stress disorder”) and burnout (“a state of
physical, emotional, and mental exhaustion caused by long-term involvement in emotionally
demanding situations.”).
https://vtt.ovc.ojp.gov/what-is-vicarious-trauma
13

Such symptoms include difficulty managing emotions, difficulty making good decision, problems
managing boundaries, sleep disruption and/or nightmares, generalized hypervigilance/anxiety, substance
abuse and engaging in high risk behaviors, changes in appetite, negativity, and social isolation. Some
studies estimate that up to nearly 20% of police officers in the United States meet DSM-V diagnostic
criteria for PTSD; other studies estimate that up to 34% suffer symptoms associated with PTSD, although
they may not meet criteria for full diagnosis. See, e.g., Charles R. Marmar, et al., Predictors of
Posttraumatic Stress in Police and Other First Responders, 1071 ANNALS N.Y. ACAD. SCI. 1, 18 (2006);
Regehr., C. and LeBlanc, VR (2017, PTSD, Acute Stress, Performance and Decision-Making in Emergency
Service Workers. J. Am. Acad. Psychiatry and Law. 45(2) 184-192.
14

See, e.g., President’s Task Force on 21st Century Policing, Washington DC Office of Community Oriented
Policing Services, Final Report of the President’s Task for on 21st Century Policing (2015)
http://www.cops.usdoj.gov/pdf/taskforce/taskforce_finalreport.pdf (noting the toll of stress on the life
expectancy of law enforcement officers).
15

See generally Regehr and LeBlanc (2017), discussing the impact of stress conditions on perception,
cognition, risk assessment, and response.
26

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 27 of 30

a. tied to indicators that are at minimum logically, but where possible, empirically,
predictive of future performance deficits; and
b. structured so as to provide meaningful mitigation strategies that address both the
nature of the indicators and, to the extent that the Department may influence,
physiological/psychological factors that may negatively impact an officer’s
performance, professional growth, development, and well-being.
This vision is not only confidently aligned with core principles around preserving
performance ideals in first responders, but importantly, incorporates feedback received
from officers, supervisors, and commanders around advancing an EIS that truly succeeds
in identifying and mitigating against predictors of future problematic performance.
Indeed, there were positive points elicited from focus groups that must continue to be
nurtured in future iterations of EIS. As one example, supervisors expressed a general
hunger for aggregated information around their employees’ overall activity and a
mechanism for identifying trends within their squads. Officers expressed appreciation for
a system that encouraged conversation with their supervisors; supervisors likewise
expressed a desire for greater interaction with their officers but lamented the limited
time available given the amount of paperwork that they are also responsible to complete.
And importantly, officers across the ranks of focus groups – but particularly those in
senior positions – emphasized the significance, in time and over time, of vicarious trauma,
critical incident stress, and the toll of this experience on officer well-being and
performance, and openly advocated for a system that would shift an EIS in this direction.
Over the longer term, SPD intends to continue its engagement with organizations
nationally, with research partners, with its employees, and with the Inspector General to
develop a new model for EIS that more accurately meets the goals of the system.

VALIDATION
In Phase I of the work of under the Consent Decree, DOJ and the Monitoring Team
reviewed SPD’s compliance with the requirements of the Consent Decree through 10
assessments, covering the roughly six topic areas of the Consent Decree: force
investigation and reporting, crisis intervention, supervision, Early Intervention System
(“EIS”), use of force, and stops and biased policing. By the end of 2017, the Monitoring
Team and DOJ found the City of Seattle to be in compliance with each area. On that basis,
the Court issued a finding of “full and effective compliance” with the requirements of the
Consent Decree. By the terms of the Consent Decree, the City of Seattle is now required

27

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 28 of 30

to demonstrate that it can sustain compliance with those requirements for a period of
two years.
During Phase II of the Consent Decree work, the City of Seattle has taken over the lead
role in conducting assessments of the six core topic areas of the Consent Decree. By taking
this lead role, SPD must demonstrate not only sustained compliance, but also a
willingness and ability to critically self-assess their own progress in these areas, which are
central to effective and constitutional policing.
This does not mean, however, that the work of DOJ and the Monitoring Team is done. In
Phase II, DOJ and the Monitoring Team are reviewing the City’s proposed methodologies
for each audit and are conducting their own independent research and analysis or “look
behind” the City’s review.
For this audit, DOJ and the Monitoring Team consulted with SPD and ultimately approved
the methodology used by SPD in conducting its own self-assessment. Further, as part of
their own validation review, DOJ and the Monitoring Team interviewed the EIS
Coordinator regarding (1) the process and decisionmaking involved in converting EIS
triggers to assessment assignments for supervisors; (2) oversight of supervisory decisions
to create or not create PMPs from assessments; and (3) oversight over supervisory
decisions regarding the content of PMPs. DOJ and the Monitoring Team also participated
in discussions with SPD at the end of 2018 regarding revisions to the EIS triggers and,
ultimately, approved policy changes related to those triggers in order to support SPD’s
efforts to make the system more usable for risk management purposes.
In addition, DOJ and the Monitoring Team requested and received a randomly selected
sample of EIS case files, including the initial assessments, Performance Mentoring Plans,
Status Reports, and Performance Review Committee Disposition Forms, for the time
period January 1, 2017 to June 30, 2018. The sample set was comprised of 15 of the 33
Performance Mentoring Plans reviewed by SPD’s Performance Review Committee during
the study period. DOJ and the Monitoring Team, together with their subject matter
experts, reviewed these materials for compliance with the terms of the Consent Decree
relating to EIS and SPD’s policies regarding the same. DOJ and the Monitoring Team
subsequently conferred about their findings and, based on their Phase II reviews,
concluded as follows:
• The City of Seattle has demonstrated that it has sustained compliance with the EIS
requirements of the Consent Decree and SPD’s policies governing the same,
including requirements that the EIS system continue to be used for risk
management (and not disciplinary) purposes and that supervisors periodically
28

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 29 of 30

review EIS activity of officers in their chain of command. Further, the City has
demonstrated (in particular, recently) that it has monitored and made adjustments
to the EIS system to ensure that it is meeting its risk management objectives.
• In addition, DOJ and the Monitoring Team commend the City for seeking out and
collaborating with those in the field of EIS to stay abreast of best practices and
developments in how better to anticipate and address issues with officers in their
command prior to their development into major events. DOJ and the Monitoring
Team will continue to follow these developments as well and will include them in
conversations about the future of EIS during the next audit period, which is due to
result in a December 2019 filing with this Court.
In the interest of continuous improvement, DOJ and the Monitoring Team offer the
following technical assistance to SPD based upon issues spotted during their respective
reviews. Although none of these issues rose to the level of systemic non-compliance with
the terms of the Consent Decree, DOJ and the Monitoring Team strongly encourage SPD,
and the Inspector General who will take over the audit function for this topic area in the
future, to give attention to these matters going forward:
• From our sampling of PMPs, there appeared to be at least a few occasions where
supervisors were not clear on the amount of discussion they could engage in with
respect to officers who “triggered” under the EIS system because of (or in part
because of) complaints that were made about them to the Office of Police
Accountability (“OPA”). While we understand that collective bargaining rules
regarding discipline may limit the amount of discussion supervisors can have
regarding a specific incident while it is being investigated by OPA, supervisors
should be made aware that they are able to, and should, still thoroughly discuss
the underlying issues of the complaint.
• DOJ and the Monitoring Team were particularly impressed with the thoughtful and
robust manner by which the current EIS Coordinator approaches his role with
respect to identifying and assigning out EIS “assessments” and evaluating the work
of supervisors in selecting and crafting PMPs. However, we believe that in order
for the system to continue to function effectively and with the benefit of lessons
learned from the EIS Coordinator, there should be more robust systems in place to
institutionalize that role. To that end, we would encourage SPD to adopt a practice
of semi-yearly meetings or trainings in which the EIS Coordinator provides a
detailed account of how his or her work is done to at least 2-3 personnel who
would be in the position to step into the role of the EIS Coordinator as needed.
29

 Case 2:12-cv-01282-JLR Document 550-1 Filed 04/15/19 Page 30 of 30

• During our review of the EIS Status Reports and the Performance Review
Committee Disposition Forms it was not always clear what actions had been taken
in response to the PMP and when those actions were completed. While the PRC
has the ability to communicate directly on these points, including through Blue
Team, it may be difficult for reviewers/auditors to easily view what has occurred.
Accordingly, we would recommend issuing clarifying instruction to supervisors and
the PRC regarding clear documentation of the completion of PMP action items
using the Status Report and Performance Review Committee Disposition Forms,
including dates completed.
• Finally, while we acknowledge the importance of the work the City and those at
SPD have done to fine tune the EIS systems and its triggers to better serve the City’s
risk management needs, we believe that there is an additional approach that has
not yet been explored. SPD’s findings related to the “over-triggering” of proactive
officers is consistent with what other police departments have found in the early
years of their own EIS systems. One well-established fix to this issue is to embed a
“peer to peer” analysis into the triggers. In other words, instead of setting up a
trigger for numbers of uses of force across all officers, triggers would be calibrated
to compare only officers with like assignments (e.g. by a particular watch/precinct,
or a particular assignment like the West Bike Squad). This approach is much better
suited to truly identifying outliers. Accordingly, we strongly encourage SPD to
analyze this issue and to commit it to practice through its next EIS policy review.

30