Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 1 of 28 ` Early Intervention System Audit Report November 2019 1 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 2 of 28 Introduction Under Paragraph 223 of the Consent Decree, the Court retains jurisdiction over this matter “until such time as the City has achieved full and effective compliance and maintained such compliance for no less than two years.” On January 10, 2018, the Court entered an order finding the Department to be in “full and effective compliance” as of the date of the Order, thus commencing at that point the two-year “sustainment period.” Dkt. #439. The Court further ordered the parties and the monitor to “meet, confer, and prepare a plan for discharging their obligations under the Consent Decree” during this two-year period. On March 13, 2018, the Court entered an order approving the Sustainment Period Plan developed pursuant to the Court’s January 10th order. This plan, and an attached matrix of deadlines, became the governing documents for this Sustainment Period. As relates to SPD’s obligations under the Consent Decree, the matrix contains deadlines for SPD self-reporting or assessment of specific topical areas (“Commitments”) of the Consent Decree. These assessments, implicated paragraphs of the Consent Decree, and associated deadlines are provided in the chart that follows: Sustainment Matrix Deliverable Nos. 6 114 120 134 142 99 111 172 180 25 33 42 50 Systemic Report/Audit/Assessment Filing Deadline Community Engagement Stops and Detentions – Outcome Report Stops and Detentions – Audit Report Crisis Intervention – Outcome Report Supervision May 31, 2018 June 30, 2018 May 31, 2019 January 31, 2019 October 31, 2019 October 31, 2018 October 31, 2019 October 31, 2018 November 30, 2019 October 31, 2018 October 31, 2019 October 31, 2018 July 31, 2019 Force Reporting, Review, and Investigation – Type I Force Reporting, Review, and Investigation – Type II 2 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 3 of 28 Sustainment Matrix Deliverable Nos. 105 56 62 79 71 194 202 156 164 Systemic Report/Audit/Assessment Crisis Intervention/Use of Force Use of Force – Outcome Report Force Review Board Use of Force – Comprehensive Early Intervention Filing Deadline December 15, 2018 January 31, 2019 January 10, 2020 July 31, 2019 October 31, 2019 March 15, 2019 December 1, 2019 April 30, 2019 Bias-Free Policing – Disparity Review (Use of Force and Search/Seizure) Bias Free Policing – Disparity December 31, 2019 Review (Use of Force/Search and Seizure/other activity) This report covers SPD’s obligations with respect to Sustainment Matrix Deliverable No. 202. Following the agreed methodology attached as Appendix A to this report, this report demonstrates SPD’s sustained compliance with those commitments of the Consent Decree (paragraphs 157-163) that relate to Early Intervention. I. EIS – Consent Decree Requirements The Consent Decree between the City of Seattle and the Department of Justice contains seven paragraphs relating to EIS, reproduced here: 157. “The City’s EIS system will continue to be used for risk management purposes and not for disciplinary purposes. SPD will monitor the EIS to ensure it is meeting its objective of providing SPD with notice before behaviors become problematic. 158. SPD will review and adjust, where appropriate, the threshold levels for each of the current EIS indicator criteria, and the EIS indicators. The Monitor will review and approve the revised EIS threshold levels and indicators. 159. SPD will revise its EIS policy to include a mechanism for review of an officer whose activity has already triggered a threshold for one of the EIS indicator 3 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 4 of 28 criteria, so that the threshold level is lower if EIS is triggered again, where appropriate. For example, if an officer has participated in a certain number of uses of force in a six-month period, SPD will design a protocol for lowering the threshold for subsequent review. 160. SPD will collect and maintain information related to supervisor, precinct, squad and unit trends, consistent with the provisions of this section. 161. SPD will collect, maintain, and retrieve information related to the following precinct-level activity: a) uses of force; b) OPA complaints and their dispositions; c) number of individual officers who have triggered EIS reviews; and d) supervisor reviews with officers. 162. Supervisors should periodically review EIS activity of officers in their chain of command. 163. SPD will revise its EIS policy and procedure, as necessary, so that interventions assist officers in avoiding potentially troubling behavior. Specifically, SPD policies and procedures will ensure that (1) the intervention strategy is implemented in a timely manner; (2) data regarding the implementation of the intervention is tracked in EIS; and (3) if necessary, the employee’s supervisor reviews the progress of the intervention strategy.” Consent Decree ¶¶ 157-163. SPD’s compliance with these provisions is discussed below. II. SPD’s EIS Policy To satisfy these requirements of the Consent Decree, SPD implemented a new EIS policy in mid-2015. See Monitor’s Third Semiannual Report (dkt. 123) at 76-89. The policy was approved by the U.S. Department of Justice, the Monitor and the Court. Id. SPD’s EIS policy relies on triggers, thresholds, and alerts. The following definitions are instructive. 4 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 5 of 28 Trigger: This is the basic building block of EIS. The system monitors the occurrence and frequency of triggers for each officer. Triggers are either individual actions by an officer (e.g., an officer uses force) or an event related to an officer’s actions (e.g., an OPA complaint or lawsuit is filed against an officer). SPD’s EIS policy refers to these actions or events as “triggers,” “indicators,” and “indicator criteria.” Alert: Alerts occur when an employee reaches a predetermined number of triggers within a set time period. The EIS module of IAPro (one of SPD’s case management systems) generates the alert, which is received electronically by the EIS Coordinator. The Alert contains a listing of the triggers that caused the Alert and details about the employee, such as name and assignment. Threshold: SPD’s EIS is threshold based. The threshold is the number of triggers and the time period that cause an Alert. For example, the threshold for preventable vehicle collisions is 2 collisions within 6 months. Early Intervention Assessment: The EIS Coordinator receives an Alert and then assigns an Assessment that must be completed by the chain of command. The Assessment document makes a recommendation as to whether the officer should be placed on a mentoring plan. As explained in Section III.C, not all Alerts lead to an Assessment. Early Intervention Mentoring Plan: When a chain of command determines that the reviewed triggers represent a need for performance improvement. A mentoring plan will be created when, in the judgement of the chain, the performance could be mitigated with a defined curriculum. The plan will be written by the employee’s direct supervisor and receive command approval to the deputy chief level. In addition, the chain of command can initiate a mentoring plan at any time based on their own observed concerns, even if an employee has not alerted. The Performance Review Committee (PRC): The PRC includes both members of Command Staff and Department representatives across multiple bureaus, including the Chief’s designee (Deputy Chief), the Executive Director of Human Resources, the Executive Director of Legal Affairs, the Early Intervention 5 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 6 of 28 Coordinator, and representatives from Patrol Operations, the Audit, Policy and Research Section, and the Education and Training Section. The PRC meets monthly and, per policy and by practice, is responsible for: • Ensuring Department-wide consistency and uniformity in the implementation of the EIS, including timeliness of Assessments consistent with policy requirements; • Determining whether Assessments and proposed Early Intervention Mentoring Plans are adequate to address concerns about risk factors, demonstrated indicators of stress, or training opportunities, and to provide the employee with any necessary tools and support. EIS Coordinator: a supervisor-level position housed in Human Resources, responsible for tracking and routing EIS Alerts, monitoring timelines associated with EIS review deadlines, and tracking the dispositions of EIS Assessments (also available in DAP). The Consent Decree requires SPD to monitor, review, and adjust, where appropriate, the EIS triggers and thresholds. ¶¶ 157-58, 163. On January 1, 2019, the triggers and thresholds were revised, with approval from the U.S. Department of Justice, the Monitor, and the Court. Dkts. 502 & 510. During the Study Period—January 1, 2019 to June 30, 2019—the triggers and thresholds were as follows: Trigger Threshold Levels A. Chain-of-command recommendations / discretionary assessments Each is reviewed B. Use-of-force (Type I) Top 1% of officers exceeding five incidents within a six-month period C. Use of-force (Type II) Top 5% of officers exceeding three incidents within a six-month period D. Use-of-force (Type III) Top 5% of officers exceeding ne incident within a six-month period E. Vehicle collisions Two (2) preventable collisions within six months OR four (4) collisions total within 12 months 6 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 7 of 28 F. Receipt of OPA complaints / Supervisory Action Four (4) complaints within 12 months G. Receipt of OPA complaints / Frontline Investigation Four (4) complaints within 12 months H. Receipt of OPA complaints / OPA Investigation Four (4) complaints within 12 months I. Receipt of EEO complaints Two (2) complaints within 12 months J. Named in police actions claims or Police action lawsuit against the City Two (2) within 24 months Additionally, an Early Intervention Assessment will be conducted at the aggregate level when an employee has a total of ten (10) triggers during any six-month window when the employee is not on a mentoring plan. III. Objectives and Methodology This report covers a study period of January 1, 2019, to June 30, 2019. Data for this report are derived from three sources. Data on supervisor reviews and individuals who have triggered EIS reviews are contained in records maintained by the EIS Coordinator. Data regarding precinct and squad trends with respect to use of force and closed OPA complaints/dispositions are sourced from the Data Analytics Platform (DAP, as described in detail in prior reports). Data is also drawn from IAPro, the Department’s database solution for records and case management relating to activities that may result in EIS triggers. IAPro provides the technical administration of EIS (e.g., it generates alerts and tracks workflow); the DAP collects EIS alert and workflow information from IAPro and renders it for analysis. While some additional data may be joined to provide additional analytical context for the EIS “incident,”1 the record of the alert is discrete and generated automatically by IAPro, resulting in a high-quality / high-confidence record that is not subject to functional error IAPro captures threshold exceedances through the creation of new “incidents” and adds every new EIS alert to the incidents table in the IAPro database. 1 7 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 8 of 28 handling rules.2 Of note, when the EIS code was delivered to the Department by Accenture, the contracted vendor of the DAP in Fall 2016, the code was accepted only after passing exit criteria3 for technical testing and User Acceptance Testing (UAT). A. Objective 1 – Assess Whether EIS Is Accomplishing Its Goals (Paragraphs 157-159; 163) SPD is committed to continuously evaluating the effectiveness of EIS and making evidence-based improvements as warranted. The EIS program is successful at developing supervisory skills and providing effective coaching and mentoring to officers. Each Early Intervention Assessment that is completed requires the officer and his or her supervisor to meet in person to discuss the incidents that caused the alert. These Assessments, alone, document a large volume of active, engaged supervision that otherwise would not have occurred. Mentoring plans also require supervisors to problem solve and engage with their employees on an ongoing basis, often over a period of several months. Another feature of the program is that the EIS Coordinator serves as an important resource to supervisors. In addition to reviewing and providing feedback on supervisors’ written Assessments and mentoring plans, the EIS Coordinator also maintains a repository of reference materials including training videos, books, and sample mentoring plans. As noted in the previous EIS Audit report, SPD identified some shortcomings in the EIS program. In an extensive effort to address these issues, SPD implemented numerous, substantial policy changes which were approved by DOJ, the Monitoring Team, and the Court. The information reported in the current study period reflects EIS information after the implementation of these changes. 1. Has SPD reviewed and adjusted EIS indicator criteria and thresholds as appropriate? (Paragraphs 157-158 and 163) The DAP maintains rules for functional error handling intended to protect the analysis from “orphan” or partial records. For data concept areas such as UoF, where multiple complete elements are required for a valid count (i.e. unique incident, officer and community member), a functional error may be applied, flagging the record for correction. With regard to EIS, data convergence occurs entirely within the IApro system (a trigger cannot occur without a complete record of a UoF, for example), functional error handling is therefore not applicable. 2 3 Technical testing involved source-to-target testing at every layer of the DAP system. UAT acceptance criteria required the delivered product (code and semantic layer) function analytically, sufficient to respond to questions necessary to demonstrate compliance. 8 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 9 of 28 After extensive consultation with and input from the Department of Justice and the Monitoring Team, SPD amended its policies related to EIS and to Use of Force. Those policies went into effect on January 1, 2019. The first set of policy changes involved revising the definition of Type I Force to clarify that discomfort caused by wearing (properly applied) handcuffs is not reportable as force. SPD determined that such reports of discomfort or minor pain made up an increasingly high percentage of reported use-of-force incidents (approximately 44% of Type I force in 2017 and two-thirds of Type I force in 2018).4 The policy now distinguishes between pain resulting from the use of force involving handcuffs (e.g., improper application of handcuffs) versus pain that can be attributed solely to the inherent discomfort of wearing handcuffs. SPD’s revised Use of Force policy defines “handcuff discomfort” as “discomfort or minor pain caused solely by wearing handcuffs properly applied” Accompanying that change was a revision requiring that that such complaints must be reported and screened by a supervisor. These revisions were intended to ensure that all complaints of pain from handcuffs are investigated as either a Type I use of force (application pain) or as Handcuff Discomfort (wearing handcuffs) and properly tracked by the Department. These changes were discussed at length with DOJ and the Monitor and approved by the Court. Dkt. 477. In addition, the Department deployed new handcuffs that are beveled and made of lighterweight aluminum, and are as effective while causing less discomfort. These policy and equipment changes have reduced the amount of time and effort spent by the chain of command on instances where handcuffs, properly applied, cause unavoidable discomfort. In addition, this handcuff-discomfort policy revision was a critical step in bringing the EIS program back in line with its goals, namely, ensuring that the EIS program is providing SPD with notice before an officer’s behaviors become problematic. Type I force was and continues to be a major driver of EIS alerts. In the Part I audit, Type I force accounted for approximately one third of all alerts, and that share has grown in the current study period. Eliminating “handcuffing discomfort” as a use of force, and therefore a trigger for EIS purposes, allows SPD to focus the EIS program on true uses of force and not on handcuffing discomfort. This is an important improvement to the EIS program5. The second set of policy changes implemented in January 2019 adjusted and revised EIS thresholds and indicators as required by Paragraphs 158 and 163 of the Consent Decree. As with the use of force policy revisions, SPD engaged in multiple rounds of discussions See SPD’s 2017 and 2018 Use of Force Outcome Reports, previously filed with the Court. Dkts 443 at 47 and 524-1 at 5. 5 In 2018, of the 1.860 Type I uses of force reported, 805 (43%) were handcuffing discomfort. 4 9 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 10 of 28 with the Monitor and the Department of Justice in developing the proposed revisions, and—with their acceptance—received approval from the Court on December 17, 2018 (Dkt. No. 510). SPD continues to be in compliance with this requirement. The revised EIS policy modified the description of the program’s purpose, triggers, and risks to better align the policy with the philosophy of the program which is not intended to be punitive or disciplinary. For example, instead of describing the EIS program solely as a “strategy to address at-risk behavior,” the revised policy identifies a more holistic set of goals, including to support employee wellness and professional growth and to address underlying factors that can lead to negative performance issues (including incident-based stress or training needs). The revised policy also clarifies that the primary purpose of a mentoring plan under EIS is to support the employee in achieving professional goals. In addition, the EIS policy revisions adjusted the trigger levels. To respond to the analysis and conclusions summarized above, the triggers for use of force are adjusted so that they are still based on percentages, but now are also subject to a baseline number of uses of force at or beneath which there will be no alert. For Type I force, the trigger is the top 1% of officers who have at least five incidents. Similarly, for Type II the percentage stayed at 5%, with at least three incidents. The threshold for triggering review is now 4 OPA complaints. The adjustments were intended to exclude from review the officers who fall within the top percentages for use of force, but still have used force very rarely (to account for the fact that Department-wide the rate of force being used is extremely low). Four of the trigger categories were eliminated, because they are duplicative of other policy provisions or review processes: (1) the unexcused failure to appear at training category was eliminated, because supervisors are already required elsewhere in policy to discuss any and all missed trainings with officers; (2) the K9 apprehension bite category was eliminated, because these incidents already trigger review under the Type II use of force category; (3) the vehicle pursuit category was eliminated, because it is already extensively reviewed by the chain of command pursuant to SPD’s vehicle eluding/pursuits policy; and (4) similarly, the officer involved shooting (OIS) category was eliminated as an EIS trigger6 because the Department has adopted a far more extensive process for OIS incidents— after every OIS incident the Department places the involved officer on mandatory leave, the specially trained Force Investigation Team conducts an investigation, and an in-depth inquiry is undertaken by the Force Review Board. At the time of these policy revisions, the Department expected that these changes would more closely focus the EIS program on activities that are relevant to risk and achieve better 6 Type III force is still a trigger, and under the current threshold any officer who exceeds one use of Type III force (including more than one OIS) within six months will be subject to an EIS alert . 10 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 11 of 28 alignment with the program’s goals. As discussed below, the findings of this report validate that expectation. Finally, the parties and the Monitor are currently discussing a third set of proposed policy revisions. The Sustainment Plan provides that SPD’s 2019 EIS policy review is due to the Court 30 days from today, on December 31, 2019. The proposed policy revisions under discussion are intended to address a recommendation made by DOJ and the Monitor in the validation section of the Part I EIS Audit—to calibrate EIS trigger thresholds to compare officers to other peer officers with similar assignments. 2. Has SPD monitored EIS to determine whether EIS is meeting its risk management objectives? (Paragraphs 157 and 163) SPD found, during the Part I Audit of EIS, reported and filed in April 2019, that the previous thresholds were over-capturing triggering incidents. For instance, under the previous thresholds, nearly 25% of all SPD officers received at least one alert, and the vast majority of those were Type I uses of force. Many of those Type I uses of force were for handcuffing pain only. During 2017, OPA did not sustain any allegations of out-of-policy use of Type I force. SPD found that EIS review of so many Type I uses of force was a drain on supervisory resources without an accompanying benefit of accurately capturing behaviors before they become problematic. During this Part II Audit, SPD has begun to see the effects of its policy changes that went into effect on January 1, 2019, as discussed above. While Type I uses of force still account for a large percentage of the alerts, the number of Type I uses of force has decreased, thereby significantly reducing the number of SPD officers who received an alert. SPD is now able to focus its supervisory resources more appropriately. In 2018, OPA sustained no out-of-policy uses of Type I. Notably, there was an increase in the percentage of mentoring plans when compared with the Part I Audit, which found that approximately 10% of all Assessments included a mentoring plan. During the study period for Part II, 17% of Assessments included a mentoring plan. Thus, although the number of Assessments and mentoring plans is smaller, SPD believes that the new threshold triggers better capture the SPD employees who are at risk of negative performance issues, employee discipline, and/or employee or department liability. 11 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 12 of 28 3. Does the EIS policy contain a mechanism by which the threshold for review is lower after an officer has already alerted? (Paragraphs 159 and 163) SPD continues to be in compliance with paragraphs 159 and 163. The EIS policy as approved by the Court on December 17, 2018, retains the language of the Department’s prior (Court approved) policy with respect to this requirement and states as follows: This policy is implemented by the EIS Coordinator. By design, once an officer has alerted, and therefore is already being assessed or on a mentoring plan, then the EIS Coordinator receives electronic notification for every new trigger that occurs for that officer.7 (As noted above, policy uses the word “trigger” to refer to the occurrence of an incident or action that is tracked by EIS, e.g., a use of force or the filing of an OPA complaint.) The EIS Coordinator reads each new alert and reviews the employees’ record to ensure that the alert is not repeating previous triggers (e.g., one new of pointing a firearm, when the officer is already being assessed or on a mentoring plan based on similar Type I uses of force). If the new alert is based on three or more new triggers, the officer’s supervisor is required to complete an updated Assessment (using a form titled Subsequent Alert Update). However, if an officer who is already on a mentoring plan receives even one new trigger, then the officer’s lieutenant or manager is required to perform an additional review. The purpose of an update Assessment is to inform a supervisor that an employee for whom the supervisor has recently completed an Assessment on has at least three new This occurs by design, because the EIS alert system does not reset or “zero out” after an officer alerts until the officer is outside the measuring period (e.g., if an officer alerts for six Type I uses of force within six months, then each new Type I force will cause a new alert until the system “resets” six months after the first incident). The purpose is to ensure close and continuous monitoring of the officer’s activities. 7 12 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 13 of 28 un-reviewed triggers. The supervisor is required to review the new triggers and assess whether or not the new information alters his or her Assessment of the employee’s performance when analyzed in conjunction with the previous history. To illustrate, assume an officer receives an “aggregate” alert based on six triggers: e.g., two instances of pointing a firearm, two complaints of handcuffing discomfort, and two OPA complaints.8 An Assessment is assigned to the officer’s supervisor. Within two months of the alert, the officer has three additional Type I uses of force. The EIS Coordinator would assign the supervisor to complete an update Assessment (if the officer had not already alerted, it would take six Type I uses of force to cause an Assessment). SPD accordingly continues to be in compliance with this requirement of the Consent Decree. B. Objective 2 – Assess Whether SPD Is Tracking EIS-Related Outcomes (Paragraphs 160 & 161) In its Phase I assessment of the EIS program, the Monitor found SPD to be in compliance with all EIS requirements in the Consent Decree, including data collection and tracking. See Monitor’s Eighth Systemic Assessment, Dkt. 374 (March 23, 2017). SPD’s data collection processes have continued in the same manner since that time. The specific mechanisms employed to collect and track EIS-related data are described below. • Has the number of individual officers who have triggered EIS reviews been completely and accurately collected, maintained, and retrieved at the precinct level? • Has the number of supervisor reviews of officers based on EIS triggers been completely and accurately collected, maintained, and retrieved? SPD has maintained compliance with these requirements of paragraphs 160 and 161. The required data on individual officers and supervisor reviews is collected and maintained by the EIS Coordinator and recorded in IAPro. During the study period, the Performance Review Committee (PRC) reviewed 89 Assessments.9 Fifteen (15) of During the study period, EIS policy, section 3.070-POL-2, provided that “an Early Intervention Assessment will be conducted at the aggregate level when an employee has a total of ten (10) [triggers] between A – M during any six-month window when the employee is not on a mentoring plan.” 9 As part of this Audit, SPD reviewed the number of Assessments the PRC reviewed between January 1 and June 30, 2019. Many of those Assessments began in 2018. 8 13 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 14 of 28 those Assessments contained mentoring plans – approximately 17% of all Assessments. This is an increase in the percentage of mentoring plans when compared with the Part I Audit, which found that approximately 10% of all Assessments included a mentoring plan. SPD believes this indicates that the new threshold triggers are better capturing the SPD employees who are at risk of negative performance issues, employee discipline, and/or employee or department liability. Over the study period, just 51 unique officers received at least one alert. More than 96% percent of sworn officers did not receive any alerts. Nearly all (88.2%) of the officers who received at least one alert received fewer than five alerts during the period; no officers received 20 or more alerts. See Figure 1. The total number of unique officers who received an alert is smaller than the number of completed Assessments (89), likely because the 51 new alerts all were generated under the higher thresholds of the new EIS policy. Due to inherent delays in the system some number of the Assessments completed during the study represent alerts generated toward the end of 2018 under the previous policy. Figure 1: EIS Alerts Per Officer • Has use of force information been completely and accurately collected, maintained, and retrieved at the precinct level? 14 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 15 of 28 The Department has been collecting, maintaining, and reviewing information relating to use of force at the precinct level for many years. The Monitor recognized these practices in Phase I of the Consent Decree. See Dkts. 231 & 360. More recently, the Department has demonstrated its gathering and tracking of such information through its annual Use of Force Outcome Reports, which contain a comprehensive overview of all use of force for a given year, see Dkt. 524-1, through its Use of Force Reporting and Investigation Audit Report (dkt. 497-1), and through its Use of Force dashboard, a version of which is publicfacing10. • Has information on closed OPA complaints and their dispositions been completely and accurately collected, maintained, and retrieved at the precinct level? As discussed in the Monitor’s Fourth Systemic Assessment (Dkt. 259-1 at 18), OPA began using IAPro as its day-to-day case management system in 2014. OPA issues annual, public reports on this data and its other activities. OPA reports are available at https://www.seattle.gov/opa/opa-reports Through the EIS module of IAPro, OPA complaints that lead to EIS alerts are tracked and reviewed by the chain of command and the PRC for the purpose of making Assessments and mentoring plans. Because coaching and mentoring are most effective when they are conducted close in time to the underlying incident, the EIS process is not delayed to await the outcome of OPA’s review. Therefore, an officer’s Assessment or mentoring plan is frequently completed before OPA has finished investigating or reviewing the underlying complaints. Separate and aside from the EIS program, OPA findings are used in other ways by SPD. After a recommended finding of sustained has been issued by the OPA Director on an allegation of misconduct, the Chief of Police decides what discipline to impose on the named employee. If the Chief decides not to follow one or more of the OPA Director’s recommended findings, the Chief must provide a written statement of the reasons for the decision within 30 days of making that decision. OPA training recommendations are always followed by SPD. Sometimes the disciplinary action recommendation includes a training requirement, which SPD requires the officer to complete regardless of the ultimate disciplinary resolution. In addition, for not 10 https://www.seattle.gov/police/information-and-data/use-of-force-data/use-of-force-dashboard) 15 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 16 of 28 sustained complaints, OPA may recommend that an officer undergo additional training. Officers must complete the recommended training in that situation as well. • Has information related to supervisor, precinct, squad and unit trends with respect to EIS been completely and accurately collected, maintained, and retrieved? In the following section, the Department reports on data relating to EIS tracking and alerts as required under Paragraphs 160 and 161 of the Consent Decree. During the 6-month study period (January 1, 2019 to June 30, 2019), a total of 119 EIS Alerts (“alerts”) were generated. As described above in Section II, alerts occur when an employee reaches a predetermined number of triggers within a set time period. (Triggers are events, such as a use of force or an OPA complaint.) Accordingly, there are many more alerts than triggers. A breakdown of these alerts, by trigger, is presented in Figure 2; for clarity, Figure 2 shows only the trigger that generated the alert. Figure 2: EIS Alerts by Trigger 70 61 60 50 40 30 23 20 11 8 10 1 1 1 Level 3 - Use of Force Vehicle pursuit EEO 13 0 OPA Level 2 - Use Investigation of Force Firearm discharge Vehicle Collision Level 1 - Use of Force Over the study period, 51 of the 119 alerts have led or will lead to an assessment of the officer. That is because many of the alerts are combined into one Assessment, since the EIS system does not reset after an officer has alerted (as explained above in footnote 5). In addition, officers can receive multiple alerts for the same underlying incident. To illustrate, assume an officer uses Type I force. That single use of force causes the officer to receive three alerts: one alert because it is the officer’s eighth Type I use of force in 16 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 17 of 28 six months; one alert because it is the officer’s tenth trigger (of any type) in six months; and one alert because the subject of force made an OPA complaint. In this instance, the officer’s supervisor is assigned to complete one Assessment—not three Assessments. Finally, some alerts are exact duplicates of prior alerts (largely due to how the EIS module handles reclassifications of incidents.) The EIS Coordinator reviews each and every new alert and the underlying triggers and ensures that multiple EIS Assessments or mentoring plans are not completed for the same officer based on the same underlying triggers. The two incident types that caused the largest portion of alerts are “Level 1 – Use of Force” and “Vehicle Collision.” “Level 1 – Use of Force” refers to Type I (least serious) force. During the study period, Type I force included, for example, the pointing of a firearm, use of a hobble restraint, and other force that causes transitory pain or a complaint of transitory pain.11 Seattle Police Manual § 8.050. As shown in Figure 2, Type I uses of force caused 81 EIS alerts. “Vehicle Collision” refers to both preventable and non-preventable collisions involving department owned, leased, rented, or otherwise department acquired vehicle. Vehicle Collisions accounted for 39 EIS alerts. The vast majority (86.6%) of alerts were generated by officers assigned to the Operations Bureau, with North Precinct officers represented most frequently (36.1%), followed by West Precinct officers (31.9%). See Table 1. 11 As explained above, minor discomfort caused by properly applied handcuffs is no longer classified as Type I force. 17 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 18 of 28 Table 1: Distribution of EIS Alerts by Organizational Assignment The distribution of alerts by organization assignment closely mirrors calls for service, crime, force, Terry stops and crisis activity, suggesting no obvious pattern. In other words, the highest volume of EIS alerts occurred in West Precinct and North Precinct, which are the precincts with by far the highest volume of 911 calls, crime rates, and thus policy activity. By rank/position of employment, Police Officers accounted for all but one alert generated during the study period (99.2%). Just one alert was generated for a single civilian Parking Enforcement Officer. See Table 2. Table 2: Distribution of EIS Alerts by Rank/Employee Position 18 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 19 of 28 Descriptive analysis suggests no obvious pattern at the Department-wide level, for EIS alert generation. Distribution of alerts by assignment, across age and experience closely mirror the demographics of the Operational Bureau, suggesting the most frequently observed alert activity is the predictable result of police service (complaints, force, vehicle operations). C. Objective 3 – Assess How EIS Is Being Implemented (Paragraphs 162 & 163) • Are supervisors periodically and appropriately reviewing EIS activity of officers in their chain of command? Aside from mentoring plans, EIS Assessments are the primary way that supervisors review the EIS activity of their employees. Sergeants are also required to monitor their employees’ EIS activity on electronic dashboards weekly, and in this way, they learn when an officer is approaching EIS thresholds. However, Assessments make up the vast majority of all work produced by the EIS program. Each Assessment requires an in-person meeting between the employee and their supervisor in which the supervisor reviews each of the triggering events with the employee. This type of supervisor-employee engagement provides the opportunity for a two-way conversation about the employee’s work performance, well-being, and more. Supervisor-employee engagement is an important component for supervisors to guide, counsel, and mentor their employees. An Assessment may or may not result in a mentoring plan; however, the supervisoremployee engagement is central to employee management. Once the Assessment is completed by the employee’s supervisor, it is routed through the chain of command. The chain of command – from the supervisor through the 19 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 20 of 28 commanders – must review it and make a determination on the following basic question. Do the events indicate that an improvement in performance could be achieved with a mentoring plan? The majority of Assessments come back with the “No action box checked,” meaning that no formal mentoring plan is required, which is expected in a threshold-based system. However, checking the no action box does not mean that nothing was done to address issues and or coach and mentor the employee. To conduct a proper and accurate Assessment, a supervisor must review the cases, speak with the employee about the triggers, discuss policy, document employee understanding and provide justification for decisions made. If the supervisor and chain of command does not properly perform and document an Assessment, the PRC (the executive committee which reviews and approves all Assessments) will return the Assessment for further work. This system fosters careful attentive leadership skills in our supervisors. All of these Assessments are reviewed through the chain of command and all must meet PRC approval. PRC review yields a fairly consistent return rate of about 10 percent for Assessments that are not approved. Typical reasons for Assessment returns are for minor corrections on format, or inadequate review of triggers. In addition, the chain of command and the PRC review the supervisor’s determination as to whether or not a mentoring plan is warranted. If the supervisor determines there should not be a mentoring plan and the chain of command or the PRC disagrees, then the supervisor must write a mentoring plan. The supervisor must then return the Assessment or mentoring plan within a specified time period. It will then be reviewed again by the chain of command and the PRC and only proceed if the requested corrections were made in a satisfactory way. Throughout the study period, the EIS Coordinator has observed a steady increase in acceptance and recognition of the goals and worthwhileness of mentoring plans by the chain of command, demonstrating a positive development in supervision and leadership. • Do supervisors review the progress of assigned intervention strategies as appropriate? • Are EIS intervention strategies being implemented in a timely manner? SPD policy and the practices of the PRC ensure that EIS intervention strategies are implemented in a timely manner. Once an EIS Assessment is assigned to a supervisor, the supervisor is required to complete the Assessment within fourteen days; the lieutenant 20 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 21 of 28 and captain have 3 and 7 days respectively for their review. If a mentoring plan is warranted, then the supervisor is assigned to complete it within a specified amount of time (ranging from 10-24 days depending on complexity). Each mentoring plan contains a timeline within which the supervisor and chain of command must complete the intervention strategy; the most common mentoring plan timeline is 30 days and the longest is 90 days. The EIS coordinator reviews and monitors all in-progress mentoring plans regularly to ensure that they are closed out in a timely manner. Out of all 15 mentoring plans from the study period (each for a different officer), the tasks specified in 7 of them had been completed by the officer and supervisor within 4 months. Three additional mentoring plans took between four and six months to complete. Three mentoring plans were completed between six months and one year. These timeframes were appropriate, given the subject matter being addressed. For example, a mentoring plan to address routine driving issues can be completed relatively quickly, as it typically involves taking a driving course that is offered regularly. Interventions assigned to address more complex or weighty topics appropriately take longer and involve substantially more interactions over a longer period of time between the officer and his or her supervisor. One of the fifteen plans was not completed because the officer went on an extended administrative leave, and the mentoring plan was placed on hold. 21 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 22 of 28 For the 89 Assessments that were assigned to a supervisor and completed by the supervisor during the study period, more than one third were completed with two weeks of the initial EIS alert. All but 6 were completed within two months. For those cases that were not completed within two months, the EIS coordinator granted supervisors extensions in which to complete their Assessments. The EIS coordinator generously granted extensions, as SPD believes that a thorough Assessment is more critical to SPD’s intended outcomes than meeting the fourteen-day timeline. The EIS coordinator closely monitors the progress of the Assessments to ensure that the supervisor is striving to complete them. Table 3: EI Assessments – Time between Notification of the Alert and Completion of the Assessment Days between Notification & Completion of Assessment 20 19 18 16 13 14 14 12 9 10 9 8 6 6 4 2 0 0-7 Days 8-14 Days 15-21 Days 22-30 Days 31-60 Days 60+ Days As with the underlying Assessments, mentoring plans provide an opportunity for continued supervisor-employee engagement. The mentoring plan process requires that the supervisor meet in person with the officer and, where appropriate, the supervisor must participate in the assigned intervention. For example, the recommended action step for an Assessment related to receipt of an OPA Complaint is “[r]ide along with employee for a minimum duration of one hour per shift, twice per work week.” The EIS AssessmentMentoring Plan Form contains other suggested intervention strategies and is attached to this report as Appendix C. However, there is no formula that a supervisor can apply when determining an appropriate intervention; the decision must be based on the supervisor’s interactions with and knowledge of the welfare and status of their employees. In addition, 22 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 23 of 28 the EIS Coordinator maintains a large repository of high-quality Assessments and mentoring plans that have been anonymized to provide instructive examples. • Is data regarding interventions being tracked in EIS? For every EIS Assessment or mentoring plan, a “final disposition form” is entered into IAPro. Using this form, the Department records the specific interventions assigned, if any, and the dates on which the Assessments or interventions are completed. In sum, the Department has continued to sustain compliance with each of Paragraphs 157 through 163 of the Consent Decree. FUTURE STEPS As noted in the last report, EIS continues to pose challenges and areas for improvement. The social science around early intervention theory has evolved substantially over the past several years, and SPD must ensure that its system evolves to keep pace. Committed to doing so, the Department describes below the work that it is doing in this regard, independently and in collaboration with partners, nationally. The Department is also taking a leadership role nationally by exploring ways to leverage multi-disciplinary social science research to improve upon the EIS model. In concept, Early Intervention Systems are designed to leverage data-based metrics to identify officers at risk of future performance issues, providing police supervisors an opportunity to intervene through mentoring, counseling, additional training, or other non-disciplinary measures at a point before the officer is involved in a significant adverse event. Such systems are intended to promote chain of command accountability for employee performance while also providing supervisors a foundation to foster and recognize the success and professional growth of their employees. In recent years, the model of tying EIS review to the occurrence of policing activities, such as the use of force, has been criticized in academic literature as ineffective, often counterproductive, and perceptually, as punitive and stigmatizing. See, e.g., Worden et al. (2013) Intervention with Problem Officers: An Outcome Evaluation of an EIS Intervention. Crim. Justice and Behavior, Vol. 40, No. 4 409-437; http://www.latimes.com/local/la-me-lapd-problem-officers-20140826-story.html. SPD fully endorses the importance of developing an effective early intervention model, from both an officer safety/well-being and Department risk management perspective. Accordingly, while appreciating that designing an accurate model for attempting to 23 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 24 of 28 predict how any one individual will behave in dynamic, unpredictable, and often dangerous situations is an inherently complex and iterative task, SPD is actively engaged in efforts on multiple fronts to further advance the social science underlying this goal. At the national level, SPD is a core contributing member of a multi-disciplinary advisory committee, comprising representatives of law enforcement, medical and mental health fields, and research institutions from around the country, working in collaboration with the Chicago Police Department, the Los Angeles Police Department, and the University of Chicago Crime Lab to design a next-generation EIS. At the organization level, continuing its work with research partners from Washington State University, SPD is exploring how it might best leverage the increasing sophistication of its data and analytic tools to allow for more validated identification of relevant variables to an EIS, both psychological and situational. Paramount in both efforts is a shift of focus away from the outcome of an incident (outcomes that, again, are subject through separate processes for timely and rigorous critical review) and instead towards the nature of incidents to which officers are called to respond. The premise for this shift is grounded in the impacts – physiologically, psychologically, and behaviorally – on cumulative incident stress and vicarious trauma on first responders. In simplistic form, the Department’s longer-term vision for EIS is built on the following premises: 1. By nature of their occupation, law enforcement officers are regularly exposed to incident stress likely to manifest, acutely or over time, in secondary traumatic stress/vicarious traumatization.12 12 “Vicarious trauma” has been explained as [A]n occupational challenge for people working and volunteering in the fields of victim services, law enforcement, emergency medical services, fire services, and other allied professions, due to their continuous exposure to victims of trauma and violence. This work-related trauma exposure can occur from such experiences as listening to individual clients recount their victimization; looking at videos of exploited children; reviewing case files; hearing about or responding to the aftermath of violence and other traumatic events day after day; and responding to mass violence incidents that have resulted in numerous injuries and deaths. Vicarious trauma overlaps with concepts of secondary traumatic stress (the “natural consequent behaviors and emotions that often result from knowing about a traumatizing event experienced by another and the stress resulting from helping, or wanting to help, a traumatized or suffering person[,] with symptoms [that] can mimic those of posttraumatic stress disorder”) and burnout (“a state of physical, emotional, and mental exhaustion caused by long-term involvement in emotionally demanding situations.”). 24 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 25 of 28 2. Repeated stressful and/or traumatic events can chronically elevate the body’s stress response, leading to poorer physiological and psychological functioning, up to and including post-traumatic stress disorder (PTSD).13 3. Recognizing, mitigating, and ideally preventing vicarious trauma and its sequelae is both a critical officer safety issue14 and an organizational risk management priority.15 4. If the goal of an EIS is to identify potentially problematic performance indicators before they culminate in an adverse (out of policy, or worse) event, then such a system must be: a. tied to indicators that are at minimum logically, but where possible, empirically, predictive of future performance deficits; and b. structured so as to provide meaningful mitigation strategies that address both the nature of the indicators and, to the extent that the Department may influence, physiological/psychological factors that may negatively impact an officer’s performance, professional growth, development, and well-being. This vision is not only confidently aligned with core principles around preserving performance ideals in first responders, but importantly, incorporates feedback received https://vtt.ovc.ojp.gov/what-is-vicarious-trauma 13 Such symptoms include difficulty managing emotions, difficulty making good decision, problems managing boundaries, sleep disruption and/or nightmares, generalized hypervigilance/anxiety, substance abuse and engaging in high risk behaviors, changes in appetite, negativity, and social isolation. Some studies estimate that up to nearly 20% of police officers in the United States meet DSM-V diagnostic criteria for PTSD; other studies estimate that up to 34% suffer symptoms associated with PTSD, although they may not meet criteria for full diagnosis. See, e.g., Charles R. Marmar, et al., Predictors of Posttraumatic Stress in Police and Other First Responders, 1071 ANNALS N.Y. ACAD. SCI. 1, 18 (2006); Regehr., C. and LeBlanc, VR (2017, PTSD, Acute Stress, Performance and Decision-Making in Emergency Service Workers. J. Am. Acad. Psychiatry and Law. 45(2) 184-192. See, e.g., President’s Task Force on 21st Century Policing, Washington DC Office of Community Oriented Policing Services, Final Report of the President’s Task for on 21st Century Policing (2015) http://www.cops.usdoj.gov/pdf/taskforce/taskforce_finalreport.pdf (noting the toll of stress on the life expectancy of law enforcement officers). 14 15 See generally Regehr and LeBlanc (2017), discussing the impact of stress conditions on perception, cognition, risk assessment, and response. 25 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 26 of 28 from officers, supervisors, and commanders around advancing an EIS that truly succeeds in identifying and mitigating against predictors of future problematic performance. Indeed, there were positive points elicited from focus groups that must continue to be nurtured in future iterations of EIS. As one example, supervisors expressed a general hunger for aggregated information around their employees’ overall activity and a mechanism for identifying trends within their squads. Officers expressed appreciation for a system that encouraged conversation with their supervisors; supervisors likewise expressed a desire for greater interaction with their officers but lamented the limited time available given the amount of paperwork that they are also responsible to complete. And importantly, officers across the ranks of focus groups – but particularly those in senior positions – emphasized the significance, in time and over time, of vicarious trauma, critical incident stress, and the toll of this experience on officer well-being and performance, and openly advocated for a system that would shift an EIS in this direction. Over the longer term, SPD intends to continue its engagement with organizations nationally, with research partners, with its employees, and with the Inspector General to develop a new model for EIS that more accurately meets the goals of the system. DOJ AND MONITORING TEAM VALIDATION In Phase I of the work of under the Consent Decree, DOJ and the Monitoring Team reviewed SPD’s compliance with the requirements of the Consent Decree through 10 assessments, covering the roughly six topic areas of the Consent Decree: force investigation and reporting, crisis intervention, supervision, Early Intervention System (“EIS”), use of force, and stops and biased policing. By the end of 2017, the Monitoring Team and DOJ found the City of Seattle to be in compliance with each area. On that basis, the Court issued a finding of “full and effective compliance” with the requirements of the Consent Decree. By the terms of the Consent Decree, the City of Seattle is now required to demonstrate that it can sustain compliance with those requirements for a period of two years. 16 16 Although the Court found that the City has fallen partially out of full and effective compliance with the Consent Decree in its May 21, 2019 Order, the Court did not find that the City has fallen out of compliance in any area covered in the Phase II Sustainment Plan. See Dkt. 562 at 2. The Court indicated that it “remains hopeful that the City can complete these assessments and discharge these areas of the Consent Decree within the two-year sustainment period.” Id. These assessments, and DOJ’s and the Monitoring Team’s review of these assessments, are therefore unaffected by the Court’s May 21, 2019 Order. 26 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 27 of 28 During Phase II of the Consent Decree work, the City of Seattle has taken over the lead role in conducting assessments of the six core topic areas of the Consent Decree. By taking this lead role, SPD must demonstrate not only sustained compliance, but also a willingness and ability to critically self-assess their own progress in these areas, which are central to effective and constitutional policing. This does not mean, however, that the work of DOJ and the Monitoring Team is done. In Phase II, DOJ and the Monitoring Team are reviewing the City’s proposed methodologies for each audit and are conducting their own independent research and analysis or “look behind” the City’s review. For this audit, DOJ and the Monitoring Team consulted with SPD and ultimately approved the methodology used by SPD in conducting its own self-assessment. In addition, DOJ and the Monitoring Team requested and received a randomly selected sample of EIS case files, including the initial assessments, Performance Mentoring Plans, and Status Reports, for the time period January 1, 2019 to June 30, 2019. The sample set was comprised of fifteen of the Performance Mentoring Plans reviewed by SPD’s Performance Review Committee during the study period. DOJ and the Monitoring Team, together with their subject matter experts, reviewed these materials for compliance with the terms of the Consent Decree relating to EIS and SPD’s policies regarding the same. DOJ and the Monitoring Team subsequently conferred about their findings and, based on their Phase II reviews, concluded as follows: • The City of Seattle has demonstrated that it has sustained compliance with the EIS requirements of the Consent Decree and SPD’s policies governing the same, including requirements that the EIS system continue to be used for risk management (and not disciplinary) purposes and that supervisors periodically review EIS activity of officers in their chain of command. Further, the City has demonstrated through revisions to SPD’s policies that it has monitored and made adjustments to the EIS system to ensure that it is meeting its risk management objectives. • In the last audit of the EIS system (filed April 15, 2019), we identified some apparent confusion on the part of supervisors regarding how and to what extent they could discuss performance issues with officers who “triggered” under the EIS system because of (or in part because of) complaints that were made about them to the Office of Police Accountability (“OPA”). In our validation of this audit, it appears this issue has been addressed -- supervisors appear to be thoroughly discussing the underlying issues of the complaint with the officers even when a matter remains pending with OPA. 27 Case 2:12-cv-01282-JLR Document 595-1 Filed 11/29/19 Page 28 of 28 • Revisions to the EIS policy in 2018 eliminating some trigger categories and adjusting the trigger levels for other categories appear to be having the intended effect of re-focusing the EIS program to more accurately identify underlying issues that can lead to problematic officer behavior, as well as supporting employee wellness and professional growth. In the interest of continuous improvement, DOJ and the Monitoring Team offer the following technical assistance to SPD based upon issues spotted during their respective reviews. Although none of these issues rose to the level of systemic non-compliance with the terms of the Consent Decree, DOJ and the Monitoring Team strongly encourage SPD, and the Inspector General who will take over the audit function for this topic area in the future, to give attention to these matters going forward: • The forms used in the EIS system could be improved by capturing more information, in particular to assist future auditors in tracking and evaluating the EIS process. For instance, we would encourage SPD to include spaces for identifying deadlines for performance plan action items, and dates by which the actions were actually taken. We would also recommend identifying and tracking when performance plans are initiated by supervisors versus when they are initiated by the PRC (which will help auditors to assess how well supervisors are identifying issues). In addition, in cases in which the performance plan arises due to OPA triggers, to include an update about the status of the OPA investigation or the outcome of the OPA investigation in order to assist supervisors (and later auditors) in assessing the underlying issues. • We note that SPD will soon propose EIS policy revisions to the Court that adopt “peer to peer” analysis to address the issue of “over-triggering” of proactive officers. We encourage SPD to continue monitoring the EIS system to in this regard, while also ensuring that the system does not under-trigger after such adjustments. In addition, we encourage SPD to continue to track national best practices regarding early intervention systems as they continue to evolve and to modify SPD’s own system as appropriate to better predict problematic behavior. 28