CIVILIAN OV E R S I G H T O F L A W ENFORCEMENT ASSESSING THE EVIDENCE Joseph De Angelis Richard Rosenthal Brian Buchner Table of Contents Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Research and Resources Related to Civilian Oversight of Police . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Brief History of Oversight in the U .S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Contemporary Models of Civilian Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Organizational Structure and Effectiveness in Citizen Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Trending Issues in Civilian Oversight of Law Enforcement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 What Do We Know About Civilian Oversight of Law Enforcement? Assessing the Evidence . . 13 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Research Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 Electronic Organizational Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18 Brief History of Civilian Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Early Efforts at Establishing Civilian Review, 1920s-1960s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 Emergence of Investigative Models of Civilian Oversight, 1970s-1980s . . . . . . . . . . . . . . . . . . . . . . . . .20 Contemporary Models of Civilian Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Investigation-Focused Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 Review-focused Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 Auditor/monitor-focused Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29 Organizational Structure and Effectiveness in Citizen Oversight . . . . . . . . . . . . . . . . . . . . . . 32 Organizational Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 Core Elements of Successful Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36 Trending Issues in Civilian Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Using Performance Metrics to Capture Oversight Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Problem-oriented Approaches to Civilian Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46 Expanding Alternative Dispute Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47 Conclusions and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Oversight Is a Complex Organizational Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 There is Clear Variation in the Budgets Among Oversight Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 Some Models of Oversight May be More Effective at Getting Recommendations Implemented . . . . . .50 Important Considerations when Implementing or Reforming a Local Oversight Program . . 52 Jurisdictions Should Focus on “Best-Fit” Rather Than “Best Practices” When Considering How to Structure Civilian Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52 Oversight Should Employ the Least Force Necessary to Accomplish its Goals . . . . . . . . . . . . . . . . . . . . .52 A Number of Resources are Available to Jurisdictions Considering Implementing Oversight or Reforming their Current Oversight Framework . . . . . . . . . . . . . . . . . . . . . .53 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Appendix A . Research Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Appendix B . Statistical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4 Civilian Oversight of Law Enforcement Assessing the Evidence Executive Summary Over the last several decades, issues of trust and accountability have moved to the forefront of community-police relations and a great deal of scholarship has been devoted to enhancing police performance through the strengthening of law enforcement oversight functions. During this same period, highly publicized officer-involved encounters have led to the proliferation of organizational mechanisms for reviewing and improving officer conduct (Walker 2001; Alpert et al. 2016). One such mechanism is the development of civilian oversight of law enforcement. Sometimes referred to as citizen oversight, civilian review, external review and citizen review boards (Walker 2001; Alpert et al. 2016), this form of police accountability is often focused on allowing non-police actors to provide input into the police department’s operations, often with a focus on the citizen complaint process. In some jurisdictions, this is sometimes accomplished by allowing oversight practitioners (both paid and volunteer) to review, audit or monitor complaint investigations that were conducted by police internal affairs investigators. In other jurisdictions, it is done by allowing civilians to conduct independent investigations of allegations of misconduct lodged against sworn law enforcement officers. It can also be accomplished through the creation of mechanisms that are authorized to review and comment on police policies, practices, training and systemic conduct. Some oversight mechanisms involve a combination of systemic analysis and complaint handling or review . The goal of this publication is to offer a broad examination of the key issues facing civilian oversight of law enforcement in the U.S. Drawing from a review of the available research, as well as organizational data collected from 97 police oversight executives, this report is designed to help local policy-makers, police executives and members of the local community explore the key issues that can accompany the implementation and sustainability of civilian oversight of police mechanisms at the municipal and county levels. More specifically, this report addresses a number of key areas in relation to police oversight, including: 1 . A comprehensive review of the key resources and research relating to civilian oversight of police; 2. A brief review of the historical evolution of oversight in the U.S.; 3 . A detailed examination of three different models of oversight: investigation-focused, review-focused and auditor/monitor-focused; 4. An assessment of the key factors that promote organizational effectiveness in civilian oversight; and, 5 . An exploration of trending issues in relation to oversight, particularly the debate over how to measure performance of police oversight agencies, the potential value of problem-solving methodologies and the increasing emphasis on the value of alternative dispute resolution techniques for resolving complaints against police officers. This report concludes by identifying issues that jurisdictions may want to consider if they are evaluating whether to implement oversight or revise their current oversight framework. This report also identifies critical areas in need of further research. Research and Resources Related to Civilian Oversight of Police A reasonably large body of academic and policy literature on civilian oversight exists that dates back to the 1970s. For jurisdictions that are considering implementing police oversight or are considering reforming the current oversight functions in their jurisdiction, there are a number of key publications that can be consulted. For example, the following reports and books provide general guidance on the issues that can accompany the implementation of civilian oversight of police: • Civilian Oversight of Police: Advancing Accountability in Law Enforcement (Prenzler and den Heyer 2016) • The New World of Police Accountability (Walker and Archbold 2014) • Citizen Oversight of Law Enforcement Agencies (Perino 2006) Civilian Oversight of Law Enforcement Assessing the Evidence 5 • Citizen Review of Police: Approaches and Implementation (Finn 2001) • Police Accountability: The Role of Citizen Oversight (Walker 2001) Recent publications have begun to examine the broad organizational characteristics of different models of civilian oversight in the U.S. They include: • Citizen Oversight in the United States and Canada: Applying Outcome Measures and Evidence-Based Concepts (Alpert et al. 2016) • Overview of Civilian Oversight of Law Enforcement in the United States (Attard and Olson 2013) • Civilian Oversight of Policing: Lessons from the Literature (Miller 2002) • Models of Oversight: A Critique (Prenzler and Ronken 2001) A limited number of publications, listed below, focus on effectiveness and performance measurement in civilian oversight of law enforcement: • Holding Police Accountability Theory to Account (Harris 2012) • Performance Indicators for Police Oversight Agencies (Prenzler and Lewis 2005) • Evaluating the Performance of External Oversight Bodies (Brereton 2000) • The Use of Performance Measurement in Civilian Oversight in the United States (Mohr 2007) Several websites operated by non-profit organizations contain useful resources, links to police oversight agencies, historical blue ribbon reports and other types of toolkits that can serve as a starting point for jurisdictions considering implementing police oversight or revising their current oversight framework. • National Association for Civilian Oversight of Law Enforcement (NACOLE), www .nacole .org • Police Assessment Resource Center (PARC), www .parc .info • Center for Policing Equity, www .policingequity .org Brief History of Oversight in the U .S . Samuel Walker (2001; 2006), Merrick Bobb (2003) and Alpert et al. (2016) provide detailed historical descriptions of the evolution of civilian oversight in the U.S. The history of civilian oversight in the U.S. can be broken down into several distinct developmental waves . Early Police Commissions, 1880s-1920s. Early forerunners of modern police oversight initially took the form of police commissions, which were implemented by Progressive Era reformers in the late 19th and early 20th centuries as a strategy to break the hold of political machines on local policing (PARC 2005; Walker 1977; Walker 2001). Most early commissions failed to provide meaningful oversight of local police departments, partly due to the fact that early commission members were political appointees, had little expertise in policing and tended to become highly deferential to police executives (Walker 2001; Bobb 2003: 4; Attard and Olson 2013). Early Efforts at Establishing Modern Civilian Review, 1920s-1960s. The first modern forms of civilian oversight in the U.S. began in several large cities in the middle of the 20th century (e.g., Washington, D.C., Philadelphia and New York City). Early efforts often developed out of conflicts between police and local communities of color and generally took the form of under-resourced review boards that played a role in receiving complaints and reviewing completed internal police investigations into allegations of misconduct filed against officers (Hudson 1971; Terrill 1988; Walker 2001; Alpert et al. 2016). All of the early review boards were ultimately abolished in the face of limited resources and fierce resistance from police unions and local politicians (Bayley 1991; Walker 2001; Walker 2006). 6 Civilian Oversight of Law Enforcement Assessing the Evidence Emergence of Investigative Models of Civilian Oversight, 1970s-1980s. Though all of the initial review boards failed, another wave of development began in the late 1960s and carried through to the 1980s (Walker 2001; Walker 2006; Alpert et al. 2016). This wave has its roots in the Civil Rights Movement and developed, in large part, as a result of pushback and protests over police treatment of African-Americans. While the first modern oversight agencies were structured as civilian review boards, this wave saw the development of several full investigative oversight agencies that had enhanced resources, greater durability and expanded organizational authority. Examples of full investigative agencies in this wave included the Public Review Commission in Berkeley, CA and the Office of Citizen Complaints in San Francisco, CA. Emergence of Auditor/monitor and Hybrid Models of Civilian Oversight, 1990s-Present. Another wave development began in the 1990s and continues to present day (Walker 2001; 2006; Alpert et al. 2016). During this period, there was a rapid expansion of the number of police oversight agencies subsequent to the March 1991 videotaped beating of Rodney King by the Los Angeles Police Department (LAPD). While review-focused agencies remained the most common form of oversight, this period also saw the development of the first police auditors, including the San Jose Police Auditor and the Special Counsel for the Los Angeles County Board of Supervisors in 1993. Hybrid form of oversight agencies, where elements of different models of oversight are purposely combined in a single agency, increased during this period. While no one measured the exact numerical increase in oversight agencies in the 1990s, it is clear the number grew from a handful in the early 1990s to over 100 in 2001 (Walker 2001).1 The research team for this report was able to identify 144 oversight agencies operating at the local level in the U.S. as of March 2016. Contemporary Models of Civilian Oversight Over the years, there have been multiple attempts to create schemas to classify civilian oversight of law enforcement programs. One of the challenges in accomplishing this feat is that it almost appears that no two civilian oversight agencies in the U .S . are identical. Each jurisdiction has its own political, social and cultural tensions that influence the development of each oversight entity’s legal authority and organizational structure. In this publication, oversight agencies in the U.S. have been grouped into one of three oversight categories based on input they provided in conjunction with an evaluation of their public mission statement and foundational legal authority. The three categories include: Investigation-focused Models. A form of oversight that operates separately from the local police or sheriff’s department. While the structure, resources and authority of these types of agencies can vary among jurisdictions, these agencies are tied together by their ability to conduct independent investigations of allegations of misconduct against police officers. Review-focused Models. A type of oversight that focuses its work on reviewing the quality of completed internal affairs investigations. Many review agencies take the form of volunteer review boards or commissions and are designed around the goal of providing community input into the internal investigations process. Instead of conducting independent investigations, review agencies may evaluate completed internal affairs investigations, hear appeals, hold public forums, make recommendations for further investigation and conduct community outreach . Auditor/monitor-focused Agencies. One of the newest forms of police oversight. While there can be variation in the organization structure of this type of civilian oversight, auditor/monitor agencies tend to focus on promoting large-scale, systemic reform of police organizations while often also monitoring or reviewing individual critical incident or complaint investigations . 1 Due to the municipal nature of these agencies, it is difficult, if not impossible to identify all such agencies throughout the U.S. Even with the advent of modern information technology, it can be difficult to identify oversight agencies when they have no web presence. Civilian Oversight of Law Enforcement Assessing the Evidence 7 Organizational Structure and Effectiveness in Citizen Oversight To examine variation in civilian oversight, the research team collected organizational data from 97 civilian oversight agencies in the U.S. The goal was threefold: 1) to identify national patterns in oversight; 2) to compare organizational characteristics across common oversight models; and 3) to examine effectiveness in oversight (See Appendix A for a description of the research methods). Common Goals of Civilian Oversight of Police. An important issue for any municipality considering implementing or revising an oversight agency is to determine the goals for that agency. Based on the organizational data collected, the most commonly shared goals across all models of oversight are: 1 . Improving public trust; 2. Ensuring accessible complaint processes; 3 . Promoting thorough, fair investigations; 4. Increasing transparency; and, 5 . Deterring police misconduct . Even though those five goals were common to almost all oversight agencies that submitted organizational data, differences between oversight models emerged in relation to other types of organizational goals. Auditor/monitor agencies tended to rate goals associated with policy review, auditing, statistical pattern analysis and issuing recommendations as critical or important for their agency. Investigation-focused agencies were more likely than other types of agencies to indicate that improving accessibility of the complaint process, ensuring thorough investigations, complainant satisfaction and ensuring that complainants received justice were critical or important for their agency. Review-focused agencies tended to identify the importance of goals associated with the complaint investigation process, but were less likely to rate goals associated with the complaint investigation process as critical or important . Triggers for Police Oversight in the U.S. Much of the writing on civilian oversight indicates that it tends to emerge out of local crises involving local police and community complaints over racially disparate policing and excessive uses of force (Walker 2001). The data supplied support that view. The most common trigger involved force-related incidents (49 percent). Just under 30 percent of the agencies indicated that racially biased policing played a role in the development of oversight. Notably, only 11 percent of the agencies indicated that they implemented their oversight agency as a proactive or preventive policy effort and not as the result of a specific critical incident. Age of Oversight Agencies. The 97 civilian oversight agencies reported how long their agency has been in operation. The largest proportion of respondents (55 percent) indicated that their agency has been in continuous operation for 16 or more years, which is not surprising since the largest increase in the number of oversight agencies occurred in the 1990s. This pattern suggests that the field of oversight has begun to mature and has reached a level of some stability. Types of Law Enforcement Agencies Overseen. A large proportion of the oversight agencies that provided data indicated that they were responsible for overseeing municipal police departments (82 percent). The second largest area of jurisdiction was for county sheriffs (15 percent). Size of Law Enforcement Agencies Overseen. The largest proportion of oversight programs oversaw law enforcement agencies (45 percent) that had fewer than 500 sworn employees. However, there was clear variation across the three models of oversight. Review agencies were much more likely to report overseeing smaller law enforcement agencies (500 or fewer officers), while investigative agencies and auditor/monitor agencies were more commonly found in jurisdictions with larger law enforcement agencies . 8 Civilian Oversight of Law Enforcement Assessing the Evidence Core Elements of Effective Civilian Oversight. Overall, little research has been devoted to examining factors that influence the effectiveness of civilian oversight on police. More critically, relatively little is known as to whether different types of jurisdiction or authority have a greater or lesser impact on patterns in officer misconduct or organizational reform. Even though there is relatively little social science research on this issue, this report draws from an existing body of policy writing (see Walker 2003) and organizational data collected from 97 programs to examine 12 related principles that may strengthen civilian oversight programs. The principles reviewed include: 1 . Independence; 2. Adequate jurisdictional authority; 3 . Unfettered access to records; 4. Full cooperation; 5 . Access to law enforcement executives and internal affairs staff; 6 . Support of process stakeholders; 7 . Adequate resources; 8 . Public reporting/transparency; 9 . Use of statistical pattern analysis; 10 . Community outreach; 11 . Community involvement; and 12. Respect for confidentiality. Trending Issues in Civilian Oversight of Law Enforcement This report explores some of the key recent developments in civilian oversight of police. Using Performance Metrics to Capture Oversight Effectiveness. Even though there is a growing need for civilian oversight to demonstrate its organizational value, there is currently no consensus on how to measure organizational performance in the field of civilian oversight. Even so, a number of studies have sought to explore the value of different performance measures (Brereton 2000; Walker 2001; Prenzler and Lewis 2005; Mohr 2007; Filstad and Gottschalk 2011; Faull 2013). In general, a number of issues appear clear. First, “sustain rates” cannot be used to reliably compare performance across civilian oversight agencies (Walker 2001). Second, there is a growing consensus that other measures can be useful for capturing workload and organization performance in oversight. These measures include: 1 . Trends in reported police misconduct (e.g., number of cases filed); 2. Workload measures (e.g., number of open cases, cases reviewed, number of investigations conducted, number of appeals, identification of case deficiencies); 3 . Patterns in findings and outcomes on case investigations; 4. Timeliness and process efficiency measures (e.g., number of cases assessed in a given time period); 5 . Public opinion survey measures (e.g., satisfaction rates for complainants and officers, community confidence in the police); Civilian Oversight of Law Enforcement Assessing the Evidence 9 6 . Review and recommendation effectiveness (e.g., number and type of recommendations made, number and type implemented); and 7 . Accessibility measures (e.g., accessibility of the office to the public; accessibility of the complaint process overall) (see Prenzler and Lewis 2005: 78-82). Adapting Problem-Oriented Approaches to Civilian Oversight. Problem-oriented policing has been an important approach to law enforcement for more than 30 years. In this approach, patterns of crime and disorder are analyzed carefully to develop new programs that can reduce the incidence of the identified crime (Porter 2016). Even though this strategy has been popular with law enforcement, it has not been widely adopted for use in civilian oversight of law enforcement. Adapting this approach for use in civilian oversight has the potential to shift the focus of civilian oversight away from a reactive focus on investigating officers who may have engaged in misconduct toward a proactive, preventive focus that seeks to explore patterns in problems and identify solutions than will prevent misconduct from occurring in the future (Walker 2001; Harris 2012; Porter 2016). Expanding Alternative Dispute Resolution. The use of mediation by civilian oversight to resolve police complaints has become an increasingly popular strategy over the last 15 years. Altogether, 29 agencies reported that they had a mediation program, up from 20 agencies in 2008 (PARC 2008). There is a growing body of research that supports the argument that mediation can have positive benefits for complainants and officers, especially in relation to complaints alleging less serious misconduct (Walker et al. 2002; PARC 2008: 45; Schaible et al. 2013). Conclusions Given that there has been relatively little empirical research on civilian oversight, this project drew strongly from organizational data collected from 97 civilian oversight programs to identify important issues within the field. Using those data, this paper catalogs a number of key issues that jurisdictions may consider when evaluating whether to implement oversight or revise their current oversight framework. Oversight is common in U.S with at least 144 oversight agencies and almost all large cities having one or more forms of civilian oversight of police. These patterns, when taken as a whole, seem to indicate a field that is approaching maturity. Large jurisdictions that do not have oversight now stand as outliers, rather than representing the norm. Oversight is a complex organizational field. Review-focused agencies were the most common in the data collected from the program sites (40 percent), but auditor/monitor and investigation-focused agencies are now common in the field as well (25 percent and 35 percent, respectively). More importantly, there is an increasing blurring of the boundaries between different models of oversight. Even though the oversight models tended to have clear organizational foci, there was substantial crossover between models. For example, a majority of auditor/monitor agencies (61 percent) reported the ability to conduct independent investigations .2 Roughly 43 percent of review-focused agencies and 50 percent of investigative agencies reported having the authority to audit complaint investigations by the police. Taken as a whole, this indicates that jurisdictions are mixing-and-matching forms of oversight authority to meet the needs of their local environment (McDevitt et al. 2005: 11). There is clear variation in the budgets among oversight types. In general, the data provided by the 97 civilian oversight agencies demonstrates that some models of oversight have larger budgets than others. Full investigative agencies tended to report the largest budgets, followed by auditor/monitor and review agencies. Similar patterns were evident in terms of staffing. Investigative agencies reported having the largest number of full-time, paid staff while review-focused agencies were the least likely to report having full-time paid staff. This is likely due in part to variation in the size of jurisdictions and the different 2 Olson and Attard (2016), in a review of oversight models in California, found that auditors were often granted the legal authority to conduct independent investigations, but seldom exercised that authority. 10 Civilian Oversight of Law Enforcement Assessing the Evidence mandates of the models. Investigative and auditor/monitor agencies were much more likely to be found in big cities/counties with large law enforcement agencies . Some models of oversight may be more effective at getting recommendations implemented. Almost all of the oversight agencies reported that police executives listened carefully to the recommendations made by oversight staff (78 percent). However, auditor/ monitor agencies were much more likely to report that police or sheriff’s agencies implemented their recommendations frequently or very frequently (72 percent) as compared to investigative (42 percent) and review agencies (34 percent). Recommendations The following recommendations are designed to highlight additional critical issues for implementing oversight and identify ways to expand what is known about civilian oversight of law enforcement. Recommendations for Jurisdictions Considering Whether to Implement or Revise Civilian Oversight Jurisdictions should focus on the “Best-Fit” rather than “Best Practice” when considering how to structure civilian oversight. A key lesson that can be learned from the literature and the organizational data is that there is not necessarily any “best practice” in the creation of a civilian oversight of law enforcement program. Local jurisdictions vary across a range of areas and it is unlikely that any single model will work well for all jurisdictions. Instead, jurisdictions should focus on specifying the key goals for their oversight program and then identify the “best fitting” model of oversight. Oversight should employ the least force necessary to accomplish its goals. It can be argued that “the least intrusive means of oversight” (Bobb 2003) necessary to achieve police accountability is the best means of approaching the oversight function long term. Just as the police are expected to use only that amount of force that is proportionate, necessary and reasonable to accomplish their task, so it can be argued that jurisdictions creating or reforming an oversight function should similarly accomplish the feat of ensuring police accountability (Bobb 2003). In other words, a jurisdiction seeking to create or update an oversight function should choose the least intrusive model of oversight necessary to accomplish the task. If the model chosen does not accomplish that objective, a more aggressive form of oversight would then be required. A number of resources are available to jurisdictions considering implementing oversight or reforming their current oversight framework. Local jurisdictions who are evaluating whether to implement police oversight should consult several relatively recent reports that provide detailed cases studies of existing oversight agencies (see PARC 2005; McDevitt et al. 2005; Noe 2013; Olson 2016; PARC 2016). To help local jurisdictions gain access to examples of oversight policies, legal language and key organizational documents, the National Association for Civilian Oversight of Law Enforcement (NACOLE) has created a companion website to this report that includes up-to-date profiles for model police oversight agencies. This website toolkit can be accessed by visiting: www.nacole.org/agency_profiles Recommendations for Future Research More systematic, comparative research is needed. While some researchers have worked to extend what we know about oversight (see Walker and Archbold 2014; Prenzler and den Heyer 2016; Alpert et al. 2016), there are still tremendous gaps in what we know regarding the effectiveness in civilian oversight. More systematic, empirical research is needed before strong conclusions can be drawn about what works and what does not work in relation to civilian oversight of police. Additional research is needed on the impact that citizen oversight has on public trust in the police and local government. Civilian oversight often emerges as a result of conflict between the police and communities of color and is often implemented with the goal of improving community trust and confidence in the police and local government (Walker 2001: 55). While some scholars have begun to study the relationship between civilian oversight and public opinion (see Tarling and Dowds 1997; Worden and McLean 2010; Prenzler 2016), little is known about the impact that the implementation of civilian oversight Civilian Oversight of Law Enforcement Assessing the Evidence 11 has on public opinion or whether the presence of civilian oversight can improve public confidence. Future comparative evaluations of effectiveness need to be based on primary data collected by outside researchers and not just on public reports produced by oversight agencies. Prenzler and Lewis (2005) and Ferdik et al. (2013) have conducted some of their most important work on oversight performance by reviewing publicly available agency reports and documents. Unfortunately, the lack of standardization in reporting and measurement between agencies makes it difficult to compare across jurisdictions. In addition, public reports created by oversight agencies may tend to understate challenges and overstate performance. Future research should adopt mixed-methods methodological approaches that combine the use of quantitative outcome measurement, qualitative interviews and community members and officers to generate forms of data that allow for comparison across jurisdictions (Ferdik et al. 2013:18). More support for independent research on oversight will be necessary before key questions relating to the effectiveness of oversight can be answered. Federal and state agencies that support applied research in law enforcement should consider developing new and sustained funding streams for research into civilian oversight of police (Buchner 2015). More work on standards for oversight is needed. Professional oversight associations, like NACOLE, should continue to work collaboratively with researchers and other oversight stakeholders to develop standards and guidelines for oversight agency performance and the evaluation of oversight programs . Local jurisdictions should be willing to evaluate the effectiveness of their local oversight agencies proactively and independently. In conducting those evaluations, jurisdictions need to have realistic expectations when it comes to the impact civilian oversight may have on local police accountability. Oversight is—and should be—only one component of a jurisdiction’s police accountability framework. As such, local jurisdictions should seek to evaluate effectiveness based on the factors over which local oversight agencies actually have control . 12 Civilian Oversight of Law Enforcement Assessing the Evidence What Do We Know About Civilian Oversight of Law Enforcement? Assessing the Evidence Introduction Over the last several decades, issues of trust and accountability have moved to the forefront of community-police relations, and a great deal of scholarship has been devoted to enhancing police performance through the strengthening of law enforcement oversight functions. During this same period, officer-involved shootings, in-custody deaths and claims of racially and ethnically biased police practices has led to the proliferation of organizational mechanisms for reviewing and improving officer conduct. One such mechanism has been the development of civilian oversight of law enforcement mechanisms. Variously referred to as citizen oversight, civilian review, external review and citizen review boards (Walker 2001; Alpert et al. 2016), this form of police accountability is often focused on creating a framework that allows non-police actors to provide input into police department operations, with a historical—and often primary—focus on the citizen complaint process. Civilian oversight may be defined as one or more individuals outside the sworn chain of command of a police department whose work focuses on holding that department and its officers and employees accountable.3 In some jurisdictions, members of the public review, audit or monitor complaint investigations that were conducted by police internal affairs investigators. In other jurisdictions, civilians conduct independent investigations of allegations of misconduct lodged against sworn law enforcement officers (Walker 2001; Finn 2001). Civilian oversight can also be accomplished through the creation of oversight mechanisms that are authorized to review and comment on police policies, practices, training and systemic conduct. Some mechanisms involve a combination of systemic analysis and complaint handling or review (Alpert et al. 2016). Civilian oversight mechanisms are usually implemented based on the assumption that members of the community do not have faith in the ability of a police or sheriff’s department to police itself (Walker 2001; Miller 2002; Bobb 2003; Harris 2012). When the public believes that officers are not being held accountable for violating the law or department policy, then a consensus may develop that misconduct allegations can be more effectively handled by a civilian organization external to the police (Bobb 2003; PARC 2005; Alpert et al. 2016). Underlying this view is the belief that having non-police actors play a role within the process for handling complaints can lead to more thorough, complete and impartial investigations and findings (Walker 2001; Miller 2002; Harris 2012). A second common assumption is that involving non-sworn individuals in the oversight of the police has the potential to increase public confidence and trust in the police, or at least trust in local government more generally (Walker 2001; Harris 2012). The implementation of oversight has long been a source of controversy and political conflict. Early on in the development of oversight, police unions, police chiefs, local political actors and law enforcement-related professional associations vehemently objected to the implementation of oversight (Gordon-Reed, 1995; Walker, 2006). Although police unions and some line officers associations still tend to resist oversight (Tal 2003; Wilson and Buckler 2010), other influential stakeholders, such as the International Association of Chiefs of Police, have accepted the idea that civilian oversight can play a crucial role in policing (Finn 2001; International Association of Chiefs of Police 2000; Farrow and Pham 2003). In some cases, police unions have supported what has been perceived to be more professionalized forms of oversight, such as a monitor or auditor model, 3 Civilian oversight, as it is defined and applied throughout this report, is distinctly different from internal forms of police accountability, such as internal affairs, risk management executive committees, and constitutional policing advisors (NACOLE 2015). It is also distinct from community oversight of the police. Community oversight involves grassroots or non-profit organizations that (attempt to) directly monitor, evaluate, or influence police conduct through means such as video recording police activity, submitting public records or Freedom of Information Act requests, attending public meetings during which police actions are discussed, or other tactics. Community oversight would include, but would not be limited to, groups such as Cop Watch, Cop Block, or, in some cases, Black Lives Matter. These groups are neither non-governmental and do not have legal or administrative authority to oversee the police, and do not act on behalf of a local jurisdiction nor do they have legal authority over cases they investigate. Civilian Oversight of Law Enforcement Assessing the Evidence 13 over the creation of citizen review committees (Prenzler and Ronken 2001; Finn 2001). In Los Angeles, the Los Angeles Police Protective League, the union representing the nearly 10,000 sworn LAPD officers, sponsors an annual civilian oversight-specific training workshop facilitated by staff from the Office of the Inspector General, which provides civilian oversight of the LAPD. More importantly, the rapid increase in the number of oversight agencies in the U.S. indicates that police unions have been unsuccessful at opposing the spread of oversight. While there were only a handful of civilian oversight agencies in the early 1990s (Walker 2001), there are now more than 140 civilian oversight agencies at the county and municipal level in the U.S. Almost all large cities and an increasing number of mid-sized cities have implemented some form of civilian oversight (Walker 2001; Harris 2012; Alpert et al. 2016). Moreover, this pattern is not just limited to the U.S.; civilian oversight is also widespread in Canada (Ferdik et al. 2013.), Australia and New Zealand (den Heyer and Beckley, 2016), Israel, the United Kingdom (Seneviratne 2004), the continent of Europe (Smith 2016) and the Caribbean. Oversight has also become increasingly common in Asia (Nalla 2016) and some parts of Africa (Berg and Howell 2016). Police oversight has received significant media and political attention since 2014. The President’s Task Force on 21st Century Policing (2015) listed civilian oversight as one crucial pillar of reform and argued that, “Some form of civilian oversight of law enforcement is important in order to strengthen trust with the community. Every community should define the appropriate form and structure of civilian oversight to meet the needs of that community.” (President’s Task Force on 21st Century Policing 2015:26). In response to media reports and public demonstrations that followed the police-related incidents,4 new forms of police oversight are being implemented in places as diverse as Ferguson, MO, New York City, NY and Sonoma County, CA. Efforts are also underway across a wide range of cities and counties to implement police oversight mechanisms, including Fairfax County, VA, Newark, NJ and Oxnard, CA. Not all recent media and political attention devoted to police oversight has been positive; however, and some jurisdictions have struggled to improve the effectiveness of their oversight systems. The Chicago Police Accountability Taskforce, which was convened following the public release of a dashboard camera video of the police shooting death of Laquan McDonald in November 2015, recently released a report that was highly critical of Chicago’s police oversight and accountability framework, arguing that “Chicago’s police accountability system does not work. The system should identify and investigate misconduct and then impose appropriate punishment. But at every step, there are enormous barriers.” (Police Accountability Task Force 2016: 63). The Chicago Task Force made a significant number of recommendations directed at strengthening Chicago’s police oversight system. In response to the public scandal and final Task Force report, Chicago Mayor Rahm Emmanuel elected to disband Chicago’s full investigative police oversight agency—the Independent Police Review Authority, which was created in 2007—and replace it with a combination of a police inspector general’s office and a community safety oversight board. In a similar set of circumstances, a 2014 task force review of police accountability in Albuquerque, NM resulted in substantial changes to its system of police oversight. This review, which was triggered by a series of controversial officer-involved shootings, resulted in the implementation of a new independent, non-police agency focused on overseeing investigations into allegations of officer misconduct (Ad hoc Task Force on Civilian Police Oversight 2014). Even though police oversight is a common outcome from local controversies relating to officer-involved shootings and other critical incidents, there has been relatively little systematic, comparative research on the effectiveness of civilian oversight. As a result, the goal of this publication is to examine the current state of knowledge regarding civilian oversight of law enforcement in the U .S . This report reviews the research literature on oversight in conjunction with data collected from 97 civilian oversight programs to help municipal executives, policy-makers, community leaders and the general public consider the current issues facing civilian oversight of the police . 4 Incidents include the deaths of Michael Brown (who was shot by a police officer in Ferguson, MO), Eric Garner (who died after the application of a choke hold used by a New York City police officer) and Andy Lopez (a 13-year old who was carrying a replica firearm when he was shot by a Sonoma County Sheriff’s deputy). 14 Civilian Oversight of Law Enforcement Assessing the Evidence With that goal in mind, this publication contains six major sections. • The first section includes a brief discussion of the methodologies used to review the research literature and collect the organizational data from the civilian oversight agencies. This section also contains pointers to several key empirical reports and research studies on police oversight and offers links to key online resources that may be of use to jurisdictions that are evaluating whether to implement civilian oversight or are undergoing the process of reviewing their existing oversight framework. • The second section examines the history of civilian oversight, with a particular focus on the evolution of the organizational forms of civilian oversight in the U .S . • The third section provides a more detailed review of the different models of oversight, including investigation-focused, review-focused and auditing/monitoring-focused models of oversight. • The fourth section turns to the organizational data provided by 97 civilian oversight agencies to explore key issues and obstacles to effectiveness in police oversight. It draws from the policy literature on oversight to identify the core indicators of effectiveness in police accountability. This section then uses the organizational data to explore variation in the presence or absence of those effectiveness indicators across different models of civilian oversight. • The fifth section of the report explores trending issues in relation to oversight, particularly the debate over how to measure performance in police oversight, the potential value of problem-solving methodologies for oversight and the increasing emphasis on the value of alternative dispute resolution techniques for resolving complaints against police officers. • The sixth and final section concludes the report by offering recommendations intended to identify ways that can expand what is known about civilian oversight of police. Ultimately, this report offers a broad discussion of the important issues currently facing the field of civilian oversight of law enforcement. The review and discussion are intended to provide a broad framework that can help policy-makers, police executives and local community members consider the benefits and challenges that the implementation or reform of oversight can have in their jurisdiction . Research Methodology To develop this report, the research team5 used two different methodologies. First, the research team conducted a review of the academic and policy literatures on civilian oversight of the police. Second, to understand the state of the field better, the research team requested organizational data from police oversight executives in the U.S. with the goal of describing variation in the organizational structure, authority and practices of civilian oversight.6 In this section, the team briefly presents the methods used to accomplish these data collection tasks.7 Literature Review To examine the literature on civilian oversight of the police, the authors conducted key word searches on the Internet, using 5 The primary research team was composed of Joseph De Angelis, Richard Rosenthal, and Brian Buchner. NACOLE staff members Cameron McEllhiney and Liana Perez and University of Idaho students Andrea Pikes and Nicole Waldorf provided extremely valuable research assistance. 6 The research team also conducted a similar survey of civilian oversight agencies in Canada. However, the data collection phase of the Canadian survey was not completed in time to be included in this report. 7 The research team for this project also conducted a web scrape of the substantive website content for 144 police oversight agencies in the U.S. For this part of the project, the research team collected all available substantial website material for the oversight bodies, including mission statements, reports, brochures, ordinance/statute language, complaint forms, and any other textual content. The goal of this data collection was to develop documentation to supplement an electronic survey of oversight executives. Civilian Oversight of Law Enforcement Assessing the Evidence 15 Google and Google Scholars search engines. The team conducted key word searches in common academic databases, including LexisNexis Academic, Criminal Justice Abstracts, WorldCat, Academic Search Premier and JSTOR. The authors reviewed reference lists and citations for previous academic papers and policy reports. Finally, the authors also reviewed reference lists provided on the websites for non-profit oversight and law enforcement-related professional associations, such as the National Association for Civilian Oversight of Law Enforcement (NACOLE), Police Assessment Resource Center (PARC), CATO Institute and International Association of Chiefs of Police (IACP).8 A reasonably large body of academic and policy writing on civilian oversight dates back to the 1970s. For jurisdictions that are considering implementing police oversight or reforming the current oversight functions in their jurisdiction, there are a number of key publications that can be consulted. Several reports and books provide general guidance on the issues that can accompany the implementation of civilian oversight of police. The following book-length publications provide detailed, foundational advice across a range of concerns relating to civilian oversight:9 • Citizen Review of Police: Approaches and Implementation (Finn 2001) • Police Accountability: The Role of Citizen Oversight (Walker 2001) • Citizen Oversight of Law Enforcement Agencies (Perino 2006) • The New World of Police Accountability (Walker and Archbold 2014) • Civilian Oversight of Police: Advancing Accountability in Law Enforcement (Prenzler and den Heyer 2016) • Common Sense about Police Review (Perez 1994) A body of more recent work has begun to examine the broad organizational characteristics of different models of civilian oversight in the U.S.: • Citizen Oversight in the United States and Canada: Applying Outcome Measures and Evidence-Based Concepts (Alpert et al. 2016) • Getting It Right: Building Effective Civilian Review Boards to Oversee Police (Ofer 2016) • Overview of Civilian Oversight of Law Enforcement in the United States (Attard and Olson 2013) • Civilian Oversight of Policing: Lessons from the Literature (Miller 2002) • Models of Oversight: A Critique (Prenzler and Ronken 2001) • Building Legitimacy and Public Trust Through Civilian Oversight (NACOLE 2015) A number of jurisdictions have also produced (or contracted for) the development of independent evaluations of local civilian oversight agencies. Several of these reports, listed below, contain useful comparative case studies of police oversight agencies in the U.S.: • Review of National Police Oversight Models for the Eugene Police Commission (PARC 2005) • Enhancing Citizen Participation in the Review of Complaints and Use of Force in the Boston Police Department (McDevitt et al. 2005) 8 The authors also conducted key word searches of newspapers and other print media sources in the LexisNexis Academic databases. However, initial searches generated several thousand articles spanning the period between 1990 and 2016. As a result, a review of print media sources is not included in this publication. 9 The International Association of Chiefs of Police (IACP) also published a guide for the implementation of civilian oversight in 2000 titled: Police Accountability and Citizen Review. Although it is brief, it does offer potentially useful advice about issues that jurisdictions may want to consider when implementing oversight. The guide is also notable because it was designed as a best practices guide for law enforcement chief executives faced with the prospect of civilian oversight being implemented in their jurisdiction. 16 Civilian Oversight of Law Enforcement Assessing the Evidence • Findings on Models of Civilian Police Oversight Authorities (Noe 2013) • Peer Review of the New Orleans Office of Independent Police Monitor (PARC 2016) • Law Enforcement Oversight: Limited Independence, Authority, and Access to Information Impede Effectiveness (Anderson et al. 2015) • Analysis of Police Oversight Models for the City of Pasadena (Olson and Attard 2016)10 One of the core—and currently unanswered—questions in relation to oversight is the extent to which local jurisdictions can measure the effectiveness and impact of police oversight on police reform. Overall, there have been few systematic, social scientific studies focusing on the impact of citizen oversight (Walker 2001; Prenzler and Lewis 2005; Harris 2012). There are likely a number of reasons for this. First, it can be difficult for researchers to gain access to data relating to police complaints and internal investigations . State law often protects these types of personnel records and law enforcement and police oversight agencies are often unwilling (or unable) to grant outside researchers access to these types of records. More importantly, there are considerable difficulties in designing research that can parcel out the distinct influence that oversight has on policing and police reform. The work of oversight agencies is often intertwined with the work of internal police accountability mechanisms (Brereton 2000; Prenzler and Lewis 2005). As such, it can be difficult to establish the direct impact that oversight has had on the quality of internal investigations or improvements to police operations, policies or procedures. That said, there are a limited number of publications focused on effectiveness and performance measurement in civilian oversight of law enforcement. These include: • Holding Police Accountability Theory to Account (Harris 2012) • Effectively Implementing Civilian Oversight Boards to Ensure Police Accountability and Strengthen Police-Community Relations (King 2015) • Arrested Oversight: A Comparative Analysis and Case Study of How Civilian Oversight of the Police Should Function and How it Fails (Clarke 2009) • Performance Evaluation of Police Oversight Agencies (Filstad and Gottschalk 2011) • Performance Indicators for Police Oversight Agencies (Prenzler and Lewis 2005) • Evaluating the Performance of External Oversight Bodies (Brereton 2000) • The Use of Performance Measurement in Civilian Oversight in the United States (Mohr 2007) There are currently no agreed upon national standards for police oversight. Even so, there have been several efforts, documented in the citations below, to memorialize national guidelines for certain types of police oversight, including police auditors and monitors: • Core Principles for an Effective Police Auditor’s Office (Walker 2003) • National Guidelines for Police Monitors (Bobb et al. 2008) Beyond these academic and policy publications on police oversight, a number of websites operated by non-profit organizations contain useful resources, links to police oversight agencies and other types of toolkits that can serve as a starting point for jurisdictions considering implementing police oversight or revising their current oversight framework: • NACOLE, www .nacole .org • Police Assessment Resource Center (PARC), www .parc .info • Center for Policing Equity, www .policingequity .org 10 Olson (2016) also recently conducted a review of key issues confronting the Spokane County Sheriff’s Citizens Advisory/Review Board, which performs some oversight functions for the Spokane County Washington Sheriff’s Department. This report may be of particular interest to jurisdictions that are thinking about transitioning a purely internal advisory board into a broader form of independent, external citizen oversight. Civilian Oversight of Law Enforcement Assessing the Evidence 17 Electronic Organizational Data Collection Since much of the recent research on civilian oversight has been based on either case studies or anecdotal accounts of individual agency practices, there are substantial gaps in what is known about the state of civilian oversight in the 21st century. To develop better understanding of the general state of civilian oversight in the U.S., the research team requested organizational data from 97 civilian oversight agencies in the U.S. The overall goal was to capture and describe variation in the organizational structure, legal authority, organizational practices and financial resources for contemporary oversight. Several of the attitudinal questions were adapted from early survey research on oversight (Walker and Bumphus 1991). The research team designed the data request to measure organizational and attitudinal information in 12 key areas: 1 . The history of the oversight agency; 2. The legal authority/organizational powers granted to the agency; 3 . Agency goals; 4. Organizational structure; 5 . Budget; 6 . Staffing; 7 . Presence and structure of a board or community members; 8 . Role in the complaint intake and investigation process; 9 . Community outreach practices; 10 . Perceived support from process stakeholders; 11 . Data collection and analysis practices; and 12. Key successes or obstacles that the agency has experienced in the previous three years. See Appendix A for further details about the methodology. Brief History of Civilian Oversight Early forerunners of modern police oversight initially took the form of police commissions, which were implemented by Progressive Era reformers in the late 19th and early 20th centuries as a strategy to break the hold of political machines on local policing (PARC 2005; Walker 1977; Walker 2001; Walker 2006). To reduce the influence of partisan politics and nepotism on police departments, these commissions were populated by part-time, non-police volunteers and often granted the power to hire and fire the local police chief. The goal was to support the development of a professionalized, independent police force by having the commission function as a buffer between local political actors and the police chief (Walker 2001; Walker 2006). Unfortunately, early commissions never functioned as effectively as initially envisioned, partly due to the fact that commission members were political appointees and had little expertise in policing (Walker 2001; Bobb 2003). As a result, they tended to become highly deferential to police executives and failed to provide meaningful oversight of local police departments (Bobb 2003: 4; Attard and Olson 2013).11 Later efforts to establish police oversight arose out of the Civil Rights 11 A number of police commissions remain in existence today, such as Detroit’s Board of Police Commissioners, the Kansas City Board of Police Commissioners, the Honolulu Police Commission and the Los Angeles Board of Police Commissioners. These modern examples of police commissions perform some of the same functions as Progressive Era commissions (e.g., commissioners are typically volunteers and they hire/fire the police chief). However, they have also evolved considerably from early commissions, operating with greater independence, large budgets, full-time professional staff and playing a significant role in overseeing police policy, complaint investigation and disciplinary processes. 18 Civilian Oversight of Law Enforcement Assessing the Evidence Movement, in direct response to the treatment of the black community by the police (Walker 2001). The intersection of race and policing is central to the history of civilian oversight of law enforcement in the United States; issues of race, equality, justice and trust in law enforcement have had a prominent role in police reform over the last 50 years . Early Efforts at Establishing Civilian Review, 1920s-1960s Support for the implementation of civilian oversight can be seen as early as the 1920s, with most of the efforts focused on creating bodies that could better respond to complaints against the police. In 1928, the Los Angeles Bar Association created a non-governmental committee to record complaints against police officers (Walker 2006: 3; Ferdik et al. 2013). More substantive proposals to create independent, governmental mechanisms for handling complaints against police officers can be seen in a series of national commission reports in the 20th century.12 In 1931, the Wickersham Commission Report argued that “some disinterested agency” should be created in “every locality” to take community complaints against the police (Wickersham Commission 1931(1968): 192; see also Walker 2006). In 1968, the Kerner Commission (created as the result of race-related rioting in United States cities between 1964 and 1968) recommended the creation of external oversight, specifically, “a specialized agency with adequate funds and staff, [to] be created separate from other municipal agencies” (United States Kerner Commission 1968: 312). The Kerner Commission also recognized that complaints could have value for organizational policy analysis and that the wider public would be less likely to trust the result of complaint investigations if those investigations were handled internally by a police department (p. 312; see also Walker 2006). Initial efforts to implement modern forms of civilian oversight began in several large cities in the middle of the 20th century, with a focus on creating volunteer review boards that would play a role in receiving complaints and reviewing completed internal police investigations into allegations of misconduct filed against officers (Hudson 1971; Terrill Key Research on Oversight History 1988; Walker 2001). For example, a Complaint Review • The History of Citizen Oversight (Walker 2006) Board (CRB) was created in Washington, D.C. in 1948 as a result of community concern about police • Civilian Oversight of the Police in the United States (Bobb 2003) racism and use of excessive force against Afri• Civilian Oversight of Police in Philadelphia: The First 50 Years (Tal can-Americans (Beattie and Weitzer 2000). The CRB 2003) existed for almost fifty years, but was poorly funded by the city and unable to keep up with an increasing • Race, Democracy and Law: Civilian Review of Police in Washington, caseload (Beattie and Weitzer 2000). The CRB was DC (Beattie and Weitzer 2000) eventually replaced by the Office of Police Complaints • Police Accountability in Philadelphia: Restrospects and Prospects (Walker, 2001). In Philadelphia, a Police Advisory (Terrill 1988) Board was created in 1958, but was shut down in 1969 due to a lack of funding (Terrill, 1988; Walker 2001). The New York City Civilian CRB was initially established in 1953 as an internal review body staffed by three Deputy Commissioners (Walker 2006). It was subsequently made into an external review body, but was disbanded shortly thereafter following a public referendum orchestrated by the Policeman’s Benevolent Association (Hudson 1971). Examining the early U.S. oversight agencies, the research team found several key characteristics common to each of these efforts. First, the implementation of each of these oversight mechanisms emerged from local crises and civil unrest relating to conflict between the police and local communities of color. Second, these oversight agencies were designed around a civilian review model—that is, they were largely composed of volunteer members with relatively little expertise in police issues, 12 There were also a number of proposals offered by non-governmental actors that outlined both the rationale and possible organizational features of possible independent, non-police complaint intake and investigation procedures. See, for example, Beral and Sisk’s (1964) law review note comparing the internal police investigation procedures to potential models of citizen review of complaints. Civilian Oversight of Law Enforcement Assessing the Evidence 19 had small or non-existent budgets and little staff support (Jones 1994; Walker 2001). Their core function was to receive complaints and review completed internal affairs investigations (Walker 2006) but their extremely limited resources compromised their ability to keep up with increasing complaint workloads (Ferdik et al. 2013). Third, these agencies all encountered significant resistance from police unions, local politicians and policy makers, which ultimately resulted in their abolition (Bayley 1991; Walker 2001; Walker 2006). Emergence of Investigative Models of Civilian Oversight, 1970s-1980s Although all of the oversight agencies implemented during the initial wave failed, another wave of development began in the late 1960s and carried through to the 1980s (Walker 2001; Walker 2006; Alpert et al. 2016). As with the first wave, these oversight agencies were primarily concentrated in large cities and their implementation was not the product of slow, deliberate policy-making but instead driven by local crises as the result of civil unrest and police-community conflict (Walker 2001). However, while the initial wave of oversight agencies were structured around limited review board models, oversight agencies implemented in the 1970s and 1980s tended to have enhanced resources, greater durability and expanded organizational authority (Walker 2006). Moreover, while only a small number of oversight agencies were created in the 1960s through the 1980s, most of these agencies have remained operational in the long term. A city ordinance in 1969 created the Kansas City, MO Office of Community Complaints (OCC) and it has remained in continuous operation through the present day. Unlike earlier board models, the OCC had expanded professional staff support that allowed it to respond more effectively to increasing workload demands. The OCC’s legal authority was also enshrined in ordinance, which served as an obstacle to easy repeal (Walker 2001; Ferdik et al. 2013). Shortly after the OCC was created, a number of other oversight agencies were also implemented and granted the power to conduct investigations that were entirely independent of the police. A city ordinance in Berkeley, CA in 1973 created the Police Review Commission (PRC) and granted it the ability to independently investigate complaints filed by members of the public against police officers (Walker 2001). Nearly ten years later in 1982, an amendment to the City Charter created the Office of Citizen Complaints in San Francisco, CA. The Office of Citizen Complaints completely replaced the police internal affairs function in relation to citizen complaints and was granted the authority to both receive and investigate all citizen complaints (the police department continued to investigate internally-generated complaints against officers) (Walker 2001; Ferdik, Alpert, and Rojek 2013). Finally, an ordinance in 1986 reconstituted the Civilian Complaint Review Board (CCRB) in New York City. The CCRB initially employed civilian investigators who worked alongside police employees and were supervised by police managers; it became an all-civilian investigative oversight agency in 1993 (Walker 2006). Emergence of Auditor/monitor and Hybrid Models of Civilian Oversight, 1990s-Present Another wave of development began in the 1990s and continues to present day (Walker 2001; Walker 2006; Alpert et al. 2016). During this period, there was a rapid expansion of the number of police oversight agencies in the U.S. subsequent to the March 1991 videotaped beating of Rodney King by the LAPD and the resulting report published four months thereafter by the Independent Commission on the LAPD (“Christopher Commission”). 20 Civilian Oversight of Law Enforcement Assessing the Evidence Key Events in the Development of Civilian Oversight • 1931: National Commission on Law Observance and Enforcement (Wickersham Commission) Report on “Lawlessness in Law Enforcement” • 1948: Creation of Washington D.C. Complaint Review Board. • 1968: Kerner Commission Report recommending External Oversight for Police . • 1969: Creation of the Kansas City, MO Office of Community Complaints (OCC); the longest continuously operating oversight agency in the U .S . • 1973: Creation of the first Independent Investigations Office in Berkeley, CA. • 1991: Rodney King beating and Christopher Commission Report, LAPD • 1993: Implementation of the first police auditor program in the U.S.—San Jose Independent Police Auditor. • 1993: Creation of the first monitor program in the U.S. – Special Counsel for the Los Angeles County Sheriff Department. • 1993: The New York City CCRB becomes completely civilianized . • 1994: Enabling legislation for federal “Pattern and Practice” civil lawsuits by the Department of Justice, Civil Rights Division (42 U.S.C. § 14141) While no one has measured the exact increase in the number of oversight agencies since the 1990s, it is clear the number increased from a handful in the early 1990s to over 100 in 2001 (Walker 2001).13 One of the core characteristics of this wave was the increasing diversity in the organizational structure and legal powers of the different organizational forms of police oversight . Moreover, the 1990s saw the emergence of a new model of oversight—the auditor/monitor model (Bobb 2003; Walker 2006). Unlike earlier models of oversight that tended to focus on either reviewing or investigating individual complaints, these auditor/monitor agencies had the mandate to examine systemic patterns in complaints or other types of officer conduct (including uses of force, pedestrian and traffic stops, arrests and detentions), and to offer recommendations for improving police policies, practices and training. While the implementation of review-focused and investigatory oversight agencies remained common in the 1990s,14 a number of oversight entities began to focus increasingly on systemic causes of misconduct as opposed to focusing solely on outcomes of individual complaint investigations (Lewis 2000; Livingston 2004; Harris 2012). The first auditor-focused oversight agency was implemented in 1993 in San Jose, CA and was • 2001: Over 100 oversight agencies identified in the U.S. followed a short time later by the Seattle Police Auditor (Walker 2006; Ferdik et al. 2013). Simi2016:144 oversight organizations identified in the U.S. larly, as the result of the response of the City of Los Angeles to the videotaped beating of Rodney King, a voter initiative in 1995 created the Office of the Inspector General (OIG) for the City of Los Angeles and gave it authority to investigate and audit complaint (and other) investigations conducted by the LAPD to ensure that they complied with accepted standards (Office of the Inspector General n.d.). Shortly thereafter, and in the wake of the Kolts Report and recommendations, the County of Los Angeles adopted the Special Counsel model with jurisdiction over the Los Angeles County Sheriff’s Department. The Special Counsel was an independent entity that had the ability to audit and investigate any area of the Sheriff’s Department’s operations, policies and procedures, as well as to monitor the implementation of reforms outlined in the Kolts Report . 13 This wave of development was accompanied by the growing recognition that citizen oversight constituted a professional field of work and saw the development of professional associations that brought together different oversight agencies from both inside and outside of the United States (Walker 2006). The International Association of Civilian Oversight of Law Enforcement (IACOLE) was created in 1985, NACOLE was created in 1995 and the Canadian Association for Citizen Oversight of Law Enforcement (CACOLE) established in 1997. Websites for the last two professional associations can be visited at: www.nacole.org and www.cacole.ca. 14 While police auditors emerged as an important form of oversight during this period, it is important to note that there continued to be a rapid increase in the number of citizen review boards, which comprised the most common form of oversight implemented during the late 1990s in the United States. Civilian Oversight of Law Enforcement Assessing the Evidence 21 By the late 1990s and early 2000s, the numerical rise in the number of oversight agencies continued. This time period also saw two critical developments in independent police oversight. First, the U.S. began to see the development of a second generation of hybridized forms of civilian oversight, which often emerged as replacements for earlier civilian review boards. For example, Portland, OR implemented the Independent Police Review Division (IPR) in 2001 as a replacement for an earlier review board. The IPR was consciously designed to draw its organizational structure from different models of oversight (Office of the City Auditor 2001). It was empowered to serve as an initial intake point for community complaints against police officers; monitor on-going serious investigations; audit complaint investigations conducted by police internal affairs staff; and develop data-driven recommendations for improving police policies, practices and training (Independent Police Review Division 2002). A later revision to the IPR’s ordinance also granted the agency the ability to conduct independent investigations and ultimately monitor police critical incident investigations (Independent Police Review Division 2012: 1). The IPR was also created with a citizen oversight board, the Citizen Review Committee, which had the authority to review and audit closed complaints, make policy recommendations, hold public hearings and serve as an appellate body for appeals of non-sustained findings made by the Portland Police Bureau (Independent Police Review Division 2003: ix). Similarly, other large cities, including Denver (2005) and New Orleans (2009) implemented hybridized auditor/monitor oversight agencies. Contemporary Models of Civilian Oversight Over the years, there have been multiple attempts to create schema to classify approaches to civilian oversight of law enforcement. One of the challenges in accomplishing this feat is that it appears that almost no two civilian oversight agencies in the U .S . are identical. Each jurisdiction has its own political, social and cultural tensions that influenced the development of each oversight entity’s legal authority, structure and organizational practices (NACOLE 2015). Oversight agencies may receive, review, audit and monitor police investigations into officer misconduct. Oversight agencies may conduct investigations that are wholly independent of the police. They may analyze and issue public reports on misconduct investigations, lawsuits, uses of force or detentions and arrests. Oversight investigators or auditors may provide on-scene monitoring of critical incidents, such as officer-involved shootings or protests; and they may evaluate whether an officer’s actions were appropriate under the circumstances or showed a need for individual accountability or systemic reform. In addition, oversight entities may also assess a police department’s policies, supervision and management practices and training, hiring and recruitment standards, among other procedures. Nonetheless, it is possible to review the previous literature and make several initial observations about contemporary forms of civilian oversight. These include: • High Variability in Organizational Structure. Currently, there is a tremendous amount of variation in the structure of different oversight agencies (Walker and Kreisel 1996; Walker 2001; Bobb 2003; Alpert et al. 2016). Some agencies operate almost completely by a small number of community volunteers while others have a large number of paid professional staff. Some oversight agencies have no operating budget while other agencies have multimillion-dollar budgets. • Wide Differences in Organizational Authority. Substantial variation exists in the jurisdictional authority granted to police oversight agencies. Elements such as the role that oversight agencies play in relation to the intake of complaints, the relationship they have to the complaint investigation process and their level of access to police records vary . A long list of other characteristics differ greatly including whether they can make recommendations as to findings and discipline, or make recommendations relating to police policies, practices and training (Walker and Kreisel 1996; Walker 2001; Bobb 2003; Alpert et al. 2016). • Organizational “Hybrids” are Common. While early forms of oversight tended to operate as “citizen review boards,” and focused on reviewing and commenting on completed internal affairs investigations, many contemporary oversight agencies combine different organizational forms and types of organizational authority in relatively complex ways (Walker 2001; Finn 2001; Attard and Olson 2013; Alpert et al. 2016). 22 Civilian Oversight of Law Enforcement Assessing the Evidence Even given the variability in the structure of oversight, there have been multiple efforts to classify police oversight agencies based on key characteristics. Walker (2001) developed one of the earliest and most sophisticated classification systems for oversight.15 Describing the different models as Class I, Class II, Class III and Class IV systems, Walker argues that models of oversight should be considered along a continuum that range from forms of oversight that are the most independent from police departments to oversight systems that are the least independent. He defines Class I systems as agencies that are independent of police departments and conduct fully independent investigations into allegations of officer misconduct. Class II systems review and comment on internal investigations conducted by the police. Class III systems function as appellate bodies, with complainants filing appeals with the oversight agency when they are dissatisfied with the outcomes on complaints investigated by local law enforcement. Class IV systems are agencies that have the ability to audit, monitor or review the police/sheriff department’s complaint handling system. In addition to these classes, Walker also recognizes “hybrid” oversight agencies that do not fit easily within any of the four categories (Walker 2001: 62). Since Walker developed this classification scheme in the late 1990s, a number of others attempts to update it have occurred. Ferdik, Alpert and Rojek (2013) adapted Walker’s (2001) classification schema to explore organizational variation in U.S. and Canadian oversight agencies. Using publicly available reports and documents from several large civilian oversight agencies, Ferdik et al. (2013) concluded that there was substantial organizational variation across different oversight agencies and new “holistic” forms of oversight increasingly blur the boundaries between traditional models of oversight. They argue that these forms of oversight can perform traditional oversight roles (e.g., investigation, auditing, and review), but can also conduct proactive pattern analysis to offer recommendations for improving policy, practices, and training. Ferdik at al. (2013) contended that this new proactive policy recommendation ability can be found across three of the four “classes” that Walker developed . In 2005, PARC conducted a research project for the city of Eugene, OR. In it, they outlined a three-part classification scheme (PARC 2005). 1 . Review and Appellate models, which are designed to review completed police internal investigations or hear appeals from the public on investigation findings; 2. Investigative and Quality Assurance models, which replace the police internal affairs process in whole or in part; and 3 . Evaluative and Performance-based models, which adopt a holistic approach to evaluating patterns in police risk management, performance, operations or other organizational systems to promote systemic reform. In another recent review of models of oversight, Attard and Olson (2013) revised Walker’s oversight schema and grouped oversight agencies based on their role in the complaint handling process as well as their organizational structure. Accordingly, they grouped oversight agencies into three categories: 1 . Investigative agencies, which conduct independent investigations of complaints filed against police officers; 2. Auditing/monitoring agencies, which systematically review and examine police internal investigations and other law enforcement activity to make recommendations around policy and training; and, 3 . Review boards and commissions, which includes a diverse range of agencies headed by volunteer community members who may hold community forums, hear appeals or issue findings on investigations completed by paid staff (Attard and Olson 2013: 3-5). 15 2005). A number of other academics and practitioners have developed classification schema that group police oversight agencies in slightly different ways (Prenzler and Ronken 2001; PARC Civilian Oversight of Law Enforcement Assessing the Evidence 23 This report adopts an oversight classification scheme that is a slightly revised version of Walker’s (2001). It groups oversight agencies into three categories based on the core agency functions: 1 . Investigation-focused, 2. Review-focused,16 and 3 . Auditor/monitor-focused. Grouping agencies into these categories required a two-step process. First, an initial quantitative analysis was conducted using the organizational data provided by the oversight agencies. More specifically, agencies were initially grouped based on data provided, which focused on how frequently the oversight agencies conducted independent investigations, audited internal affairs investigations, monitored open internal affairs investigations and reviewed completed internal affairs investigations. During the second stage of the analysis, the agencies were grouped into different organizational categories after the organizational data provided by the oversight agencies was evaluated against their agency’s public mission statement and foundational legal authority (when available).17 This two-stage process allowed each agency to be grouped into one of three categories: auditor/monitor-focused, investigation-focused and review-focused. Table 1: Classification Schema for Agencies Providing Data for this Report Auditor/monitor-focused Investigation-focused Frequency 24 34 Percent 24.7% 35.1% Review-focused 39 40.2% Total 97 100% In the remainder of this section, the key characteristics of the three different models of oversight are examined. Investigation-Focused Models One relatively common model of oversight is the investigation-focused agency that operates separately from the local police or sheriff’s department (Walker 2001: 62). While the structure, resources and authority of these types of agencies can vary among jurisdictions, these agencies are tied together by their ability to conduct independent investigations of allegations of misconduct against police officers (Prenzler and Ronken 2001; Walker 2001). These oversight agencies may either completely replace the police internal affairs function or they may conduct investigations that parallel or duplicate the work of internal affairs (Finn 2001; PARC 2005). As one example, San Francisco’s Office of Citizen Complaints (OCC) is an entirely civilian, governmental agency solely responsible for investigating complaints filed by community members against sworn members of the San Francisco Police Department (OCC 2016). The organizational structure of investigative agencies can vary significantly. In some cases, an investigative agency may be governed by a volunteer board (and supported by a professional staff of investigators). In small jurisdictions, an investigative agency may be staffed by a single investigator or consultant (Finn 2001; PARC 2005). The available literature on full investigative oversight agencies identifies a common set of organizational functions (Walker 16 Unlike Walker’s oversight classification scheme, this publication did not include appellate agencies (Class III systems) as a separate category. Patterns in the organizational data indicated appellate functions can be found across a range of agencies, including investigative and auditor/monitor systems that operate with a combination of professional staff and oversight boards composed of community volunteers. 17 Auditor/monitor agencies were collapsed into one category due to significant overlap between agencies that reported performing both auditing and monitoring functions either “all the time” or “very frequently.” 24 Civilian Oversight of Law Enforcement Assessing the Evidence and Kreisel 1996; Prenzler and Ronken 2001; Walker 2001; Finn 2001; Bobb 2003; Attard and Olson 2013; Ferdik et al. 2013; Walker and Archbold 2014). Investigative agencies may: • Serve as the intake point for public complaints against police officers (Bobb 2003). • Review and classify the nature of the complainants’ allegations (King 2015). • Conduct independent interviews of complainants, officers and witnesses (Attard and Olson 2013). • Be staffed by non-police “civilian” investigators, although some agencies may employ retired or former police officers (Finn 2001).18 • Be led by a community board or commission that may hold hearings, issue subpoenas or make findings on investigations conducted by professional non-police investigative staff (Attard and Olson 2013). Potential Strengths of Investigation-focused Agencies An investigation-focused agency with appropriately trained staff can complete thorough and impartial investigations (Prenzler and Ronken 2001; PARC 2005). Investigation-focused agencies are the most independent forms of oversight (Walker 2001) and tend to have more resources and larger staffs than other types of oversight. Their investigators are also likely to have had highly specialized training and experience in relation to investigations, particularly as the organization matures. Thus, where investigation-focused agencies are sufficiently resourced, have well-trained, competent staff and are granted sufficient access to department personnel and records, they may be able to improve the quality of internal investigations. Even though this is a commonly identified strength of the investigation-focused oversight agency, more rigorous comparative research is needed on this issue . A related potential strength of the investigation-focused model is its ability to increase public faith in the integrity of the investigations process, especially in the aftermath of significant public scandals involving the police. Available public opinion research demonstrates strong public support for the independent investigation of serious complaints against police officers (Prenzler 2016). Most investigation-focused agencies utilize civilian staff to conduct fact-finding investigations and operate a multi-member community board that may hold hearings, issue findings and/or make recommendations to the police department. As a result, this model may reassure a community that investigations are unbiased, thorough and that civilian perspectives are represented both within the complaint investigation process and upon review of completed investigations (PARC 2005). Finally, relative to other forms of oversight that rely on community volunteers, investigative agencies are able to hire full-time investigators who often have highly specialized training, greater resources and more time to perform their oversight work (Finn 2001). Potential Model Limitations Another potential limitation of the investigative model is the significant costs and resources necessary to conduct competent, timely investigations, including large staffing requirements, as well as the complex organizational issues that can accompany the implementation of a stand-alone investigative oversight agency. Full investigative agencies are more expensive than other models of oversight, largely due to the increased personnel costs that accompany the hiring of professional investigators (Finn 2001: vii). While the cost of an investigation-focused oversight agency is, by necessity, higher than the other models of oversight, the higher cost could be mitigated by the savings realized from a reduction or the elimination of personnel needed to conduct police internal investigations. For example, evaluation research conducted in the 1990s on the now-defunct Minneapolis 18 Some Canadian independent investigation agencies employ “seconded” officers who are currently serving police officers assigned as full-time investigators serving at the pleasure of the oversight agency director. (e.g., the Alberta Serious Incident Response Team (ASIRT) and the Nova Scotia Serious Incident Response Team (SiRT)). Civilian Oversight of Law Enforcement Assessing the Evidence 25 Civilian Police Review Authority (CRA) found that CRA investigations were far less costly to conduct on a per case basis than investigations conducted by the police internal affairs unit ($3,649 per CRA investigation versus $6,278 per police internal affairs investigation). This was due, in part, to the lower costs associated with using volunteers and civilian investigators (Minneapolis Civilian Review Authority 1997: 18-17; Summary of Investigation-focused Agencies Walker 2001: 137). Another potential weakness is that attempts to implement investigation-focused agencies tend to generate significant resistance from police unions and their allies (King 2015). Unions have routinely argued that civilian investigators do not have the technical background or professional experience to conduct complex investigations into allegations of officer misconduct (Prenzler and Ronken 2001; Walker 2001). Police unions have also opposed the implementation of full investigatory oversight agencies, arguing that they will be biased against police officers (King 2015). In addition, it can be argued that the use of former police officers or even civilian investigators who have not previously served as police officers may not eliminate pro-police bias in complaint investigations. Civilian investigators may harbor either pro or anti-police bias, depending on their own personal background and experiences. Another argument is that civilian investigators may also be less willing to challenge an officer’s account of events then a peer investigator or a police supervisor. Similarly, while some former police officer investigators may have the ability to investigate without bias, others may approach their jobs with the intent or desire to justify or defend the conduct of other officers, regardless of whether those officers are investigating an agency where they were previously employed . Key Characteristics 1 . Routinely conducts independent investigations of complaints against police officers. 2. May replace or duplicate the police internal affairs process . 3 . Staffed by non-police, “civilian” investigators. Potential Key Strengths 1 . May reduce bias in investigations into citizen complaints. 2. Full-time civilian investigators may have highly specialized training . 3 . Civilian led investigations may increase community trust in the investigations process . Potential Key Weaknesses 1 . Most expensive and organizationally complex form of civilian oversight . 2. Civilian investigators may face strong resistance from police personnel . 3 . Disillusionment among the public may develop overtime when community expectations for change are not met . Independent investigation-focused agencies in large cities have also been plagued with budgetary and personnel limitations that have resulted in untimely investigations. The New York City CCRB has often been criticized for lack of timely investigations as well as efforts taken by that agency to reduce its workload through re-allocation of resources (Clarke 2009). Some researchers have argued that while the community may have great confidence in full investigative models initially, community confidence can wane over time if these models are perceived as not leading to the reforms promised during implementation (McDevitt et al. 2005: 5). For example, the public may expect that more citizen complaints will be sustained and stronger punishments imposed after full investigative oversight models are implemented. However, there is currently no systematic evidence to support this expectation and it is currently unclear what impact full investigative models have on patterns in findings and discipline for police officers alleged to have engaged in misconduct. As a result, disillusionment among the public may develop over time when community expectations for change are not met. This is, in part, one of the reasons for the recent recommended dissolution of Chicago’s Independent Police Review Authority (Police Accountability Task Force 2016). 26 Civilian Oversight of Law Enforcement Assessing the Evidence One final criticism sometimes offered in relation to investigation-focused agencies is that they have the potential to undermine the responsibility of police chiefs and sheriffs to maintain discipline (McDonald 1981; Prenzler and Ronken 2001.) In police agencies where internal affairs units are reduced or eliminated, the opportunity for officers to obtain experience in conducting personnel investigations and recognizing the extent to which misconduct can negatively impact the agency becomes limited or nonexistent. A robust internal affairs process can result in investigators and supervisors who are more committed to ensuring a police culture of integrity and accountability within their own organization. Table 2 presents examples of investigation-focused models in the U.S. Table 2: Examples of Investigation-focused Models in the U.S. Agency Jurisdiction Website Office of Citizen Complaints San Francisco, CA www .sfgov .org/occ Office of Police Complaints Washington, D.C. www .policecomplaints .dc .gov Citizen Complaint Review Board New York, NY www.nyc.gov/html/ccrb Citizens’ Law Enforcement Review Board San Diego County, CA www.sandiegocounty.gov/clerb.html Citizen Police Review Board Pittsburgh, PA www.cprbpgh.org Office of Municipal Investigations Pittsburgh, PA www.pittsburghpa.gov/omi Review-focused Models A second category of oversight focuses its work on reviewing the quality of internal investigations, primarily those conducted by police internal affairs units. Many review agencies take the form of volunteer review boards or commissions and are designed around the goal of providing community input into the internal investigations process (PARC 2005). Instead of conducting independent investigations, review agencies may evaluate completed internal affairs investigations, hear appeals, hold public forums, make recommendations for further investigation or conduct community outreach (Attard and Olson 2013). As with “Boards and commissions can greatly enhance community full investigative agencies, review-focused agencies vary in their confidence in a law enforcement agency. When members of the organizational structure and can perform a range of functions public are empowered to hear concerns and weigh in on (Walker and Kreisel 1996; Prenzler and Ronken 2001; Walker 2001; complaints...public trust is increased.” (Attard and Olson Finn 2001; Bobb 2003; Attard and Olson 2013; Ferdik et al. 2013; 2013: 5) Walker and Archbold 2014). The available literature on oversight indicates that review agencies can: 1 . Conduct complaint intake. Some review boards may serve as an alternate intake point for community members who would like to file a complaint against an officer, but are uncomfortable visiting a police facility (King 2015). After receiving the complaint, review agencies will generally send the complaints to the police or sheriff internal affairs unit for investigation. 2. Review completed police investigations of externally generated complaints. Review-focused agencies typically only evaluate completed police internal affairs investigations. They tend to have neither independent investigatory authority nor do they typically have the power to subpoena witnesses or records (Walker 2001; Finn 2001; Attard and Olson 2013). Moreover, they tend to have authority to only review complaints filed by community members and are less likely to review internal police complaints (e.g., complaints filed by a supervisor) or critical incident investigations (e.g., officer-involved shootings, in-custody deaths) in the absence of a complaint (Bobb 2003). Civilian Oversight of Law Enforcement Assessing the Evidence 27 3 . Make recommendations to the police executive. After evaluating completed investigations, review agencies may have the ability to make recommendations regarding findings on cases or request that further work be done on cases to remedy deficiencies in initial internal affairs investigations (PARC 2005: 11). 4. Hear appeals. Review-focused agencies may also hear appeals from members of the public who were dissatisfied with the outcome of their complaint that was filed with, and/or investigated by, the police agency (Finn 2001; PARC 2005). In most cases, findings made by a review-focused agency come in the form of recommendations to the police executive. In some cases, the agency’s findings may have the force of law.19 5 . Gather and review public input. Another common function of review-focused agencies is to hold public forums and gather input on the quality of complaint investigations and other law enforcement-related issues. After gathering and reviewing public input, the review agencies may communicate those concerns to police executives and other policy-makers. Potential Model Strengths Some researchers have argued that the public may perceive review boards and commissions as more representative of the community than programs that are staffed by full-time professionals (Finn 2001; Attard and Olson 2013). As such, community members may be more likely to perceive the oversight entity as supporting and protecting community interests (Walker 2001). Beyond public perception, review-focused agencies have the benefit of allowing community representatives to bring an outsider’s perspective to the complaint investigations process . This may help jurisdictions identify and correct deficiencies within individual complaint investigations (PARC 2005). Where review boards have diverse community representation, there may be a stronger motivation on the part of police investigators to ensure that not only is there no bias in the conduct of their investigations, but that any appearance of bias is also removed . With respect to the review of policy and officer conduct, review boards have the ability to identify deficiencies in policy or training as they apply to individual cases being reviewed. A diverse board will have the ability to provide different perspectives on police policy and training and make recommendations for change that could result in improved community-police relations. Summary of Review-focused Agencies Key Characteristics 1 . Often focuses on reviewing the quality of completed police internal affairs investigations. 2. May make recommendations to police executives regarding findings or request that further investigation be conducted. 3 . Commonly headed by a review board composed of citizen volunteers . 4. Often hold public meetings to collect community input and facilitate police-community communication. Potential Key Strengths 1 . Ensures that the community has the ability to provide input into the complaint investigation process . 2. Generally the least expensive form of oversight since it typically relies on the work of volunteers. 3 . Community review of complaint investigations may increase public trust in the process. Potential Key Weaknesses 1 . May have limited authority and few organizational resources . 2. Review board volunteers may have significantly less expertise in police issues and limited time to perform their work. 3 . May be less independent than other forms of oversight. 19 In those cases where an agency can compel a police executive to change a finding, an officer would, as a matter of law, have the ability to appeal a sustained finding of misconduct through a civil service process or through the courts. 28 Civilian Oversight of Law Enforcement Assessing the Evidence Finally, review focused agencies tend to be the least expensive form of oversight. They are often operated by volunteers and may have no stand-alone budget (PARC 2005). As a result, this type of oversight is popular in smaller jurisdictions that have limited resources . Potential Model Limitations Like all models of oversight, review-focused agencies may have certain limitations. First, review-focused oversight agencies tend to have limited authority and, like full investigative agencies, typically focus on individual case investigations. As a result of that reactive focus, their ability to promote large-scale systemic organizational change may be limited (Walker 2001; PARC 2005). Moreover, review agencies may not have the authority to systemically evaluate police policies or procedures, make policy recommendations or examine aggregate patterns in officer conduct (PARC 2005: 11). Depending on the structure of the review agency, they may be less independent from the police than other oversight models. These types of oversight agencies may be more likely to report to the police chief, have a small or no stand-alone budget, have limited or no staff support and board members tend to be political appointees (Walker 2001; PARC 2005; Olson 2016). Moreover, they may have to rely on the police or sheriff’s department for meeting space, administrative support and training. Since they do not always have the power to conduct independent investigations, review agencies are also more likely to rely on the police or sheriff’s department for their information (McDevitt et al. 2005; Olson 2016). Table 3 presents examples of review-focused models in the U.S. Table 3. Examples of Review-focused Models in the U.S. Agency Jurisdiction Website Citizens’ Police Review Board Albany, NY www.albanylaw.edu/cprb Citizens’ Police Complaint Board Indianapolis, IN www .indy .gov/egov/city/dps/cpco Civilian Police Review Board Urbana, IL www.urbanaillinois.us/boards/civilian-police-review-board Citizen Review Committee St. Petersburg, FL www.stpete.org/boards_and_committees/civilian_police_ review_committee Citizens’ Review Board on Police Practices San Diego, CA www.sandiego.gov/citizensreviewboard Since review board members are generally volunteers drawn from a range of professional backgrounds, they may have less expertise than paid professional oversight staff and have limited time to perform oversight functions. This may reduce the efficiency of a jurisdiction’s oversight function and lead to a shallow impact on the quality of internal investigation (Finn 2001; Olson 2016). Auditor/monitor-focused Agencies One of the newest forms of police oversight is the auditor model of oversight referred to by several different names, including police monitor20 or inspector general. As noted earlier in this report, this model of civilian oversight began to develop in the 1990s and generally emerged as a type of political compromise to satisfy police-community concerns about bias and professionalism (Walker 2006; Walker and Archbold 2014: 180). While local community and civil rights activists tended to 20 It is important to distinguish between court appointed monitors, who are limited term appointees charged with overseeing the implementation of a court-sanctioned reform agreement, and municipal or county civilian monitors who are local oversight professionals or consultants employed by the local jurisdiction. For purposes of this report, the term monitor is used to refer to locally employed police monitors. The role of court appointed monitors in promoting police reform is beyond the scope of this publication, but has been explored elsewhere (Davis et al. 2002; Chanin 2015). Civilian Oversight of Law Enforcement Assessing the Evidence 29 argue in favor of citizen review boards or full investigative models, police unions tended to be strongly opposed to those models. As a result, the auditor model emerged partly as a mechanism for bridging the disparate goals held by the different stakeholders to the complaint process (Walker and Archbold 2014). “Policy review, as it is called, is While there can be variation in the organizational structure of this type of civilian potentially the most important oversight, auditor/monitor agencies tend to focus on promoting large-scale, systemic accountability function that any reform of police organizations (PARC 2005). Accordingly, this type of organization tends citizen oversight agency can to have a unique set of goals that distinguish it from investigative and review models of perform because it is directed oversight (Walker 2001; Finn 2001; PARC 2005; Attard and Olson 2013; Ferdik et al. toward organizational change 2013; Walker and Archbold 2014). that hopefully will prevent 1 . Audit or monitor internal complaint investigations process. One of the core functions of this future misconduct” (Walker and type of oversight agency is to ensure that a jurisdiction’s processes for investigating Archbold 2014: 182). allegations of misconduct are thorough, complete and fair (Attard and Olson 2013: 4). Unlike review or investigative-focused agencies that tend to focus their organizational resources on reviewing or investigating individual complaints, auditor/monitors are more likely to review broad patterns in citizen complaints or internally-generated complaints (e.g., complaints filed by supervisors or other officers) (PARC 2005). While focused on systemic change, some police auditor or monitor agencies spend significant resources on the monitoring and auditing of individual complaints to ensure integrity in individual investigations, findings and the appropriate imposition of discipline. Some police auditors or monitors may also play a broader role in relation to investigations by responding to the scene of critical incidents to ensure integrity in those investigations . 4. Conduct evaluations of police policies, practices and training. Auditor/monitor-focused agencies may use social science research methods or generally accepted auditing methodologies to evaluate the effectiveness of policies, practices or training and may issue recommendations for organizational improvement (Walker and Archbold 2014; Olson and Attard 2016). Those policy evaluations can be triggered by individual complaints, patterns of complaints or allegations or broader investigations into issues that arise outside of the complaint process. In some jurisdictions, auditor/monitor-focused models have the authority to conduct data-driven evaluations of police policies, procedures and practices that go beyond the internal investigations process (Walker and Archbold 2014: 181; Olson and Attard 2014). 5 . May actively participate in open investigations. Some agencies that fall into the auditor/monitor category may also play a role within the complaint investigations process, including the ability to monitor or participate in open internal affairs investigations (sometimes both internally initiated and civilian initiated), observe investigative interviews and work alongside police internal affairs staff as investigations are conducted (PARC 2005; King 2015). Some auditor/monitor models may also serve as intake points for community complaints and have the ability to conduct initial intake investigations. 6 . Robust public reporting. Given their focus on bringing about systemic organizational change, auditor/monitor agencies may be more likely than other oversight models to have robust public reporting systems (Walker and Archbold 2014). Like full investigative agencies, most auditors and monitors issue public reports on aggregate patterns and trends in complaints, allegations, findings and discipline and make recommendations for process and organizational improvement. Others report on process performance, including workload, investigation thoroughness and timeliness. Some auditor/monitor reports include in-depth, data-driven evaluations of police operations, policies, procedures and training (PARC 2005). Potential Model Strengths Since these agencies tend to focus on exploring patterns in complaints, auditor/monitor models may have broader access to police and sheriff’s department records, case files and electronic databases than review-focused agencies (McDevitt et al. 2005; Olson and Attard 2016). While review boards tend to have only limited access to individual closed internal affairs files, au- 30 Civilian Oversight of Law Enforcement Assessing the Evidence ditors/monitors tend to be granted more expansive access to police department records (Walker and Archbold 2014). Moreover, auditors/ monitors tend to be (or become) policing experts, have larger budgets and may have more extensive training than might be found in volunteer-based oversight agencies (McDevitt et al. 2005). Additionally, it is possible that the auditor/monitor-focused model may be more effective at promoting long-term, systemic change in police organizations, in part because they can focus on broader trends and patterns in complaints and make public recommendations for how the police department can improve (Walker and Archbold 2014). Unlike investigative agencies, auditor/monitor models do not generally take the investigations process away from the police department, but instead use systematic auditing, monitoring or reviews to ensure the department is holding its officers accountable (PARC 2005). Auditor/monitor agencies also have the ability to track whether police departments implement their recommendations and whether those changes have resulted in organizational improvement over time (PARC 2005; Walker and Archbold 2014). Some scholars have also argued that the independence of auditor/ monitor agencies may also increase their credibility with the public, leading to more effective public outreach (Walker and Archbold 2014: 183). Moreover, the more robust public reporting authority and greater staffing resources may enhance the ability of auditor/ monitor agencies to conduct effective community outreach when compared to review-focused agencies that rely on community volunteers or even independent investigation agencies that focus on specific, individual complaints of misconduct. Potential Model Limitations Summary of Auditor/ monitor-focused Agencies Key Characteristics 1 . Often focuses on examining broad patterns in complaint investigations, including patterns in the quality of investigations, findings and discipline. 2. Some auditors/monitors may actively participate in or monitor open internal investigations . 3 . Often seek to promote broad organizational change by conducting systematic reviews of police policies, practices or training and making recommendations for improvement . Potential Key Strengths 1 . Often have more robust public reporting practices than other types of oversight . 2. Often less expensive than full investigative agencies, but more expensive than review-focused agencies. 3 . May be more effective at promoting long-term, systemic change in police departments . Potential Key Weaknesses Local civil rights or community activists may oppose this type of civil1 . Auditor/monitor focus on examining ian oversight, viewing this model’s reliance on full-time, paid staff with broad patterns rather that individual cases skepticism. The focus of auditors/monitors on aggregate patterns in be treated with skepticism by some local complaints and other metrics within law enforcement agencies may rights activist . also leave some community members and civil rights activists dissatisfied, 2. Conducting broad, systematic policy since they may desire that discipline be imposed in specific cases of evaluations requires significant expertise. officer misconduct (Walker and Archbold 2014). In fact, the very nature of the auditor/monitor model concept may put the police auditor/ 3 . Most auditors/monitors can only make monitor at odds with community demands or expectations in high recommendations and cannot compel law profile and controversial cases. The concept behind the auditor/monitor enforcement agencies to make changes. model is that the office is fair, unbiased and evidence-based in its decision-making (Walker and Archbold 2001). Such decision-making may result in criticism of the oversight program by the community, the police or both. Further, to achieve long-term success, a police auditor or monitor may reach compromises on individual cases with the police executive to ensure a healthy long-term relationship is developed between the agencies. In some Civilian Oversight of Law Enforcement Assessing the Evidence 31 cases, a monitor or auditor may decide that officers should not be held accountable in a particular case in lieu of systemic policy or training curriculum changes that would result in less resistance from the agency being overseen. This can result in an individual case not receiving the result demanded by members of the community; even though in the long-term, police practices may be improved overall. In some cases, an auditor/monitor agency may choose to allow the police executive to take credit for a reform initiative, again to maintain long-term relationships with police leadership. Such actions, while they may support positive reform in a police organization, may result in a lack of understanding amongst the community as to the actual effectiveness of the oversight program . Like other models of oversight, most auditors/monitors can only make recommendations and cannot compel law enforcement agencies to make changes (Walker and Archbold 2014: 195). One of the core functions of this model is to use a combination of rigorous, data-driven analytical methods, evidence-based recommendations and public reporting to promote organizational change within police departments. In cases where a police department does not implement oversight agency recommendations, the auditor/monitor will traditionally use its public reporting function to inform policy makers and the wider public about the department’s decision. While the threat of public exposure may affect police executive decision-making in favor of implementation in some cases, a department may publicly take the oversight agency to task on some recommendations or risk short-term public controversy to avoid implementing recommendations with which they disagree. However, in this model the police executive remains the decision-making authority on police policy, procedure and discipline, though the auditor/monitor may have significant influence as a result of the ability to use “the bully pulpit” in favor of oversight program initiatives. Table 4 presents examples of audit/monitor-focused models in the U.S. Table 4. Examples of Auditor/monitor-focused Agencies Agency Jurisdiction Website Independent Police Auditor San Jose, CA www .sanjoseca .gov/ipa Office of the Independent Monitor Denver, CO www .denvergov .org/oim Independent Police Monitor New Orleans, LA www .nolaipm .gov Los Angeles Board of Police Commissioners Office of the Inspector General Los Angeles, CA www .oig .lacity .org Office of the Inspector General for the New York City Police Department New York, NY www .nyc .gov/oignypd One final limitation is that the auditor/monitor model is strongly dependent on the quality of the staff hired to do the work (Walker and Archbold 2014). Analyzing patterns in complaints, findings, discipline or conducting performance evaluations of other police policies and practices requires a high level of technical sophistication and training, as well as a commitment to objective, evidence-based evaluation. The hiring of staff without relevant experience or a commitment to objective, dispassionate evaluation methods may cause significant tension between the oversight agency and police executives, as well as with rank-and-file officers. Organizational Structure and Effectiveness in Citizen 32 Civilian Oversight of Law Enforcement Assessing the Evidence Oversight In this section, the research team uses data collected from 97 civilian oversight agencies in the U.S. to compare and identify patterns in the structure, authority, practices and resources across civilian oversight agencies in the U.S. The goal of this comparison is threefold: 1 . Identify National Patterns in Oversight. Identify broad patterns in organizational characteristics across different models of oversight. By exploring broad patterns, this report seeks to help local jurisdictions think about how their potential oversight agency may fit within the larger field of oversight. 2. Compare Organizational Characteristics Across Common Oversight Models. Allow for the comparison of organizational characteristics across different models of oversight. Much of the case-study research has demonstrated that many contemporary oversight agencies exhibit hybrid characteristics and do not fit easily within pre-existing classification schemes. 3 . Examine Effectiveness in Oversight. Examine the distribution of organizational characteristics at the national level that have been associated with effective civilian oversight of police. This section of the report has two subsections. The first subsection draws from the organizational data collected from the 97 civilian oversight agencies to examine broad organizational features of oversight in the U.S., including organizational goals, organizational age and the types of law enforcement agencies subject to oversight. The second subsection examines key indicators of effectiveness across different oversight models, including independence, legal authority, financial resources, political support and public reporting.21 Organizational Characteristics In this subsection, the research team draws from the survey data to examine patterns in key organizational features of oversight. Common Goals of Civilian Oversight of the Police An important issue for any jurisdiction that is considering implementing or revising an oversight agency is to determine the goals for the program. A number of organizational goals are commonly cited across a range of academic and policy writing on oversight (Walker 2001: 55; Finn 2001: 6-11; Bobb 2003; Harris, 2012; Attard and Olson 2013; King 2015; Prenzler 2016; Alpert et al. 2016). Common goals include: • Ensure that the police complaints process is accessible to all and to remove impediments to the filing of lawful complaints . Most commonly shared goals across all models of oversight: 1 . Improve public trust 2. Ensure accessible complaint processes 3 . Promote thorough investigations 4. Increase transparency 5 . Deter police misconduct • Ensure that internal investigations are fair and thorough, findings are reasonable and evidence-based and discipline is appropriate . • Improve public confidence in the police and local government by demonstrating that internal investigations are fair and thorough and findings and discipline are reasonable and appropriate. 21 Care must be taken when applying these types of observational survey data. These data can be used to identify patterns of association between models of oversight and the characteristics of the jurisdiction in which they operate. However, these data do not allow us to establish whether different models of oversight cause the development of different jurisdictional characteristics or whether those characteristics are the result of other unmeasured factors. Civilian Oversight of Law Enforcement Assessing the Evidence 33 • Enhance the transparency of police organizations by publicly reporting on the department’s efforts in holding its officers accountable. • Strengthen police organizations by analyzing patterns in complaints and other police-related data to improve policies, practices, training and management. • Deter officers from engaging in misconduct through the creation of more effective and consistent investigation and disciplinary processes . • Reducing legal liability from officer misconduct. • Improve the public’s understanding of police policy, training and practices. While these are commonly cited as goals for the implementation of oversight, it was not clear how commonly oversight agencies actually subscribed to these goals. As a result, one of the initial sets of questions asked the oversight executives to rate the importance of a series of goals for their oversight agencies .22 The top five most common critical or important goals identified in the data collected largely mimic the most commonly cited goals in the academic literature (See Appendix B, Table B1). Almost all oversight agencies reported that improving public trust (98 percent), ensuring accessible complaint processes (93 percent), ensuring thorough investigations (88 percent), increasing transparency (86 percent) and deterring police misconduct (85 percent) were among their most critical programmatic goals. However, clear differences between oversight models emerged in relation to other types of organizational goals: • Auditor/monitor agencies tended to rate goals associated with policy review, auditing, statistical pattern analysis and issuing recommendations as critical/important for their agency .23 • Investigation-focused agencies were more likely than other types of agencies to indicate that improving accessibility of the complaint process, ensuring thorough investigations, complainant satisfaction and ensuring complainants received justice were critical or important for their agency . • Review-focused agencies tended to rate the importance of goals associated with the complaint investigation process highly. What Led to the Implementation of Oversight? Much of the writing on oversight indicates that it tends to emerge out of local crises involving local police and particularly community conflicts over racially disparate policing and excessive Much of the writing on overuses of force (Walker 2001). To examine patterns in the development of oversight, the agensight indicates that it tends to cies identified the types of incidents that catalyzed the creation of their oversight agency (see emerge out of local crises inAppendix B, Table B2). The most common trigger involved force-related incidents (49 percent). volving local police, and parJust under 30 percent of the agencies indicated that racially biased policing played a role in ticularly community conflicts the development of oversight. Notably, only 11 percent of the agencies indicated that they over racially disparate policing implemented their oversight agency as a proactive or preventive policy effort and not the result and excessive uses of force. of a specific critical incident. Overall, there were few differences between oversight agencies in terms of the incidents that triggered their implementation—they all tended to have similar origins. One important exception was that review-focused agencies were more likely to report that they were implemented proactively than auditor/monitor or investigative agencies . 22 The possible responses ranged from Critical, Important, Moderate, Minor and No Jurisdiction. 23 Investigative agencies fell between auditor/monitor and review agencies on this issue, but were closer to auditor/monitor agencies in their identification of these issues as critical/important for their organization. 34 Civilian Oversight of Law Enforcement Assessing the Evidence Age of Oversight Agencies An important and commonly cited concern within oversight has been a lack of stability of the field. As was evident in the historical section of this report, many early 20th century forms of oversight failed in the face of resistance by police unions and politicians (Walker 2006). However, the organizational data from the civilian oversight agencies suggest that pattern no longer holds. Executives reported on how long their agency has been in operation. As shown in Figure 1, the largest proportion of respondents (55 percent) indicated that their agency has been in continuous operation for 16 or more years. This finding is not surprising since the largest increase in the number of oversight agencies occurred in the 1990s. Although a large proportion of current agencies were older than 16 years, the field continues to grow, but at a somewhat slower pace today than in the 1990s. Approximately 16 percent of respondents indicated that their agency had been in operation for five years or less .24 Types of Law Enforcement Agencies Overseen A large proportion of the oversight agencies indicated that they were responsible for overseeing municipal police departments (82 percent). The second largest area of jurisdiction was for county sheriffs (15 percent). Several agencies indicated that they were responsible for overseeing multiple jurisdictions (e.g., municipal police and county sheriff’s agencies) (See Appendix B, Table B3). While patterns in jurisdiction were similar across oversight models, auditor/monitor-focused models were somewhat more likely to report that they had jurisdiction over county-level agencies than either investigation-focused or review-focused agencies. Size of Law Enforcement Agencies Overseen The largest proportion of oversight programs oversaw law enforcement agencies (45 percent) that had fewer than 500 sworn employees. However, there was clear variation across the three models of oversight. Review agencies were much more likely to report overseeing smaller law enforcement agencies (500 or fewer officers), while investigative agencies and auditor/monitor agencies were more commonly found in jurisdictions associated with larger law enforcement agencies ( See Appendix B, Table B4). Agency Leadership Figure 1: How long has your agency been in existence? Years in Operation Data was provided that helped identify the characteristics of the administrative head of the 55% agency. A majority of the agencies (70 percent) 16 or more years had either a full or part-time paid executive, 16% 11 to 15 years though there was noticeable variation between 13% types of agencies. Auditor/monitor-focused 6 to 10 years models overwhelmingly had paid executives 16% Less than 5 years leading the organization (78 percent). Investigation-focused agencies also tended to have full-time Percentage of Agencies paid executives leading the organizations (71 percent), though almost a quarter were led by multi-member boards. In contrast, a majority of review-focused agencies were led by multi-member boards (52 percent) (See Appendix B, Table B5). 24 There is an important nuance to this pattern. Approximately 29 percent of the oversight agencies that were 15 years or younger had replaced a previously existing oversight body. Only 4 percent of agencies that were 16 years or older had replaced a previous oversight agency. So while the absolute number of oversight agencies continues to grow, it is clear that a not insignificant proportion of the new agencies are replacements for previously existing oversight bodies. Civilian Oversight of Law Enforcement Assessing the Evidence 35 Core Elements of Successful Oversight What combination of organizational components, authority, practices and resources lead to successful civilian oversight of law enforcement? While there is some disagreement within the literature, there is a developing consensus around the core components of oversight that are necessary to implement and sustain effective oversight (Perez 1994; Walker and Bumphus 1992; Prenzler and Ronken 2001; Walker 2001; Walker 2003; Bobb et al. 2008; Attard and Olson 2013; Walker and Archbold 2014; King 2015; Alpert et al. 2016). Walker and his colleagues (Walker 2003) developed one of the more comprehensive frameworks outlining the key organizational components of an effective oversight agency. They developed the initial framework out of a series of conferences with police auditors, where roundtable discussions were held regarding the key elements of effective oversight (Walker and Archbold 2014: 199). As a result of those conferences, Walker published a set of oversight “principles” in a report titled: “Core Principles for and Effective Police Auditor’s Office.” (Walker 2003). Since its publication, several authors have sought to extend slightly revised versions of Walker et al.’s original 12 principles to other models of oversight (Olson and Attard 2013; King 2015).25 Based on that literature and Walker’s (2003) core principles, effectiveness in oversight can be grouped into 12 related areas: 1 . Independence 2. Adequate jurisdictional authority 3 . Unfettered access to records 4. Access to law enforcement executives and internal affairs staff 5 . Full cooperation 6 . Support of process stakeholders 7 . Adequate resources 8 . Public reporting or transparency 9 . Pattern analysis 10 . Community outreach 11 . Community involvement 12. Respect for confidentiality requirements For the remainder of this section, the report draws from the organizational data to estimate the frequency with which these components are present within the larger oversight field. Independence A consensus exists in the literature relating to the crucial nature of independence for the implementation of successful oversight of law enforcement (Prenzler and Ronken 2001; Walker 2001; Walker 2003; Olson and Attard 2013; (Savage 2016). The extent to which oversight is independent of police, political actors and other special interests has been argued to be strongly related to effectiveness of oversight. Within the literature, several types of independence have been identified, though no social science research has examined the impact that different forms of independence have on the effectiveness of oversight . 25 In addition, a similar guidance document for police monitors was developed in 2008 by the PARC (Bobb et al. 2008). While comprehensive, the guidance in this document covers both municipal and county monitors and litigation-based monitors. As a result, the guidelines in this document are highly specific and focused on the monitoring role, and less easily adapted to other models of oversight. Nonetheless, this set of guidance would be useful for jurisdictions considering implementing a civilian oversight agency with a monitoring component. 36 Civilian Oversight of Law Enforcement Assessing the Evidence Structural Independence. Structural independence has been defined as one potential element of independence (Perez, 1994; Walker 2001; Savage 2016). Walker (2001) argues that oversight agencies that are structurally separate from the police departments they oversee will be able to operate with more independence than agencies whose work is intertwined or dependent on local law enforcement. There are multiple ways to achieve this: • Prenzler (2000; 2016) contends that investigation-focused models that replace police internal affairs units (which he terms “civilian control” models) are the most independent forms of oversight, since non-police citizens take control of all the different stages of the complaint investigation process. • Oversight agencies that report to political bodies separate from those that oversee police agencies may have greater independence (Prenzler and Ronken 2001; Savage 2016). Legal Protection. A second measure of independence relates to the protections offered by the oversight agency’s founding legal authority (Walker 2001; Harris 2012). The more difficult it is to eliminate, censor or shutter the oversight agency, the more likely it is the agency will be able to act with some measure of independence (Harris 2012). For example, agencies whose authority was created through an amendment to a city’s charter or enshrined in ordinance will likely be more independent than agencies that were created through an executive policy change (Walker 2001; Ferdik et al. 2013). Table B6 (Appendix B) identifies the different sources of legal authority enjoyed by the oversight agencies that provided data for this report. More than two-thirds of the agencies reported that their legal authority was enshrined in either ordinance or charter documents. Job Protections. Specific language can be entered into an enabling ordinance that prevents agency heads from being removed except for cause (Walker 2003). Table B7 (Appendix B) reports the percentage of agencies that had job protections for agency leaders. Overall, 78 percent of full and part-time oversight agency leaders were “at will.” Anti-Influence Language. Agencies can have legal language in their enabling ordinances that prohibits retaliation based on the lawful acts taken by oversight agencies in the course of their work (Walker 2003; Attard and Olson 2013). Limit Involvement of Police Officers. One controversial approach to independence relates to whether current or former police officers should be able to work or volunteer for civilian oversight agencies. Some oversight agencies do not allow current or former police officers to work or volunteer for their agency. Other agencies may employ current or former officers who have worked for other jurisdictions, but not for the police agency being overseen. Other departments allow or require the oversight agency to use retired or currently serving officers. There is no social science research that systematically establishes whether allowing police officers to work or volunteer for oversight increases or decreases the quality of an oversight agency’s work (Savage 2016) or affects the public’s perception of its independence. The research team asked 97 civilian oversight agencies whether they allowed former police officers to serve as their agency executive or director. Altogether, 50 percent of the agencies responded that a former police officer could serve as their agency executive/director. An additional 36 percent reported that a former police officer could serve as the executive/director “under some circumstances” (Appendix B, Table B8). Adequate Jurisdiction and Authority Little is currently known about whether An important component of effective oversight identified in the research and different types of jurisdiction or authority policy literature related to the jurisdictional authority granted to oversight agenhave a greater or lesser impact on patterns cies (Attard and Olson 2013). Agencies need to have adequate jurisdiction that in officer misconduct or organizational will allow them to achieve their organizational goals. Of course, the goals for reform. oversight can vary significantly between jurisdictions depending on local issues, the political climate and a jurisdiction’s financial resources. Little is known about whether different types of jurisdiction or authority have a greater or lesser impact on patterns in officer misconduct or organizational reform. Civilian Oversight of Law Enforcement Assessing the Evidence 37 Adequate Jurisdiction. One issue local jurisdictions need to decide on is whether the local oversight agency will have the ability to review allegations of misconduct that emerge from sources outside of citizen complaints. Table B9 (Appendix B) reports the core areas of jurisdictional authority across the three different models of oversight. Almost all of the agencies submitting data indicated that they have jurisdiction in relation to citizen complaints. A majority of the agencies stated they “always/sometimes” have jurisdiction in relation to: • Officer-involved shootings. • Serious force . • In-custody deaths. There were some differences, however, between oversight models in other areas. For example, auditor/monitor–focused agencies were more likely than other models to report that they had jurisdiction in relation to officer-initiated complaints, supervisor complaints, high-speed pursuits and criminal investigations, as compared to review and investigative agencies. Adequate Authority. A second critical issue in this area relates to authority that oversight agencies have in relation to the handling of complaints. Given that most oversight agencies perform an oversight function in relation to citizen complaints, the agency executives provided data to identify the specific role their agency plays in relation to the processing of citizen complaints. In looking at Table B10 (Appendix B), it is clear that there are important differences among the oversight models in terms of the role they play in the handling of citizen complaints . One remarkable pattern related to the breadth of the organizational authority enjoyed by auditors/monitors. Almost all of the auditor/monitor agencies reported that they were able to: • Review police complaint investigations (96 percent). • Audit police complaint investigations for compliance with investigative standards (92 percent). • Monitor open police complaint investigations (92 percent). • Refer complaints back to internal affairs for additional investigation (78 percent). • Conduct independent investigations (61 percent). • Recommend findings on investigations (70 percent). • Recommend discipline (61 percent). Investigative agencies reported somewhat more focused forms of authority than auditor/monitor agencies and tended to report that they were able to: • Conduct independent investigations (100 percent). • Decide how complaints would be handled (97 percent). • Classify complaints (94 percent). • Conduct intake investigations (94 percent). • Review completed police complaint investigations (67 percent). • Recommend and issue findings (84 percent and 88 percent). Review agencies reported the most focused forms of authority and reported strong involvement in: • The review of police complaint investigations (81 percent). • Referring complaints back to internal affairs for further investigation (65 percent). 38 Civilian Oversight of Law Enforcement Assessing the Evidence Unfettered Access to Records An organizational power commonly identified as a critical component of effective oversight relates to an oversight agency’s access to police department records (PARC 2005; Olson and Attard 2013; King 2015). Both Attard and Olson (2013) and King (2015) identify access to records as one of the most important components of effective oversight. Similarly, Walker (2003) includes unfettered access as one of the core principles for an effective police auditor (Walker and Archbold 2014: 200). To explore the extent to which contemporary oversight agencies have access to police records, the data received from the 97 civilian oversight agencies included a series of questions that asked executives about their level of access to police records. Table B11 (Appendix B) identifies the proportion of oversight agencies reporting that they always or sometimes have access to specific kinds of records. More than two-thirds of the oversight agencies reported that they always or sometimes had access to closed internal investigations. There was substantial variation; however, across the different models of oversight in relation to records access. Auditor/monitor agencies tended to report high levels of records access, with more than three quarters reporting that they always or sometimes had access to all internal affairs files and internal affairs databases. More than half of the auditor/ monitor agencies also reported having access to all police databases. In contrast, 38 percent of review-focused agencies had access to all internal affairs records, 30 percent reported access to internal affairs databases and 19 percent reported access to all police databases. Investigative agencies were more likely than other models to report the ability to subpoena witnesses (56 percent) and records (59 percent). A large proportion (74 percent) of all agencies indicated that they had access to police body-worn/in-car video. Access to Law Enforcement Executives and Internal Affairs Staff A fifth component of effective oversight relates to the extent to which oversight actors have the ability to influence appropriate law enforcement officials. Walker (2003) notes that oversight agencies should have “…direct access to the chief executive of the law enforcement agency under review.” (p. 7). From the perspective of the authors, while gaining access to the police chief or sheriff is important, it is also critical that the police chief or sheriff and their internal affairs staff are willing to both consider and implement the oversight agencies recommendations. Given that, the organizational data collected included information about whether: (1) police executives “listened carefully” to the recommendations made by the oversight agency; and (2) police departments were willing to implement the recommendations made by the oversight agency. To be effective, oversight agencies must have regular access to police executives. However, effectiveness hinges not just on access, but also on the willingness of police executives to both consider and implement the recommendations made by oversight agencies. Listening to oversight recommendations. There was little variation between models of oversight in relation to whether police executives considered the views of oversight actors. Roughly 78 percent of auditor/monitor, investigative and review-focused agencies indicated that police executives listened carefully to their recommendations (see Table B12 in Appendix B). Implementing oversight recommendations. There was more variation in relation to whether police executives were willing to implement the recommendations made by oversight agencies (see Table B13 in Appendix B). As shown in Figure 2, auditor/monitor agencies were much more likely to report that police executives very frequently or frequently implemented their recommendations (72 percent), while 42 percent of the responding investigative agencies and 34 percent of the review agencies indicated that police or sheriff’s departments frequently implemented their recommendations. Some of this difference Civilian Oversight of Law Enforcement Assessing the Evidence 39 may relate to the type of work different agencies tend to perform. Auditor/monitor agencies are more likely to have specialized staff who focus on analyzing patterns in complaints and other police data and they are more likely to issue special topics reports as a result of that work (which Harris (2012) describes as a “holistic’ approach). It is possible that that type of analytical work makes it more likely that police departments will implement recommendations made by oversight. More systematic comparative research is needed before strong inferences can be drawn on this issue. Full Cooperation Oversight Model Another component of effective oversight Figure 2: Percent of Oversight Agencies Indicating that Police Stakeholders relates to the ability of an oversight agency “very frequently or frequently” Implement Oversight Recommendations? to gain the cooperation of police agency executives and officers (Walker 2003; 34% Review Walker and Archbold 2014). Some agencies 42% Investigative attempt to achieve voluntary cooperation by developing a collaborative working re72% Auditor/Monitor lationship with the departments they 0% 20% 40% 60% 80% 100% oversee and attempting to engage departPercent ments, officers and unions within the oversight process (Miller 2002). Other jurisdictions attempt to strengthen cooperation by having legislative requirements for cooperation built into the oversight agency’s enabling ordinance, charter or statute. To date, there is no social science evidence that systematically tests whether one or both of these approaches improve the effectiveness of oversight. There are two common variations in legislated cooperation requirements. First are general cooperation requirements that direct all law enforcement employees within a particular jurisdiction to cooperate with the oversight agency during the course of its work, but do not clearly articulate sanctions for non-cooperation. A second type of cooperation requirement builds in the possibility of a sanction for non-cooperation, where law enforcement employees are required to cooperate with the oversight agency as a condition their employment (i.e., non-cooperation could constitute grounds for termination). The organizational data received from the civilian oversight agencies (see Table B14 in Appendix B) showed investigation-focused agencies tended to have more robust cooperation requirements than other forms of oversight (see Figure 3). Altogether 73 percent of investigative agencies reported that their jurisdiction included a general cooperation requirement, while 52 percent of auditor/monitor agencies and 40 percent of the review agencies reported a cooperation requirement. A much smaller proportion of the agencies reported that their enabling ordinance/statute made cooperation a condition of employment. Less than half of investigative agencies indicated that cooperation was a condition of employment for officers or command staff. Only 26 percent of auditor/monitor agencies and 20 percent of review agencies had the same authority. Support from Process Stakeholders Oversight Model Figure 3: Percent of Oversight Agencies Reporting that their Jurisdiction has a General Cooperation Requirement 40% Review 73% Investigative 52% Auditor/Monitor 0% 40 20% 40% Percent 60% 80% 100% A sixth key component of effective oversight relates to the importance of process stakeholder support for the oversight agency’s work. A lack of political support has the potential to undermine an otherwise well designed system of oversight. Government officials and office holders, if not supportive of an oversight agency, can reduce its effectiveness in a variety of ways, including by failing to provide the agency with adequate resources or authority or by appointing ineffective managers or board members (Attard and Civilian Oversight of Law Enforcement Assessing the Evidence Olson 2013: 6). Similarly, opposition from police unions, local district attorneys or police executives has the potential to complicate the work of oversight agencies (Wilson and Buckler 2010). While conflict is not necessarily avoidable in all cases, the extent to which an agency can achieve its organizational goals will be partly dependent on whether it has support from local office holders, the police executive and other key process stakeholders. Table B15 (Appendix B) describes the percentage of agencies reporting that key process stakeholders have shown very strong or strong support for oversight . A large percentage of the responding agencies across all models reported very strong or strong support from most key process stakeholders, including police executives, city attorneys or county counsels, mayors, city managers, city council and the local civil rights community. Reported support was noticeably lower among police unions. There was little variation in reported support across the three models of oversight,26 with three exceptions. Review-focused agencies were somewhat more likely to report support from police unions. Auditor/monitor agencies were less likely to indicate support from city attorneys/county counsels, but were slightly more likely to report support from the local civil rights community . Investigative agencies reported the least support from police unions . Adequate Resources Many oversight agencies reported very A seventh organizational component critical for the effectiveness of oversight relates strong or strong support from stakeholders to the resources available to the oversight agency. In general, an oversight agency’s such as mayors, city council and the local resourcing is considered one of the most important potential indicators of effeccivil rights community, but less support tiveness (Finn 2001; Walker 2001; Walker 2003; Attard and Olson 2013). In disfrom police unions. cussing resources, most scholars and policy experts distinguish between the importance of having a budget that matches an agency’s goals and the presence of professional staff who have the time and expertise to support the work of the oversight agency (Finn 2001; Walker 2003). For example, Olson (2016) recently conducted a limited examination of one review-focused agency and found that local agency was unable to perform certain important oversight functions or fully engage with important stakeholders to the complaint investigations process due to limited budgetary and financial resources (Olson 2016: 6). Even though there is some commonsense appeal to the notion that budgetary and staffing resources are related to effectiveness, no social science research has systematically examined the impact that varying budgets have on the effectiveness of oversight. That said, there is noticeable variation between models of oversight in two areas: budgets and staffing. Adequate Budget. Table B16 (Appendix B) examines patterns in oversight agency resources. It shows a breakdown in the annual budgets for the agencies across the three model categories. As was seen in the review of the previous literature on models of oversight, review-focused agencies tended to have much smaller budgets than auditor/monitor and investigative oversight agencies. Just under 60 percent of the review agencies reported having no operating budget and another 20 percent of review agencies reported budgets of less than $100,000 per year. In comparison, roughly half of the auditor/monitor and investigative agencies had budgets that were greater than $500,000 per year. Overall, this pattern in funding can be at least partially explained by the fact that investigation-focused and auditor/monitor-focused agencies are more likely to be implemented in jurisdictions with bigger populations and larger law enforcement agencies. In addition, where programs have more limited mandates, it can be expected that the resources needed to achieve that mandate would require less funding. Adequate Staffing. Table B17 (Appendix B) evaluates patterns in the staffing resources available to oversight agencies across the three models. The table shows the distribution of full-time, paid staff across the three different types of civilian oversight. As was seen with agency budgets, review-focused agencies tended to have the fewest staff, with 61 percent of the review agencies reporting that they had no full-time, paid staff. Investigative agencies tended to report a larger number of full-time, 26 It is important to note that there was a noticeable increase in the number of agencies that chose not to respond to questions about stakeholder support as compared to other questions. Civilian Oversight of Law Enforcement Assessing the Evidence 41 paid staff with 21 percent of the investigative agencies reporting they had 16 or more full-time staff members. Auditor/ monitors fell in-between review and investigative agencies, with 26 percent of auditor/monitors reporting no full-time staff and an additional 43 percent indicating that they had one to five full-time staff. As with patterns in relation to budgets, variation in staffing is at least partly the result of jurisdiction size—that is, investigative agencies and auditor/monitor agencies are more likely than review-focused agencies to oversee large police departments in bigger cities. Public Reporting or Transparency The eighth component of effectiveness in oversight relates to public reporting and the level of transparency that the oversight agency brings to the complaint handling process and other areas of police operations (Walker 2001; Finn 2001; Attard and Olson 2013). As we saw with the data on agency goals, increasing the transparency of the complaint investigations process was one of the most commonly cited organizational goals identified by the civilian oversight agency executives. While there is no comparative, systematic social science research on the impact of public reporting on community perceptions relating to the transparency of the internal investigations process, the civilian oversight executives provided data regarding their agency’s public reporting and data analysis practices. Issue Public Reports. (See Table B18 in Appendix B). Most civilian oversight agencies reported that they publish public reports (78 percent), although there was slight variation among oversight agencies that provided data for this report, with a slightly smaller proportion of review boards reporting that they publish reports (69 percent) as compared to auditor/monitor (80 percent) and investigative agencies (85 percent). Pattern Analysis Another potentially important element of an oversight agency’s work is the extent to which it analyzes and reports on aggregate patterns in relation to complaint handling processes and outcomes. Police departments have traditionally been reluctant to report publicly on patterns in internal investigations and an important role of oversight has been to use public reporting to inform the community about patterns in complaints, investigative outcomes and indicators of process or procedural effectiveness (e.g., timeliness of the process) (Walker 2001; Finn 2001; Attard and Olson 2013). Given this, the civilian oversight agencies shared information that help examine data management and pattern analysis practices across different models of oversight (see Table B19 in Appendix B). A larger proportion of auditor/ monitor agencies reported conducting analyses of patterns in officer-involved shootings, officer discipline, in-custody deaths and police data relating to stops, searches and arrests. The most common types of issues analyzed very frequently or frequently by oversight agencies were complaint handling workload (i.e., number of complaints), allegations and findings on allegations. Only a small proportion of agencies (20 percent) frequently conducted analyses of issues of critical concern to the public and communities of color, such as patterns in police stops, searches and arrests. There were a number of noticeable differences across types of agencies in relation to the use of pattern analysis. A larger proportion of auditor/monitor agencies reported conducting analyses of patterns in officer-involved shootings, officer discipline, in-custody deaths and police data relating to stops, searches and arrests. Community Outreach A tenth component of civilian oversight relates to the ability of agencies to conduct outreach to community members and other process stakeholders. Overall, there is a strong consensus in the research and policy literatures that community outreach is an essential element of any effective oversight agency (Walker 2001; Finn 2001; Stewart 2006; Walker and Archbold 2014). While the resources that oversight agencies have can vary substantially, outreach activities provide oversight agencies with 42 Civilian Oversight of Law Enforcement Assessing the Evidence the opportunity to: 1 . Publicize the different processes for handling complaints; 2. Reach out to disenfranchised members of the community who might be fearful or distrustful of the police; 3 . Talk with that the community about police policies, procedures or training; and 4. Gather input from a range of community members and groups. A successful oversight agency will typically include a robust outreach program. In terms of the research literature, a few authors have identified several components of an effective outreach program, including holding public outreach events involving different community groups, engaging in a range of diverse outreach activities and making attempts to conduct outreach with difficult-to-reach or underserved populations (Walker 2001; Stewart 2006; King 2015). Outreach to Groups. The first element of an outreach program relates to the whether the oversight agency is engaging in outreach activity at all (Walker 2001). Table B20 (Appendix B) identifies patterns in outreach across oversight models. The more common groups targeted for community outreach included community leaders (77 percent); neighborhood associations (69 percent); policy-makers (65 percent); and police command staff (59 percent). As with other outreach activities, review-focused agencies reported conducting somewhat less outreach to stakeholders than either auditor/monitor or investigative agencies . Outreach Activities. An additional important component of outreach relates to the type and range of activities (Walker 2001). To explore the kinds of outreach activities that different models of oversight were using, the civilian oversight agencies provided data regarding their outreach activities (see Table B21 in Appendix B). The operation of an agency website was Reaching Out to Special Populations the most common form of outreach (69 percent), followed by The types of outreach that oversight agencies conduct tabling at community fairs (48 percent), offering complaint will depend not just on their resources, but also on the forms at multiple community locations (46 percent) and holding specific social and cultural characteristics of populations community forums (45 percent). in which they operate . Some oversight agencies have Outreach to Special Populations. An additional measure developed sophisticated community outreach strategies of outreach effectiveness relates to the extent to which oversight tailored to their local communities . agencies direct outreach resources toward special populations For example, the San Jose Office of the Indepen(see Table B22 in Appendix B). The civilian oversight agencies dent Police Auditor makes many of their key overprovided information on whether they translated any of their sight documents available in English, Spanish, and agency documents into languages other than English as one Vietnamese. They also maintain an up-to-date Facemeasure of this form of outreach . Just under half of all agenbook© page, distribute a resource guide to local students, cies reported translating complaint forms (44 percent) and developed a training video for teachers and produced a agency brochures (48 percent) into languages other than local cable access television show that explored importEnglish. Only a small proportion of agencies translated webant law enforcement and oversight issues facing the San pages or agency reports. Review agencies were noticeably less Jose community. Example material can be found here: likely to report that they translated outreach documents than www.sanjoseca.gov/index.aspx?nid=200 other models of oversight . Community Involvement An eleventh component of effective oversight is community involvement. McDevitt et al. (2005) note that it will likely be impossible for effective oversight to be implemented without sufficient community involvement. Involving community stakeholders in Civilian Oversight of Law Enforcement Assessing the Evidence 43 the process allows the oversight agency to identify and address the key accountability issues that the jurisdiction is facing (McDevitt el al. 2005: 7). Different types of civilian oversight agencies involve community members in a wide variety of ways—some oversight agencies involve community volunteers in the core work (review-focused agencies), while other highly professionalized agencies may have community members sitting on governing boards, police commissions or advisory boards. However, there is no current empirical research that examines whether one approach is more effective than another. Given the arguments for the importance of involving community members in oversight, it is not surprising that a very large proportion of the oversight agencies who provided data indicated that they operated with a board composed of community members (See Appendix B, Table B23). Respect for Confidentiality Requirements Some oversight practitioners have argued that another important component of effective oversight relates to the ability of civilian oversight agencies to protect the confidentiality of employee personnel records (Walker 2003). Many states and local jurisdictions have legal rules that prevent the disclosure of certain kinds of personnel records. While these vary considerably by state, the ability of a civilian oversight agency to be effective in its work will depend, at least in part, on its ability to respect confidentiality rules (Attard and Olson 2013). Failure to respect state statutes relating to confidentiality may constitute a serious violation of professional ethics, undermine trust between the oversight agency and the local police department and may cause the oversight agency to lose access to confidential records. To date, no systematic comparative research has examined how effectively civilian oversight agencies have been at ensuring the confidentiality of the records they handle. 44 Civilian Oversight of Law Enforcement Assessing the Evidence Trending Issues in Civilian Oversight In the previous section, the team drew from the research literature and organizational data provided by 97 civilian oversight agencies to explore 12 broad organizational components identified as being critical for ensuring effectiveness in oversight. In this section of the report, the team explores some of the key recent developments in civilian oversight of police. Using Performance Metrics to Capture Oversight Effectiveness Over the last ten years there has been a growing interest in the question of how the effectiveness of police oversight agencies can be measured (Filstad and Gottschalk 2011; Faull 2013). More specifically, there is a developing body of research focused on exploring whether “performance measures” can be used to capture the impact oversight agencies have in their work (Brereton 2000; Miller 2002; Prenzler and Lewis 2005; Mohr 2007; Filstad and Gottschalk 2011; Faull 2013). Performance measures are indicators of how successful an organization has been in achieving an organizational objective (Hatfield 1994). Performance measures are typically expressed statistically (i.e., as numbers or rates) and usually focus on the impact, timeliness, quality or productivity of an organization. Put another way, performance measures should tell the public, managers and policymakers how well an organization is doing, whether it is achieving its organizational mission and whether changes are necessary to improve the organization’s effectiveness. There is currently no consensus on how to measure organizational performance in the field of civilian oversight (Faull 2013). Evaluating the impact of oversight can be difficult for several reasons. First, many oversight agencies make recommendations based on their work reviewing, monitoring or auditing police internal investigations. Thus, their work is intertwined with internal police accountability systems (Brereton 2000; Prenzler and Lewis 2005). Second, evaluating the distinct impact that oversight has is complicated by the fact that there are a large number of variables involved and patterns in complaint-related data can be interpreted in multiple ways (Walker and Bumphus 1992; Walker 2001; Prenzler and Lewis 2005). For example, increases in the number of citizen complaints filed could indicate that officer misconduct is increasing or it could indicate that community members are more willing to file complaints due to increasing confidence in the process. Finally, there is wide variation among oversight agencies in their organizational structure, jurisdiction, resources, data collection practices and authority, making comparison between and among agencies difficult (Walker 2001; Prenzler and Lewis 2005). Even if there is no current agreement about how best to measure organizational effectiveness in police oversight, a number of studies have sought to explore the value of different performance measures (Walker 2001; Prenzler and Lewis 2005; Filstad and Gottschalk 2011). David Brereton (2000) argues that there are four key questions that should be asked when assessing the performance of civilian oversight agencies: • How successful have oversight agencies been in improving the process for investigating complaints against the police? • Are civilian oversight agencies more successful at increasing public satisfaction than systems without oversight? • Has the implementation of civilian oversight agencies improved police practices and behavior? • Has the implementation of oversight negatively impacted officer morale or police operations (Brereton 2000: 106; see also Faull 2013)? Prenzler and Lewis (2005) used an analysis of public reports and a brief survey to explore the performance measures used by civilian oversight agencies in Australia in the early 2000s. They offered a number of conclusions. First, they observed that most Australian oversight agencies were attempting to evaluate their performance, but that they only used relatively simple measures of effectiveness, such as: • The number of complaints filed and closed per year. Civilian Oversight of Law Enforcement Assessing the Evidence 45 • Time taken to close complaints. • Patterns in outcomes, such as substantiated complaints, referrals, mediations and discipline. Prenzler and Lewis (2005) contend that while these simple measures are useful for capturing an agency’s workload, they do not capture much about an agency’s performance or impact. They then identified a large number of other measures used by oversight agencies to measure performance (p. 78-82): • Trends in reported police misconduct (e.g., number of cases filed). • Workload measures (e.g., number of open cases, cases reviewed, number of investigations conducted, number of appeals, identification of case deficiencies). • Patterns in findings and outcomes on case investigations. • Timeliness and process efficiency measures (e.g., number of cases assessed in a given time period). • Public opinion survey measures (e.g., satisfaction rates for complainants and officers, community confidence in the police). • Review and recommendation effectiveness (e.g., number of recommendations made, number implemented). • Accessibility measures (e.g., accessibility of the office to the public). More recently, other scholars have used slight different variations on the performance indicators identified by Prenzler and Lewis (Filstad and Gottschalk 2011: 102-104; Mohr 2007; Faull 2013). However, there is a general consensus that strong caution should be used when attempting to use “sustain” or “conviction” rates to measure an oversight agency’s performance or to compare performance across oversight systems .27 Walker (2001) points out a number of pitfalls associated with using sustain rates, including an absence of standards for “acceptable” sustain rates; the lack of understanding of potential differences between complaints filed with oversight agencies and complaints filed with police department; the fact that investigations are only one potential function of oversight and the presence of profound data problems that make calculating; and comparing sustain rates unworkable (Walker 2001: 120-122 and 134-135). Problem-oriented Approaches to Civilian Oversight A second trending issue has to do with an increasing focus on shifting some of the emphasis within oversight away from reactive forms of case review and investigation and toward more proactive reform efforts (Porter 2013; Harris 2012). Part of this trend has been visible since the 1990s with the development of auditor/monitor models of oversight (Bobb 2003). However, there has been a developing focus on adapting problem-oriented policing strategies for use in civilian oversight. Problem-oriented policing has been an important approach to law enforcement for more than 30 years. In this approach, patterns of crime and disorder are analyzed carefully to develop new programs that can reduce the incidence of the identified crime (Goldstein 2003; Braga 2008; Porter 2016). This approach to law enforcement is notable because it is designed to be both preventive and collaborative. That is, it is designed to reduce future crime problems and bring all of the stakeholders together to develop a solution (Goldstein 2003). Even though this strategy has been popular with law enforcement, it has not been widely adopted for use in civilian oversight of law enforcement. Recent research by Porter (2016) indicates that it is possible to adapt problem-oriented approaches to increase the effectiveness of civilian oversight .28 Often referred to as the SARA problem-solving model, this strategy has four stages: 27 In this context, a sustain rate would be calculated as the percentage of complaints in a given period that result in one or more sustained findings. 28 See also Walker (2001) for an early statement on the value of problem-oriented policing for civilian oversight of law enforcement. 46 Civilian Oversight of Law Enforcement Assessing the Evidence 1 . Scanning: A specific problem is identified; 2. Analyzing: Data is collected and analyzed so the breadth and causes of the problem can be identified; 3 . Responding: A potential solution for the problem is developed and implemented; and 4. Assessing: A rigorous follow-up examination is conducted to ensure the new program is working and to determine whether changes to the intervention are necessary (Braga 2008; Porter 2016: 172). In arguing for the potential value of the problem-solving model for oversight, Porter (2016) offers two case studies of oversight programs where the SARA model was used successfully. First, she reviewed a collaborative project on use of force undertaken by the Portland Independent Police Review Division (IPR) and the Portland Police Bureau. She reported that this collaborative task force conducted a deep analysis of uses of force and made 16 recommendations for changing the agency’s policies, procedures, practices and training around use of force. The task force then conducted a follow-up assessment and reported that uses of force, excessive force complaints and injuries to officers and community members had declined after the recommendations were implemented (Porter 2016: 178; citing to Force Task Force 2007 and 2009). In a second case study, Porter reviewed a collaborative problem-solving effort that focused on TASER use in Australia. Porter (2016) reported that the project, which was undertaken by the Queensland, Australia Crime and Misconduct Commission (CMC) and the Queensland Police Service (QPS), resulted in a measurable decline in the use of TASERs by the police over time (Porter 2016: 178). Expanding Alternative Dispute Resolution The use of mediation by civilian oversight to resolve police complaints has become an increasingly popular strategy over the last 15 years . The organizational data collected from 97 civilian oversight agencies for this project included information on whether they mediate complaints against the police. Altogether, 29 agencies reported that they had a mediation program, up from 20 agencies in 2008 (PARC 2008). Unlike traditional investigation-focused forms of complaint processing, mediation is often described as the informal resolution of a dispute through a confidential face-to-face meeting facilitated by a neutral third party (Walker and Archbold 2000). While different jurisdictions implement mediation in slightly different ways, one of the themes that ties almost all programs together is the goal of using facilitated, face-to-face meetings between police officers and community members to resolve less serious complaints, such those that involve discourtesy or misunderstandings about police procedure (Walker et al. 2002; PARC 2008; Schaible et al. 2013). Underlying this theme is the belief that mediation can help overcome some of the limitations of traditional police complaint handling procedures for certain kinds of cases. For example, traditional complaint investigations processes for less serious complaints can be inefficient, rarely result in sustained findings or the imposition of discipline and tend to leave both complainants and officers highly dissatisfied (PARC 2008). In contrast, mediation may be a more cost-effective and efficient method of resolving minor complaints than more traditional complaint investigation processes (Walker et al. 2002). Moreover, the use of mediation is designed to help agencies improve officer and complainant satisfaction by bringing community members and police officers together in a non-adversarial setting to discuss their concerns (Quinn 2006). There is a growing body of research that provides support for the argument that mediation can have some positive benefits for complainants and officers, especially in relation to complaints alleging less serious misconduct (Walker et al. 2002; PARC 2008: 45). Schaible et al. (2013) used attitudinal survey data collected in one jurisdiction to compare patterns in attitudes between complainants and officers who went through a mediation process to a comparable set of complainants and officers whose complaints were handled through a traditional investigation process. The study found that both police and community participants in the mediation program reported significantly higher levels of satisfaction than those who went through Civilian Oversight of Law Enforcement Assessing the Evidence 47 the traditional process. Moreover, the study found that mediation had even stronger effect on Latino complainants and female police officers. For local jurisdictions considering implementing a mediation program, several U.S. oversight agencies operate well-known and long-running community-police mediation programs. Each of these agencies publish detailed, useful information about the philosophy and operation their individual programs (Table 5 for links to each program’s mediation webpage). Table 5 . U .S . Civilian Oversight Agencies that Operate Model Mediation Programs Jurisdiction New York, NY San Francisco, CA Washington, D.C. Denver, CO Oversight Agency Civilian Complaint Review Board Office of Citizen Complaints Office of Police Complaints Office of the Independent Monitor Mediations Mediations Sworn Completed per 1,000 Officers in 2015 Officers Agency Website: 34,500 205 5 .9 www.nyc.gov/html/ccrb/html/ mediation/mediation .shtml 2,208 45 20.4 www .sfgov .org/occ/mediation 3,789 42 11 .1 1,442 38 26.4 www .policecomplaints .dc .gov/ service/mediation-service www .denvergov .org/content/ denvergov/en/office-of-the-independent-monitor/mediation.html Note: This table and mediation data were adapted from The Denver Office of the Independent Monitor’s (2016), 2015 Annual Report (pp. 45). 48 Civilian Oversight of Law Enforcement Assessing the Evidence Conclusions and Recommendations To help municipal executives, policy-makers, community leaders and the general public consider some of the important issues currently facing police oversight, this report conducted a broad review of the current state of knowledge regarding civilian oversight of law enforcement in the U.S., drawing from a review of the research literature on oversight and organizational data provided by 97 police oversight executives. There has been a fair amount of writing on civilian oversight and it is currently a topic of substantial national interest. Unfortunately, there has been far less systematic, comparative social science research on the effectiveness of oversight. That said, there are a number of resources that can act as a jumping-off point for jurisdictions considering implementing civilian oversight or who are evaluating the effectiveness of their current oversight mechanisms. The early seminal work by Walker (2001) and Finn (2001) still stand as critical resources for jurisdictions weighing the costs and benefits of implementing civilian oversight. More recent research by Prenzler and his colleagues (Prenzler and den Heyer 2016) update what is known about the state of the field and offer useful information about novel developments outside of the U.S. Finally, there have been a series of recent program evaluations of agencies that consider some of the strengths and weaknesses of individual civilian oversight agencies in Chicago (Police Accountability Task Force 2016), King County (Anderson et al. 2015), New Orleans (PARC 2016), San Francisco (2016) and Spokane County (Olson 2016). Given that there has been relatively little research on civilian oversight, this project drew strongly from data provided by civilian oversight agencies to identify important issues within the field. Using those data, it was possible to catalogue a number of key issues that jurisdictions may want to consider when evaluating their oversight mechanisms. Oversight Is Common in the U .S . These patterns, when taken as a whole, seem Patterns in the development and persistence of oversight indicate that civilian to indicate a field that is approaching oversight has been increasingly institutionalized as a regular feature of policing maturity. Large jurisdictions that do not have in cities and counties across the U.S. In 1975, there were seven oversight agencies oversight now stand as outliers, rather than in the U.S. That figure grew to over 100 agencies by 2000 (Walker 2001: 6). For representing the norm. this report, we identified more than 140 civilian oversight agencies, with almost all large cities having some sort of civilian oversight mechanism in place . More critically, roughly 55 percent of the oversight agencies that provided data for this report indicated that they had been in existence for 16 or more years. These patterns, when taken as a whole, seem to indicate a field that is approaching maturity. Large jurisdictions that do not have oversight now stand as outliers, rather than representing the norm. Oversight Typically Develops Through Local Crises The history of oversight in the U .S . demonstrates that the early models of oversight emerged in response to local crises relating to conflict between the police and local communities of color, often catalyzed by controversial officer-involved shootings, in-custody deaths or concerns about racially disparate policing (Walker 2001). Few historical examples of oversight exist of proactive implementation. The data provided by civilian oversight agencies for this report supports this view. Only 11 percent of the agencies reported proactive implementation of their oversight system. While not common, this form of development suggests that it is possible for local jurisdictions to implement civilian oversight in a proactive versus reactive manner. Oversight Is a Complex Organizational Field In looking at the historical evolution of oversight, it is clear that earlier forms of oversight tended to be organized around the review of completed internal affairs investigations. The field has evolved considerably, however. Review-focused agencies Civilian Oversight of Law Enforcement Assessing the Evidence 49 were the most common among the respondents to provide data (40 percent), but auditor/monitor and investigation-focused agencies are now common in the field (25 percent and 35 percent, respectively). This means that local jurisdictions have a much broader range of options when considering the characteristics of the oversight mechanism they would like to implement. In the data shared by the civilian oversight agencies, it was clear that each of the models identified tended to have common organizational characteristics and forms of authority . • Review-focused agencies, on the whole, were more likely to report that they had jurisdiction in relation to the review of completed internal affairs investigations. • Investigative agencies were more likely to report that they had authority to decide how to handle citizen complaints and conduct independent investigations . • Auditor/monitor agencies were more likely to report having strong authority in relation to reviewing, auditing and monitoring investigations, as well as authority to analyze patterns and make recommendations across a range of police/sheriff data . Even though the oversight models tended to have clear organizational foci, there was substantial crossover among models. For example, a majority of auditor/monitor agencies (61 percent) reported the ability to conduct independent investigations.29 Roughly 43 percent of review-focused agencies and 50 percent of investigative agencies reported having the authority to audit complaint investigations by the police. Taken as a whole, this indicates that jurisdictions are mixing-and-matching forms of oversight authority and jurisdiction to meet the needs in the local environment (McDevitt et al. 2005: 11). As such, regardless of whether a local jurisdiction has a review board or auditor/monitor, it is clear that just about any type of oversight model can include hybridized characteristics. There is Clear Variation in the Budgets Among Oversight Types In general, the data provided by the civilian oversight agencies demonstrated that some models of oversight have larger budgets than others. Full investigative agencies tended to report the largest budgets, followed by auditor/monitor and review agencies. Similar patterns were evident in terms of staffing. Investigative agencies reported having the largest number of full-time, paid staff while review-focused agencies were the least likely to report having full-time paid staff. This is likely due in part to variation in the size of jurisdictions and variation in the mandate of the different models. Investigative and auditor/ monitor agencies were much more likely to be found in jurisdictions with large populations and big police agencies. Even so, jurisdictions need to carefully evaluate the trade-offs between different types of oversight agencies. Review-focused agencies are the least expensive agencies to operate. Yet, the pattern in data provided by the civilian oversight agencies indicate that review-focused agencies also tend to have the most limited range of organizationEven though a large majority al authority . Local jurisdictions will need to carefully match their goals for oversight to the of all oversight agencies reportresources they have available when considering the type of oversight to implement. ed that police executives listened Some Models of Oversight May be More Effective at Getting to their recommendations, Recommendations Implemented auditor/monitor agencies were more likely to report that law Almost all of the oversight agencies reported that police executives listened carefully to the enforcement frequently implerecommendations made by oversight staff (78 percent). However, auditor/monitor agencies mented their recommendations. were much more likely to report that police or sheriff’s agencies implemented their recom29 Olson and Attard (2016), in a review of oversight models in California, found that auditors were often granted the legal authority to conduct individual investigations, but seldom exercised that authority. 50 Civilian Oversight of Law Enforcement Assessing the Evidence mendations very frequently or frequently (72 percent) as compared to investigative (42 percent) and review agencies (34 percent). It is critical to note that some caution is warranted when it comes to drawing conclusions from this finding. These are self-report data and thus only measure the perceptions of the oversight executives who provided input. Part of the difference, however, may relate to the type of work different agencies tend to perform. Auditor/monitor agencies are more likely to have specialized staff who focus on analyzing patterns in complaints and other police data and they are more likely to issue special topics reports as a result of that work (which Harris (2012) describes as a “holistic’ approach). It is a possibility that using data-driven evaluation methods increases the likelihood that police departments will implement recommendations made by oversight. More systematic comparative research is needed; however, before strong inferences can be drawn on this issue. Oversight Can Serve as a Conduit of Positive Information about Police Department Performance Even though much of the public reporting on police accountability and oversight tends to focus on instances of wrong-doing or misconduct on the part of individual officers, police oversight agencies can also serve as an important conduit of information to the public about the positive forms of police-community interaction. A number of jurisdictions actively seek to use their public reporting frameworks to highlight circumstances where police officers have engaged in outstanding public service. For example, Albuquerque’s Civilian Police Oversight Agency (2015) and Denver’s Office of the Independent Monitor (2015) developed sections of their annual reports to highlight commendations and other notable positive forms of police-community interaction . Civilian Oversight of Law Enforcement Assessing the Evidence 51 Important Considerations when Implementing or Reforming a Local Oversight Program Over the past 30 years, local experimentation with different types of oversight models, to include the hybridization of these different models, has result in a complex heterogeneous organizational field. While the data provided by 97 civilian oversight agencies included in this report explores organizational variation across different oversight agencies, it cannot answer the fundamental question of: • Which forms of oversight are the most effective? • Under what circumstances should a jurisdiction implement a review-focused model of oversight as opposed to an investigative or auditor/monitor-focused model? Even though the question of what type of model constitutes a “best” form of oversight remains unanswered, that does not mean that nothing can be learned from these patterns. In fact, the growing hybridization of police oversight and the blurring of the boundaries between different models of oversight by itself carries with it an important lesson for local jurisdictions that are exploring whether to implement oversight or considering revising their current oversight framework. Jurisdictions Should Focus on “Best-Fit” Rather Than “Best Practices” When Considering How to Structure Civilian Oversight “Evidence that that any one civilian oversight approach or mechanism is more effective than another does not yet exist, although the role and authority of a civilian oversight function often grows over time to meet emerging community needs and expectations.” (Anderson et al. 2015: 3) One key lesson that can be learned from the history of oversight in the U.S. is that there is not necessarily any “best practice” in the creation of civilian oversight of a law enforcement program, but rather what a jurisdiction should be looking for is a “best fitting” model of oversight. Every jurisdiction has its own social, cultural and political issues and every police agency has its own unique organizational, history, traditions and sub-cultural characteristics. While some police agencies may be proficient at holding their officers to account with respect to certain types of conduct, other police agencies may struggle. Some large jurisdictions have ample financial resources to implement highly professionalized, organizationally complex forms of oversight while smaller jurisdictions may have far fewer resources with which to implement and sustain police oversight . Given these differences between cities and counties in the U.S., it is likely that no single model of oversight is going to work for all jurisdictions. As a result, the best form of oversight for individual jurisdictions simply depends on the circumstances faced by the jurisdiction that is either creating or updating its oversight processes. This is why it is important to review the strengths and challenges of a police department before selecting an oversight model. Oversight Should Employ the Least Force Necessary to Accomplish its Goals Even though law enforcement resistance to the concept of police oversight has diminished over time, it can still be argued that “the least intrusive means of oversight” (Bobb 2003) necessary to achieve police accountability is the best means of approaching the oversight function in the long term. Just as we expect the police to use only that amount of force that is proportionate, necessary and reasonable to accomplish their task, so it can be argued that jurisdictions creating or reforming an oversight function should similarly accomplish the feat of ensuring police accountability (Bobb 2003: 165). In other words, a jurisdiction seeking to create or update an oversight function should choose the least intrusive model of oversight necessary to accomplish the task. If the model chosen does not accomplish that objective, a more aggressive form of oversight would then be required. As such, it is impossible to suggest that any one model of oversight is better than another. Each jurisdiction must evaluate its own police 52 Civilian Oversight of Law Enforcement Assessing the Evidence agency, including its culture, leadership, capacity to police itself before choosing the most appropriate form of oversight. A Number of Resources are Available to Jurisdictions Considering Implementing Oversight or Reforming their Current Oversight Framework One of the key challenges for local jurisdictions considering whether to implement oversight is to find examples of jurisdictions that have successfully implemented and sustained effective oversight agencies. It can also be difficult and resource intensive for local jurisdictions to collect examples of legal language, organizational procedures and other “nuts-and-bolts documents that they can use as models after they decide to implement oversight . Several relatively recent reports have sought to overcome these problems by providing detailed cases studies of existing oversight agencies (PARC 2005; McDevitt et al. 2005; Noe 2013; Olson 2016; PARC 2016). These reports contain key details about oversight agency powers, organizational, structure, funding and staffing and should be consulted by local jurisdictions considering oversight or interested in reforming their local oversight agency . Recommendations for Further Research The following recommendations, which emerged from the review of the literature and organizational data provided by civilian oversight agencies, are intended to identify ways to expand what is currently known about civilian oversight of law enforcement. To help local jurisdictions gain access to examples of oversight policies, legal language and key organizational documents, the National Association for Civilian Oversight of Law Enforcement (NACOLE) has created a companion website to this report that includes up-to-date profiles for model police oversight agencies. This website toolkit includes examples of ordinance/charter language, oversight policies and procedures, annual reports, special topics reports, complaint forms, outreach brochures and examples of other documents that can serve as examples for new oversight agencies. This website toolkit can be accessed by visiting: www.nacole.org/agency_profiles. 1 . More systematic, comparative research needs to be conducted on the effectiveness of civilian oversight of law enforcement. While some researchers are working to extend what we know about oversight (see Walker and Archbold 2014; Prenzler and den Heyer 2016; Alpert et al. 2016), there are still tremendous gaps in what we know about the issue of effectiveness in civilian oversight. Moreover, we still know little about the relative effectiveness of different models, strategies and approaches. While the data provided by nearly 100 civilian oversight agencies helps to identify and examine broad differences among models of oversight, it does not allow for the measurement of differences in the impact that different types of oversight agencies have on police accountability, police effectiveness or patterns in police misconduct. It is clear that more systematic, empirical research is needed before we will be able to draw strong conclusions about the organizational factors that determine effectiveness in citizen oversight. 2. Additional research is needed on the impact that citizen oversight has on public trust in the police and local government. Civilian oversight often emerges as a result of conflict between the police and communities of color and is typically catalyzed by one or more local police scandals (e.g., racially biased policing, officer-involved shootings) (Walker, 2001, p. 55). Moreover, oversight is often implemented with the goal of improving community trust and confidence in the police and local government. While some scholars have begun to study the relationship between civilian oversight and public opinion (Tarling and Dowds 1997; Worden and McLean 2010; Prenzler 2016; De Angelis and Wolf 2016), we still know little about the impact that the implementation of civilian oversight has on public opinion. We also know little about the extent to which the presence of civilian oversight can repair the damage to public confidence caused by instances of police misconduct. More research is needed on the impact of civilian oversight on public trust in policing and local government . 3 . Future comparative evaluations of effectiveness need to be based on primary data collected by outside researchers. Prenzler and Lewis (2005) and Ferdik et al. (2013) have conducted some of their most important work by reviewing publicly-avail- Civilian Oversight of Law Enforcement Assessing the Evidence 53 able agency reports and documents. However, the lack of standardization in reporting and measurement among agencies makes it difficult to compare across jurisdictions. Future research should adopt mixed-methods methodological approaches that combine the use of quantitative outcome measurement with qualitative interviews with community members and officers to generate forms of data that allow for comparison across jurisdictions (Ferdik et al 2013:18). 4. More support for independent research on oversight will be necessary before key questions relating to the effectiveness of oversight can be answered. Federal and state agencies that support applied research in law enforcement should consider developing new and sustained funding streams for research into civilian oversight of police (Buchner 2015). The same applies to private funding sources that support social science research and study . 5 . More work on standards for oversight is needed. Professional oversight associations, like NACOLE, should continue to work collaboratively with researchers and other oversight stakeholders to develop standards and guidelines for evaluating oversight . 6 . Local jurisdictions should consider participating in the “open data” movement. Over the last several years, a number of jurisdictions have begun to make some of their non-identifying police-related data available to the public, including data related to complaints against officers, critical incidents and other data relating to police operations and activity. The goal of this has been to build public trust in the police and local government by increasing the transparency of police operations and local government. For example, the Indianapolis Metropolitan Police Department and Citizen’s Police Complaint Office collaborated with a local non-profit, Code for America, to create a web portal through which members of the public can download datasets relating to officer uses of force, officer-involved shootings and complaints filed by members of the public (https://www.projectcomport.org). 7 . Local jurisdictions should be willing to evaluate the effectiveness of their local oversight agencies proactively and independently. In conducting those evaluations, jurisdictions need to have realistic expectations when it comes to the impact civilian oversight may have on local police accountability. Oversight is—and should be—only one component of a jurisdiction’s police accountability framework. As such, local jurisdictions should seek to evaluate effectiveness based on the factors over which local oversight agencies have control. As the IACP noted: “Citizen review is but one tool among many that can be used to promote and ensure accountability. It is neither a cure-all not likely to promote desired results unless accompanied by a full package of accountability-building strategies” (IACP 2000: 2). 54 Civilian Oversight of Law Enforcement Assessing the Evidence References Ad Hoc Police Oversight Task Force. 2014. “Report on the Activities of the Ad Hoc Police Oversight Task Force.” Albuquerque City Council . http://www.cabq.gov/council/documents/police-oversight-task-force-documents/POTF%20Report_ Final%20013114.pdf Alpert, Geoffrey P., Tyler Cawthray, Jeff Rojek and Frank V. Ferdik. 2016. “Citizen Oversight in the United States and Canada: Applying Outcome Measures and Evidence-Based Concepts.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, 179–204. New York: CRC Press. Anderson, Justin, Larry Brubaker, Sean DeBlieck, Brooke Leary and David Dean. 2015. Law Enforcement Oversight: Limited Independence, Authority & Access to Information Impede Effectiveness. King County, Washington: King County Auditor’s Office. Attard, Barbara and Kathryn Olson. 2013. “Oversight in the United States.” http://nacole.org/wp-content/uploads/Oversight-in-the-United-States-Attard-and-Olson-2013.pdf Bayley, David. 1991. “Preface.” In Complaints Against the Police: The Trend to External Review, edited by Andrew Goldsmith, v–vii. Oxford: Clarendon. Beattie, Cheryl and Ronald Weitzer. 2000. “Race, Democracy and Law: Civilian Review of Police in Washington, DC” (From. In “Civilian Oversight of Policing: Governance, Democracy and Human Rights,” edited by Andrew Goldsmith and Colleen Lewis, 41–61. Portland, OR: Hart Publishing. Beck, Charlie. 2015. “Civilian Oversight of the Police in Los Angeles: Collaboration, Activism and Outreach.” In President’s Task Force on 21st Century Policing Listening Session. U.S. Department of Justice. http://www.cops.usdoj.gov/pdf/taskforce/ submissions/Beck-Charlie-Testimony.pdf Beral, Harold and Marcus Sisk. 1964. “The Administration of Complaints by Civilians against the Police.” Harvard Law Review 77 (3): 499–519. doi:10.2307/1339031. Berg, Julie and Simon Howell. 2016. “Civilian Oversight of Police in Africa: Trends and Challenges.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, edited by Tim Prenzler and Garth den Heyer, 121–38. http:// www.tandfonline.com/doi/abs/10.1080/15614263.2013.767093 . Bobb, Merrick. 2003. “Civilian Oversight of the Police in the United States.” Saint Louis University Public Law Review 22: 151 . Bobb, Merrick, Michael Bromwich, Gary Blackmer, Ralph Godbee, Michael Graham, Richard Rosenthal, Jeff Schlanger and Julio Thompson. 2008. “National Guidelines for Police Monitors.” Police Assessment Resource Center. Braga, Anthony. 2008. Problem-Oriented Policing and Crime Prevention. Monsey, NY: Criminal Justice Press. Brereton, David. 2000. “Evaluating the Performance of External Oversight Bodies.” In Civilian Oversight of Policing: Governance, Democracy and Human Rights, edited by Andrew Goldsmith and Colleen Lewis, 105–24. Portland, OR: Hart Publishing. Buchner, Brian. 2015. NACOLE President Brian Buchner’s Oral Testimony for the President’s Task Force on 21st Century Policing, Policy and Oversight Listening Session, in Cincinnati, Ohio. National Association for Civilian Oversight of Law Enforcement . Civilian Oversight of Law Enforcement Assessing the Evidence 55 Chanin, Joshua M. 2015. “Examining the Sustainability of Pattern or Practice Police Misconduct Reform.” Police Quarterly 18 (2): 163–92. doi:10.1177/1098611114561305. Civilian Police Oversight Agency. 2016. “2015 Annual Report.” City of Albuquerque, New Mexico. www.cabq.gov/cpoa/ documents/oc-16-13-cpoa-2015-annual-report.pdf Clarke, Stephen. 2009. “Arrested Oversight: A Comparative Analysis and Case Study of How Civilian Oversight of the Police Should Function and How It Fails.” Columbia Journal of Law and Social Problems 43 (1): 1–49. Davis, Robert C., Christopher Ortiz, Nicole Henderson, Joel Miller and Michelle Massie. 2002. Turning Necessity into Virtue: Pittsburgh’s Experience with a Federal Consent Decree. New York: Vera Institute of Justice. www .cops .usdoj .gov/ html/cd_rom/inaction1/pubs/TurningNecessityintoVirtue.pdf De Angelis, Joseph and Brian Wolf. 2016. Perceived accountability and public attitudes toward local police. Criminal Justice Studies, 0(0), 1–21. doi.org/10.1080/1478601X.2016.1158177 den Heyer, Garth and Alan Beckley. 2016. “Police Independent Oversight in Australia and New Zealand.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, edited by Tim Prenzler and Garth den Heyer, 205–29. New York: CRC Press. Farrow, Joe and Trac Pham. 2003. “Citizen Oversight of Law Enforcement: Challenge and Opportunity.” The Police Chief 70 (10): 22–29. Faull, Andrew. 2013. “Monitoring the Performance of Police Oversight Agencies.” APCOF Policy Paper 8. http://www. apcof.org/files/185_MonitoringPerformanceofOversight.pdf Ferdik, Frank V., Jeff Rojek and Geoffrey P. Alpert. 2013. “Citizen Oversight in the United States and Canada: An Overview.” Police Practice and Research 14 (2): 104–16. doi:10.1080/15614263.2013.767089. Filstad, Cathrine and Petter Gottschalk. 2011. “Performance Evaluation of Police Oversight Agencies.” Policing and Society 21 (1): 96–109. doi:10.1080/10439463.2010.540653. Finn, Peter. 2001. Citizen Review of the Police: Approaches and Implementation. Office of Justice Programs, National Institute of Justice . Force Task Force. 2007. Use of Force by the Portland Police Bureau: Analysis and Recommendations. Portland, OR: Force Task Force to Chief of Police Rozie Sizer. ———. 2009. Use of Force by the Portland Police Bureau Follow-Up: Progress Report and Analysis of Further Data. Portland, OR: Force Task Force to Chief of Police Rozie Sizer. Goldsmith, Andrew John. 1991. Complaints against the Police: The Trend to External Review. Clarendon Press. Goldstein, Herman. 2003. “On Further Developing Problem-Oriented Policing: The Most Critical Need, the Major Impediments and a Proposal.” Crime Prevention Studies 15: 13–47. Gordon-Reed, Annette. 1995. “Watching the Protectors: Independent Oversight of Municipal Law Enforcement Agencies.” New York Law School Law Review 40: 91–105. Harris, Frank. 2012. “Holding Police Accountability Theory to Account.” Policing 6: 240–49. doi:10.1093/police/pas003. Hatfield, Jerry. 1994. Developing Performance Measures for Criminal Justice Programs. Department of Justice, Bureau of Justice Assistance . Retrieved from https://www.ncjrs.gov/pdffiles1/Digitization/148453NCJRS.pdf 56 Civilian Oversight of Law Enforcement Assessing the Evidence Hudson, James R. 1971. “Police Review Boards and Police Accountability.” Law and Contemporary Problems 36 (4): 515. doi:10.2307/1190933. Independent Police Complaints Commission. 2015. “Access to the Police Complaints System.” Independent Police Complaints Commission . https://www.ipcc.gov.uk/sites/default/files/Documents/research_stats/Access_to_the_police_complaints_system.pdf. Independent Police Review Division. 2002. Annual Report. Portland, OR: Office of the City Auditor. _____. 2003. Annual Report. Portland, OR: Office of the City Auditor. International Association of Chiefs of Police. 2000. Police Accountability and Citizen Review. International Association of Chiefs of Police . http://www.iacp.org/Police-Accountability-and-Citizen-Review Ivkovic, Sanja. 2005. Fallen Blue Knights: Controlling Police Corruption. Oxford: Oxford University Press. https://global. oup.com/academic/product/fallen-blue-knights-9780195169164 . Jones, Richard. 1994. “Processing Civilian Complaints: A Study of the Milwaukee Fire and Police Commission.” Marquette Law Review 77: 505–19. King, Kevin. 2015. “Effectively Implementing Civilian Oversight Boards to Ensure Police Accountability and Strengthen Police-Community Relations.” Hastings Race and Poverty Law Journal 12 (91–259). https://litigation-essentials.lexisnexis. com/ Lewis, Colleen. 2000. “The Politics of Civilian Oversight: Serious Commitment or Lip Service.” In Civilian Oversight of Policing: Governance, Democracy and Human Rights, edited by Andrew Goldsmith and Colleen Lewis, 19–41. Portland: Hart Publishing. Livingston, Debra. 2004. “The Unfulfilled Promise of Citizen Review.” Ohio State Journal of Criminal Law 1 (2): 653–67. McDevitt, Jack, Amy Farrell and W Andresen. 2005. “Enhancing Citizen Participation in the Review of Complaints and the Use of Force in the Boston Police Department.” Institute on Race and Justice, Northeastern University. http://www.nlg-npap. org/sites/default/files/Northeasternreport12-05.pdf McDonald, D. 1981. Royal Commission on Inquiry Into Certain Activities of the Royal Canadian Mounted Police. Ottawa: Canadian Publishing Government Center. Miller, Joel. 2002. “Civilian Oversight of Policing: Lessons from the Literature.” Vera Institute of Justice. http://vera.org/ sites/default/files/resources/downloads/Civilian_oversight.pdf Minneapolis Civilian Review Authority. 1997. “Redesign Team Report.” Minneapolis Civilian Review Authority. Mohr, Beth. 2007. The Use of Performance Measurement in Civilian Oversight in the United States. Retrieved from http:// www.civilianoversight.net/NACOLE-Perf_mgmt.pdf Nalla, Mahesh. 2016. “Police Accountability and Citizen Oversight in Emerging Democracies in Asia.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, edited by Tim Prenzler and Garth den Heyer, 139–58. http:// www.tandfonline.com/doi/abs/10.1080/15614263.2013.767093 . National Association for Civilian Oversight of Law Enforcement. 2015. “Building Legitimacy and Public Trust through Civilian Oversight: Written Testimony for the President’s Task Force on 21st Century Policing.” National Association for Civilian Oversight of Law Enforcement . Civilian Oversight of Law Enforcement Assessing the Evidence 57 Noe, George. 2013. Findings on Models of Civilian Police Oversight Authorities. Aurora, CO: Office of the City Manager. Ofer, Udi. 2016. “Getting It Right: Building Effective Civilian Review Boards to Oversee Police.” Seton Hall Law Review 46 (4). http://scholarship.shu.edu/shlr/vol46/iss4/2 . Office of Citizen Complaints. 2016. “Frequently Asked Questions.” In. San Francisco, CA: Office of Citizen Complaints. www.sfgov.org/occ/frequently-asked-questions Office of the City Auditor. 2001. Addressing Citizen Complaints about Police: A Proposal for Change. Portland, OR: Office of the City Auditor . Olson, Kathryn. 2016. “Citizen Advisory/Review Board Spokane County Sheriff’s Office: Oversight Review.” Change Integration Consulting, LLC. http://www.spokesman.com/documents/2016/may/16/spokane-county-sheriffs-office-citizen-advisory-bo/ . Office of the Independent Monitor. 2016. 2015 Annual Report. Denver, CO: City & County of Denver, Office of the Independent Monitor . Office of the Inspector General. n.d. About Us. http://www.oig.lacity.org/about-us Olson, Kathryn and Barbara Attard. 2016. “Analysis of Police Oversight Models for the City of Pasadena.” Change Integration Consulting, LLC. pasadenanow .com/documents/models .pdf Perez, Douglas. 1995. Common Sense Police Review. Temple University Press. Perino, Justina. 2006. Citizen Oversight of Law Enforcement Agencies. American Bar Association. http://shop.americanbar. org/eBus/Store/ProductDetails.aspx?productId=214974 . Police Accountability Task Force. 2016. Police Accountability Task Force Full Report: Restoring Trust between the Chicago Police and the Communities They Serve. Chicago, IL: City of Chicago, Illinois. https://chicagopatf.org/wp-content/ uploads/2016/04/PATF_Final_Report_4_13_16-1.pdf Police Assessment Resource Center. 2005. “Review of National Police Oversight Models for the Eugene Police Commission.” Police Assessment Resource Center . http://www.parc.info/eugene _____. 2008. Evaluation of a Pilot Community Policing Program: The Pasadena Police-Community Mediation and Dialog Program. Washington, D.C.: Office of Community Oriented Policing Services, U.S. Department of Justice. ———. 2016. “Peer Review of the New Orleans Office of Independent Police Monitor.” Police Assessment Resource Center. http://www.parc.info/blog/2016/2/27/parc-completes-peer-review-of-new-orleans-office-of-independent-police-monitoroipm . Porter, Louis. 2016. “Police Professionalism Units and External Oversight Agencies: Can There Be Productive Collaboration?” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, edited by Tim Prenzler and Garth den Heyer, 229–50. http://www.tandfonline.com/doi/abs/10.1080/15614263.2013.767093 . Prenzler, Tim. 2000. “Civilian Oversight of Police: A Test of Capture Theory.” British Journal of Criminology 40 (4): 659–74. doi:10.1093/bjc/40.4.659. ———. 2016. “Democratic Policing, Public Opinion and External Oversight.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, edited by Tim Prenzler and Garth den Heyer. CRC Press. 58 Civilian Oversight of Law Enforcement Assessing the Evidence Prenzler, Tim and Garth den Heyer. 2016. Civilian Oversight of Police: Advancing Accountability in Law Enforcement. CRC Press . Prenzler, Tim and Colleen Lewis. 2005. “Performance Indicators for Police Oversight Agencies.” Australian Journal of Public Administration 64 (2): 77–83. doi:10.1111/j.1467-8500.2005.00443.x. Prenzler, Tim and Carol Ronken. 2001. “Models of Police Oversight: A Critique.” Policing and Society 11 (2): 151–80. doi :10.1080/10439463.2001.9964860. President’s Task Force on 21st Century Policing. 2015. Final Report of the President’s Task Force on 21st Century Policing. Washington, D.C.: Office of Community Oriented Policing Services. Savage, Stephen. 2013. Thinking Independence Calling the Police to Account through the Independent Investigation of Police Complaints. British Journal of Criminology, 53(1), 94–112. doi.org/10.1093/bjc/azs056 Savage, Stephen. 2016. “Independent Minded: The Role and Status of Independence in the Investigation of Police Complaints.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, 29–50. New York: CRC Press. Schaible, Lonnie, Joseph De Angelis, Brian Wolf, and Richard Rosenthal. 2013. Denver’s Citizen/Police Complaint Mediation Program Officer and Complainant Satisfaction. Criminal Justice Policy Review, 24(5), 626–650. doi . org/10.1177/0887403412455327 Seneviratne, Mary. 2004. “Policing the Police in the United Kingdom.” Policing and Society 14 (4): 329–47. doi:10.1080/1 043946042000286056. Smith, Graham. 2013. “Oversight of the Police and Residual Complaints Dilemmas: Independence, Effectiveness and Accountability Deficits in the United Kingdom.” Police Practice and Research 14 (2): 92–103. doi:10.1080/15614263.2013.7 67088 . Smith, Megan and Roy Austin. 2015. “Launching the Police Data Initiative.” https://www.whitehouse.gov/blog/2015/05/18/ launching-police-data-initiative Stewart, Lori. 2006. “Community Outreach and Public Education in Citizen Oversight.” In Citizen Oversight of Law Enforcement : Legal Issues and Policy Considerations, 127–46. Chicago: ABA Publishing. Tal, Bruria. 2003. “Civilian Oversight of Police in Philadelphia: The First 50 Years.” http://www.phila.gov/pac/PDF/ HistoryofOversight .pdf Tarling, Roger and Lizanne Dowds. 1997. “Crime and Punishment.” In British Social Attitudes, 197–204. A;dershot: Ashgate. Terrill, Richard. 1988. “Police Accountability in Philadelphia: Retrospects and Prospects.” American Journal of Police 7 (2): 79–99. United States Kerner Commission. 1968. Report of the National Advisory Commission on Civil Disorders. Washington: United States, Kerner Commission. Walker, Samuel. 1977. Critical History of Police Reform: The Emergence of Professionalism. Lexington, Mass: Lexington Books. Walker, Samuel and Carol A. Archbold. 2000. “Mediating Citizen Complaints against the Police: An Exploratory Study.” Walker, Samuel, Carol Archbold and L Herbst. 2002. Mediating Citizen Complaints against Police Officers: A Guide for Police and Community Leaders. Washington, D.C.: US Department of Justice, Office of Community Oriented Policing Services . Civilian Oversight of Law Enforcement Assessing the Evidence 59 Walker, Samuel and Vic Bumphus. 1991. “Civilian Review of Police: A National Survey of the 50 Largest Cities.” Criminal Justice Policy Research Group 9 (13): 1–6. Walker, Samuel and Vic W. Bumphus. 1992. “The Effectiveness of Civilian Review: Observations on Recent Trends and New Issues Regarding the Civilian Review of the Police.” American Journal of Police 11: 1. Walker, Samuel E. 2003. “Core Principles for an Effective Police Auditor’s Office.” University of Nebraska. http://samuelwalker.net/wp-content/uploads/2010/06/coreprinciples.pdf Walker, Samuel E. and Carol A. Archbold. 2014. The New World of Police Accountability. 2nd ed. edition. Los Angeles: SAGE Publications, Inc. Walker, Samuel and Betsy Wright Kreisel. 1996. “Varieties of Citizen Review: The Implications of Organizational Features of Complaint Review Procedures for Accountability of Police.” American Journal of Police 15 (3): 65–88. doi:10.1108/07358549610129640. ———.2001. Police Accountability: The Role of Citizen Oversight. Belmont, CA: Wadsworth Publishing. ———. 2003. “Core Principles for an Effective Police Auditor’s Office.” University of Nebraska. http://samuelwalker.net/ wp-content/uploads/2010/06/coreprinciples.pdf ———. 2006. “The History of the Citizen Oversight.” In Citizen Oversight of Law Enforcement Agencies. ABA Book Publishing. _____. 2016. “The Interface between Human Rights and Police Complaints in Europe.” In Civilian Oversight of Police: Advancing Accountability in Law Enforcement, edited by Tim Prenzler and Garth den Heyer, 159–178. New York: CRC Press . Journal of Dispute Resolution 2 (2): 1–14. Wickersham, George. 1931. Enforcement of the prohibition laws: Official records of the National Commission on Law Observance and Enforcement pertaining to its investigation of the facts as to the enforcement, the benefits and the abuses under the prohibition laws, both before and since the adoption of the eighteenth amendment ot the Constitution. Washington: U.S. Govt. Print. Office . Wilson, Steve and Steve Buckler. 2010. “The Debate over Police Reform: Examining Minority Support for Citizen Oversight and Resistance by Police Unions.” American Journal of Criminal Justice 35: 184–97. Worden, John and Sarah McLean. 2010. Citizen Oversight of the Albany Police, 2010. Report to the Albany Law School Government Law Center. Albany, NY: John F. Finn Institute for Public Safety, Inc. About the Authors Joseph De Angelis, Ph.D. is an assistant professor in the Department of Sociology and Anthropology at the University of Idaho . He has also served as a policy director and an analyst for two civilian oversight agencies in the U .S . His research has appeared in a wide variety of scholarly and policy-focused journals, including the Journal of Criminal Justice, Criminal Justice Review, Criminal Justice Studies, Police Quarterly, Criminal Justice Policy Review, Qualitative Report and Policing: An International Journal of Police Strategies and Management . Richard Rosenthal is the first Chief Civilian Director for the Independent Investigations Office of British Columbia. He is currently enrolled in graduate studies at Simon Fraser University. Previously, he was the first director of the Independent Police Review Division of the Portland, Oregon City Auditor’s Office as well as the first Independent Monitor for the City and County of Denver. Mr. Rosenthal served 15 years as a deputy district attorney in Los Angeles County, specializing in the 60 Civilian Oversight of Law Enforcement Assessing the Evidence investigation and prosecution of public officials, judges and police officers. He has been credited for uncovering the LAPD Rampart Scandal in 2000. Mr. Rosenthal has held teaching positions at various universities, including Loyola Law School, Portland State University and the University of Colorado at Denver. He served for five years on the board of directors for the NACOLE and currently serves on the board of directors for the Canadian Association for Civilian Oversight of Law Enforcement (CACOLE). Brian Buchner is a Policy Director in Los Angeles Mayor Eric Garcetti’s Office of Public Safety. Prior to joining the Mayor’s Office, he was a Police Special Investigator with the Los Angeles Police Commission’s Office of the Inspector General, which provides civilian oversight to the LAPD. Before joining the OIG in 2007, Mr. Buchner was the Policing Specialist at PARC, where he assisted Merrick Bobb, former Special Counsel to the Los Angeles County Board of Supervisors and PARC’s Executive Director, in monitoring and critically reviewing the Los Angeles County Sheriff’s Department. Mr. Buchner is the current president of NACOLE, the nation’s largest and premier police oversight organization. Mr. Buchner also serves as an Adviser to the American Law Institute’s Principles of the Law, Police Investigations Project; as a member of the Open Police Complaints advisory board; and is a founding board member of the Integrated Recovery Network. He has spoken about policing and oversight issues in a variety of forums, including testifying before President Obama’s Task Force on 21st Century Policing and at United Nations hearings on torture and police accountability in Vietnam. Mr. Buchner holds a bachelor’s degree in criminal justice from Bowling Green State University and a master’s degree in criminology from the University of Missouri-St. Louis. Appendix A. Research Methods Since much of the recent research on civilian oversight has been based on either case studies or anecdotal accounts of individual agency practices, there are substantial gaps in what is known about the state of civilian oversight in the 21st century. To develop a better understanding of the state of civilian oversight in the U.S., the research team administered an electronic organizational survey to 97 civilian oversight agencies in the U.S. The overall goal of the survey was to capture and describe variation in the organizational structure, legal authority, organizational practices and financial resources for contemporary oversight in the U.S. Several of the attitudinal questions were adapted from early survey research on oversight (Walker and Bumphus 1991). The survey instrument contained 86 item groups that were designed to measure organizational and attitudinal information in relation to 12 key areas: • • • • • • • • • • • The history of the oversight agency . The legal authority/organizational powers granted to the agency . Agency goals organizational structure . Budget . Staffing. Presence and structure of a board or community members. Role in the complaint intake and investigation process. Community outreach practices . Perceived support from process stakeholders. Data collection and analysis practices . Key successes or obstacles that the agency has experienced in the previous three years. Civilian Oversight of Law Enforcement Assessing the Evidence 61 Prior to the distribution of the surveys, the authors first sought to identify local and state-level civilian oversight agencies currently operating in the U.S. For the purposes of this portion of the project, civilian oversight agencies were defined as governmental bodies that involve non-police community members in the review, auditing, monitoring or investigation of allegations of misconduct filed against law enforcement officers (see Walker 2001). Generating a list of oversight agencies proved to be a substantial challenge. There are currently no governmental or regulatory bodies that precisely track or maintain current lists of all oversight agencies in the U .S . The project team used several methods to identify civilian oversight agencies . 1 . Reviewed organizational membership lists for NACOLE; 2. Reviewed the academic literature and all available policy evaluation reports for references to oversight agencies; 3 . Used U.S. Census data to identify all U.S. cities with greater than 100,000 residents (N=295) (United States Census Bureau 2014). Once the list was developed, the research team conducted a series of key word searches of the Internet using Google and a combination of the city names and terms relating to civilian oversight. Reviewed the resulting websites to determine whether a civilian oversight agency was in operation and to collect contact information (when available). 4. Conducted key word searches of newspaper articles and other print media sources using Google News and LexisNexis Academic . 5 . Consulted subject matter experts knowledgeable about civilian oversight (e.g., individuals with experience working in the area of civilian oversight). Once an initial list of oversight agencies was developed, the team collected and stored all available substantive website content, including contact information, electronically. For those identified agencies where contact information for the oversight executive could not be determined, used emails and phone interviews to: 1 . Determine whether the oversight agency was in operation; and, 2. Collect the contact email information for the oversight executive (or his or her designee). Altogether, the team identified 144 oversight agencies using these methods. Twenty-one of those jurisdictions either refused to provide electronic contact information for the oversight agency or failed to respond to multiple email and phone call requests for contact information. The research team did not send surveys to four agencies because the contractor operating their oversight body performs oversight duties in another city (each contractor was surveyed only once). This left the research team with viable contact information for 119 civilian oversight agencies. Once the team identified the agencies, the team developed and administered the survey using Qualtrics™ survey research software. The University of Idaho Institutional Review Board reviewed and approved the survey research design, protocol and protections for human subjects. Each oversight agency received up to five separate contacts. Each agency received an initial notification email from the president of NACOLE that explained the nature, goals and likely outcome of the survey. The oversight agencies then received a solicitation email with a link to the survey three days later. The team delivered three additional solicitations via email to non-responding agencies. After the fourth solicitation, the research team and NACOLE board members and staff placed a final round of follow-up phone calls to the non-responding agencies with a request for their participation. In total, 97 of the 119 oversight agencies that received a survey solicitation submitted an electronic survey, resulting in a response rate of 81.5 percent. Once the team received the survey data, they extracted the data from Qualtrics™ and analyzed it in SPSS . When reviewing the results of the survey, it is important to recognize several important limitations. First, even though the team identified 144 oversight agencies, the research team was unable to distribute surveys to 25 of those agencies. In addition, 62 Civilian Oversight of Law Enforcement Assessing the Evidence another 22 agencies received, but did not submit, a survey. This indicates that some caution should be used when attempting to draw strong inferences about the larger population of civilian oversight agencies in the U.S. Second, the team distributed the electronic survey to only one representative from each agency. It is not clear whether the attitudes reported by the agency representatives reflect the attitudes of other employees or volunteers working for those jurisdictions. Appendix B. Statistical Results Table B1: Percentage of Organizational Goals Rated as “Critical” or “Important” by Oversight Agencies Auditor/ Investigative Review All Agencies monitor 100% 97% 97% 98% Ensure that complaint processes are accessible to all members of the community 87% 100% 91% 93% Ensure thorough investigations of complaints 87% 97% 82% 89% Increase transparency of the complaint investigation system 91% 85% 85% 87% Deter police misconduct 91% 91% 74% 85% 74% 85% 71% 77% 100% 62% 71% 75% 70% 91% 65% 76% 87% 85% 52% 73% 87% 76% 38% 64% Issue public reports that describe patterns in complaints 73% 68% 44% 60% Ensure that officers who engage in misconduct are punished 61% 65% 44% 56% Improve police officer satisfaction with the complaint process 52% 53% 53% 53% Audit internal investigations for investigatory compliance 74% 47% 32% 48% 52% 56% 35% 47% 57% 29% 30% 37% 18% 26% 35% 28% 23 34 34 91 Improve public trust in the local police/sheriff department Ensure that individuals who experience police misconduct receive justice Make recommendations to the police about how to improve the investigations process Improve complainant satisfaction with the complaint process Look for patterns in complaints to identify the underlying causes of police misconduct Analyze patterns in police data to make recommendations for organizational improvement Use alternative dispute resolution techniques to resolve complaints (e.g., mediation) Use performance audits to strengthen police practices, policies or training Sustain a proper percentage of complaints Number of Agencies Civilian Oversight of Law Enforcement Assessing the Evidence 63 Table B1: Percentage of Organizational Goals Rated as “Critical” or “Important” by Oversight Agencies Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. Table B2: What Local Circumstances Triggered the Creation of your Oversight Agency? Auditor/ monitor Officer-Involved Shooting/Other Excessive Force Racially Disparate Policing (e.g., racial profiling) Proactive/Preventive Effort In-Custody Death Police Corruption Scandal Change by Legislative Body Sexual Misconduct Mishandled internal Affairs Investigations Dissatisfaction with Previous Oversight Agency Other Issue Number of Agencies Investigative Review 50% 25% 8% 13% 13% 0% 8% 4% 8% 21% 24 56% 38% 0% 6% 12% 12% 0% 0% 0% 9% 34 All Agencies 44% 23% 23% 10% 5% 13% 5% 3% 0% 13% 39 49% 29% 11% 9% 9% 9% 4% 2% 2% 13% 97 Table Notes: Agencies could select multiple responses, so percentages will sum to more than 100%. The table reports the valid percent and excludes missing values from the calculation . Table B3: What type of Law Enforcement Organization(s) do you Oversee? Auditor/ monitor Municipal Police Department County Sheriff Campus Law Enforcement Special Police Agency Probation Number of Agencies 67% 25% 4% 4% 4% 24 Investigative 85% 12% 3% 6% 3% 34 Review All Agencies 90% 13% 8% 3% 0% 39 82% 15% 5% 4% 2% 97 Table Notes: Agencies could select multiple responses, so percentages will sum to more than 100%. The table reports the valid percent and excludes missing values from the calculation . 64 Civilian Oversight of Law Enforcement Assessing the Evidence Table B4: Approximately how Many Sworn Officers Work for the Agencies you Oversee? Auditor/ monitor 500 or Fewer 501 to 1,000 1,001 to 2,000 2,001 to 3,000 3,001 to 4,000 4,001 or more Number of Agencies 29% 21% 29% 4% 0% 17% 24 Investigative 34% 22% 16% 13% 9% 6% 32 Review All Agencies 64% 21% 10% 0% 3% 3% 39 45% 21% 17% 5% 4% 7% 95 Review All Agencies 39% 6% 52% 3% 33 61% 9% 29% 1% 90 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Table B5: Who Leads your Oversight Agency? Auditor/ monitor Full-Time Paid Executive Part-Time Paid Executive Multi-Member Board Other Number of Agencies 78% 17% 4% 0% 23 Investigative 71% 6% 24% 0% 34 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Civilian Oversight of Law Enforcement Assessing the Evidence 65 Table B6: What is the Source of your Agency’s Authority? Auditor/ monitor County/Municipal Ordinance City/County Charter Both Ordinance/Charter State Statute Contract Executive Order Law Enforcement Policy City/County Policy Legal Agreement Other Number of Agencies Investigative 25% 29% 8% 8% 17% 4% 4% 4% 0% 0% 24 41% 34% 0% 13% 3% 6% 3% 0% 0% 0% 32 Review All Agencies 36% 38% 0% 13% 0% 0% 3% 3% 5% 3% 39 35% 35% 2% 12% 5% 3% 3% 2% 2% 1% 95 Table Notes: Agencies could select multiple responses, so percentages will sum to more than 100%. The table reports the valid percent and excludes missing values from the calculation . Table B7: What Type of Job Protection Does the Head of your Agency have? Auditor/ monitor At Will Contract Term of Office Removed by Legislature/City Council Civil Service Other Number of Agencies Investigative Review All Agencies 73% 18% 9% 9% 67% 11% 15% 7% 48% 17% 22% 9% 78% 15% 15% 8% 0% 18% 22 7% 0% 27 4% 17% 23 4% 21% 72 Table Notes: This question was only administered to oversight agencies who reported that they were led by a full or part-time executive. Agencies could select multiple responses, so percentages will sum to more than 100%. The table reports the valid percent and excludes missing values from the calculation . 66 Civilian Oversight of Law Enforcement Assessing the Evidence Table B8: Are Former Police Officers Allowed to Serve as your Agency Executive/Director? Auditor/ monitor Yes Under some circumstances No Number of Agencies 52% 33% 14% 21 Investigative 63% 33% 4% 27 Review All Agencies 32% 41% 27% 22 50% 36% 14% 70 Table Notes: This question was only administered to oversight agencies who reported that they were led by a full or part-time executive. The table reports the valid percent and excludes missing values from the calculation . Table B9: Percentage of Agencies Reporting that they “Always/Sometimes” have Jurisdiction in Specific Areas of the Personnel Investigations Process Auditor/ All Investigative Review monitor Agencies Citizen-Initiated Complaints Officer-Initiated Complaints Internal/ Supervisor Complaints Officer-Involved Shootings Other Serious Force In-Custody Deaths High Speed Pursuits Complaints against Non-Sworn Employees Criminal Investigations Tort Claims Number of Agencies 92% 83% 87% 86% 86% 86% 86% 67% 61% 52% 24 100% 49% 39% 76% 85% 69% 56% 36% 16% 38% 33 92% 50% 49% 71% 77% 66% 63% 20% 27% 23% 38 95% 58% 55% 77% 82% 72% 66% 38% 32% 36% 95 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. Civilian Oversight of Law Enforcement Assessing the Evidence 67 Table B10: Percent of Agencies Responding with a “Yes” or “Sometimes” that they Have Specific Forms of Authority in Relation to Citizen Complaints Auditor/ All Investigative Review monitor Agencies 57% 52% 94% 94% 39% 30% 63% 58% 48% 97% 27% 57% 96% 67% 81% 80% Send complaint investigations back to the police for further investigation 78% 53% 65% 64% Audit police complaint investigations for compliance with investigative standards 92% 50% 43% 58% 92% 61% 70% 61% 61% 4% 46% 52% 52% 23 52% 100% 84% 88% 53% 10% 39% 33% 33% 33 38% 32% 57% 54% 30% 3% 14% 16% 16% 37 57% 63% 70% 67% 46% 6% 31% 31% 31% 93 Classify the nature of a complaint Conduct an initial intake investigation on complaints Decide how a complaint will be handled (e.g., investigated, administratively closed, mediated) Review police complaint investigations (e.g., for thoroughness, completeness, accuracy) Monitor open police complaint investigations Conduct investigations that are independent of the police Recommend investigation findings to the police Issue findings on investigations Recommend discipline to the police Impose Discipline Attend police disciplinary hearings Attend police use of force hearings Assist in the operation of a police early intervention system Number of Agencies Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 68 Civilian Oversight of Law Enforcement Assessing the Evidence Table B11: Percentage of Agencies Reporting that they “Always/Sometimes” Have the Following Types of Authority in Relation to Police Records? Auditor/ All Investigative Review monitor Agencies Access to CLOSED internal affairs police files Access to ALL internal affairs police files Access to internal affairs electronic databases Access to all electronic police databases Access to Body Worn/In-Car Video Right to subpoena witnesses Right to subpoena records Number of Agencies 96% 88% 75% 58% 83% 30% 26% 24 59% 52% 48% 39% 78% 56% 59% 33 57% 38% 30% 19% 65% 32% 35% 37 68% 55% 48% 36% 74% 40% 41% 94 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. Table B12: Percent of Oversight Agencies Indicating “Strongly Agee/Agree” that Police Stakeholders Listened Carefully to Recommendations Made by Oversight Staff Auditor/ All Investigative Review monitor Agencies Police executives listen carefully to recommendations made by oversight 83% 76% 78% 78% staff Number of Agencies 18 33 32 83 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Table B13: Percent of Oversight Agencies Indicating that Police Stakeholders “Very Frequently/Frequently” Implemented their Recommendations Auditor/ All Investigative Review monitor Agencies How often does the police/sheriff implement recommendations made 72% 42% 34% 46% by your oversight agency? Number of Agencies 18 33 32 83 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Civilian Oversight of Law Enforcement Assessing the Evidence 69 Table B14: Percentage of Oversight Agencies Indicating that their Enabling Statute/Ordinance Explicitly Includes a Requirement that Law Enforcement Employees Cooperate with their Agency Auditor/ All Investigative Review monitor Agencies That police officers/command staff cooperate with your oversight 52% 73% 40% 55% agency? That police officers/command staff cooperate with your oversight 26% 47% 20% 31% agency work as a condition of their employment? 23 Number of Agencies 33 35 91 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Table B15: Percentage of Oversight Agencies who Indicated “Very Strong/Strong” Support from Key Process Stakeholders Auditor/ All Investigative Review monitor Agencies Police Executive Police Command Staff Internal Affairs Command Staff Internal Affairs Investigators Police Union City Attorney/County Counsel Mayor City Council Civil Rights Community Number of Agencies 83% 78% 88% 82% 35% 47% 75% 92% 82% 18 87% 81% 73% 73% 21% 74% 81% 82% 71% 31 93% 93% 92% 83% 55% 75% 82% 88% 65% 29 88% 85% 84% 79% 35% 67% 80% 86% 72% 78 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 70 Civilian Oversight of Law Enforcement Assessing the Evidence Table B16: Yearly Operating Budget for Responding Oversight Agencies Auditor/ monitor 12% 0% 12% 6% 24% 18% 29% 17 No budget $1 - $10,000 $10,001 - $100,000 $100,001 - $250,000 $250,001 - $500,000 $500,001 - $1,000,000 More than 1,000,000 Number of Agencies Investigative 3% 3% 7% 13% 20% 17% 37% 30 Review All Agencies 59% 10% 10% 7% 10% 0% 3% 29 26% 5% 9% 9% 17% 11% 22% 76 Review All Agencies 61% 27% 3% 3% 6% 0% 33 33% 36% 13% 4% 6% 8% 90 Review All Agencies 69% 31% 32 78% 22% 85 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Table B17: Number of Full Time, Paid Staff by Type of Oversight Agency Auditor/ monitor 26% 43% 9% 9% 4% 9% 23 0 Staff 1-5 Staff 6-10 Staff 11-15 Staff 16-20 Staff 21 or more Staff Number of Agencies Investigative 12% 38% 26% 3% 6% 15% 34 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Table B18: Does your Agency Publish Public Reports? Auditor/ monitor Yes No Number of Agencies 80% 20% 20 Investigative 85% 15% 33 Table Notes: The table reports the valid percent and excludes missing values from the calculation. Civilian Oversight of Law Enforcement Assessing the Evidence 71 Table B19: Types of Issues Analyzed “Very Frequently/Frequently” by Oversight Agencies Auditor/ monitor Total number of complaints filed Types of allegations contained in complaints Findings on allegations Timeliness data Other uses of force Officer-involved shootings Discipline imposed In-custody death Traffic stop data Demographic information of individuals subject to stops, searches or arrest Vehicle pursuits K-9 bites Tort claims filed against officers Agency Does Not Analyze Statistical Patterns Number of Agencies Investigative Review All Agencies 67% 71% 71% 48% 57% 62% 52% 76% 76% 67% 58% 48% 33% 36% 42% 39% 36% 33% 27% 30% 18% 61% 61% 56% 46% 43% 39% 33% 57% 29% 30% 12% 18% 21% 32% 20% 29% 21% 12% 20% 29% 24% 19% 10% 21 18% 15% 15% 18% 33 15% 21% 18% 33% 33 20% 20% 17% 22% 87 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 72 Civilian Oversight of Law Enforcement Assessing the Evidence Table B20: Did Representatives from your Agency Conduct Outreach Activities with any the Following Groups in the Last Year? Auditor/ All Investigative Review monitor Agencies Community leaders Neighborhood associations Policy-makers (e.g., city council members) Police command staff Police officers Schools Religious organizations Youth group Mental health community Police unions At-risk youth Homeless community Health care providers Inmates Others Number of Agencies 83% 72% 78% 72% 67% 61% 44% 56% 61% 61% 50% 50% 17% 17% 6% 18 88% 76% 70% 64% 55% 45% 39% 42% 27% 33% 27% 18% 15% 12% 6% 33 63% 59% 53% 47% 50% 38% 38% 25% 19% 9% 16% 16% 6% 6% 31% 32 77% 69% 65% 59% 59% 46% 40% 39% 31% 30% 28% 24% 12% 11% 16% 83 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. Civilian Oversight of Law Enforcement Assessing the Evidence 73 Table B21: Most Common Outreach Activities Reported by Oversight Agencies in the Previous Year Auditor/monitor Operated an agency website Tabled at community fairs Provided complaint forms at multiple locations Held community forums Operated an agency Facebook page Mailed brochures/business cards to community groups Held press conferences Mailed brochures/business cards to neighborhood centers Operated an agency Twitter account Posted signs in police stations Televised hearings Other Number of Agencies Investigative 72% 56% 56% 50% 28% 28% 33% 28% 33% 22% 11% 11% 18 88% 52% 58% 48% 45% 30% 30% 27% 24% 24% 24% 9% 33 Review All Agencies 47% 41% 28% 38% 25% 31% 13% 16% 9% 9% 13% 28% 32 69% 48% 46% 45% 34% 30% 24% 23% 20% 18% 17% 17% 83 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. Table B22: Percentage of Agencies Indicated that they have Translated Outreach Documents into Languages Other Than English? Auditor/monitor Complaint forms Agency brochures Agency web page Agency reports Other Number of Agencies Investigative 53% 59% 24% 12% 6% 17 59% 56% 22% 6% 9% 32 Review All Agencies 25% 34% 3% 3% 0% 32 44% 48% 15% 6% 5% 81 Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. Table B23: Does your Agency have a Board Composed of Community Members? Auditor/monitor Yes No Number of Agencies 65% 35% 23 Investigative Review All Agencies 85% 100% 15% 0% 34 37 86% 14% 94 Table Notes: The table reports the valid percent and excludes missing values from the calculation. 74 Civilian Oversight of Law Enforcement Assessing the Evidence Acknowledgements The authors would like to thank a number of individuals for the assistance they provided on this project. NACOLE staff members Cameron McEllhiney and Liana Perez offered incredible administrative, technical and editorial support. University of Idaho students Andrea Pikes and Nicole Waldorf supplied outstanding research assistance. Members of the NACOLE Board of Directors provided useful advice during the development of the organizational survey, including: Ainsley Cromwell, Karen Ullery Williams, Avice Evans Reid, Donald Casimere, Kelvyn Anderson, Brian Corr, Margo Frasier, Nicholas Mitchell, Dawn Reynolds and Mark P. Smith. Susannah Numa and her team at the OJP Diagnostic Center offered critical support during the conceptualization and writing of the report. Finally, the members of the OJP expert peer review panel and the anonymous U .S . DOJ reviewers provided helpful advice for strengthening the report . The peer review panel included: • Steve Rickman, Senior Advisor (Contractor) for the DOJ Office of Justice Programs Diagnostic Center; • Jane Castor, Chief (Ret.), Tampa Police Department; • Robert Davis, Senior Social Scientist, Police Foundation; • Philip Eure, Inspector General, Office of the Inspector General for the NYPD; • Kathryn Olson, Principal, Change Integration Consultants; • Ursula Price, Deputy Police Monitor, New Orleans Office of the Independent Police Monitor; • Professor John Skinner, Towson University; • Darrel Stephens, Executive Director, Major Cities Chiefs Association; and • Dr. Robert Worden, Associate Professor, University at Albany, State University of New York and the Finn Institute for Public Safety. U.S. Department of Justice Disclaimer. This project was supported by Contract No GS-23F-9755H awarded by the Office of Justice Programs, U.S. Department of Justice to Booz Allen Hamilton, and its partners: the Institute for Intergovernmental Research and CNA. Points of view or opinions in this document are those of the author and do not necessarily represent the official position or policies of the U.S. Department of Justice. September 2016 CENTER Data-Driven Justice Solutions Overs?o? I ?91113310913 ?33