Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial David Yokum Anita Ravishankar Alexander Coppock Working Paper October 20, 2017 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial David Yokum1* Anita Ravishankar1,2,3 Alexander Coppock4 WORKING PAPER OCTOBER 20, 2017 1 The Lab @ DC, Office of the City Administrator, Executive Office of the Mayor, Washington, DC, USA. * Corresponding author. Email: David.Yokum@dc.gov. 2 Executive Office of the Chief of Police, Metropolitan Police Department, Washington, DC, USA. 3 Public Policy and Political Science Joint PhD Program, University of Michigan, Ann Arbor, MI, USA. 4 Department of Political Science, Yale University. New Haven, CT, USA. Contents 1 5 7 11 18 22 23 28 Introduction Study Design Measurement Strategy Results Discussion Conclusion References Acknowledgments Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial David Yokum1*, Anita Ravishankar1,2,3, Alexander Coppock4 ABSTRACT: Police officer body-worn cameras (BWCs) have been promoted as a technological mechanism that will improve policing and the perceived legitimacy of the police and legal institutions. While there is a national movement to deploy BWCs widely, evidence of their effectiveness is limited. To estimate the average effects of BWCs, we conducted a randomized controlled trial involving 2,224 Metropolitan Police Department (MPD) officers in Washington, DC. Our pre-analysis plan was publicly registered in advance. We compared officers randomly assigned to wear BWCs to officers in the control condition who did not wear BWCs. The primary outcomes of interest were documented uses of force and civilian complaints, although we also measure a variety of additional policing activities and judicial outcomes. We estimated very small average treatment effects on all measured outcomes, none of which rose to statistical significance. These results suggest that we should recalibrate our expectations of BWCs’ ability to induce large-scale behavioral changes in policing, particularly in contexts similar to Washington, DC. 1 The Lab @ DC, Office of the City Administrator, Executive Office of the Mayor, Washington, DC, USA. * Corresponding author. Email: David.Yokum@dc.gov. 2 Executive Office of the Chief of Police, Metropolitan Police Department, Washington, DC, USA. 3 Public Policy and Political Science Joint PhD Program, University of Michigan, Ann Arbor, MI, USA. 4 Department of Political Science, Yale University. New Haven, CT, USA. Introduction Police body-worn camera (BWC) programs are rapidly spreading across the United States. In 2015, the U.S. Department of Justice awarded over $23 million in funding to support the implementation of BWC programs throughout the country,1 and a nationwide survey found that 95% of large police departments either have already implemented or intend to implement a BWC program.2 Much of the expansion has been motivated by a series of high-profile, officer-involved shootings, many of which were captured in bystander video and shared across social media. Stakeholders such as the American Civil Liberties Union (ACLU), Campaign Zero, and Black Lives Matter3 have urged the police to equip BWCs as technological solution to improve policing.4 One of the theories of change underlying the anticipated effects of BWCs is that both officers and civilians on the street will behave differently if under the watchful lens of a camera. A wide range of research,5 dating back to the classic experiments at Hawthorne Works,6 has suggested that people act differently when they believe they are being watched, from increasing work productivity and charitable giving,7,8 to 1 U.S. Department of Justice. “Justice Department Awards over $23 Million in Funding for Body Worn Camera Pilot Program to Support Law Enforcement Agencies in 32 States”. September 21, 2015. Available : https:// www.justice.gov/opa/pr/justice-department-awards-over-23-million-funding-body-worn-camera-pilotprogram-support-law 2 Maciag, M. (2016).Survey: Almost All Police Departments Plan to Use Body Cameras. Governing.com, Available: http://www.governing.com/topics/public-justice-safety/gov-police-body-camera-survey.html 3 Stanley, J. (2015). Police Body-Mounted Cameras: With Right Policies in Place, a Win For All. American Civil Liberties Union. Available https://www.aclu.org/sites/default/files/assets/police_body-mounted_cameras-v2. pdf and Campaign Zero. Body Cams/Film the Police. Available https://www.joincampaignzero.org/film-thepolice. 4 Hudson, D. (2014). “Building Trust Between Communities and Local Police,” Whitehouse.gov, Available https://obamawhitehouse.archives.gov/blog/2014/12/01/building-trust-between-communities-and-local-police (President Obama’s Body-Worn Camera Partnership Program invests $75 million to underwrite state and local deployment costs for 50,000 BWCs). 5 For a recent review of research on the Hawthorne effect, see McCambridge, J, Witton, J.,and Elbourne, D. R.. (2014). Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. Journal of clinical epidemiology, 67(3), 267-277. 6 Izawa, M. R., French, M. D., and Hedge, A. (2011). Shining new light on the Hawthorne illumination experiments. Human factors, 53(5), 528-547. 7 Ekström M. (2011). Do watching eyes affect charitable giving? Evidence from a field experiment. Experimental Economics, 15, 530–546. 8 Haley, K. J., and Fessler, D. M. (2005). Nobody’s watching?: Subtle cues affect generosity in an anonymous economic game. Evolution and Human behavior, 26(3), 245-256. 1 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Introduction encouraging honesty,9 promoting adherence to recycling rules,10 stimulating voter turnout,11 and reducing crime.12 Across these settings, monitoring appears to shift behavior into alignment with socially acceptable conduct.13 In the policing context, cameras are expected to encourage officer adherence to departmental protocols and deter police from engaging in unprofessional behavior or misconduct, especially unjustified use of force. Similarly, civilians interacting with a BWC-equipped officer may be less likely to engage in inappropriate or combative behavior. The underlying pathways between BWCs and behavior could include greater self-awareness, heightened threat of being caught, or a combination of the two. Whatever the exact mechanisms, commentators sometimes allude to a so-called “civilizing effect,” wherein BWCs are predicted to calm all parties involved and reduce the likelihood that violence occurs.14 Secondly, by capturing the police-civilian interaction, the cameras are also expected to have evidentiary value, both for internal affairs and criminal investigations.15 Camera footage could help resolve cases in a more timely, judicious manner that makes more efficient use of investigative resources. 9 Bateson, M., Nettle, D., and Roberts, G.(2006). “Cues of being watched enhance cooperation in a realworld setting.” Biology letters , 2(3), 412-414. 10 Francey, D., Bergmüller, R. (2012). Images of Eyes Enhance Investments in a Real-Life Public Good. PLoS ONE 7, e37397. 11 Gerber, A.S., Green,D.P. ,and Larimer, C. W. (2008). Social pressure and voter turnout: Evidence from a large-scale field experiment. American Political Science Review 102(1),: 33-48. 12 Nettle, D., Nott, K., and Bateson, M. (2012). ‘Cycle Thieves, We Are Watching You’: Impact of a Simple Signage Intervention against Bicycle Theft. PLoS ONE, 7(12), e51738. 13 Farrar, W., and Ariel, B. (2013). Self-awareness to being watched and socially-desirable behavior: A field experiment on the effect of body-worn cameras and police use-of-force. Washington, DC: Police Foundation.; Ariel, B., Farrar, W.A., and Sutherland, A. (2015). The effect of police body-worn cameras on use of force and citizens’ complaints against the police: A randomized controlled trial. Journal of Quantitative Criminology, 31(3), 509-535. However, for reviews concluding the absence of Hawthorne effects, see Adair, J.G. (1984). The Hawthorne Effect: A Reconsideration of the methodological artifact. Journal of Applied Psychology, 69(2) and Vist, G. E., Bryant, D., Somerville, L., Birminghem, T., and Oxman, A. D. (2008). Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. The Cochrane Library. 14 Katz, C.M., Kurtenbach, M., Choate, D.E., and White, M.D. (2015) Phoenix, Arizona, Smart Policing Initiative: Evaluating the Impact of Police Officer Body-Worn Cameras. Washington, DC: Bureau of Justice Assistance, 2. 15 Ibid. 2 Furthermore, the existing evidence on whether BWCs have the anticipated effects on policing outcomes remains relatively limited, with mixed results.18 Several observational studies have evaluated BWCs by comparing the behavior of officers before and after the introduction of BWCs into the police department; others compared officers who happened to wear BWCs to those without. The causal inferences drawn in those studies depend on the untestable assumption that (possibly after statistical adjustments) BWCs were as-if randomly assigned.19 There have also been several randomized evaluations but the findings from these studies have been mixed. For example, in a series of RCTs conducted across several sites in the United Kingdom and the United States, researchers did not find a consistent effect of BWCs on police use of force (which increased at some sites and decreased at others) or on citizen complaints despite using the same experimental design across all trials.20 Several other experiments found no detectable effects of BWCs on measured outcomes when comparing control and treatment groups (though they do observe statistically significant effects in pre/ 16 See generally Police Executive Research Forum, Implementing a Body-Worn Camera Program: Recommendations and Lessons Learned, (U.S. Department of Justice, Community Oriented Policing Services, 2014): Chapter 2, “Considerations for Implementation.” Available https://www.justice.gov/iso/ opa/resources/472014912134715246869.pdf. 17 Mateescu, A., Rosenblat, A., and Body, A. (2015). Police Body-Worn Cameras. Data and Society Research Institute, Available https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2569481 18 Cubitt, T. I., Lesic, R., Myers,G.L., and Corry, R. (2017). Body-worn video: A systematic review of literature. Australian and New Zealand Journal of Criminology 50, no. 3379-396. See also Miller, L., and Toliver, J. (2014) Implementing a Body-Worn Camera Program: Recommendations and Lessons Learned, Police Executive Research Forum and Lum, C. M., Koper, C. S., Merola, L. M., Scherer, A., & Reioux, A. (2015). Existing and ongoing body worn camera research: Knowledge gaps and opportunities. George Mason University. 19 Ellis, T., Jenkins, C., and Smith, P. (2015). Evaluation of the Introduction of Personal Issue Body Worn Video Cameras (Operation Hyperion) on the Isle of Wight. Final Report to the Hampshire Constabulary. Available http://www.bwvsg.com/news/operation-hyperion-final-report/; ODS Consulting (2011). “Body Worn Video Projects in Paisley and Aberdeen - Self Evaluation.” Available http://www.bwvsg.com/wpcontent/uploads/2013/07/BWV-Scottish-Report.pdf.; Stratton, M., Clissold, P., and Tuson, R. (2015) “Body Worn Video: Considering the Evidence.” Available http://www.bwvsg.com/wp-content/uploads/2015/06/ Edmonton-Police-BWV-Final-Report.pdf.; Hedberg, E. C., Katz, C.M., andChoate, D.E. (2017) “Bodyworn cameras and citizen interactions with police officers: Estimating plausible effects given varying compliance levels.” Justice quarterly ,34(4),627-651. 20 Ariel et al. (2016). In a follow-on study (Ariel 2016), researchers examine whether officer discretion to use BWCs might explain the variation in results across different sites, and find support for this theory. 3 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Introduction Given these purported benefits, the early adoption of this technology is perhaps unsurprising. Yet BWCs present a number of challenges: they are financially costly and involve new administrative complexities, such as officer training and procedures for retention of and access to video footage.16 Difficult questions related to privacy and state power also remain unresolved.17 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Introduction post comparisons, which again have the causal inference issues noted above).21 Moreover, these studies typically have been limited in one of two ways. The first is small sample size, leading to low statistical power and difficulty in making precise causal estimates. The second is randomization at the level of shift rather than officer. While this design increases statistical power, it raises questions of spillover effects when the same officers are engaged in both control and treatment shifts.22 (We discuss potential spillover across officers in the same shift—a limitation of the present study—below.) We collaborated with the Metropolitan Police Department of the District of Columbia (MPD) to design and implement an RCT to evaluate the effects of BWCs citywide. Specifically, as part of MPD’s deployment of BWCs to its entire police force, approximately half of all full duty patrol and station officers were randomly assigned to wear BWCs, while the other half remained without BWCs until December 2016. The evaluation period ran from June 28, 2015 to December 15, 2016, at which time MPD also began issuing BWCs to control group officers. We tracked outcomes associated with police activity that occurred during the treatment period until March 31, 2017. Before seeing the data, we developed a detailed write-up of the methodology and planned statistical analyses—a pre-analysis plan—and publicly shared it on the Open Science Framework.23 Writing this plan before we had the data ensured that we did not, intentionally or unintentionally, have discretion to conduct the analyses merely to fit preconceived expectations. We also participated in 11 events to discuss the plan with stakeholder groups and a wider public audience. We used administrative data to measure average effects. The primary outcomes of interest were documented uses of force and civilian complaints, although we also measure a variety of additional policing activities and judicial outcomes. We also report exploratory analyses examining the rate of officer adherence to departmental policies on camera activation and use. With 2,224 MPD members participating in the study, this is one of the largest randomized evaluations of BWCs conducted to date. 21 See, for example, Ariel et al (2016a) and Ariel et al. (2015) discussion of effects of BWCs on citizen complaints. 22 For example, see Ariel, B., Farrar, W.A., and Sutherland, A.(2015). The effect of police body-worn cameras on use of force and citizens’ complaints against the police: A randomized controlled trial. Journal of quantitative criminology 31, 3, 509-535. 23 Yokum, D., Ravishankar, A., and Coppock, A.(2017). Pre-Analysis Plan - A Randomized Controlled Trial of the Police Body-Worn Camera Program in the District of Columbia. Open Science Framework. Available https://osf.io/yjyng/. 4 Study Design MPD is one of the largest police departments in the country, with over 3,800 sworn members serving a resident population of over 680,000 (plus many more daily commuters from surrounding states). The department is organized into seven police districts covering 68 square miles, and is unique in its role as the local, state, and federal law enforcement authority in Washington, DC. The RCT encompassed the entire department and included geographic coverage of the entire city. We identified eligible officers within each of the seven police districts (as well as several specialized units) based on the following criteria: the officer was on active, full duty administrative status and did not have a scheduled leave of absence during the study period; held a rank of sergeant or below; and was assigned to patrol duties in a patrol district or to a non-administrative role at a police station. Eligible officers within each district or special unit were then randomly assigned to one of two groups: (1) no BWC (“control”) or (2) with BWC (“treatment”). Specifically, treatment entails assignment of an eligible participant to wear and use a BWC in accordance with MPD policy. MPD General Order SPT-302.13 specifies that “[m]embers, including primary, secondary, and assisting members, shall start their BWC recordings as soon as a call is initiated via radio or communication from OUC [Office of Unified Communications] on their mobile data computer (MDC), or at the beginning of any self-initiated police action.”24 The general order enumerates the range of events for which officers were required to activate their BWCs; this list is included in the Supplementary Materials. 24 Metropolitan Police Department. General Order - Body Worn Camera Program. GO-SPT-302-13. 11 Mar 2016. Available: https://go.mpdconline.com/GO/GO_302_13.pdf. In addition, MPD issued an executive order on September 15, 2016 requiring members to “verbally acknowledge activation of their BWC” to an Office of Unified Communications dispatcher when they acknowledge “receipt of a radio run over the air or are advising the dispatcher of self-initiated police action” (1). In addition, OUC agreed to have its dispatchers “confirm over the air that members’ BWCs are activated, when practicable” (2). 5 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Study Design Randomization was implemented using a block randomized assignment procedure. This approach, which uses pre-treatment information to group officers into blocks before randomly assigning a fixed number of cameras to officers in each block, helps increase the statistical power of the experimental design and enforce treatment-versus-control balance on the covariates according to which blocking occurs. We applied a two-level blocking approach: “major” blocks are the seven police districts and three special units, and the minor blocks were constructed using a clustering algorithm based on the background characteristics of the officers.25 Based on the eligibility requirements noted above, our sample consisted of 2,224 MPD members, with 1,035 members assigned to the control group, and 1,189 members assigned to the treatment group. As anticipated in our pre-analysis plan, some officers who were assigned cameras might not install or use them, and some officers who were not assigned cameras might have nevertheless obtained them.26 We estimate two compliance measures: the number of videos uploaded to the video database by treatment officers, and the average length of the videos in minutes. If officers complied with the randomization protocol, we would expect that officers assigned BWCs would make vastly more videos per year, as well as have a longer average length of videos. On average, treatment officers uploaded about 665 videos annually (compared with 14 videos uploaded among control officers). The average video recorded by a treatment officer was over 11 minutes long, while the average video recorded by a control officer was just 0.8 minutes long. For both manipulation check measures, the treatment assignment is both substantively and statistically significant (at the p<0.01 level). We conclude that compliance with the study protocol was high. Following best practices in settings encountering two-sided noncompliance,27 we conducted all of our analyses according to the original random assignment. Our experiment thus recovers estimates of the effect of being assigned to a BWC on a variety of outcomes (the intention-to-treat effect, or ITT). 25 Moore, R. T., and Schnakenberg, K. (2012). blockTools: Blocking, assignment, and diagnosing interference in randomized experiments. Version , 0.5-6, 26 See Yokum et al. (2017), 8. 27 See Gerber and Green (2012). Chapter 6. 6 Measurement Strategy We assessed the effect of BWCs on four families of outcome measures: police use of force, citizen complaints, policing activity, and judicial outcomes. • Police use of force is based on officers’ self-reported use of force (in accordance with MPD policy).28 It includes a count of all use of force incidents, as well as subsetted measures of serious uses of force (as defined by MPD policy),29 other uses of force,30 and use of force incidents by the race of the subject of force. • Outcome measures related to civilian complaints include a count of all complaints filed against MPD officers (either through MPD or through the independent Office of Police Complaints), as well as complaints disaggregated by disposition (sustained, not sustained, or insufficient facts to resolve). • The policing activity category includes traffic tickets and warnings issued, reports taken from particular types of calls for service, arrests on specific charges (e.g., disorderly conduct, traffic violations, assaults against a police officer), and injuries sustained by officers in the line of duty. We use these measures to evaluate the effects of BWCs on officer discretion and activity, as well as on civilian behavior. 28 Metropolitan Police Department. General Order - Use of Force. GO-RAR-901.07. https://go.mpdconline.com/ GO/GO_901_07.pdf. 1 Dec 2016. Per this policy, “The following actions are designated “reportable uses of force”: (1) Deadly force; (2) Serious use of force; (3) Use of a less-than-lethal weapon; (4) Any use of force indicating potential criminal conduct by a member; and (5) Any use of force resulting in injury or a complaint of injury or pain where the injury or pain is directly associated with a member’s use of force. The following actions are designated “reportable force incidents” as long as the use of force does not result in injury or a complaint of injury or pain: (1) All solo or team takedowns, where there is no complaint of pain or injury; and (2) The drawing and pointing of a firearm at, or in the direction of, another person when no other force was used. Minor injury or discomfort resulting from the application and general wearing of handcuffs is not, in and of itself, considered a “reportable use of force” or a “reportable force incident” (p. 4). 29 Metropolitan Police Department. General Order - Use of Force. GO-RAR-901.07. https://go.mpdconline. com/GO/GO_901_07.pdf. 1 Dec 2016. Serious use of force are defined as “actions by members including: (1) All firearm discharges by a member with the exception of range and training incidents, and discharges at animals; (2) All uses of force by a member resulting in a serious physical injury; (3) All head strikes with an impact weapon; (4) All uses of force by a member resulting in a loss of consciousness, or that create a substantial risk of death, serious disfigurement, disability or impairment of the functioning of any body part or organ; (5) All incidents where a person receives a bite from an MPD canine; (6) All uses of force by an MPD member involving the use of neck restraints or techniques intended to restrict a subject’s ability to breathe; and (7) All other uses of force by a member resulting in a death.” (p. 3) 30 We define “other uses of force” as all other uses of force not indicated as “serious” in MPD policy. 7 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Measurement Strategy • Finally, we examine the effects of BWCs on judicial outcomes, measured by whether MPD arrest charges are prosecuted by the U.S. Attorney’s Office (USAO) or the Office of the Attorney General (OAG) and the disposition of those charges. Our examination of this set of outcomes is constrained by limitations in the available data. Namely, we did not have access to the full datasets managed by the USAO, OAG, and the courts. We instead had access to a subset of this data available to MPD, which captures only the initial charges31 on which an individual was arrested. A consequence is that we are unable to track court outcomes for any changes to those initial charges. As this limitation applies to both control and treatment groups, however, we were still able to conduct a preliminary analysis on the evidentiary value of BWCs. Due to logistical constraints, MPD deployed cameras on a district-by-district basis over the course of 11 months. Officers in two of the seven police districts received cameras in late June 2015, with the deployment to the remaining districts taking place from March to May 2016. By integrating randomization directly into the BWC deployment process, we were able to conduct this study at marginally low cost to MPD. To address the staggered deployment process, the data collection period varies for each police district, based on the start date of BWC deployment in that district. All outcomes were obtained at the officer level and translated into yearly rates. These rates were calculated from the date that the cameras were first deployed in each district (i.e., the date on which treatment begins in that district). We calculate these rates before and after the intervention based on a window of k days, where k is the number of days between deployment and the end of the study period for the district that was the last to receive cameras.32 The pre- and post-treatment periods are of the same length for all districts; the pre-treatment measurements come from the same k-day window (in the previous year) as the post measurements to account for seasonality in policing and desensitization to the treatment over time. This measurement strategy is depicted in Figure 1 below: 31 For example, if MPD makes an arrest for a felony, and USAO changes those charges to a misdemeanor, then this event is only reflected in our data as a felony not prosecuted. The misdemeanor charge is not captured in our data. This limitation was known at the outset, as documented in the pre-analysis plan. 32 The last district to receive cameras was 2D, which deployed BWCs on May 17, 2016. This gives us a window of 212 days (from May 17-December 15, 2016), or approximately a seven-month study duration for each district. Thus, we effectively measure outcomes for the first seven months in each district, and report the results averaged across all seven police districts. See Supplementary Materials for the start date of BWC deployment in each district. 8 This chart depicts the pre- and post-treatment periods for each police district. The length of the treatment period is based on a window of k days, where k is the number of days between first BWC deployment in a district and the end of the study period for the district that was last to receive cameras—in our case, k = 212 days. We thus measure the same pre- and post-treatment period length for all districts, with pre-treatment measures coming the corresponding window of time in the previous year for a given district. Pre-Treatment Post-Treatment We employ two estimators of the average treatment effects: (1) difference-inmeans with inverse probability weights to account for differential probabilities of assignment by block; and (2) regression of outcome on treatment assignment with controls for pretreatment characteristics and inverse probability weights. Specifically, we control for the pre-treatment value of the outcome (e.g., past use of force), pre-treatment covariates for the officer, and indicators for each major block. Equation 1, below, provides the exact specification, as pre-registered on the Open Science Framework before the realization of outcomes.33 33 Yokum et al. (2017). 9 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Measurement Strategy FIG. 1. Main measurement strategy. Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Measurement Strategy YPOST = β0 + β1 Z + β2 YPRE + β3 Block + β4 X + ε (1) where Z is the treatment indicator (officer assigned camera or not); YPRE is the pretreatment value of the outcome under study; Block is a categorical variable for an officer’s home district or special unit; X is a vector of pre-treatment covariates that includes race, gender, and length of service; and ε is the error term. We estimate Equation 1 using weighted least squares (WLS) regression with inverse probability weights, which are calculated as the inverse of the probability of each unit being in its observed condition (see Supplementary Materials).34 We employ HC2 robust standard errors for variance estimation.35 Our primary analysis was conducted among officers in 1D, 2D, 3D, 4D, 5D, 6D, and 7D (N = 1,922). As specified, we excluded officers in special units from this analysis as policing activities and camera use patterns may differ between these units and the district officers. We conduct this analysis at the officer level, and report results as a yearly rate per 1,000 officers.36 Our analysis were conducted by two independent statistical teams, to help avoid coding errors and as a check of convergence in results. 34 Gerber, A.S., and Green., D.P. (2012). Field experiments: Design, analysis, and interpretation. WW Norton, 35 Samii, C., and Aronow, P.M.(2012). On Equivalencies Between Design-Based and Regression- Based Variance Estimators for Randomized Experiments. Statistics and Probability Letters, 82(2), 365 – 370. 36 To obtain results as a yearly rate per 1,000 officers, we first translate the raw count of outcomes for each officer in the k-day period (e.g., raw B1) to an equivalent number of outcomes over 365.25 days (the yearly rate) and then multiply that rate by a factor of 1000. 10 Results Across each of the four outcome categories, our analyses consistently point to a null result: the average treatment effect on all of the measured outcomes was very small, and no estimate rose to statistical significance at conventional levels. Tables 1 and 2 below provide the average treatment effects of BWCs on select police behaviors and judicial outcomes, respectively (tables documenting the effects of BWCs on all outcomes are included in Supplementary Materials). TABLE 1. AVERAGE TREATMENT EFFECTS OF BWCS ON POLICE BEHAVIORS USE OF FORCE (1) CITIZEN COMPLAINTS (2) ARRESTS FOR DISORDERLY CONDUCT (3) Officer Assigned BWC 73.6 (87.0) 57.3 (41.4) -127.7 (277.2) Constant 807.2 (59.2) 280.1 (29.6) 1,416.5 (186.3) N 1,922 1,922 1,922 R2 0.000 0.001 0.000 *p < .1; **p < .05; ***p < .01 Outcomes are yearly event rates per 1000 officers Robust standard errors are in parentheses TABLE 2. AVERAGE TREATMENT EFFECTS OF BWCS ON JUDICIAL OUTCOMES PROSECUTED (1) FOUND GUILTY (2) NOT FOUND GUILTY (3) ENTERED PLEA (4) NOT PURSUED (5) Officer Assigned BWC 2,421.6 (2,632.7) 13.5 (20.0) -15.6 (22.2) 62.8 (353.1) -114.3 (102.2) Constant 33,139.1 (1,814.6) 39.6 (14.1) 49.3 (17.7) 1,348.5 (182.0) 390.1 (95.9) N 1,922 1,922 1,922 1,922 1,922 R2 0.000 0.000 0.000 0.000 0.001 *p < .1; **p < .05; ***p < .01 Outcomes are yearly event rates per 1000 officers Robust standard errors are in parentheses 11 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Results Figures 2 and 3 below plot our results for a subset of the outcomes, with the remainder included in the Supplementary Materials. Figure 2 plots the estimated average treatment effect (as a yearly rate per 1,000 officers) of BWCs on police use of force, citizen complaints, and officer discretion (as measured by arrests for disorderly conduct). Figure 3 shows the effects of BWCs on whether a criminal case was prosecuted and on the possible case dispositions. We chose to present these outcomes in the main text as they are of most interest to practitioners and other scholars of BWCs, not because the results for these outcomes are exceptional. The equivalent figures for other measured outcomes (shown in Supplementary Materials) look approximately the same. 12 For example, among 1,000 officers over a year, our best estimate is that there would be about 75 more uses of force if they had BWC than not; but the 95% CI expresses our uncertainty, and that actually anywhere from about 95 fewer to 250 more uses of force would be consistent with the data. We show findings from both our difference-in-means estimator (in red) and ordinary least squares regression including pretreatment covariates (in blue). A. Use of force Average Treatment Effect (per 1000 officers, per year) 13 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Results FIGS. 2A-C. Average difference (with 95% confidence interval) between BWC and non-BWC groups, per 1,000 officers over a year on police use of force, complaints filed against officers, and arrests for disorderly conduct. Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Results B. Complaints Average Treatment Effect (per 1000 officers, per year) C. Arrests for disorderly conduct Average Treatment Effect (per 1000 officers, per year) 14 Average difference (with 95% confidence interval) between BWC and non-BWC groups, per 1,000 officers over a year, on court outcomes. We show findings from both our difference-in-means estimator (in red) and ordinary least squares regression including pretreatment covariates (in blue). A. Prosecuted Average Treatment Effect (per 1000 officers, per year) 15 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Results FIGS. 3A-E. Average difference (with 95% confidence interval) between BWC and non-BWC groups, per 1,000 officers over a year, on court outcomes. Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Results B. Not found guilty Average Treatment Effect (per 1000 officers, per year) C. Found guilty Average Treatment Effect (per 1000 officers, per year) 16 Average Treatment Effect (per 1000 officers, per year) E. Entered plea Average Treatment Effect (per 1000 officers, per year) 17 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Results D. Not pursued Discussion We are unable to reject the null hypotheses that BWCs have no effect on police use of force, citizen complaints, policing activity, or judicial outcomes. Because our study has a large enough sample size to detect small effect sizes, these failures to reject the null are unlikely to be due to insufficient statistical power, at least for uses of force and complaints. (Our estimates of average effects in the judicial outcomes category occasionally have much wider confidence intervals.) We consider here a few possible explanations for our null findings. First and most obviously, it is possible the null finding needs no explanation: the devices, in fact, have no effect on the measured behaviors, and the video footage they produce has no effect on judicial outcomes. Perhaps neither the officer nor citizen involved in an interaction are actually aware of the camera, either due to attention being diverted elsewhere or desensitization over time to the presence of the cameras. Alternately, the officer and citizen may notice the cameras, but other factors in the heat of the moment may override any deterrent effect the cameras may have had. In addition to examining average differences in outcomes for the treatment and control groups, we also conducted a time-series analysis that tracks both groups over time. This analysis allows us to consider whether strong initial effects may have washed out as officers became accustomed to the cameras; or, in the other direction, whether effects began weak but then strengthened as officers learned to use the cameras. However, as shown in Figures 4 and 5 below, we find that control and treatment group behaviors were similar both before and after the deployment of BWCs. 18 Uses of force filed per 1000 officers This figure plots pre- and post-treatment uses of force for both control and treatment group officers. As the chart indicates, there is no statistically significant difference between the two groups in either the 90-day period before or after the deployment of BWCs (which occurs on day 0). Days since cameras deployed Z Control Officer assigned BWC FIG. 5. Complaints per 1,000 Officers, 90 days before and after BWC deployment. Complaints filed per 1000 officers This figure plots pre- and post-treatment complaints filed for both control and treatment group officers. As the chart indicates, there is no statistically significant difference between the two groups in either the 90-day period before or after the deployment of BWCs (which occurs on day 0). Days since cameras deployed Z Control Officer assigned BWC 19 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Discussion FIG. 4. Uses of Force per 1,000 Officers, 90 days before and after BWC deployment. Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Discussion Another explanation for these findings concerns the particular context of Washington, DC relative to other jurisdictions around the United States. As the primary police agency for the nation’s Capital, MPD has distinct responsibilities and experience in supporting a variety of large-scale events, from presidential inaugurations to frequent concerts, festivals, and protests on the National Mall. The training officers undergo to operate in this high-stakes environment on a daily basis may equip them with a unique set of skills that translate into improvements in their interactions with District residents and visitors. In addition, over the two decades prior to the introduction of BWCs, MPD has undertaken several reforms to reduce police misconduct.37 Thus, where BWCs might help improve behavioral outcomes in departments with notable misconduct issues, the elevated scrutiny MPD encounters as the police force for the capital city, combined with a sustained effort to implement reforms under external monitoring, may have already helped root out many of these types of issues at MPD, limiting the added effect of BWCs. A third explanation for the null findings considers the possibility that other factors are masking the true effect of the BWCs: the cameras do affect the measured outcomes, but these effects are being hidden by interference across units. For example, a control officer without a BWC may be affected by his or her awareness of a nearby colleague in the treatment group who is equipped with a BWC.38 Such spillover may also affect officers indirectly, as the introduction of BWCs into MPD may have caused a shift in the norms of the broader force even though devices were only deployed to a subset of the officers. Finally, the true effect of BWCs may also be masked by the widespread presence of non-police cameras (e.g., citizens’ cell phones). Civilians regularly record encounters with MPD members with their 37 Following a Department of Justice (DOJ) investigation into MPD practices in the late 1990s that found “a pattern or practice of excessive use of force by MPD,” the city, MPD, and DOJ entered into a Memorandum of Agreement (MOA) in June 2001 to reform the department and address the findings of the DOJ investigation. These reforms were implemented under DOJ oversight, which was terminated in 2008 at the recommendation of the Independent Monitor based on “MPD’s having achieved substantial compliance with the vast majority of the MOA’s 126 substantial provisions and requirements.” A subsequent audit of MPD’s performance was conducted in 2015 at the request of the Office of the District of Columbia Auditor (ODCA), and found that the department has maintained its compliance with the 2001 MOA, with the reforms still in place. See U.S. Department of Justice. Letter from William R. Yeomans to Honorable Anthony Williams and Chief Charles Ramsey (“DOJ Findings Letter”) Available https://www.justice.gov/crt/findings-letter-re-use-force-washington-metropolitan-police-department.; The Bromwich Group LLC. (2016). “The Durability of Police Reform - The Metropolitan Police Department and Use of Force: 2008-2015.” Available http://apps.washingtonpost.com/g/documents/local/audit-of-dcpolice-use-of-force/1848/ Office of the District of Columbia Auditor. 38 An analysis of calls for service generating central case numbers showed that about 30% of these calls had no treatment officers recorded on scene, suggesting some support for this explanation. 20 Other researchers have suggested that BWCs may fail to affect outcomes because of nonadherence: officers, for a variety of reasons, may not use their assigned cameras according to departmental policy.39 They may fail to turn on the camera, for example. We have no indication that non-adherence was a widespread problem in this study. For 98% of the days in 2016, MPD averaged at least one video (and often many more) per call for service associated with a treatment officer. Further, even for the 2% of days in 2016 in which the number of videos uploaded was less than the number of incidents for which we would expect them, the difference is minimal, with 96% average adherence based on our measure. Finally, we consider the possibility that there may be effects of BWCs that are simply not captured in the administrative data. For example, it may be the case that there were uses of force that were previously going unreported, and those have now dropped with the introduction of BWCs. However, because our data do not capture unreported uses of force, we are unable to detect this kind of change. As a matter of speculation, however, we find it implausible that we would measure very small effects on reported outcomes but that the true average effect on unreported outcomes is large. Because use of force is, by definition, a self-reported measure, we note the possibility that BWCs may increase the likelihood of an officer self-reporting use of force, rather than the actual incidence of force. The measured effect in such a situation would suggest that BWCs actually increase uses of force, when in fact the underlying rate has remained the same—it is the reporting that has improved. This limitation applies to any BWC study relying on administrative data, and was anticipated from the onset (see pre-analysis plan).40 39 For example, see Ariel et al 2016b, Katz et al 2015, and Hedberg et al 2017. 40 Yokum et al. (2017), 17. 21 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial Discussion own cameras, and CCTV is widespread. It may be the case that these already existent cameras affect behavior to the extent possible, and adding BWCs into the mix has no statistically significant marginal effect, simply because there is no more room to have an effect. To explore this possibility (we note this analyses was not pre-registered), we examined use of force at night—the idea being there are fewer cameras (and less visibility) at night. Yet we still found no detectable effect of BWCs. Conclusion We measured the average effects of BWCs on documented uses of force and civilian complaints, as well as a variety of additional policing activities and judicial outcomes. Our sample size was unusually large, enhancing our ability to detect changes should they exist. In addition, our comparison groups were constructed from an individuallevel officer randomization scheme, which avoids several problems of inference present in other methodologies used to date. We are unable to detect any statistically significant effects. As such, our experiment suggests that we should recalibrate our expectations of BWCs. Law enforcement agencies (particularly in contexts similar to Washington, DC) that are considering adopting BWCs should not expect dramatic reductions in use of force or complaints, or other large-scale shifts in police behavior, solely from the deployment of this technology. We would also temper expectations about (and suggest further research into) the evidentiary value of BWCs. The administrative court data we had access to has certain limitations, but preliminary analyses do not uncover any clear benefits. Body-worn cameras may have great utility in specific policing scenarios, but we cannot conclude from this experiment that they can be expected to produce large, department-wide improvements in outcomes. 22 References Adair, J.G. (1984). The Hawthorne Effect: A Reconsideration of the methodological artifact. Journal of Applied Psychology, 69(2) Ariel, B., Farrar, W.A., and Sutherland, A. (2015). The effect of police body-worn cameras on use of force and citizens’ complaints against the police: A randomized controlled trial. Journal of Quantitative Criminology, 31(3), 509-535. Ariel, B., Sutherland, A., Henstock, D., Young, J., Drover, P., Sykes, J., and Henderson, R. (2016a). “Contagious Accountability” A Global Multisite Randomized Controlled Trial on the Effect of Police Body-Worn Cameras on Citizens’ Complaints Against the Police. Criminal justice and behavior, 44(2), 293-316. Ariel, B., Sutherland, A., Henstock, D., Young, J., Drover, P., Sykes, J., and Henderson, R. (2016b). Report: increases in police use of force in the presence of body-worn cameras are driven by officer discretion: a protocol-based subgroup analysis of ten randomized experiments. Journal of Experimental Criminology, 12(3), 453-463. Ariel, B., Sutherland, A., Henstock, D., Young, J., Drover, P., Sykes, J., and Henderson, R. (2016c). Wearing body cameras increases assaults against officers and does not reduce police use of force: Results from a global multi-site experiment. European Journal of Criminology, 13(6), 744-755. Bateson, M., Nettle, D., and Roberts, G.(2006). Cues of being watched enhance cooperation in a real-world setting. Biology letters, 2(3), 412-414. Cubitt, T. I., Lesic, R., Myers,G.L., and Corry, R. (2017). Body-worn video: A systematic review of literature. Australian and New Zealand Journal of Criminology 50, no. 3379-396. 23 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial References Ekström M. (2011). Do watching eyes affect charitable giving? Evidence from a field experiment. Experimental Economics, 15, 530–546. Ellis, T., Jenkins, C., and Smith, P. (2015). Evaluation of the Introduction of Personal Issue Body Worn Video Cameras (Operation Hyperion) on the Isle of Wight. Final Report to the Hampshire Constabulary. Available http://www.bwvsg.com/news/ operation-hyperion-final-report/; Farrar, W., and Ariel, B. (2013). Self-awareness to being watched and sociallydesirable behavior: A field experiment on the effect of body-worn cameras and police use-of-force. Washington, DC: Police Foundation. Francey, D., Bergmüller, R. (2012). Images of Eyes Enhance Investments in a RealLife Public Good. PLoS ONE 7, e37397. Gerber, A.S., and Green., D.P. (2012). Field experiments: Design, analysis, and interpretation. WW Norton, Gerber, A.S., Green, D.P. ,and Larimer, C. W. (2008). Social pressure and voter turnout: Evidence from a large-scale field experiment. American Political Science Review 102(1),: 33-48. Grossmith, L., Owens, C., Finn, W., Mann, D., Davies, T., and Baika, L. (2015). Police, camera, evidence: London’s cluster randomised controlled trial of Body Worn Video. London: College of Policing and the Mayor’s Office for Policing and Crime (MOPAC). Haley, K. J., and Fessler, D. M. (2005). Nobody's watching?: Subtle cues affect generosity in an anonymous economic game. Evolution and Human behavior, 26(3), 245-256. Hedberg, E. C., Katz, C.M., and Choate, D.E. (2017) "Body-worn cameras and citizen interactions with police officers: Estimating plausible effects given varying compliance levels." Justice quarterly, 34(4),627-651. 24 Hudson, D. (2014 )“Building Trust Between Communities and Local Police,” Whitehouse.gov, Available https://obamawhitehouse.archives.gov/blog/2014/12/01/ building-trust-between-communities-and-local-police Izawa, M. R., French, M. D., and Hedge, A. (2011). Shining new light on the Hawthorne illumination experiments. Human factors, 53(5), 528-547. Jennings, W. G., Lynch, M. D., and Fridell, L. A. (2015). Evaluating the impact of police officer body-worn cameras (BWCs) on response-to-resistance and serious external complaints: Evidence from the Orlando police department (OPD) experience utilizing a randomized controlled experiment. Journal of Criminal Justice, 43(6), 480-486. Katz, C.M., Kurtenbach, M., Choate, D.E., and White, M.D. (2015) Phoenix, Arizona, Smart Policing Initiative: Evaluating the Impact of Police Officer Body-Worn Cameras. Washington, DC: Bureau of Justice Assistance, 2. Lum, C. M., Koper, C. S., Merola, L. M., Scherer, A., & Reioux, A. (2015). Existing and ongoing body worn camera research: Knowledge gaps and opportunities. George Mason University. Maciag, M.(2016). “Survey: Almost All Police Departments Plan to Use Body Cameras,” Governing.com, available:http://www.governing.com/topics/publicjustice-safety/gov-police-body-camera-survey.html Mateescu, A., Rosenblat, A., and Body, A. (2015). Police Body-Worn Cameras. Data and Society Research Institute, Available https://papers.ssrn.com/sol3/papers. cfm?abstract_id=2569481 25 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial References Henstock, D., and Ariel, B. (2017). Testing the effects of police body-worn cameras on use of force during arrests: A randomised controlled trial in a large British police force. European Journal of Criminology, 1477370816686120. Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial References McCambridge, J, Witton, J.,and Elbourne, D. R.. (2014). Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. Journal of clinical epidemiology, 67(3), 267-277. Metropolitan Police Department. (2016). General Order - Use of Force. GORAR-901.07. https://go.mpdconline.com/GO/GO_901_07.pdf. . Per this policy, “The following actions are designated “reportable uses of force” Metropolitan Police Department.(2016). General Order - Body Worn Camera Program. GO-SPT-302-13. Available https://go.mpdconline.com/GO/GO_302_13.pdf. Miller, L., and Toliver, J. (2014) Implementing a Body-Worn Camera Program: Recommendations and Lessons Learned, Police Executive Research Forum Moore, R. T., and Schnakenberg, K. (2012). blockTools: Blocking, assignment, and diagnosing interference in randomized experiments. Version, 0.5-6, Nettle, D., Nott, K., and Bateson, M. (2012). ‘Cycle Thieves, We Are Watching You’: Impact of a Simple Signage Intervention against Bicycle Theft. PLoS ONE, 7(12), e51738. ODS Consulting (2011). “Body Worn Video Projects in Paisley and Aberdeen - Self Evaluation.” Available http://www.bwvsg.com/wp-content/uploads/2013/07/BWVScottish-Report.pdf. Owens, C., Mann, D., and Mckenna, R. (2014). The Essex body worn video trial: The impact of body worn video on criminal justice outcomes of domestic abuse incidents. College of Policing, UK. Police Executive Research Forum, (2017). Citizen Perceptions of Body-Worn Cameras: A Randomized Controlled Trial. Final Report. Available http://www. policeforum.org/assets/bodyworncameraperceptions.pdf. 26 Ready, J. T., and Young, J. T. (2015). The impact of on-officer video cameras on police–citizen contacts: findings from a controlled experiment in Mesa, AZ. Journal of Experimental Criminology, 11(3), 445-458. Samii, C., and Aronow, P.M.(2012). On Equivalencies Between Design-Based and Regression- Based Variance Estimators for Randomized Experiments. Statistics and Probability Letters, 82(2), 365 – 370. Stanley, J. (2015). Police Body-Mounted Cameras: With Right Policies in Place, a Win For All. American Civil Liberties Union. Available https://www.aclu.org/sites/ default/files/assets/police_body-mounted_camera Stratton, M., Clissold P., and Tucson, R. (2015). Body worn video: Considering the Evidence. Available http://www.bwvsg.com/wp-content/uploads/2015/06/ Edmonton-Police-BWV-Final-Report.pdf. White, M. D., Gaub, J. E., and Todak, N. (2017). Exploring the Potential for BodyWorn Cameras to Reduce Violence in Police–Citizen Encounters. Policing: A Journal of Policy and Practice, paw057. 27 Evaluating the Effects of Police Body-Worn Cameras: A Randomized Controlled Trial References Police Executive Research Forum, (2014). Implementing a Body-Worn Camera Program: Recommendations and Lessons Learned, (U.S. Department of Justice, Community Oriented Policing Services, 2014): Chapter 2, “Considerations for Implementation.” Available https://www.justice.gov/iso/opa/ resources/472014912134715246869.pdf. We thank the Laura and John Arnold Foundation for generous financial support. The research and views expressed in this report are those of The Lab @ DC and do not necessarily represent the views of the foundation. We extend our gratitude to Katherine Barnes, JD, PhD, Donald Green, PhD, Bill Egar, PhD, Jennifer Doleac, PhD, and Donald Braman, JD, PhD, as well as a variety of reviewers from The Lab @ DC and its partners, for valuable feedback. We also wish to thank the many individuals who participated in briefings and shared their thoughtful insights and opinions with us. This study would not have been possible without the Metropolitan Police Department of the District of Columbia. They welcomed our research team and were committed to understanding, as rigorously as possible, the impacts of the BWC program and how those learnings can be leveraged into program improvements. Special thanks to Chief Cathy Lanier (ret.), Chief Peter Newsham, Matthew Bromeland, Commander Ralph Ennis, Heidi Fieselmann, Derek Meeks, and all the sworn members who dutifully adapted to a new, complicated program and participated in the study. Many thanks to Objectively for layout and web design. We also thank the Executive Office of the Mayor, especially Mayor Muriel Bowser, City Administrator Rashad Young, Deputy Mayor for Public Safety and Justice Kevin Donahue, and Chief Performance Officer Jenny Reed, for dedicating their support, time, and resources to advancing evidence-based governance and policy in the District.