The Sobering Center Comprehensive Program Evaluation and Evaluation Plan October 2019 Acknowledgements This report was prepared by Woollard Nichols & Associates and submitted to the Sobering Center in September 2019, in Austin, Texas. Components of the comprehensive program evaluation and evaluation plan were completed in partnership with Luminosa Research (process and outcome evaluation) and Kim Wilson, DrPH (economic evaluation framework). Sierra Castedo de Martell provided project management support. Lauren Hinkel coded and entered data used for the evaluation. 2 Table of Contents October 2019 .......................................................................................................................................................... 1 Acknowledgements .............................................................................................................................................. 2 Executive Summary.............................................................................................................................................. 4 History, Background and Purpose ............................................................................................................. 4 Methodology ...................................................................................................................................................... 4 Findings................................................................................................................................................................ 5 Conclusions and Recommendations ......................................................................................................... 8 Part I. Background ................................................................................................................................................ 9 History .................................................................................................................................................................. 9 Program Evaluation ..................................................................................................................................... 10 Part II. Process and Outcome Evaluation.................................................................................................. 12 Background and Purpose ........................................................................................................................... 12 Literature Review ......................................................................................................................................... 12 Methodology ................................................................................................................................................... 14 Findings: Process Evaluation .................................................................................................................... 15 Findings: Outcome Evaluation ................................................................................................................. 34 Building Capacity for Data Collection, Evaluation, and Reporting ............................................. 41 Process & Outcome Evaluation Insights ............................................................................................... 44 Part III. Economic Evaluation Framework ............................................................................................... 46 Background...................................................................................................................................................... 46 Economic Literature Review..................................................................................................................... 46 Costs to External Parties ............................................................................................................................ 47 Averted Costs (Benefits) ............................................................................................................................. 50 Methods for Conducting a Cost-Benefit Analysis .............................................................................. 51 Data Issues and Recommendations ....................................................................................................... 55 Part IV. Conclusion and Recommendations ............................................................................................. 57 References ............................................................................................................................................................ 58 Appendix I: Map of Austin Police Department Sectors ....................................................................... 60 Appendix II: Codebook for Core Data Set ................................................................................................. 61 Appendix III: Core Data Set ............................................................................................................................ 65 Appendix IV: Cost-Benefit Analysis Workbook ...................................................................................... 66 3 Executive Summary History, Background and Purpose Sobering centers are an emerging public health intervention intended to divert publicly intoxicated adults from becoming involved in the criminal justice system, and, where appropriate, to prevent the use of emergency medical services in cases of public intoxication (Jarvis, Kincaid, Weltge, Lee, & Basinger, 2019). Further, these sobering centers can serve as a more effective link to recovery support services for individuals struggling with substance use disorders. The Sobering Center serving the City of Austin and Travis County was first examined for feasibility in 2002 and underwent several rounds of further feasibility studies and implementation planning phases, the final of which concluded in 2015. The 2015 implementation plan identified five primary functions of the planned Sobering Center: 1) safe sobering; 2) medical screening for appropriateness and safety of patients; 3) referrals to treatment; 4) peer recovery support; and 5) security for staff and patients. Staffing practices were initially modeled after the Houston Recovery Center (Houston’s sobering center) and included intake staff primarily consisting of paramedics and similar technical positions, and sobering support staff consisting of primarily peer recovery support services-oriented positions. The mission of the Sobering Center is to enhance public health and safety by providing an alternative to the emergency department and jail for publicly intoxicated individuals to safely sober, and, when appropriate, provide a safe environment to initiate recovery. The resulting facility opened its doors for limited service hours (weekends) in August 2018 and became fully operational in October 2018. In 2019, the Texas Health and Human Services commission funded a multi-faceted initiative involving a community needs assessment, prevention plan, and program evaluation of the Sobering Center. This report contains a process, outcome and efficiency evaluation plan, and a process and outcome comprehensive evaluation. Methodology The process and outcome evaluation used both quantitative and qualitative data sources to examine whether program processes are being implemented as proposed and whether these services are producing the desired outcomes. Qualitative data included two site visits to the Sobering Center and 15 in-depth interviews with Sobering Center board members, staff, and other stakeholders. Quantitative data sources included: patient data collected by Sobering Center staff, aggregate statistics on police department arrests for public intoxication (PI) from before and after the Sobering Center opened, data from all incident reports completed by Sobering Center staff, a patient satisfaction survey administered to patients admitted to the Sobering Center, and a satisfaction survey administered to law enforcement officers (LEOs) who transported publicly intoxicated people to the Sobering Center. The economic evaluation plan provides a framework and important preliminary data about external entities that will aid in the completion of a future cost-benefit analysis (CBA). The 4 economic evaluation plan takes a societal perspective, meaning that it examines costs and benefits both to the healthcare system, as well as to external entities, including public safety departments, Austin-Travis County Emergency Medical Services (EMS), emergency departments (EDs) and hospitals, and to the City of Austin and Travis County as the primary funders of the Sobering Center. A key assumption of the framework is that the time cost to individual patients is roughly equivalent regardless of where they undergo the process of sobering, and thus not included in the framework. Findings Process The planning and early implementation of the Sobering Center was well-conducted, and there was buy-in across multiple stakeholder groups and an effective educational campaign. Sobering Center staff are competent and committed to the Center’s mission. Consequently, in its first year of operation, the Sobering Center had 1,824 patient encounters with a high admission rate (88%). This led to 1,605 individuals safely sobering at the Center rather than at EDs or the jail. Additional process measures reveal: • Drop-off is an efficient process with a median time of six minutes for both AustinTravis County Emergency Medical Services (EMS) and LEOs. • Most patients were first-time visitors. Only 9% of patients were repeat visitors. Repeat visitors are significantly more likely to be male, non-student, homeless, and/or uninsured. There were no race/ethnic differences in repeat and non-repeat patients. • Approximately 89% of Sobering Center patients reported alcohol use. Approximately 10% of Sobering Center patients reported opioid use; a figure higher than that reported at the Houston sobering center (less than 3%). • Sobering Center patients are similar in age and race/ethnicity to the general Austin population; however, they tend to include vulnerable subgroups in terms of income, insurance, homelessness, and military/veteran status. • Sobering Center patients and LEOs are very satisfied with Sobering Center services. (EMS professionals were not surveyed.) • The Sobering Center should continue to 1) promote safety at the Center, 2) monitor safety concerns among employees, and 3) track significant incidents at the Center. • Sobering Center staff would like to have better communication with management, greater clarity in their roles and responsibilities, and additional training. There is also a need for more Spanish-speaking employees. • Current board members suggested recruiting a board member with a strong background in development, fundraising, and connections with foundations and individual philanthropists and a board member who is a Certified Public Accountant to serve as treasurer in the future. • Among law enforcement transports, 58% of patients are missing data on the sector the transport originated from. 5 Outcomes The primary goals of the Sobering Center are to provide a place for publicly intoxicated individuals to safely sober, gain access to treatment, and avoid the personal costs of an emergency department (ED) visit or an arrest. The Center also aims to reduce misuse of medical and/or criminal justice resources for substance use disorders. In the current evaluation, we were able to examine some of these outcomes. • The average stay was 7.5 hours, suggesting most patients stay enough time to sober up. • Sobering Center staff distributed 12 naloxone kits to patients during the study period. • Screening, brief intervention, and referral to treatment (SBIRT) increased over the study period. o By the end of the observation period, three-fourths of Sobering Center patients were screened for drug and/or alcohol use disorders using the Drug Abuse Screening Test (DAST) and the Alcohol Use Disorders Identification Test (AUDIT). o By the end of the observation period, 81% of those profiling as high risk for a drug use disorder and 76% of those profiling as high risk for alcohol use disorder received SBIRT (Screening, Brief Intervention, and Referral to Treatment). o SBIRT counselors are needed in the late evening/early morning hours and on weekends so that all patients in need of SBIRT receive it. • Evaluation data indicate that 51 Sobering Center patients went directly to treatment. Repeat patients were almost three times as likely to go to treatment as first-time patients. In fact, one patient went to treatment after their 14th visit to the Sobering Center. Thus, repeat patients do not represent “failures” of the Sobering Center. Rather, repeat patients may be ideal targets for Sobering Center services. • Law enforcement data reveal that PI arrests were declining before the Sobering Center opened. However, the rate of decline increased after the Sobering Center opened. 6 Economic Evaluation Planning Costs to all relevant external parties were estimated and are detailed in Figure ES1 below. Fig. ES1. Costs to External Parties / Partners Category Donated Medical Supplies Amount $3,407 Notes EMS data for 9/1/18 – 7/31/19 EMS Transport to SC $105.28 / transport EMS Transport from SC to Other Facility $ 68.90 / transport Calculated from EMS data (average time from dispatch to clearance = 59.2 minutes, hourly rate of one medic and one clinical specialist plus estimated fringe (30%) + loaded mileage for 3.5 miles (est. average miles traveled from pick-up to Sobering Center) Calculated from EMS data (average time from dispatch to clearance = 55.8 minutes, hourly rate of one medic and one clinical specialist plus estimated fringe (30%) + loaded mileage for 1.05 miles (est. average miles traveled from Sobering Center to final destination = 1.05) EMS Foregone Reimbursement $ 171.90 / transport EMS data, 15% of Basic Life Support charge ($831) + loaded mileage for 3.5 miles (@ $13.50 / mile) Sobering Center Rental Value $270,000-324,000/year Assumes rate of $25-30 per square foot (Class C facility adjacent to downtown Austin) Benefits, in the form of costs that would normally be incurred to public entities in Austin and Travis County, but are instead averted because of the use of the Sobering Center, are detailed in Figure ES2 below. Fig. ES2. Value of Program Benefits / Averted Costs Cost Category ED visit, alcohol intoxication w/o complications Jail booking Base Estimate $1,400 $ 153 Jail day $ Range $670-1,591 97 Source for all: Central Health Joint Technology Team, Travis County Justice Planning Dept and ECHO. In addition, for ranges of ED visits, Scheuter 2019 and Smith-Bernardin 2017. The process for conducting a comprehensive cost-benefit analysis (CBA) is detailed and a worksheet is provided in Appendix IV, including a worked example. The process for conducting the CBA and addressing the following six objectives are described: 1. To compare program costs to program benefits in monetary terms for the entire patient population; 2. To calculate how costs of administering the program to patients who use opioids compare to the benefits accrued for these patients; 3. To estimate the portion of the target population the Sobering Center is reaching; 4. To discuss the internal and external factors that impact the program’s ability to reach 100% of the target population; 5. To discuss how an increase in service provision would impact the findings; and 6. To explain how costs and benefits accrue to various stakeholders. 7 Additional salient economic considerations include: • Typically, Austin-Travis County EMS is reimbursed by private or public (Medicaid/Medicare) insurance for transportation to EDs. Currently, EMS cannot bill for transports to the Sobering Center, thus EMS transports to the Sobering Center result in foregone reimbursement income to EMS. • The benefits to individual patients, especially those connected to treatment and recovery resources for substance use disorder, were not quantified, but could be included in a future CBA. Conclusions and Recommendations The Sobering Center is an effective program that provides a substantial public health benefit to Travis County, the City of Austin, and publicly intoxicated individuals receiving Sobering Center services. While opportunities for further growth and systems improvement were detailed in the preceding sections, the Sobering Center is clearly fulfilling its mission statement by providing safe sobering and connection to treatment and recovery resources for publicly intoxicated individuals in Austin and Travis County. The Sobering Center is also fulfilling its goal of relieving some of the burden of public intoxication on jails and EDs. 8 Part I. Background History Sobering centers are an emerging public health intervention intended to divert publicly intoxicated adults from becoming involved in the criminal justice system, and, where appropriate, to prevent the use of emergency medical services in cases of public intoxication (Jarvis et al., 2019). While the current body of literature for sobering centers is sparse, the existing evidence supports the assertion that these centers will reduce the burden of problematic substance use on law enforcement and emergency medical services, and that these centers can offer a more appropriate intervention to the problem of substance misuse and substance use disorder (Jarvis et al., 2019; Scheuter, Rochlin, Lee, Milstein, & Kaplan, 2019; Smith-Bernardin, Carrico, Max, & Chapman, 2017; SmithBernardin & Schneidermann, 2012). The first feasibility study related to the implementation of a sobering center serving Austin and Travis County occurred in 2002, with a second round of planning activities in 2009 (Sobriety Center Planning Committee, 2015). In 2015, the Sobriety Center Implementation Report prepared for the Austin City Council and Travis County Commissioners Court detailed the results of an extensive feasibility study and implementation planning period (Sobriety Center Planning Committee, 2015). The 2015 planning committee recommended 5 primary functions of the Sobering Center: 1. Safe sobering lasting between four and eight hours, including liquid hydration but no food, on an assigned bed or mat; 2. Medical screening to ensure safety and appropriateness of the patient for receiving services at the Center, provided by trained paramedics; 3. Referrals to treatment for appropriate patients to specialists in the community; 4. Peer recovery support delivered by trained peer recovery support specialists rather than by licensed clinicians or counselors, with peer recovery support specialists serving as the primary staff monitoring patients during their safe sobering, and providing brief intervention, motivational interviewing, and referral to community resources; and 5. Security provided by off-duty law enforcement to ensure safety for patients and staff. The planning committee also recommended staffing practices modeled after the Houston Recovery Center (the sobering center serving Houston), which, for the primary functions of the Sobering Center, include two main parts: intake and support. Intake staff include one intake supervisor, 5 paramedic intake specialists, and 3 psychiatric technicians. Support staff include one recovery support supervisor, 4 recovery support shift leads, 6 peer recovery support specialists, and 2 case management/community support services specialists. The resulting facility, by then formally named the Sobering Center, opened its doors for limited service hours in August 2018, and officially launched on October 1st, 2018. In 2019, the Sobering Center received a multi-faceted grant from the Texas Health and Human Services Commission, intended to support opioid response activities at the Sobering Center. In addition to supporting the ongoing and expanded scope of Sobering Center 9 activities, a second goal of the grant was to support the development of an evaluation plan and implementation of a comprehensive program evaluation of the Sobering Center’s operations. Program Evaluation Program evaluation is a process of distinguishing effective social programs from ineffective ones following a systematic approach, illustrated in Figure 1 (Rossi, Lipsey, & Freeman, 2004). Effective social programs are planned in response to clearly identified needs (needs assessment) and are based on a thorough review of the evidence base for the target social issue (program theory and design; Rossi et al., 2004). In this case, the extensive research conducted in 2002, 2009, and 2015 clearly identified and quantified the need for a sobering center serving Austin and Travis County, and laid a solid foundation of program design (Sobriety Center Planning Committee, 2015). Subsequent phases of program evaluation – process evaluation, outcome evaluation, and efficiency evaluation – are undertaken once the social program has been operating for some length of time. Process evaluation answers the following two questions: 1) is the program reaching the intended population, and 2) is the program being implemented as intended. These two process questions refer to coverage and delivery, respectively (Rossi et al., 2004). Successes and challenges related to staffing practices and training, internal policies and procedures, outreach efforts, and collaboration with outside entities all contribute to process measures, thus stakeholder interviews, site visits, and observation are all critically important tools of process evaluation. Outcome evaluation is where the intended effect of a program is directly measured. In this case, the Sobering Center’s primary goals of reducing the burden of public intoxication on law enforcement and emergency medical services and providing more appropriate care for those struggling with problematic substance use are examined. The final tier of the evaluation hierarchy is efficiency evaluation. In this type of evaluation, the efficient expenditure of resources is the target of evaluation, thus it is commonly referred to as an economic evaluation. There are many methods of economic (efficiency) evaluation (e.g. cost-effectiveness analysis, return on investment, cost-benefit analysis, etc.). For each tier of the evaluation hierarchy, an evaluation plan should be created to guide evaluation efforts, both for the evaluation currently being undertaken, as well as for future evaluations. 10 Efficiency Evaluation Outcome Evaluation Process Evaluation Evaluation of Program Theory and Design Needs Assessment Figure 1. The evaluation hierarchy. Adapted from Rossi, Lipsey, & Freeman, 2004. This report contains a process, outcome, and efficiency evaluation plan, as well as a comprehensive process and outcome evaluation. Part II of this report details the process and outcome components, and Part III details the economic evaluation plan. Part IV of this report offers final recommendations and conclusions. 11 Part II. Process and Outcome Evaluation Background and Purpose The mission of Austin-Travis County Sobering Center is to enhance public health and safety by providing an alternative to the emergency department and jail for publicly intoxicated people to sober and, when appropriate, provide a safe environment to initiate recovery. Its goals include the following: freeing up law enforcement officers (LEOs) to return to their patrol to handle more serious offenses and reducing booking costs, crowding in jails, and use of emergency departments. The Sobering Center opened on August 23, 2018 on a trial run, operating mostly on weekends. On October 1, 2018, the Center officially opened, operating 24 hours a day, 7 days a week. The purpose of this project is to carry out a process and outcome evaluation using existing data and design an evaluation plan for future use by the Sobering Center. Literature Review According to Texas Statute, “Public Intoxication,” generally a Class C misdemeanor, is considered an offense when a person appears in a public place while intoxicated to the degree that the person may endanger him/herself or another (Texas Penal Code, 1994). “Intoxicated" means: A) not having the normal use of mental or physical faculties by reason of the introduction of alcohol, a controlled substance, a drug, a dangerous drug, a combination of two or more of those substances, or any other substance into the body; or B) having a blood alcohol concentration of 0.08 or more (Texas Penal Code, 1994). Public intoxication (PI) and substance misuse have significant consequences for the intoxicated individual as well as the community. One issue is the need to protect publicly intoxicated individuals from harming themselves or others. Thus, people who are publicly intoxicated are typically taken to emergency departments or arrested and placed in jail. However, research shows intoxication presents a significant burden for emergency departments (ED) and that ED visits of this nature have been increasing (White, Slater, Ng, Hingson, & Breslow, 2018). Likewise, studies show prison systems are overcrowded and inappropriately provide criminal justice solutions to what are mental health disorders, according to the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013; Subramanian, Delaney, Roberts, Fishman, & McGarry, 2015). In short, using the ED and jail for PI is costly for the individual and the community, and the underlying substance use disorder remains unaddressed. An alternative to medicalizing and criminalizing PI is the concept of sobering centers. Sobering centers are public health interventions designed to provide a place for publicly intoxicated people to sober up safely and receive brief intervention and referral to treatment. They also reduce the expense of ED visits and jail time and do not result in the negative consequences of an arrest record. While research on sobering centers is sparse, a study conducted in 2013 identified 27 sobering centers in the United States and compared services among 11 of them (Warren, Smith-Bernardin, Jamieson, Zaller, & Liferidge, 2016). The study’s primary conclusion was that there is considerable heterogeneity in how 12 sobering centers are run. Sobering centers varied in terms of policies and practices surrounding minimum length of stay, walk-ins, staff training, type of treatment, follow-up, and security. No outcome data were available on any of the sobering centers described. There are two published evaluation studies assessing sobering centers. In 2012, a study of San Francisco’s sobering center reported that over 29,000 inappropriate ED visits had been averted through the sobering center, which is primarily an ED diversion program (SmithBernardin & Schneidermann, 2012). Repeat visitors were particularly likely to benefit from continuity of care and case management. The authors also documented program challenges such as a lack of treatment options at discharge and the stress of serving such a high need, vulnerable, and hard-to-change population (Smith-Bernardin & Schneidermann, 2012). An updated evaluation of the San Francisco sobering center found that the majority of the 1,271 individuals served were male, middle-aged and ethnically diverse (Smith-Bernardin et al., 2017). Of these, 68.4% used the center once; 22.6% used it between two and five times; and 9.0% had six or more visits during the year. Average age was 44.4 (SD 0.38) and 82.2% were male. 64.2% of the center’s patients were homeless at the time of their visit and almost 72% reported being homeless at some point in the prior year. Those who had multiple visits were significantly more likely to be homeless than those who used the center’s services only once over the year (81% for 2-5 visits and 87% of those with 6+ visits vs. 55.5% of those with only 1 visit). Almost all (94.5%) had received a diagnosis of alcohol use disorder at some point in their lifetime, while almost 40% (38.8%) had a history of drug use disorder. The patient population also presented with higher levels of comorbidities than the general population. Almost 4 in 10 (39.3%) had depression; 26.1 suffered from some form of psychosis; 20.1% had liver disease and 23.1% had hypertension; 19.6% had chronic obstructive pulmonary disease (COPD; Smith-Bernardin et al., 2017). A 2019 evaluation of Houston’s sobering center reported 28,282 patient encounters from 2010 to 2017 (Jarvis et al., 2019). Approximately one-fourth (23%) were frequent visitors (Jarvis et al., 2019). Most patients (82%) reported alcohol use, and less than 3% reported opioid use (Jarvis et al., 2019). PI arrests declined by 95% once the Houston sobering center opened; however, no historical or comparison group data were presented to determine whether this decline was entirely attributable to the sobering center or if it was an acceleration of previous declines in PI arrests (Jarvis et al., 2019). The authors also argued that visits to the Houston sobering center represented a cost savings as each visit was less than half the cost of an arrest (Jarvis et al., 2019). The Houston sobering center was not yet “profitable” as it was operating significantly under capacity (19% capacity) and the researchers stated the center would have to operate at 42% capacity to break even (Jarvis et al., 2019). Almost half (48%) of patients accepted a referral for services, mostly housing and treatment, though there was no verification of whether patients followed through on these referrals (Jarvis et al., 2019). The authors acknowledged that their data are purely descriptive and do not include measures of efficacy (Jarvis et al., 2019). Existing literature suggests that sobering centers are increasingly used as ED or jail diversion programs and to bring more services to patients struggling with substance use disorder. These centers are diverse, and data on outcomes are limited. Our current 13 evaluation seeks to add to the knowledge on sobering center processes and outcomes and create an outcome-based evaluation plan for future research. Methodology The Sobering Center evaluation is a mixed method, process and outcome evaluation. We used both quantitative and qualitative data sources to examine whether program processes are being implemented as proposed and whether these services are producing the desired outcomes. Qualitative Data Source: Interviews Our qualitative data consisted of two Sobering Center site visits that took place in July and 15 in-depth interviews with key stakeholders that took place from July 22 to August 23, 2019. Eleven interviews were conducted face to face in places convenient to interviewees, usually their offices. Four interviews were conducted by phone due to scheduling difficulties. Interviews lasted an average of 45 minutes. Figure 2 shows the organizations and positions represented by the interviewees. Fig. 2. Entities and Positions Represented by Interviewees, Summer 2019 Austin Police Dept. Pflugerville Police Dept. Sobering Center Board Members Sobering Center Staff Travis County Sheriff’s Office, Central Booking UT Police Dept. Division Manager for Research and Planning Research & Planning Supervisor Lieutenant Chair Treasurer Ex-Officio, Austin Police Dept. Ex-Officio, Travis County Criminal Justice Planning Ex-Officio, Austin-Travis County EMS Clinical Manager Community Outreach Specialist Compliance Manager Paramedic SBIRT Counselor SBIRT Manager Sobering Support Specialist/Lead Director of Inmate Mental Health, Counseling and Education Services Sergeant Quantitative Data Sources Our primary source of quantitative data was patient data collected by Sobering Center staff. These data were maintained in a secure electronic format (monthly Excel spreadsheets) and provided to the evaluation team. They offer information on patients served and services offered from late September 2018 through mid-July of 2019. These data include 14 patients who were admitted to the Sobering Center as well as those transported to the Sobering Center but not admitted (taken later to the ED or jail). The data sheets were scrubbed and merged to produce a single data set. We converted the data to an SPSS format (version 25.0) for additional data coding, restructuring, and statistical analysis. We conducted frequency distributions and bivariate analyses and, where appropriate, ran statistical tests of significance. To supplement the Sobering Center patient data, we obtained aggregate statistics on law enforcement arrests for public intoxication (PI) from before and after the Sobering Center opened. We requested ED data from Seton Dell Medical Center, where during the first year of the Sobering Center’s operation, most of the ED diversions have been expected to occur. Due to hospital staffing constraints, we were unable to obtain this information. We analyzed data on all incident reports completed by the Sobering Center during the study period. These data were initially in hard-copy format. The Sobering Center staff also designed and administered a patient satisfaction survey to patients admitted to the Sobering Center, and results from this survey are summarized in this report. All patients who stay at the Sobering Center in Austin are requested to complete a satisfaction survey during discharge. Usually, they complete the online survey via SurveyMonkey on tablets. When tablets are unavailable, patients complete the survey using paper and pen, and Sobering Center staff enter the responses into SurveyMonkey. The survey takes about 3 minutes to complete and includes closed-ended questions about different aspects of services and an open-ended question asking for comments. The survey is completely confidential and anonymous. It does not include questions about personal characteristics (gender, age, race/ethnicity), nor does it include any patient IDs, chart numbers, or electronic medical record numbers. For this first summary, 541 patients completed the survey from August 24, 2018 to July 15, 2019. The response rate is 29% (541/1,875). According to staff, the main reason for an incomplete survey is that the patients are in a hurry to leave. Eighty-seven percent (n=469) of patients who took the survey said it was their first time at the Sobering Center. It is possible that some patients took the survey more than once. Law enforcement officers who transported people to the Sobering Center from September 2018 to May 2019 were requested to complete a satisfaction survey designed by Sobering Center staff upon each visit to the Sobering Center during that time period. Officers completed the survey using paper and pen. The survey took about 3 minutes to complete and included closed- and open-ended questions about different aspects of the officers’ experiences at the Center. Some 452 LEOs completed the confidential and anonymous survey, which may include more than one response from the same LEO. The LEO survey results are also summarized in this report. Findings: Process Evaluation Board and Governance The Sobering Center is a 501(c)(3) local governmental corporation. The City of Austin and Travis County each appoint board members. Ex-officio members represent partnering 15 entities such as Austin Police Department, Austin Public Health, Austin-Travis County Emergency Medical Services (EMS), City of Austin, and Travis County. As of the writing of this report, the board consists of 15 members. The main benefit of this governance structure and process of appointments is that it encourages commitment in terms of financing and oversight from the city and county. Interviewees noted several strengths of the current board. The Board Chair, Judge Nancy Hohengarten, is an effective spokesperson for the Sobering Center. Board members care deeply about issues related to public intoxication and have strong relationships with one another and with other community members. All members have expertise in specific areas that contributes to the Sobering Center’s success. The main drawback to the structure of the Sobering Center governing board is that the strict rules for operating sometimes create significant bureaucratic hurdles to getting things done. All Sobering Center board meetings are subject to the Open Meetings Act. Public notice must be given before all meetings and quorums must be met. Members must be present at meetings, and call-ins are prohibited. Amending articles of incorporation or bylaws requires a vote by both the city and county. These votes must be on their agendas. Current board members suggested recruiting a board member with a strong background in development, fundraising, and connections with foundations and individual philanthropists and a board member who is a Certified Public Accountant to serve as treasurer in the future. Planning and Launch On August 23, 2018, the Sobering Center had a soft opening with limited hours, operating mostly on weekends only. During the soft opening, staff determined the processes most likely to be successful and promoted the Sobering Center. On October 1, 2018, the Sobering Center began to operate 24 hours a day, 7 days a week. Strengths Several interviewees noted specific aspects of the planning process that contributed to the successful launch and continued operations. These included: • The city and county partnership, where both parties understood potential benefits of the Sobering Center. For example, for the county, benefits would accrue to the jail, and for the city, benefits would accrue to the police department. • The Sobriety Center Implementation Report (2015), or feasibility study, which quantified potential benefits for stakeholders • Commitment and continuity of service among Sobering Center planning members, board members, and ex-officio members • Site visits to and research on other sobering centers, especially the Houston Recovery Center, to review their plans, operations, and lay-outs. This research helped stakeholders buy into the idea of a sobering center in Austin and led to the specific design and layout of the center. 16 • • The Sobering Center building, which is a county-owned, well-maintained, professional-looking facility located downtown. It was “miraculous timing.” The county’s medical examiners’ building was in the process of being vacated, and it had historically been a city-county joint venture, like the Sobering Center. Getting this building made the transition easier and paved the way for the city and county to contribute to the Sobering Center in specific ways. Travis County leases the building to the Sobering Center at a substantial cost savings, and the City of Austin pays the operating costs associated with the building. The effective and efficient promotion, education, and roll-out to LEOs. Officers from University of Texas Police Department and Pflugerville Police Department said introductory emails and tours provided to officers at convenient times were helpful. “They made a big difference because they were hands-on. They told us that tours would be happening during certain time ranges and days, so it was very convenient for UTPD officers to drop by.” Lessons Learned Two opportunities for improvement are associated with the planning and launch of the Center. First, planning should have included the design of a data collection and reporting system. Second, while the launch and promotion went well for LEOs, it was sometimes difficult for Austin-Travis County EMS. During the launch, EMS felt that it did not receive sufficient detail about the soft and official openings. It is difficult for medics who are operating ambulances to remain flexible with a changing and constrained schedule. Once the Center opened 24/7, it became much easier for crews to “make a decision whether they're going to transport an individual there or not.” Staffing Strengths Based on interviews, staff appear mission-driven and dedicated to the Center’s patients. “All share responsibility for each patient who walks through the door.” Interviewees explained that teamwork within shifts is strong. For example, a staff person said, “Great medical director. If there's anything, we can call him, and he gives us direction.” Staff is also professional and competent: • “Well-trained staff, supported” • “Very capable implementation” • “Clinical team is very good downstairs.” The new Sobering Center is a complex organization, and staff has excelled at reacting to change and addressing challenges. They have been flexible as they work to manage the program and provide better care for patients. There have been many changes and evolving processes, and “staff has really embraced that and has done a phenomenal job… ‘What do we need to do, and how do we need to make it work?’” It has been a “huge learning process,” and now Sobering Center staff is sharing its knowledge and experiences with sobering centers being planned around the country. 17 Opportunities for Improvement Strengthen Morale. Due to recent staff transitions, including the Executive Director departure, the resulting dip in communications, and short-term budget constraints, morale among staff is low. “People are worrying about their jobs. People are worrying about their insurance. People are worried about what's the next thing we're going to read in the paper that they didn't tell us?” According to interviewees, staff meetings have not been successful at delivering information to staff. Staff suggested that the organization focus on building trust, cohesion, and morale among staff and leadership. They suggested that management visit with staff on the floor during the day and night shifts. Face-to-face meetings may “ease staff’s stress because [the information] will come directly from management. Those little things go a long way to build team orientation.” Better Define Roles and Responsibilities. There seems to be confusion or lack of clear direction on roles and responsibilities among staff. For example, a staff person said, “Our team and staff as a whole excel at meeting expectations when we know what they are.” Staff would like to see clear boundaries of positions. Some staff explained that their responsibilities have grown as a result of recent transitions and the demands of a recent HHSC grant. Some staff said they were hired for certain roles but have had to take on additional roles that sometimes seem unrelated to the main purpose of being hired. Strengthen Teamwork across Shifts. Teamwork across shifts may need to be strengthened. There are no “rotating shifts. Who's on your shift is generally the same, so every shift typically gets along well with themselves, but there seems to be a rift between shifts.” Staff emphasized a desire for accountability, and some said they feel disgruntled when they sense that they are working more than others or picking up another person’s slack. There is currently one monthly mandatory meeting, and staff suggested a “fun teambuilding event where we could all attend that would be helpful and remind everyone, ‘Hey we're on the same team. We all have the same goal.’” Hire More Employees Fluent in Spanish. Interviewees suggested hiring more employees who speak fluent Spanish. A language hotline is available, but staff is hesitant to use it due to its cost per minute, and they worry about the Center’s operating budget. Plus, with the intoxication issues, the hotline can be a “waste of money.” Training for Staff. As the Sobering Center has matured, training needs have been identified and delivered. In February 2019, Sobering Support Specialists were trained on how to administer screening tools. After screening occurs, SSS educates and monitors lowrisk patients and refers higher-risk patients to SBIRT counselors. The SBIRT counselors were trained on using brief intervention and motivational interviewing within a harm reduction model. Staff explained that these trainings were valuable. According to interviews, aspects of training for staff can be improved. Some felt that they have not been sufficiently trained to take on additional responsibilities that have resulted from staff transitions. Enhanced training can include de-escalation techniques, risk 18 prevention, restraint of adults, and how to escape from or deal with aggressive patients. Leadership can devise new ways to conduct trainings to account for scheduling realities. These new ways may include video webinars, audio- and video-recorded meetings, or multiple meetings with repeated topics. Grant from Texas Health and Human Services (HHSC) In 2019, the Sobering Center received a grant from HHSC to assist people seeking to reduce their risk of opioid overdose death and initiate medication-assisted recovery. Benefits of the HHSC Grant According to interviewees, the HHSC grant has provided benefits in addition to its funding. Improved Patient Care. Most importantly, the grant has helped the Sobering Center to improve its care of patients who are using opioids. The grant has resulted in increased knowledge related to opioids and treatment among Sobering Center staff and increased services for patients using opioids. For example: The education level across our staff has increased on what an opiate is and what signs and symptoms to look for. The paramedics already knew what to look for, but now that's trickling down to our support staff… Often people who use opiates don't know that they're using them. There seems to be a disconnect among patients between brand name prescription drugs taken for pain and ‘opiate use.’ Staff has learned where to refer patients using opioids. Staff is now better able to identify patients who are using opioids and can offer them naloxone as a result of the grant. Staff is now better able to educate people on the dangers of fentanyl and the likelihood that it is being mixed with the patient’s “normal drug of choice.” Staff now encourages people who use opioids to educate their own social networks that include other people who use opioids. The Sobering Center has developed and now distributes flyers on topics such as how to safely dispose of needles. When staff distributes naloxone to patients, they include verbal and written instructions for using it and recommendations to call 911. The HHSC grant also allowed the Sobering Center to expand its database and capture more information, which will result in better program management and patient care. Purchase and Use of Nicotine Patches. Smoking and vaping are prohibited at the Sobering Center. Often, patients who smoke cigarettes or use electronic cigarettes want to leave the Sobering Center early because they are addicted to nicotine. The HHSC grant led to the purchase and use of nicotine patches for such patients and verbal and written education about how to quit smoking. Increased Collaboration and Communication Among Partners in Travis County. The grant has led to strengthened collaboration among the Sobering Center and entities working in the continuum of care in Travis County. Relationships are stronger with Austin Harm Reduction Coalition, Communities for Recovery, Homeless Outreach Street Team 19 (HOST), Integral Care, the state’s Outreach, Screening, Assessment and Referral (OSAR) centers, and other organizations. Purchase of a Van for Transporting Publicly Intoxicated People. The HHSC allowed the purchase of a van. Currently, a paramedic and a support staff travel in the van and pick up patients cleared by an ED physician at Dell Seton Medical Center. ED staff notifies the Sobering Center that a patient fits the Sobering Center’s criteria and prepares the paperwork. According to interviews, the first van trips have been “smooth transitions” and taken about 20 minutes total. These pick-ups open bed space at the ED. Plans are underway to expand the scope of services provided by the van and the number of hospitals involved. Drawbacks Related to the HHSC Grant. The HHSC uses a reimbursement model that has delayed reimbursements to the Sobering Center and has caused some cash flow issues. Some interviewees believe that the HHSC grant has led to mission creep and distraction from the center’s main purposes. Number of Patients and Admissions Overview From September 2018 to July 2019, the Sobering Center had 1,824 patient encounters. The vast majority of these patients were unduplicated, visiting the Sobering Center only once during the observation period. Approximately 9% visited more than once. Among repeat patients, the number of visits ranged from two to twenty. Among patients transported to the Sobering Center, 88% percent were admitted (n=1,605). Admittance rates were high for both LEOs (87%) and EMS professionals (91%) after transporting publicly intoxicated people to the Center. Reasons for Non-Admittance Generally, to be admitted, patients must meet the Sobering Center’s criteria as measured by the physical and mental health screenings, and patients must refrain from combative or aggressive behaviors during triage. For those patients not admitted (12%), Figure 3 provides the reason the patient did not remain at the Sobering Center. Most non-admitted patients did not have information in the Sobering Center database regarding the reason they were not admitted (although each case was documented in the hard copy incident reports). For the 103 patients for whom this information was available, Figure 3 reveals that the most common reasons for non-admittance were medical clearance and noncompliance. 20 Fig. 3. Reasons Transports Were Not Admitted Medical issue identified during screening Non-compliance Not intoxicated Taken to jail Psychiatric issue identified during screening % 41% 36% 14% 7% 3% n 42 37 14 7 3 Data source: Patient data collected by Sobering Center staff, 9/2018-7/2019 The analysis of the Incident Report data supported the Sobering Center patient data and showed that most non-admitted PIs were excluded because they did not meet the medical or psychological criteria when screened during triage. During interviews, staff explained two barriers to intake that they have recently addressed: use of the breathalyzer and the signing of the consent form. Some patients did not want to stay at the Sobering Center for fear of using the breathalyzer in front of a police officer or for fear of the blood alcohol content (BAC) being documented and shared with authorities. Sobering Center staff changed the process in February 2019 so that now, the breathalyzer can be administered outside the view of LEOs at the Sobering Center. If a patient is still wary of using a breathalyzer, Sobering Center staff may accept the patient and continue to offer a breathalyzer on the hour. “If someone refuses a breathalyzer on intake, but they admit to drinking alcohol, we can use clinical judgement to determine how often we're going to check their vitals, how closely we monitor them. This has made a significant difference on how many people we're able to keep because some people, especially those with a language barrier, don't understand that the breathalyzer is for medical purposes only.” The signing of the consent form represents the second barrier to staying at the Center. Recently, the Sobering Center staff created flexibility on when and where the consent form is signed. Some patients have been too intoxicated to understand or sign the consent form during triage. Now, staff “ask patients to sign it during their stay. Sometimes, when patients are unable to make decisions for themselves due to intoxication, staff sign and document that this person couldn't make decisions for themselves. We take care of them.” Capacity We examined the Sobering Center capacity relative to the number of patients that use the Center on any given date (day or night). While most patients use the Center in the evening to early morning period, we use the term “day” in our analysis to represent a 24-hour period. Currently, the Center serves an average of 6 individuals per day, 4 males and 2 females. The Center has the capacity to offer beds to 12 males and 12 females per day. Thus, on a typical day approximately 34% of the beds for males and 13% of the beds for females are utilized. The number of patients per day varies considerably, ranging from 1-13 for males and 1-8 for females. Thus, the Sobering Center appears to have an appropriate 21 number of beds for meeting the existing community need. Given that there were several dates where the number of males was close to or exceeded capacity (11-13) and no days in which there were more than 8 females, the Sobering Center may want to increase the number of available beds for males and decrease the beds designated for females. Designating bed space is a relatively simple affair due to the center’s enhanced floor design. Recently, staff changed the layout of the dorm, so it is easier to maintain and monitor. Instead of two separate rooms for male and female, the center now uses one large room with mobile privacy screens. The two separate rooms required splitting up staff. In the new large room, staff can stay together, and when capacity is low on weekdays, one staff person can monitor all patients. “Especially busy nights, Friday and Saturday, when we have 10 to 14 people, having multiple staff members in the same room sharing the workload of all of the patients is significantly easier than having one Sobering Support Specialist in each dorm and the paramedic hopping back and forth because there are crises happening in both rooms.” Profile of Patients and Substance Use Figure 4 provides a demographic profile of Sobering Center patients compared to the demographic profile of Austin residents. Approximately three-fourths of Sobering Center patients are male (73%), a percentage much higher than the general population of Austin residents. The median age is 31 and ranges from 18 to 73. The racial/ethnic profile of Sobering Center patients is almost identical to that of the City of Austin. Fifty percent of Sobering Center patients are white, 35% percent are Hispanic, and 10% are African American. Approximately 8% of Sobering Center patients are military/veterans, 12% are students, and 22% are homeless. Almost half of Sobering Center patients do not have insurance (48%) and patients generally profile as low income. A very small number of Sobering Center patients are defined as “high need.” Approximately 2% were IV drug users (n=33) and .1% were pregnant (n=2). Looking specifically at repeat Sobering Center patients, the data show that repeat visitors are significantly more likely to be male, non-student, homeless, and/or uninsured. There were no race/ethnic differences in repeat and non-repeat patients. 22 Fig. 4. Descriptive Profile of Sobering Center Patients SC Patients Austin MSA Sex Male Female 73% 27% 51% 49% Median Age 31 33 Age Group 18 to 24 25 to 34 35 to 44 45 to 64 65 and older 22% 37% 19% 22% 1% 15% 21% 15% 21% 7% Race/Ethnicity White non-Hispanic Hispanic African American non-Hispanic Asian non-Hispanic Other 50% 35% 10% 2% 4% 49% 35% 8% 7% 2% Median Household Income $15,000 $73,800 Veterans 8% 5% Uninsured 48% 16% Students 12% - Homeless 22% - IV Drug User 2% - Pregnant <1% - Data derive from patient data collected by Sobering Center staff (9/2018-7/2019) and Census Bureau's 2016 and 2017 American Community Survey five-year estimates. Austin uninsured rate for residents under 65 Figure 5 provides information on the types of substances used by Sobering Center patients. Most Sobering Center patients (89%) reported consuming alcohol. Ten percent of Sobering Center patients reported using prescription and/or street opioids, e.g. heroin, Oxycontin, methadone, fentanyl. This rate of opioid use is higher than that reported for the Houston sobering center (less than 3%; Jarvis et al., 2019). Reported use of other drugs such as marijuana, methamphetamine, cocaine/crack were less common than opioid use. 23 The median BAC level of Sobering Center patients is approximately three times the legal limit for driving (.08 percent). Approximately one in five Sobering Center patients report poly-substance use, and one in four patients, according to the triage assessment, require the highest level of monitoring while sobering at the Center. Fig. 5. Substance Use Characteristics of Sobering Center Patients % n Substance Use Type (self-report) Alcohol Opioids Marijuana Methamphetamines 89% 10% 5% 2% 1619 179 93 43 Poly Substance 19% 345 Median BAC at intake (.00-.50) 0.21 92 Triage Level 1-Unable to stand 2-Unsteady 3-Normal 24% 46% 30% 396 774 497 Data source: Patient data collected by Sobering Center staff, 9/20187/2019 Patient Satisfaction Results from the Patient Satisfaction Survey show that a large majority of the patients (83%) who visit the Sobering Center are “very satisfied” with the overall experience. Almost all of those who took the survey (98%) expressed satisfaction with the overall experience (Figure 6). Fig. 6. Level of Satisfaction among Patients at the Sobering Center Very satisfied Satisfied No opinion Unsatisfied Very unsatisfied Total % 83% 15% 2% 1% 100% n 447 79 10 1 4 541 - Less than .5% Data source: Patient Satisfaction Survey, 8/24/2018-8/15/2019 24 Large majorities of patients who responded to the survey “strongly agreed” with the statements about different aspects of services received, saying they were very satisfied. Almost all (94% to 98%) “agreed” with the statements in Figure 7. Fig. 7. Percentage of Patients Expressing Satisfaction with Aspects of Services Received, August 2018 to July 2019 Strongly Agree Agree No Opinion Disagree Strongly Disagree Patient received warm and sincere greeting upon arrival 77% 17% 6% - - Purpose of center was explained to patient 75% 20% 4% 1% 1% Patient offered assistance appropriate to needs 79% 19% 2% - - Patient satisfied with recommendations offered 79% 17% 3% - - Staff was empathic and supportive toward patient and situation 83% 14% 1% - 1% Counselor seemed very knowledgeable about situation and substance abuse 80% 16% 4% - 1% Patient would feel comfortable calling back if further assistance needed 80% 16% 3% - 1% - Less than .5% Data source: Patient Satisfaction Survey Several patients (n=122) offered comments on the survey. Thirty-nine percent of these comments centered on gratitude. For example, one patient said, “Very grateful for this service, gave me counseling tips that I am going to take into consideration.” Another said, “Very glad to be alive. Today was a wake-up call.” Thirty-eight percent expressed appreciation for the center’s staff. Responses included: “Great service, great people, no judgement, thank you!”, “Very kind compassionate team, appreciate you very much!”, and “The staff was extremely helpful and kind.” Fifteen percent made other general positive comments about the Sobering Center. Some patients offered suggestions for improvement. One suggested providing newspapers or magazines to read or a television to watch. Patients can make phone calls at the end of the discharge process, but one patient said, “It would have been nice to be able to contact someone” during the stay. Transports Among patients transported to the Sobering Center, 74% were brought by LEOs, and 26% were brought by Austin-Travis County EMS. Figure 8 provides trend data on Sobering Center transports for both LEOs and EMS. Transports have been fairly stable over the ten 25 months observed, with no notable increases or decreases in use of the Sobering Center over time. Figure 8: Patient Transports Over Time 250 200 150 100 50 0 9/18 10/18 11/18 12/18 EMS 1/19 2/19 LEO 3/19 4/19 5/19 6/19 Total Data source: Patient data collected by Sobering Center staff Transports by Law Enforcement Officers (LEOs) APD Sectors. Figure 9 provides information on the Austin Police Department (APD) sectors where PIs are brought to the Sobering Center rather than jail. (See appendix for map of sectors.) Among law enforcement transports, 58% of patients are missing data on the sector the transport originated from. Some missing data are due to patients being transported by law enforcement agencies other than APD. These include but are not limited to Pflugerville PD, Travis County Sheriff’s Department, and University of Texas PD. For patients with data for APD sector, the largest number of patients came from Sector George (n=229 patients). Given that Sector George is near the Sobering Center and downtown Austin, it is logical that this sector would provide the highest number of transports. The lowest number came from Sector Charlie (n=41), which is located directly northeast of the downtown sector, e.g. East 12th street. It is unclear whether the sectors with a low number of transports reflect different rates of PI in the sector, different use rates for the Sobering Center for PI arrests, or some combination of the two. 26 Fig. 9. Transports by APD Sector A - Adam B - Baker C - Charlie D - David E - Edward F - Frank G - George H - Henry I - Ida % n 9% 13% 5% 9% 8% 11% 30% 8% 8% 70 97 41 66 59 83 229 63 63 Data source: Patient data collected by Sobering Center staff, 9/2018-7/2019 Barriers to Transports by LEOs. The existence of the Sobering Center has no bearing on whether an officer takes action in the first place and detains a person because police action is based on an intoxicated person’s behavior at the moment. While it is not against the law to be drunk in public, if a person is stumbling toward a car to drive it, stumbling in the middle of the road, or creating a disturbance, that person is considered to be a danger. Interviewees described these barriers to transporting people to the Sobering Center: • APD’s policy requires that officers take people to the Sobering Center instead of jail if they meet its criteria. The process is new, and some officers need continued education. • Officers may be unable to transport PIs to the Sobering Center when a serious crime occurs. Police departments prioritize serious crimes, re-position and re-schedule officers, and de-prioritize PIs. • According to interviews with officers from Austin PD, Pflugerville PD, and University of Texas PD, they are often unable to transport people to the Sobering Center because the person does not meet the Sobering Center’s criteria. Austin PD noted that many publicly intoxicated people are aggressive and therefore an inappropriate fit for the Sobering Center. Others are being charged with an additional crime when they are intoxicated or have outstanding warrants. In Pflugerville, the main criteria for taking someone to Sobering Center is whether they are combative or not. UT Police said that the main reason they do not take people to the Sobering Center is that they are “very sick,” and the police opt for the ED. • According to interviews, arrests for misdemeanors are at historic lows at APD. There are several reasons for the lower arrest rates. o APD changed some of the policies associated with cite and release in summer 2018. There are now only four misdemeanor offenses, down from twelve, where an officer must make an arrest. For the others, a citation may be appropriate. These include possession of marijuana and driving while license is invalid. 27 o There is a robust Forensic Assertive Community Treatment (FACT) team with 95 slots. FACT is a multi‐disciplinary, team‐based intervention that creates an alternative to incarceration of individuals living with serious mental illness. o APD is operating under capacity with regard to the number of officers. So, if officers are faced with A) arresting a publicly intoxicated person or B) obliging the person’s family member or friend to take him and return to police work, the officers will “rightfully choose option B.” o Public intoxication is not a top priority of law enforcement, and arrests of public intoxications have been decreasing. “Officers now find solutions other than taking a person to jail. They find somebody that can come get them. Everybody has a cell phone now. Officers can call a brother or whoever to come get them… These days, officers are less inclined to make arrests and more likely to work through a problem and find an alternative solution. People are with their friends during festivals, and so we find a way to get them a ride home and move on.” Addressing Transport-Related Barriers Faced by LEOs. The Sobering Center and its stakeholders can continue to educate officers and promote benefits to police departments and the community. Austin Police officers who don’t work downtown are less likely to know about the Sobering Center when they find themselves working in those sectors. Education and promotion can be tailored to officers in Del Valle, Manor, Pflugerville, and Sunset Valley. Officers are likely to appreciate hearing the positive feedback from their peers. An interviewee suggested that law enforcement agencies require additional reporting, or paperwork, from officers who choose to take offenders who meet the Sobering Center’s criteria to jail rather than the Sobering Center. The additional reporting might explain why the officer chose to take a PI to jail instead of the Sobering Center. APD is currently addressing some barriers to increase the diversions of appropriate PIs to the Sobering Center. First, APD will soon be transporting some PIs held at the processing facility during weekends at the downtown location of Caritas to the Sobering Center. Many officers working on 6th Street are on bike or foot, so they take PIs to the processing facility to await an APD van that then takes the PIs to Central Booking. The APD van will soon transport one group of PIs to Central Booking and another group of PIs to the Sobering Center. Second, APD continues to educate officers on the Sobering Center’s criteria for accepting PIs. APD prohibits officers from taking aggressive PIs to the Sobering Center, and it is each officer’s discretion that determines if a publicly intoxicated person is aggressive or not. Officers can arrest a publicly intoxicated person who is fighting, and this person is considered aggressive because he's fighting. But, as the situation and person calm down, perhaps officers can decide to transport the person to the Sobering Center. Transports by EMS Stations and Districts. As the only EMS provider that receives 911 calls in the area, Austin-Travis County EMS is the only EMS transporting PIs to the center. Figure 10 provides information on the EMS districts/stations transporting patients to the Sobering Center. District/station data is missing for most Sobering Center patients brought by EMS; however, the data that are available reveal that approximately half of the EMS transports 28 (49%) come from two stations. Station 6 brought 52 patients and Station 3 brought 39 patients. These stations are located on Red River and E. 5th street, which are areas near the Sobering Center and where PI patients may be common. Of the 49 stations, 23 (47%) have transported fewer than 10 patients to the Sobering Center, and 24 (49%) have not transported any patients to the Sobering Center. As with the PD transports, it is possible that these stations do not have a high incidence of PIs. It is also likely that medics at these stations, which are considerably farther from the Sobering Center than Stations 3 and 6, find the travel time prohibitive. Barriers to Increased Transports by EMS. During interviews, an EMS professional said, “The right thing is to move [a publicly intoxicated person] to the Sobering Center instead of a hospital because a bed can be freed up for another hospital patient.” Austin-Travis County EMS has given their employees directives and guidelines to use the Sobering Center, and EMS is promoting transports to the Sobering Center as a positive benefit for the community. The Sobering Center, rather than the healthcare system, is a more appropriate place for some intoxicated people to be. However, barriers to transports to the Sobering Center by EMS exist. The following barriers, described during interviews, are listed from minor to major: • The process of taking an intoxicated person to the ED is “pretty fast” (unlike jail bookings), and EMS crews can easily pick up food and drinks at the hospital when they’re on the run. • Often, EMS professionals, when deciding on where to take publicly intoxicated people, err on the side of caution and transport them to a hospital due to concern over a patient’s potential underlying medical conditions or complications. They are trained to believe that the ED is a safer bet for them and the patient. • For many EMS professionals, there is an incentive to avoid downtown. Once they enter the downtown area, it is difficult to return to their home districts. “It is highly unlikely that medics working the donut around Austin will take people to the Sobering Center.” • In its current configuration, the Sobering Center is not a relief for the EMS system because EMS is still taking a person from point A to point B (either the Sobering Center or Dell Seton Medical Center, which is just a few blocks away). For either trip, crews must complete the transport, complete a patient care report, clean their unit, and ensure they're ready for the next call, etc. • Most significantly, EMS cannot bill for transports to the Sobering Center. Due current billing and insurance realities, there is little to no financial savings for EMS. Even if EMS could bill individuals, it is likely that few individuals who go to the Sobering Center have the necessary insurance coverage or funds to pay out of pocket. 29 Fig. 10. Transports by EMS Stations Station 1 Station 2 Station 3 Station 4 Station 5 Station 6 Station 7 Station 8 Station 11 Station 12 Station 13 Station 14 Station 15 Station 17 Station 20 Station 28 Station 30 Station 32 Station 33 Demand 3 District 1 Other districts % 2% 5% 21% 5% 3% 28% 1% 3% 1% 4% 1% 2% 2% 4% 1% 1% 2% 1% 5% 1% 4% 5% n 4 9 39 10 5 52 2 5 2 7 1 4 4 8 1 1 4 2 9 1 8 7 Source: Patient data, 9/2018-7/2019 Sobering Center Van The new van recently began to transport patients from Dell Seton Medical Center to the Sobering Center. According to staff, it is working well. The van has audio and video recording. Staff expressed some concerns about the van. They believe the van may need: • A step to prevent staff and PIs from falling while getting in or out of the van • A divider of some kind between the driver, who is a Sobering Center employee, and the PI(s) and accompanying Sobering Center staff person who sits with the PI • Side door locks operated by the driver to prevent PIs from exiting van • A lock to prevent passengers from disengaging the back doors of the van The APD transport van has complete separation between the driver and the passengers, and it has a drain in the floor. Staff also noted that the staffing model at the Center may need updating since purchasing the van. Currently, during some shifts, the Center doesn’t have enough staff working to send two employees on a van transport. 30 Intake and Triage Intake Process Once at the Sobering Center, the process of dropping off a PI appears to be an extremely efficient one for EMS and LEOs. Data from the Sobering Center showed the average elapsed time for intake was 12 minutes. The median elapsed time was 6 minutes, and this was the case for both EMS and LEOs. Likewise, LEOs who brought PIs to the Sobering Center and who completed the satisfaction survey said the amount of time spent by officers at the Center is generally less than 10 minutes (Figure 11). Fig. 11. Amount of Time Spent by LEOs at the Sobering Center 5 or fewer minutes 6 to 10 minutes 11 to 15 minutes 16 to 20 minutes More than 20 minutes Total % 40% 50% 7% 3% 100% n 180 221 32 12 1 446 - Less than .5% Source: LEO Satisfaction Survey, 9/2018-5/2019 The intake process is “efficient and quick.” It is a timesaver for LEOs due to the quick dropoff and the medical screenings. From an Austin Police Officer's point of view, it's a great alternative to spending two to eight hours arresting a publicly intoxicated person: If you go on Friday and Saturday nights when most people are intoxicated, then you're looking at a couple of hours easily. That's an average. If you take somebody in that's diabetic, he must see a nurse. He goes to the nurse, and his blood sugar is 350. Then, you have to take him to [Dell Seton Medical Center] for the blood sugar to come down. Police officers are doing that for those transfers. That can take six to eight hours. [Officers are] completely off the street because this person's blood sugar was high, and he has unmanaged diabetes. Whereas at the Sobering Center, you just drop them off, it’s eight minutes or so, and you go back to work. A Pflugerville Police Officer agrees: Officers are back on the street relatively quickly, even with the drive. Time driving is not a barrier because overall time consumption for taking someone to the Sobering Center is relatively low compared to arrest. If a Pflugerville Police Officer takes a PI to the Holding Facility in Pflugerville, the officer must go through the booking process, staff must monitor the prisoner, feed the prisoner according to guidelines, and magistrate the prisoner. The survey and interviews show that LEOs are overwhelmingly pleased with the intake process at the center. Officers described the Sobering Center staff as “friendly and accommodating,” “knowledgeable and great,” and “extremely helpful and patient.” Figure 31 12 shows 9% are satisfied and 90% are strongly satisfied with how the Sobering Center staff treated them and the offender. Fig. 12. Law Enforcement Officers' Level of Satisfaction with Sobering Center Staff Satisfaction with how staff treated me Satisfaction with how staff acted toward offender Strongly Agree Agree No Opinion Disagree Strongly Disagree 90% 90% 9% 9% 1% - - -Less than .5% Source: LEO Satisfaction Survey, 9/2018-5/2019 According to a University of Texas Police Officer, the Sobering Center is a “boon for our agency and a boon for our community.” Reported Incidents and Staying at the Center Reported Incidents The Sobering Center meticulously documents each specific challenge confronted in serving patients and how the issue was addressed by staff. Sobering Center staff uses these incident reports to improve program processes. Tracking incidents has helped staff identify the type of patient who is likely to be a good fit for the Sobering Center. During the first few months of opening, staff sometimes persuaded intoxicated individuals exhibiting specific kinds of behaviors to stay at the Sobering Center, only to find that these patients often escalated negative behavior among others at the center. When intoxicated people arrive at the Sobering Center, they are technically still in custody, and if they “act up” in triage, and staff are unable to deescalate them, a police officer takes them to jail. Staff recorded 140 non-admissions when PIs did not meet the Sobering Center’s admission criteria and 268 other incidents. Among these 268 incidents, 20% were acts of aggression (physical or sexual) by the patient, and 3% were accidents (Figure 13). There were approximately five incidents of aggression per month. In these instances, EMS and/or APD were called to respond to the situation. The number of incidents was fairly stable across the ten-month observation period. 32 Fig. 13. Type of Incidents Reported at Sobering Center Non-Compliant behavior (no aggression) Medical incident or physical illness after admission Physical or sexual aggression Mental health issue after admission Left against medical advice before being cleared for discharge On-site accident resulting in patient or staff injury Other Total % 36% 24% 20% 7% 4% 3% 5% 100% n 97 65 53 19 12 9 13 268 Data source: Incident Reports completed by Sobering Center staff, 9/2018-7/2019 We conducted a more in-depth examination of the 53 cases of patients who were aggressive and/or sexual with staff. Bivariate statistical tests (Chi square) revealed that patients without insurance had significantly higher rates of aggression/sexual misconduct than those with insurance. There were no significant differences in the characteristics of these patients in terms of gender, race/ethnicity, homelessness, student status, military/veteran, and/or repeat visits. However, sample sizes for this analysis were small and thus only large differences in subgroups can be detected at this point. We recommend that the Sobering Center continue to monitor these incidents to better understand how to identify and respond to these safety risks. Other Barriers to Patients Remaining at the Sobering Center Hunger and Lack of Food. During interviews, staff said that lack of food is a barrier to engagement. The center can provide water and crackers, but it needs a license to serve other kinds of food. On the Patient Satisfaction Survey, patients suggested that the Sobering Center provide larger water bottles and a larger variety of food. Very Cold Building. Staff also noted that the building’s temperature is a barrier to patients staying until completely sober. “It's quite cold down here. Patients are in blankets, and employees use heaters in their offices. The building is freezing.” A patient who completed the satisfaction survey suggested making the sobering room warmer or offering more blankets. Potential Solutions to Addressing Some of the Barriers Related to Safety Sobering Center staff now maintains a “do not admit” list with names of people who have repeatedly engaged in aggressive or other negative behaviors with staff or other patients. The number of names on the list is relatively small (n=10). First, the Sobering Center puts people who act aggressively on a 30-day no admit list. If they return after 30 days and act inappropriately, the Sobering Center puts them on a 60-day no admit list. The next is a 90day no admit list. If they act inappropriately during a return visit after 90 days, they are prohibited from returning to the Sobering Center. 33 Austin Police Officers work under independent contract to provide security at the Sobering Center. Sobering Center staff describe most officers as supportive and have developed an effective rapport with many of them. They appreciate the officers who can speak Spanish and explained that often the staff and officers make an effective team. Staff suggest that all officers receive standardized training from Sobering Center staff before working at the Center to ensure that everyone understands roles, responsibilities, and how best to support the staff and patients and enforce the law at the Sobering Center. Findings: Outcome Evaluation Safe Sobering A primary goal of the Sobering Center is for at least 80% of the Sobering Center patients to stay long enough to safely sober up. Among patients transported to the Center, 88% are admitted. Of those admitted, 92% stayed at least one hour. The average patient admitted remained at the Sobering Center for an average of 7 hours and 28 minutes (range of 1 to 37.5 hours). Thus, the vast majority of patients transported to the Sobering Center were appropriate for the Sobering Center and stayed long enough to achieve a safe level of sobriety before leaving. Individuals who reported opioid use had shorter lengths of stay than those who did not, but the difference was not statistically significant (348.3 minutes vs. 398.5 minutes, p = 0.378). It is important to note, however, that some stay times may not be accurate as they exceed 24 hours. While it is a newer practice at the Sobering Center, staff have begun to provide patients at risk of opioid overdose with a naloxone kit. At the end of the study period, staff had distributed 12 naloxone kits. Substance Use Disorder Screening, Intervention, and Referral to Treatment The Sobering Center model, unlike medical or criminal justice responses to public intoxication, is specifically designed to address problems of substance misuse and substance use disorder. Once sober, Sobering Center patients are asked if they use tobacco, asked if they want to quit, and given information on smoking cessation. Patients are also given screeners for drug and/or alcohol use disorder (DAST and AUDIT). Based upon the results of these screeners, Sobering Center staff offer brief interventions for substance misuse/substance use disorder if needed and make referrals to treatment. Screening Figure 14 provides information on the percentage of Sobering Center patients who were screened for drug and alcohol use disorders, and how that changed over the course of the first nine months of operation. Examining the entire Sobering Center database from October to July, results reveal that 35% of admitted patients were screened for drug abuse using the DAST. Of those who were admitted and reported alcohol use, 70% were screened for alcohol use disorders using the AUDIT. 34 Figure 14: Patient Screening Over Time 100 90 80 70 60 50 40 30 20 10 0 10/18 11/18 12/18 1/19 2/19 DAST 3/19 4/19 5/19 6/19 7/19 AUDIT Data source: Patient data collected by Sobering Center staff Figure 14 shows that Sobering Center staff began to implement screening of patients for the DAST in February 2019 because they realized that patients brought in to the Sobering Center were using substances other than alcohol. Rates of DAST screening increased substantially over time. The AUDIT has been given more consistently during the intervention period, and rates of screening have also increased considerably over time. By mid-2019, approximately three-fourths of Sobering Center patients are screened with the DAST and AUDIT surveys. Some patients do not receive screening because they do not make it to the discharge process for a number of reasons, such as having a medical issue (experiencing withdrawal from substances), leaving against medical advice, etc. Others are not screened due to insufficient staffing at time of discharge. Figure 15 provides the results of substance use disorder screening for Sobering Center patients. Approximately 28% of Sobering Center patients who were screened with the DAST had scores indicating a drug use disorder. Of those screened with the AUDIT, 34% had scores indicating a high risk of an alcohol use disorder, all cases in which intervention is needed. Another 25% have borderline scores. Clinicians determine whether these individuals need intervention. At the least, all of these patients can benefit from preventive intervention such as information on substance use disorders. These results suggest that the Sobering Center is serving a large number of patients with, or at risk of developing, substance use disorders. 35 Fig. 15. Drug and Alcohol Screening of Sobering Center Patients % n DAST (Drug Abuse Screening Test) Positive screen (score of >1) Negative screen (score <=1) 28% 72% 157 404 AUDIT (Alcohol Use Disorders Identification Test) High risk (score of 16+) Borderline/clinical judgement (score of 8-15) Low risk (score of 0-7) 34% 25% 41% 372 267 449 Data source: Patient data collected by Sobering Center staff, 9/2018-7/2019 Intervention and Referral for Treatment Screening, brief intervention, and referral to treatment (SBIRT) is an evidence-based practice for addressing substance misuse and substance use disorder. It is a core component of the Sobering Center’s mission to offer SBIRT to patients in need of intervention. SBIRT may be administered to patients because of a positive score on the DAST or AUDIT, or if there is indication of a different need, such as domestic violence. SBIRT was implemented at the Sobering Center mid-way through the first year of operation (beginning in February of 2019). Among patients admitted during this time (n=867), 253 (29%) were given SBIRT. Not all patients warrant an SBIRT intervention, but those who screen positive for a substance use disorder on the DAST or AUDIT could be appropriate for SBIRT. Among those who scored positive on the DAST, 27% received SBIRT, and among those who scored at the highest level of risk on the AUDIT screener (score of 16+), 73% received SBIRT. Figure 15 examines these figures over time, during the period in which SBIRT was a formal part of the Sobering Center protocol and for those with a positive screen on the DAST or AUDIT. Results reveal that the percentage of Sobering Center patients who receive SBIRT has been increasing over the study period. In the last three months observed, 81% of those screening positive for a drug use disorder, and 76% of those at high risk of an alcohol use disorder (AUDIT score 16+) received SBIRT. 36 Figure 16: SBIRT Among DAST/AUDIT Positive Patients 100 90 80 70 60 50 40 30 20 10 0 2/19 3/19 4/19 DAST + 5/19 6/19 7/19 AUDIT+ Data source: Patient data collected by Sobering Center staff Concerns and Issues Related to SBIRT While the quantitative data suggest that in the most recent period, the vast majority of high-risk patients are receiving SBIRT, staff expressed some concerns. First, there were concerns with the accuracy of the SBIRT data as some entries for “yes” may include a referral to SBIRT rather than the receipt of SBIRT services. In our section on developing evaluation capacity, we draw attention to the need to make this important distinction when recording data. Some staff felt that a larger number of patients needed SBIRT services than were receiving them. This problem is reportedly due to the fact that SBIRT counselors are often not working during many of the hours in which Sobering Center patients are discharged. For example, Figures 17 and 18 provide a description of when patients arrive and when they are discharged. Fig. 17. Patient Arrival Time 12% 53% 35% 8am-5pm 5pm-12am 12am-8am Fig. 18. Patient Discharge Time 37% 14% 8am-5pm 49% 5pm-12am 12am-8am As might be expected, the vast majority of Sobering Center do not arrive during regular business hours of 8am to 5pm. However, half of Sobering Center patients also do not discharge during regular business hours. Given that SBIRT counselors are present 37 primarily during the 8am-5pm time period, it is difficult to administer SBIRT to the large number of patients being discharged outside of this time period. Sobering Center staff also attempt to help patients with substance use disorders by conducting follow-up calls. While protocol is for Sobering Center staff to attempt at least two follow-up calls to patients within 24 to 48 hours after discharge, most calls are unsuccessful. Only 2% of patients are reached through these follow-up calls. This requires a significant amount of staff time with very little successful contact. Treatment When a patient, at discharge, expresses a willingness to go to treatment, Sobering Center staff help them navigate the process. They connect them to an available treatment center and arrange transportation to the treatment center. During the ten months of operation, the quantitative data indicate that 51 patients (3%) were discharged and directly transported to treatment. This may be an inflated figure as staff reported that 28 patients were directly transported to treatment during the study period. The discrepancy may involve a lack of consistency in how “treatment” is recorded in the quantitative data set. It may be that some entries for treatment reflect referral to treatment rather than an actual transport to treatment. In the future, the way this important outcome is captured needs to be standardized. While the exact number of patients going directly to treatment is not discernable, we do know from the Sobering Center patient data that between 28 and 51 patients have gone to treatment as an outcome of having gone to the Sobering Center. This outcome is highly unlikely to have occurred had the patient been taken to jail or the ED for public intoxication. We examined if there were certain types of patients who were more or less likely to go directly from the Sobering Center to a treatment facility. Figure 19 summarizes these statistics and provides statistical tests indicating any significant differences by subgroup in treatment. Results reveal that those with positive scores on the DAST and/or AUDIT, who use opioids, and/or who receive SBIRT are more likely to go to treatment. Patients who are homeless, insured, and non-students are also more likely to go to treatment. Finally, results reveal that repeat patients were more than twice as likely to go to treatment compared to first-time visitors. In fact, one patient went to treatment after their 14th visit to the Sobering Center. Thus, repeat patients do not represent “failures” of the Sobering Center. Rather, repeat patients may be ideal targets for Sobering Center services. The Sobering Center provides information and access to treatment to individuals as they progress through the five stages of change involved in substance use disorder recovery (precontemplation, contemplation, preparation, action, and maintenance and relapse prevention; DiClemente & Prochaska, 1998). In addition to referring patients to inpatient or outpatient SUD treatment services, the Sobering Center is planning to implement a program referring individuals to peer recovery support services in partnership with a local recovery community organization, 38 Communities for Recovery. Peer recovery support services would be delivered by trained and certified peer recovery support specialists, and these services may be delivered at the Sobering Center itself. Future evaluation efforts should track the success of these types of referrals and should establish data sharing protocols that would allow the Sobering Center to assess the outcome of these referrals. Fig. 19. Patients Transported to Treatment by Subgroup Went to Treatment Opiates reported Yes No Gender Male Female Military/Veteran Yes No Student Yes No Homeless Yes No Insured Yes No DAST high risk (score >1) Yes No AUDIT Low risk (0-7 score) Moderate risk (8-15) High risk (16+) SBIRT Yes No Repeat patient Yes No % n Sig. dif. 8% 2% 15 36 *** 3% 3% 36 15 3% 3% 4 40 0% 4% 0 43 * 5% 2% 18 27 ** 4% 2% 30 13 * 5% 2% 8 7 * 1% 2% 8% 4 4 28 *** 6% 1% 16 6 *** 5% 2% 14 32 ** *p<=.05; **p<=.01; ***p<=.001; Source: Patient data collected by Sobering Center staff, 9/2018-7/2019 39 Value of the Sobering Center and its Services and Success Stories Diversions from jail and emergency departments benefit the publicly intoxicated individuals who go to the Sobering Center. Often, a person charged with a PI receives a warrant and must appear in court. If a person fails to appear for a PI (Class C misdemeanor), it can have a cascade effect. Repetitive arrests occur for the same charge, and a vicious cycle is created (Subramanian et al., 2015). According to a Pflugerville Police Officer, “With the SC [Sobering Center], PIs can avoid criminal charges, get a mental health screening, and get triaged by medical personnel. They are intoxicated to the point where we need to put them somewhere. The SC provides more compassionate treatment for the PI offender.” Some of the largest benefits accrue to those with serious substance use issues who may visit the Sobering Center more than once. According to one Sobering Center staff member: The coolest part is seeing the success stories of patients… I see patients come back time after time, and then they finally hit that point where they're ready to go to treatment. Sometimes, the timing lays out perfectly where we can get them in. We figure out the ride. We get them there. Then, I answer the phone weeks or months later when they call back to say, ‘Thank you. I've been sober for this many days, and you all made such a big impact on me, and I'm mending relationships, and I'm doing well.’ The anonymized stories of “Tom” and “Andy” (not their real names) illustrate some of the Center’s successes and its valuable services. Tom, a middle-aged man, has been homeless for quite a while. EMS found him on the street and brought him to the Sobering Center. On admission, his BAC was 0.42. SBIRT counselors met with him and offered case management. He told them that he had been in and out of treatment, and he explained that he had a caring family, a history of depression, past heroin use, and current alcohol use. APD and EMS transported him to the Sobering Center four times. Tom has visited SBIRT counselors at the Center ten times. Tom’s case management by SBIRT counselors included attempts to get him admitted to several treatment centers and communication with family members, the Downtown Austin Community Court, and case managers working at other entities in the continuum of care. An SBIRT counselor assisted Tom via several calls, emails, and face-to-face visits. They arranged treatment, transitional housing, provided prescription assistance with the help of community doctors (pro bono), and communicated with his family. The Sobering Center, other entities in the community, and Tom’s family came together to help him. Tom visited the Center for follow up and was reconnected to treatment. Unlike other treatment entities, the Sobering Center can continue to assist patients if they relapse. Sobering Center staff engaged and re-engaged with Tom. EMS and APD have transported Andy, who is in his twenties and homeless, to the Sobering Center three times. He has a history of using heroin, methamphetamines, and alcohol and continued to relapse. He has been angry and isolated from his family. 40 Andy voluntarily returned to see an SBIRT counselor three times on his own and also with assistance from the Homeless Outreach Street Team (HOST). He has health insurance in Utah but has faced difficulty in persuading a primary care provider in Utah to refer him to treatment. An SBIRT counselor provided case management and assisted him with reaching out to physicians in Utah, but they were unwilling to oblige in any way. The Sobering Center staff, along with Andy’s family, helped him secure a spot at Oxford House Slayton, a non-profit, self-supporting, and drug-free home. He is accepting treatment. Reducing Reliance on Jails and Emergency Departments for Public Intoxication According to the fiscal year data on PI bookings provided by Travis County Criminal Justice Planning, the average number of bookings for PI per week in 2016-2017 was 42. In 20172018, it was 34. With the Sobering Center in place, average weekly bookings for the period from October 2018 to July 2019 was 21. These data reveal a 19% percent decline in the year before the Sobering Center opened and a 38% decline after the Sobering Center opened. While these simple statistics are not rigorous evaluation data, they indicate that PI arrests have been declining, but the rate of decline has accelerated with the opening of the Sobering Center. Demographic data on PI arrests from before and after the Sobering Center reveal little change in the demographic characteristics of PI arrestees. For example, 16% of PI arrests before the Sobering Center opened were African American, compared to 15% after the Sobering Center was in place. It is worth noting that, while African Americans are overrepresented among those arrested, African Americans are not overrepresented among Sobering Center patients. Before the opening of the Sobering Center, approximately 82% of arrests were male compared to 78% after the center opened. Twenty-six percent of PI arrests were transient or homeless in the period before the Sobering Center opened compared to 25% after the Sobering Center opened. We were unable to obtain quantitative data on ED encounters from Dell Seton Medical Center to examine whether the ED there has experienced less crowding after the opening of the Sobering Center. Building Capacity for Data Collection, Evaluation, and Reporting Several interviewees said that the Sobering Center must improve its data collection and reporting system. Focus and consistency are important. With a robust system, the staff can easily locate and provide information to staff, board, and stakeholders, such as the City of Austin, City Council members, and Austin Police Department. Data can also be used to secure funding from grant-makers. Our assessment of existing data also revealed a need to improve the evaluation capacity of the Sobering Center. In this section, we make recommendations on how the Sobering Center can set up an effective and efficient internal evaluation system. These recommendations are based on our expertise and suggestions from staff, board members, and other stakeholders. 41 Proposed Evaluation Goals To help the Sobering Center develop evaluation capacity, we propose the following evaluation goals detailed below. These goals will provide the Sobering Center with a manageable set of research questions that get at the heart of the Sobering Center mission. We include process goals and emphasize the importance of gathering valid outcome measures. Goals can be added to or removed from this list as the priorities and needs of the center evolve. Process Evaluation Goals (Measuring Outputs) 1. Assess whether program services are being offered as proposed a. Number of transports and percentage admitted (EMS and LEO viewed separately) b. Percentage of repeat patients c. Capacity utilization (average number of patients per week relative to available beds) 2. Describe types of patients served (most/least represented groups) a. Patient counts by substance use type (with opioid use as a specific subgroup) b. Patient counts by demographic characteristics (gender, race/ethnicity, age, homelessness, income, employment, military/veteran, etc.) 3. Monitor program uptake and utilization (LEO and EMS transports over time and by sector/district of origin) 5. Identify program strengths and opportunities for improvement a. In-depth interviews and/or surveys of Sobering Center staff, LEOs, and EMS b. Patient satisfaction survey c. Types of incidents recorded and Sobering Center responses (other than non-admits) Outcome Evaluation Goals (Assessing Impact) 1. Monitor safe sobering a. Average length of stay b. Number of naloxone kits distributed 2. Document increased access to substance abuse information and services a. Percentage that receive DAST and AUDIT screeners b. Percentage at risk who receive SBIRT 3. Determine if the Sobering Center increases the odds of treatment and reduces recidivism a. Number of Sobering Center patients who are transported directly to treatment, and, when the program is implemented, the number who are connected to peer recovery support services. b. Number of Sobering Center patients who eventually go to treatment c. Recidivism rate (for PI) for Sobering Center patients compared to those taken to EDs or jail 42 4. Examine whether the Sobering Center reduces jail and ED crowding using jail and ED trend data 5. Identify types of patients most/least likely to benefit from the Sobering Center services: analyses by age, race/ethnicity, gender, type of substance, homelessness, etc. Recommendations for Data Collection and Analysis The Sobering Center collects a large amount of data on patients and services. Some of these data are in hard copy, and some are maintained in electronic format. This provides an excellent foundation for future data collection and evaluation efforts. Staff from Travis County are working with the Sobering Center to design an electronic database. Here are some recommendations to improve that database and the data collection and reporting process. 1. Create a single spreadsheet to house all quantitative data. We have reformatted the current data set from 10 different spreadsheets for each month to a single spreadsheet organized chronologically. These data will be provided to the Sobering Center to create continuity in the format of the data from program inception. 2. Streamline data collection. There is currently a large number of variables in the Sobering Center data set. The Sobering Center also maintains many sources of data in hard copy format. The Sobering Center data should include only the most important variables for monitoring processes and outcomes. It is better to have a small number of variables measured accurately than to have a large amount of data with limited validity and usefulness. 3. Standardize variables names and create numeric codes for each measure, or variable. • Create a codebook that is dated each time changes are made. The codebook should include all variables (names and labels), values with codes, and the data source for each variable. • Numeric codes should be used so that it is easier to quantify measures. For example, if a patient received SBIRT they could be coded as “1” for yes and “0” for no. This way, a summary statistic (mean score) can easily be calculated in Excel to reveal the percentage of patients receiving SBIRT. • Create a dataset template in Excel with drop down boxes. 4. Train staff in the conceptualization and operationalization of measures. • SBIRT variable: should be coded as “yes” only if the patient received SBIRT and should not include a referral to or recommendation for SBIRT. • Referral to treatment variable: should be coded as “yes” if the patient is referred to treatment. • Treatment variable: should be coded as “yes” only if the patient was transported to a treatment facility upon discharge and should not include patient willingness to go to treatment. 43 5. Develop additional outcome measures. These additional measures will require collaboration across agencies, data sharing agreements, and an evaluation plan that incorporates longer periods for following patients. Peer recovery coaches could help to develop necessary partnerships and gather data. • To assess the number of Sobering Center patients who go to treatment, collect data from treatment centers about incoming patients who have been to the Sobering Center. This will capture information about patients who express a willingness to go to treatment but were not transported at discharge. • Work more closely with EDs and law enforcement agencies to evaluate how the use of the Sobering Center relates to recidivism. What percent of PI arrests recidivate compared to Sobering Center patients? What percentage of publicly intoxicated ED patients return to the ED compared to those transported to the Sobering Center? 6. Create a reporting system. • Create formulas in Excel to automatically populate cells with summary statistics, e.g. percentage of patients who get SBIRT, percentage who go to treatment. • For key process and outcome measures, create a plan for reporting summary statistics, e.g. quarterly, biannually, annually. Provide brief summaries to staff and board members. Additional Data Requests from Stakeholders The evaluation plan outlined aims to address the key process and outcome goals of the Center. Much of what has been proposed also satisfies information requested by the Board of Directors. Some stakeholders felt that the following information would also be important for the community and funders. • Data from all law enforcement agencies (Austin PD, Manor PD, Pflugerville PD, University of Texas PD) regarding the number of PI arrests that are transported to the jail versus the Sobering Center. • Number of warm handoffs to specific entities • Number of repeat visits by patients who experienced a warm hand-off to a treatment center • History of mental illness for population served • More information about people who visit the Sobering Center frequently (What does intervention look like with these repeat visitors? Where is the Sobering Center referring them? Are they getting into treatment? Do they continue to communicate with the Sobering Center? Are they getting re-arrested? Are they homeless?) Process & Outcome Evaluation Insights The Sobering Center successfully launched in August of 2018. In the first year of operation, the center met or exceeded many of its process goals. There were a large number of transports to the center (n=1,824) and a high admittance rate (88%, n=1,605 patients). Approximately three-fourths of transports came from law enforcement and one-fourth from EMS. 44 Patients were diverse with regard to gender, age, and race/ethnicity. The Center served a sizable number of vulnerable subgroups such as the uninsured, homeless, and military/veterans. Most Sobering Center patients (89%) reported alcohol use. Ten percent of patients reported opioid use; a figure higher than that reported at the Houston sobering center (less than 3%). With an average length of stay of 7.5 hours, patients stayed long enough to safely sober and avoid using ED or jail resources. These patients also were not encumbered with the additional financial or personal cost of an ED visit or arrest record. Nine percent of Sobering Center patients were repeat visitors. Program strengths include a competent and committed staff and board. Staff requested more communication with leadership about the future directions of the Center and their roles and responsibilities. Additional training and team building exercises would be beneficial for improving competence, cohesion, confidence, and morale. Surveys of patients and LEOs reveal very high levels of satisfaction with Sobering Center services. An important goal of the Center is to increase access to substance use disorder treatment. At the beginning of the observation period, many Sobering Center patients did not receive substance use disorder screening; however, rates of screening increased steadily. In the last three months of operation, approximately three-fourths of Sobering Center patients were screened for drug and/or alcohol use disorders. During this time period, most patients at risk of a substance use disorder received SBIRT (screening, brief intervention, and referral to treatment). However, all patients profiling at high risk should receive SBIRT. To reach this goal, SBIRT counselors need to be present at all hours of operation, rather than scheduled only during the typical workweek days and hours. In the first year of operation, Sobering Center data indicate that 51 patients were transported directly to treatment. There is a need for stronger data on whether patients receive treatment. However, it is clear that many patients received treatment who would not have under the traditional medical and/or criminal justice response to public intoxication. Moving forward, one of the most important tasks for the Sobering Center will be to establish a reliable and valid data collection and reporting system. The Sobering Center needs to identify a small number of core measures, clearly operationalize these measures, standardize data collection, and house data in a single Excel spreadsheet to facilitate analysis. In addition, collaborative partnerships and data sharing agreements are needed to capture patient outcomes over time and determine if these outcomes are better than they would have been without Sobering Center services. Valid outcome measures of this kind are necessary to fully understand the value of the Sobering Center and to communicate this information to staff, funders, and the community. 45 Part III. Economic Evaluation Framework Background Sobering centers are programs that allow actively-intoxicated “clients to safely recover from acute intoxication (Warren et al., 2016, p. 1844).” They provide alternatives to emergency departments (EDs) and jail, which may be more expensive options. The Sobering Center serving Austin and Travis County plans to carry out a cost-benefit analysis (CBA) in the future to determine the extent to which the benefits it provides by diverting intoxicated individuals from EDs and jail are greater than the program’s costs. The CBA will focus both on all patients as well as those who use opioids. The following is a report of what has been learned from the literature, information provided by external sources and data provided by the Sobering Center on services provided between September 1, 2018, and June 30, 2019. The information gathered informs the parameters to be used in the organization’s CBA. This section also details the procedure for conducting a CBA and addressing the following objectives: 1. To compare program costs to program benefits in monetary terms for the entire patient population; 2. To calculate how costs of administering the program to patients who use opioids compare to the benefits accrued for these patients; 3. To estimate the portion of the target population the Sobering Center is reaching; 4. To discuss the internal and external factors that impact the program’s ability to reach 100% of the target population; 5. To discuss how an increase in service provision would impact the findings; and 6. To explain how costs and benefits accrue to various stakeholders. Economic Literature Review The literature on sobering centers is sparse, limited to five articles. One summarizes characteristics and best practices of sobering centers across the United States (Warren et al., 2016) and is explored in greater detail in the process and outcome evaluation literature review. The remaining four articles provide some examination of the economic impacts of sobering centers: one is limited to examining cost offsets (as opposed to cost savings) at the Houston sobering center (Jarvis et al., 2019), two report substantial cost savings to EDs in San Francisco as the result of their sobering center (Smith-Bernardin et al., 2017; SmithBernardin & Schneidermann, 2012), while the final study represents a more comprehensive budget impact analysis from the perspective of the US healthcare system (Scheuter et al., 2019). This final article estimates the effects on health care spending of diverting patients with acute alcohol intoxication from EDs to sobering centers (Scheuter et al., 2019). Scheuter and colleagues published an estimate of the impact on healthcare spending nationally if individuals with uncomplicated, acute alcohol intoxication were treated in sobering centers instead of EDs (Scheuter et al., 2019). The authors modeled their estimates from the perspective of the healthcare system, as opposed to the societal 46 perspective. In economic evaluations, the societal perspective examines costs and benefits both to the individuals who utilize the service, as well as to the organizations or systems providing the service. From the healthcare system perspective, in contrast, only the costs and benefits borne by the system are examined. Assuming a diversion rate of 50%, the authors estimated national healthcare system savings would range from $230 million to $1 billion annually (2017 US$). If diversion rates were as minimal as 5%, the authors estimate the healthcare system would still save as much as $99.02 million (Scheuter et al., 2019). While the authors made a number of assumptions – including a standardization of current sobering center operating practices, which is known to currently vary substantially (Warren et al., 2016) – some of their estimates of costs and benefits are relevant to the current project. Further, the Scheuter and colleagues study focuses on short-term outcomes within the healthcare system, and, similarly, the framework outlined in this report adopts a conservative approach focusing on the short-term benefit of diverting acutely intoxicated individuals from EDs and law enforcement settings. A future CBA could examine long-term benefits, as well. Examples of long-term benefits include repeat patients accessing treatment and recovery support services, though, at present, not enough is known about this phenomenon to accurately model this type of outcome. Further, these long-term benefits accrue primarily to the individual and to communities outside of the scope of the healthcare system and would be more appropriate for a societal perspective economic evaluation. The above notwithstanding, a rapid review of the Screening, Brief Intervention and Referral for Treatment (SBIRT) literature was performed to assess whether longer-term impacts as a result of SBIRT should be included. Currently the implementation of SBIRT in the Sobering Center is the primary method for making referrals to treatment and other substance use-related resources. The research from which the evidence base is derived takes place mostly in ED and hospital settings, rather than in sobering centers or other hospital diversion programs. The evidence of SBIRT impact is mixed, largely due to differences in how interventions are provided (for example, whether participants received a single brief intervention or multiple), level of fidelity, the populations targeted (e.g. severity of the alcohol problem, whether an injury was involved, readiness to change, etc.), study design and assessment procedures (Field, Baird, Saitz, Caetano, & Monti, 2010; Schmidt et al., 2016). In a meta-analysis of 28 studies on the impact of screening and brief interventions in ED settings on alcohol consumption, Schmidt and colleagues found evidence of very small effects on drinking frequency and intensity (highest SMD=0.19, 95% CI: 0.08-0.31; Schmidt et al., 2016). To date, no evidence for the effectiveness of SBIRT in sobering centers has yet been published, aside from the outcomes discussed in this evaluation. A future CBA should also consider the effects of peer recovery support services, if these services are added to the Sobering Center’s repertoire of recovery resources for those patients who screen positive for a substance use disorder. Costs to External Parties The economic evaluation outlined here will take a societal perspective. The societal perspective takes into account the costs and benefits to all parties involved. In the case of the Sobering Center, this includes public safety departments, EMS, EDs and hospitals, and 47 the City of Austin as the donor of the organization’s facility. One key assumption is that the time patients spend safely sobering at the Sobering Center is equivalent to the time they would spend at an ED or in jail; thus, their time costs are not included in the analyses. Emergency Medical Services Austin’s EMS provided data on transports to and from the Sobering Center, as well as on the pay scales of staff involved in the transports and Medicaid/Medicare rates they charge for similar transports when taking patients to an ED. The latter included loaded mileage rates. Between September 1, 2018, and June 30, 2019, Austin EMS made 427 transports to Sobering Center. On average, each transport took 59.2 minutes from dispatch to clearance. Of these, 406 (95.1%) transports ended at the Sobering Center; in the remaining 21, patients were taken to Dell Seton Medical Center (n=18), Saint David’s Medical Center (n=2) and the South Austin Hospital (n=1). Almost half (49.5%) of these transports were made on the weekend (on a Saturday or Sunday). EMS also picked up 132 patients during the same time period and transported them to hospitals. Of these 91.7% went to Dell Seton Medical Center which is less than one-half mile (0.4) away. The remainder were taken to South Austin Hospital (3.8%), St. David’s Medical Center (2.3%) and one each to the Heart Hospital, Seton Medical Center and Seton Northwest. In addition to transports, EMS donated $3,407 in medical supplies to Sobering Center during the time period which should be included as a program cost. Each ambulance is staffed by, at minimum, one medic who is paid based on an hourly rate of $19.55, and a clinical specialist who earns $25.69 per hour. These rates do not include tax and fringe benefits which amount to about 30% of the hourly rate. When EMS transports patients to the Sobering Center rather than to a hospital, they forego reimbursement from that individual’s insurance or Medicaid/Medicare. By convention, the Medicaid/Medicare reimbursement is the only reimbursement considered in economic evaluations, as private insurance rates are often not publicly available, nor are they standardized. Further, there is always a discrepancy between the amount charged and the amount paid by Medicaid/Medicare, thus a payment-to-charge percentage adjustment must be made when estimating reimbursements. Thus, the foregone reimbursement cost to EMS incurred by transporting patients to the Sobering Center instead of to the ED is not the amount charged to Medicaid/Medicare, but is, instead, the payment-to-charge adjusted final reimbursement. In this case, the charge for a Basic Life Support transport is $831 plus $13.50 per mile (loaded mileage rate). Using state-level Centers for Medicare & Medicaid Services data on 2015 hospital outpatient service submitted charges, allowed Medicare amounts and average Medicare payments, between 8.6% and 24.7% of submitted charges by Texas hospitals are paid by Medicare (mean: 14.4%; Center for Medicare and Medicaid Services, 2015). The resulting foregone reimbursement would then be between $71.47 per transport plus $1.61 per mile at the low end, and $205.26 per transport, $3.33 per mile at the high end (mean: $119.66 per transport, $1.94 per mile). Assuming a 15% paymentcharge ratio and 3.5 miles traveled per transport, Austin EMS foregoes approximately $172 48 per patient taken to the Sobering Center. This amount is included as an external program cost. Travis County Travis County donated the facility in which the Sobering Center is housed and charges the organization just $10 per year in rent. The amount that Travis County could be charging represents foregone revenue to the county. Rental rates in downtown Austin vary widely, averaging $45-$55 per square foot for Class A, $40-$45 for Class B, and $35-40 for Class C (Silas, 2019; TenantBase Inc., 2019). Outside of downtown, rates drop by $5-10 per square foot off the low end of the downtown ranges. The Sobering Center facility would be considered a Class C space and Sobering Center primary operations take place using two floors of the building (10,789 square feet). The third floor of the Sobering Center is dedicated to community outreach, and the space is shared with the Homeless Outreach Street Team (HOST). Because this third floor is not directly involved with the Sobering Center primary functions being evaluated in this report, foregone rent for this space would not be included in a future economic analysis. If the Sobering Center were to pay full rent for the facility, its annual rent would likely be between $270,000 and $324,000 per year (assuming $25-30 per square foot). Police Departments Law enforcement transported 73% of patients into the Sobering Center during the time period studied. Of these, the Austin Police Department (APD) were responsible for 88% and the Travis County Sheriff’s Office (TCSO) brought in 3%. The remainder were brought in by officers from areas outside of Austin (e.g. Pflugerville) or who were affiliated with specific neighborhoods (e.g. Rollingwood). Efforts to collect the necessary cost data from the Austin PD have not been successful; however, the Travis County Sheriff’s Office (TCSO) was able to provide cost estimates, and indicated that APD officers, which is the agency most likely to respond to a call for assistance from the Sobering Center, earn more than their counterparts at the TCSO. The estimates used here are from the TCSO data and are likely an under-estimate of actual costs. A future, rigorous CBA of the Sobering Center should use APD data in appropriate proportion to data from TSCO. The average loaded hourly cost for TCSO deputies is $39.86 and average vehicle operational costs are $0.981 per mile. The distance to the Austin City Jail from the Sobering Center is 0.4 miles. Using a broad assumption that it takes the police a similar amount of time from dispatch to clearing a patients at the jail as it does EMS to deliver to Dell Seton Medical Center (55.8 minutes on average), the costs borne by TCSO to pick up a patient at the Sobering Center, transport and hand the patient off at the Austin City Jail would be $74.53. This amount should be adjusted up to account for the higher salaries of APD officers, possibly by 10 to 15% which would amount to $81.95 to $85.65. Because transportation times for LEOs are assumed to be roughly equivalent whether the patient is transported to the Sobering Center, or transported to jail, this type of transportation cost is net zero. 49 Transportation from the Sobering Center to Other Facilities Because not all patients who are brought to the Sobering Center are admitted and successfully discharged, costs for transporting patients from the Sobering Center to either a medical facility or jail have to be included. Over the period analyzed, 126 patients were transported from the Sobering Center to medical facilities around Austin (predominantly Dell Seton Medical Center, which is approximately 0.5 miles away). In addition, 14 patients or potential patients were transported by police officers from the Sobering Center to jail over a 16-week period (October 2018 through the third week of January 2019) amounting to approximately 0.875 transports from the Sobering Center by jail per week or 38 jail transports over the 10-month period. According to data provided by Austin EMS on hourly wages, fringe benefit and tax percentages and loaded mileage, it costs $68.90 on average to transport a patient from the Sobering Center to a medical facility. This amount is relatively small since patients were transported to Dell Seton Medical Center 87% of the time, which is less than a mile away. Figure 20 below summarizes estimates of costs borne by external parties to deliver Sobering Center services. Fig. 20. Costs to External Parties / Partners Category Donated Medical Supplies Amount $3,407 Notes EMS data for 9/1/18 – 7/31/19 EMS Transport to SC $105.28 / transport Calculated from EMS data (average time from dispatch to clearance = 59.2 minutes, hourly rate of one medic and one clinical specialist plus FBT (est 30%) + loaded mileage for 3.5 miles (est average miles traveled from pick-up to SC) EMS Transport from SC to Other Facility $ 68.90 / transport Calculated from EMS data (average time from dispatch to clearance = 55.8 minutes, hourly rate of one medic and one clinical specialist plus FBT (est 30%) + loaded mileage for 1.05 miles (est average miles traveled from SC to final destination = 1.05) EMS Foregone Reimbursement $ 171.90 / transport EMS data, 15% of Basic Life Support charge ($831) + loaded mileage for 3.5 miles (@ $13.50 / mile) Sobering Center Rental Value $270,000-324,000/yr Assumes rate of $25-30 / SF (Class C facility adjacent to downtown Austin) Averted Costs (Benefits) On the other side of the ledger, averting the costs to book and jail someone with acute alcohol intoxication or to admit them to an ED are benefits of the Sobering Center program. Figure 21 below summarizes the value of the three benefits. The point estimates for all three are local estimates that were developed by the Central Health Joint Technology Team, Travis County Justice Planning Department and the Ending Chronic Homelessness Organization (ECHO) (Corporation for Supportive Housing, 2016). The range of values for an averted ED visit for acute alcohol intoxication were taken from two studies on the economic impact of sobering centers (Scheuter et al., 2019; Smith-Bernardin et al., 2017). 50 Fig. 21. Value of Program Benefits / Averted Costs Cost Category ED visit, alcohol intoxication w/o complications Jail booking Base Estimate $1,400 $ 153 Jail day $ Range $670-1,591 97 Source for all: Central Health Joint Technology Team, Travis County Justice Planning Dept and ECHO. In addition, for ranges of ED visits, Scheuter 2019 and Smith-Bernardin 2017. Methods for Conducting a Cost-Benefit Analysis Overview The goal of cost-benefit analysis (CBA) is to compare the costs of running a program to the effects (i.e. benefits) of the program in financial terms. To the extent that the program yields negative effects, these are included in the cost side of the calculation. Where possible, the analysis uses incremental costs and benefits where two groups are compared; the first group receives the program (Intervention Group) and the other does not (Control Group). The perspective used determines the scope of costs and benefits included in the analysis. The perspective used may be internal to a system (e.g. healthcare, payer and employer perspectives), public or governmental, or societal. The latter includes the costs borne and benefits accruing to individual participants such as time costs associated with program participation or impact on productivity, a benefit. For example, when a hospital implements a case management program targeting high-utilizers of its services, the organization’s administration may solely be interested in how internal implementation costs compare to reductions in utilization of more expensive services (e.g. emergency department visits) and, thus, an internally-focused health system perspective would be appropriate. In the current case, a public perspective is recommended since multiple public agencies – the City of Austin and Travis County – contribute to covering program expenses and also accrue benefits. Objectives The Sobering Center CBA has several objectives. These include: 1. To compare program costs to program benefits in monetary terms for the entire patient population; 2. To calculate how costs of administering the program to patients who use opioids compare to the benefits accrued for these patients; 3. To estimate the portion of the target population the Sobering Center is reaching; 4. To discuss the internal and external factors that impact the program’s ability to reach 100% of the target population; 5. To discuss how an increase in service provision would impact the findings; and 6. To explain how costs and benefits accrue to various stakeholders. Data Sources Figure 22 summarizes data points to be utilized, their values (where available) and their sources. Data sources are described in greater detail in the preceding section. 51 Fig. 22. Data Points, Values and Sources Data Point Service Data Patients brought to Sobering Center (total) Patients transported to Sobering Center by EMS Patients transported to Sobering Center by police departments Patients transported to Sobering Center by Other Number not admitted Clients transported from Sobering Center by EMS to medical facility Clients transported from Sobering Center by police to jail Clients successfully discharged from Sobering Center Clients not successfully discharged, course of action unknown Cost Data Medical Supplies (Donated) APD transport from Sobering Center to Austin City Jail EMS transport from Sobering Center to medical facility EMS foregone reimbursement Facility / foregone rent Value Source $1,730 Sobering Center program data $443 Sobering Center program data $1,262 Sobering Center program data $26 Sobering Center program data $217 $126 Sobering Center program data EMS $0.875 / week $1,449 Sobering Center program data $155 Sobering Center data combined w/ EMS data $310 Austin EMS, monthly average: 9/1/18 – 7/31/19 Estimates based on data provided by TCSO, unit average Austin EMS, unit average $81.95 – 85.65 $68.90 $171.90 $270,000 – 324,000 Staff: service delivery Staff: necessary operations / admin Security Utilities Internet & phone Software / technology, service-delivery related Supplies, non-medical Supplies, medical (purchased by Sobering Center) Insurance Legal and professional services, servicedelivery related Equipment depreciation expense, servicerelated Averted Costs / Benefits ED visit for uncomplicated acute intoxication Jail booking APD data Austin EMS, unit average calculated assuming 15% payment-charge ratio Estimated annual rental value for 10,789 SF of class C space ($25-30/SF) Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center Sobering Center $670-1,591 (Base: $1,400) $153 (a), Scheuter 2019, Smith-Bernardin 2017 (a) 52 Jail day $97 (a) APD: Austin Police Dept., TCSO: Travis County Sheriff’s Office, EMS: Emergency Medical Services, (a) Central Health Joint Technology Team, Travis County Justice Planning Department and ECHO. Sobering Center financial data should be aggregated over the same time period for which service delivery data are pulled. The analysis should only include costs that are necessary for running the program’s primary functions. It may be useful to conceptualize the appropriate costs from the viewpoint of program replication. If another group were to make a copy of the Sobering Center program and run it elsewhere, what costs would they necessarily encounter regardless of funding mechanism and organizational structure? These costs would undoubtedly include staff to triage and monitor patients, and it may include the costs of marketing the program to the community. However, general operating costs such as for an audit, liability insurance, etc. would not be included. Similarly, staff time spent on strategy and future program development would not be included, but time spent on program oversight could be. Depreciation expense of equipment used during the delivery of service should be included, but not that of equipment used for administrative purposes. Finally, expense categories that have service and non-service components (e.g. utilities, internet or rent) should be allocated using a standard rule. Examples include the portion of staffing expenses allocated to the program or the portion of the total facility space used by the program. The rule that is most suitable varies by organization. Sample The patient sample used for the CBA should match the time period of the financial data used. In general, analysts should select a time period during which programming and expenditures are fairly representative of the program running at full capacity. In the case of the Sobering Center, between September 2018 and June 2019, the number of visits per month varied from 137 (September) to 211 (March). If staff feel that September 2018 was a start-up month, when the Sobering Center was not in full operation, the organization might consider using the nine-month period from October 1, 2018, to June 30, 2019, instead. Reviewing monthly financial information can inform this decision. Patients who use opioids should be selected from the same time period used for selecting the primary sample above. What constitutes a patient who uses opioids may be informed by the terms of the grant related to providing services to this group. During the discovery phase of this project, patients who use opioids were identified from the month-to-month log in the following manner: (a) the Substance text variable was separated into individual substances (Substance1, Substance2, Substance3), tabulated and those self-reporting currently using one or more opioids (opioid, OP, opiate, fentanyl, Vicodin) were coded as being patients who use opioids. Using this method, 36 opioid-related visits were identified between September 1, 2018, and June 30, 2019. Clients in another 105 visits reported a history of opioid use but did not report having it currently in their system. It is not clear whether this latter set should be included in the opioid sample or not. If they are, it should be noted that 54% of visits are missing data on history of opioid use. 53 Analytical Methods This section describes the steps to be taken to complete the CBA and fulfill the objectives listed above. Objective 1. Compare program costs to program benefits in monetary terms for the entire patient population. An Excel template is provided in Appendix IV to facilitate calculations, and also includes a worked example using data from the 10-month period described previously and fictitious Sobering Center cost data. Fill out the cells shaded in green pulling from data points in Table 1 above as applicable. The worksheet will calculate Net Benefits and the Benefit-to-Cost Ratio. It is also standard practice to perform at least univariate sensitivity analysis to determine how sensitive the results are to variability in key parameters. Parameters adjusted in sensitivity analyses are usually those over which the organization has some control, such as program costs, as well as those for which values are relatively uncertain (e.g. APD costs to pick patients up at the Sobering Center and take them to jail if a safety-related incident occurs). To carry out basic sensitivity analyses, once the base model has been completed, make a copy of the CBA PRIMARY worksheet in Appendix IV and label it “Sensitivity Analyses.” In this new worksheet, adjust parameters one at a time in various directions and record the new results in a table; adjust a parameter’s value back to what it was in the base model before moving on to another parameter. Objective 2. Calculate how costs of administering the program to patients who use opioids compared to the benefits accrued for these patients. Use the CBA OPIOID worksheet in the workbook found in Appendix IV. As before, complete the cells shaded in green. Note the addition of an item about the portion of total patients who are in the Opioid subgroup. There was no evidence that patients who use opioids remain in care for a significantly different length of time than patients who do not use opioids. Thus, the results of this CBA will differ from that of the main one depending on the relative portions of successful discharges and transports to other facilities. Enter the same values from the main CBA into the shaded cells and the total annual costs allocated to patients who use opioids will be calculated based on what the portion of total patients this group makes up (approximately 2% if using the more conservative definition of a patient who uses opioids described above). Objective 3. Estimate the portion of the target population the Sobering Center is reaching. This objective will not be determined by the CBA itself. Rather the Sobering Center can use data provided by the TCSO Research & Planning Department comparing monthly public intoxication (PI) arrests to the number of PI patients brought to the Sobering Center (Source: TCSO Research & Planning. Comparison of New PI Bookings & Admissions to the Sobering Center, 9/11/2019). For the period between October 1, 2018, and August 25, 2019, it appears that the Sobering Center is receiving 64% of all PI’s picked up by the police (n=1,740). Similar data should be collected from Austin EMS to estimate the total portion of the target population reached. Objective 4. Discuss the internal and external factors that impact the program’s ability to reach 100% of the target population. This discussion will be informed by the findings 54 in Objective 3 above as well as internal analysis of the organization’s maximum capacity (itself informed by the organization’s monthly financials). One obvious external factor impeding the Sobering Center’s ability to reach 100% of the target population is the reimbursement that EMS foregoes when patients are brought to the Sobering Center rather than to a medical facility. Other external policies about how patients are admitted may also be limiting factors. Objective 5. Discuss how an increase in service provision would impact the findings. This objective will be informed by a side-to-side comparison of monthly service delivery numbers and monthly financials. The goal is to understand several critical thresholds: how many more patients could be served each month without hiring new staff, the point at which it makes sense to hire new staff and how much doing so would add to monthly financial expenditures. Those completing the CBA should consider whether any efficiencies or other (internal or external) cost reduction actions can be taken. Objective 6. Explain how costs and benefits accrue to various stakeholders. The portion of costs and benefits accruing to various stakeholder groups, broadly, is automatically calculated at the bottom of the CBA PRIMARY worksheet in Appendix IV. In addition, it is helpful to remember that the Austin PD is responsible for about 88% of patient transports to the Sobering Center by the police (i.e. benefits and costs accruing to police departments are largely received and borne by the Austin PD). With that said, report writers should acknowledge that benefit unit values include allocations of fixed costs (e.g. facilities and equipment) that are not easily eliminated. Thus, the agencies benefiting from jail and ED diversions need to do their own analyses about what reduction in utilization would need to take place in order to enable cutting equipment, facility or staff budgets. Data Issues and Recommendations Significant inconsistencies in how and what data were collected exist from month to month. Even when new data points are collected (e.g. provision of SBIRT beginning in January 2019), significant amounts of missing data persist. In the case of SBIRT provision, for example, 26% of visits between January and June have no data on whether or not the patient received SBIRT. In order to streamline the primary analyses necessary for an economic evaluation, the following data needs to be captured more consistently and systematically: • carrier category, • length of stay (LOS), • whether patients are formally admitted, • and for patients who are not admitted, whether they are taken by EMS to a health care facility. • Create categorical codes for how the patient got to the Sobering Center (i.e. EMS, Police, Walk-In, Other), • and whether the patient was successfully discharged so that data can be easily tabulated. 55 Across months, dates and times of arrival and discharge were captured differently. In addition, in some cases, LOS was not calculated for individual visits even though there are arrival and discharge times in the record. At other times, LOS was calculated even when patients are coded as not having been admitted. It is not clear which visits are opioid-related and should be included in the opioid grant reporting. Inclusion criteria that align with grant requirements need to be clearly defined (e.g. what counts as an opioid, and whether the patient needed to present with them in his/her system, self-report as being current user and/or self-report as having used them in the past) and based on this criteria, visits clearly coded as such. In general, the Sobering Center should consider investing some time in standardizing and documenting its data collection procedures, as well as training and monitoring staff to collect data correctly. The organization should consider using a platform that can force data points to be input (e.g. REDCap, which is freely available to nonprofit and academic entities). Such systems can also flag values that are out of range (for example, LOS that are out of range because arrival or discharge dates were entered incorrectly). A future, comprehensive CBA could examine both the healthcare system and societal perspectives, and could include both short-term and long-term, recovery-oriented outcomes. A future CBA may also assess benefits to individual patients who utilize Sobering Center services, whether they are one-time safe sobering patients, or repeat patients that access treatment and recovery support services. 56 Part IV. Conclusion and Recommendations The Sobering Center is an effective program that provides a substantial public health benefit to Travis County, the City of Austin, and publicly intoxicated individuals receiving Sobering Center services. While opportunities for further growth and systems improvement were detailed in the preceding sections, the Sobering Center is clearly fulfilling its mission statement by providing safe sobering and connection to treatment and recovery resources for publicly intoxicated individuals in Austin and Travis County. The Sobering Center is also fulfilling its goal of relieving some of the burden of public intoxication on jails and EDs. The Sobering Center collects a large amount of data on patients and services and maintains those data in a secure electronic format, providing an excellent foundation for future data collection and evaluation efforts. Some of these data can be streamlined into a core set of variables whose definitions are more clearly standardized. Data management tools exist that may aid in the standardization of a core data set. A map of Austin Police Department Sectors, a suggested codebook, a revised set of core measures, and a template for future economic evaluation are included as appendices. In addition to these tools, it is recommended that the Sobering Center also: • Create a single, secure database to house all quantitative data. • Standardize variable names and create numeric codes for each variable, or measure. • Train staff in the conceptualization and operationalization of measures, especially: o Measures related to longer-term patient outcomes, such as receipt of SBIRT and transportation to treatment, as opposed to merely a referral to either resource o Clear definition of what qualifies as opioid use (e.g. street opioids, prescription opioids, opioid use disorder, any opioid use?) o Consistency of AUDIT and DAST provision and triggering an automatic process of SBIRT involvement when called for. • Streamline data collection. • Develop additional outcome measures. Further, staffing practices are somewhat divergent from the original model detailed in the implementation report. Some divergence from original plans are to be expected, but a future emphasis on staff expertise in peer-based recovery support services may aid in even greater recovery-related outcomes for Sobering Center patients. In addition, because patients arrive and discharge at any time during the day, it is also important that personnel who are key to linking patients with SUD to appropriate resources and referrals – namely, SBIRT counselors in the current staffing configuration – are available during all shifts in adequate proportions. The Sobering Center has already made a substantial impact on the Austin-Travis County community, and through the process of program evaluation, opportunities for continuous improvement and the strengthening of existing effective systems have been identified and can aid in future work to improve and sustain the Sobering Center. 57 References American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th Edition). Washington, DC. Center for Medicare and Medicaid Services (Ed.). (2015). Hospital Utilization, Estimated Submitted Charges, and Medicare Payments for Comprehensive APC Groups, 2015 National and State-Level Summary Overview. In Outpatient Prospective Payment System. Corporation for Supportive Housing. (2016). Social Innovation Fund Pay for Success Feasibility Report: ECHO, Austin/Travis County. Retrieved from Corporation for Supportive Housing website: https://www.csh.org/wpcontent/uploads/2014/12/Austin-ECHO_PFS-Evaluation_CSH.pdf DiClemente, C. C., & Prochaska, J. O. (1998). Toward a comprehensive, transtheoretical model of change: Stages of change and addictive behaviors. In Applied Clinical Psychology. Treating addictive behaviors, 2nd ed (pp. 3–24). https://doi.org/10.1007/978-1-4899-1934-2_1 Field, C. A., Baird, J., Saitz, R., Caetano, R., & Monti, P. M. (2010). The Mixed Evidence for Brief Intervention in Emergency Departments, Trauma Care Centers, and Inpatient Hospital Settings: What Should We Do? Alcoholism: Clinical and Experimental Research, 34(12), 2004–2010. https://doi.org/10.1111/j.1530-0277.2010.01297.x Jarvis, S. V., Kincaid, L., Weltge, A. F., Lee, M., & Basinger, S. F. (2019). Public Intoxication: Sobering Centers as an Alternative to Incarceration, Houston, 2010–2017. American Journal of Public Health, 109(4), 597–599. https://doi.org/10.2105/AJPH.2018.304907 Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A Systematic Approach (7th ed.). Thousand Oaks, CA: Sage Publications, Inc. Scheuter, C., Rochlin, D. H., Lee, C.-M., Milstein, A., & Kaplan, R. M. (2019). Cost impact of sobering centers on national health care spending in the United States. Translational Behavioral Medicine, ibz075. https://doi.org/10.1093/tbm/ibz075 Schmidt, C. S., Schulte, B., Seo, H.-N., Kuhn, S., O’Donnell, A., Kriston, L., … Reimer, J. (2016). Meta-analysis on the effectiveness of alcohol screening with brief interventions for patients in emergency care settings. Addiction, 111(5), 783–794. https://doi.org/10.1111/add.13263 Silas, K. (2019, July 23). How Much Does It Cost to Lease Office Space in Austin, Texas? (Rental Rates, Pricing). Retrieved September 5, 2019, from AQUILA Commercial website: https://aquilacommercial.com/learning-center/cost-to-lease-office-spaceaustin-tx/ Smith-Bernardin, S., Carrico, A., Max, W., & Chapman, S. (2017). Utilization of a Sobering Center for Acute Alcohol Intoxication. Academic Emergency Medicine: Official Journal of the Society for Academic Emergency Medicine, 24(9), 1060–1071. https://doi.org/10.1111/acem.13219 Smith-Bernardin, S., & Schneidermann, M. (2012). Safe Sobering: San Francisco’s Approach to Chronic Public Inebriation. Journal of Health Care for the Poor and Underserved, 23(3A), 265–270. https://doi.org/10.1353/hpu.2012.0144 58 Sobriety Center Planning Committee. (2015). Sobriety Center Implementation Report (p. 27). Retrieved from Austin City Council, Travis County Commissioners Court website: http://www.austintexas.gov/edims/document.cfm?id=232931 Subramanian, R., Delaney, R., Roberts, S., Fishman, N., & McGarry, P. (2015). The Misuse of Jails in America (p. 56). New York, NY: Vera Institute of Justice, Center on Sentencing and Corrections. TenantBase Inc. (2019). Renting Office Space in Austin. Retrieved September 5, 2019, from https://www.tenantbase.com/austin/office-space/ Texas Penal Code. Title 10. Offenses Against Public Health, Safety, and Morals, Chapter 49. Intoxication and Alcoholic Beverage Offenses. , § 49.02 (1994). Warren, O., Smith-Bernardin, S., Jamieson, K., Zaller, N., & Liferidge, A. (2016). Identification and Practice Patterns of Sobering Centers in the United States. Journal of Health Care for the Poor and Underserved, 27(4), 1843–1857. https://doi.org/10.1353/hpu.2016.0166 White, A. M., Slater, M. E., Ng, G., Hingson, R., & Breslow, R. (2018). Trends in AlcoholRelated Emergency Department Visits in the United States: Results from the Nationwide Emergency Department Sample, 2006 to 2014. Alcoholism: Clinical and Experimental Research, 42(2), 352–359. https://doi.org/10.1111/acer.13559 59 Appendix I: Map of Austin Police Department Sectors APD t, I we I SOUTH . APD TRAINING ACADEMY . l. ?h 9?06 . ?In APD Owned Facilities "43 . Owned APD Faeirte BAKER GEORGE we. maven,? Sectors CHARLIE HENRY ADAM - DAVID DA Apr EDWARD . ?Ab? I Austin Police Department Crime Analysis Unit 60 Appendix II: Codebook for Core Data Set Codebook: Sobering Center Revised 9/12/2019 Admit1: 0. Not admitted 1. Admitted AdmitBAC: BAC level measured Age: Age of patient in years Alcohol: 0. No alcohol use reported 1. Alcohol use reported AnnualIncome ArrivalDate: Date of arrival ArrivalTime: Time of arrival AUDITScore: AUDIT3levels: 1. 0-7 low risk 2. 8-15 borderline 3. 16+ high risk Chart#: Chart number of patient City: County: DASTdummy: 0. low risk 1. high risk DASTScore: DischargeDate: Date of discharge DischargeTime: Time of discharge 61 DischargeTransportation: Uber or Lyft Friend Walk Bus Taxi Don’t know Missing Doyouusetobaccoproducts: Do you use tobacco products? 0. No 1. Yes Doyouwanttostop: Do you want to stop using tobacco? 0.No 1. Yes EMSPDElapsedTime: Male: Is sex of patient male? 0. Female 1. Male Month: Month of arrival 1. January 2. February 3. March 4. April 5. May 6. June 7. July 8. August 9. September 10. October 11. November 12. December HighNeed: Homeless 0. Not homeless 1. Homeless Insured 0. No insurance 1. Insured (any type) 62 Level: Triage level Marijuana 0. No marijuana use reported 1. Marijuana use reported Military 0. Non military 1. Military/veteran MRN: Medical Record Number of patient NARCANProvided: 0. No 1. Yes Noadmit_reason: Reason for no admittance 1. Admitted 2. Jail 3. Medical clearance 4. Non-compliant 5. Not intoxicated 6. Psych 7. General "no" (no explanation) Opioids 0. No opioid use reported 1. Opioid use reported PDEMS: APD EMS Pflugerville TCSO PDEMSSector Phone: Phone number of patient Poly1: Poly-substance use 0. No 1. Yes ProvidedInfo: 0. No 63 1. Yes raceeth1: Race/ethnicity 1. White (non-Hispanic) 2. Black (non-Hispanic) 3. Asian (non-Hispanic) 4. Hispanic/Latino 5. Mix/multiple race (non-Hispanic) 6. Other race (non-Hispanic) **Referral: Was patient referred to treatment? 1. Yes 2. No Repeatclient 0. No 1. Yes (2+ visits) SBIRT: Received SBIRT services 1. Yes - given SBIRT 0. No - not given SBIRT State: Stay: Student: Patient is currently enrolled as student 1. Student 0. Non student TimePeriod Treatmentyesonly 0. did not go to treatment 1. went to treatment UnitOfficerandNumber Year: Year of arrival Zipcode: Zipcode of patient 64 Appendix III: Core Data Set The suggested revisions to the core dataset (deidentified) are available as an Excel file at the following link: http://bit.ly/SoberingCenter_Appendix3 65 Appendix IV: Cost-Benefit Analysis Workbook The template for the cost-benefit analysis, including a worked example, is available as an Excel file at the following link: http://bit.ly/SoberingCenter_Appendix4. 66