Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A SEATTLE OFFICE OF THE MAYOR Supporting LEAD in assessing impact Proposed Scope of Work November 8, 2019 Prepared by: BENNETT MIDLAND LLC 245 West 29th Street, Floor 12A New York, New York 10001 www.bennettmidland.com Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A Supporting LEAD in assessing impact Seattle Office of the Mayor 11/8/2019 Purpose and goals Bennett Midland proposes to partner with the Office of the Mayor and the Mayor’s Innovation and Performance Team, and collaborate with the LEAD team, to review and assess Seattle’s LEAD program. The goal of this work is to establish a shared understanding of LEAD’s existing deliverables, reporting capabilities, protocols, and procedures in order to develop an appropriate set of performance metrics, inform future program evaluations, and align future budgeted resources and contract provisions with the program’s foundational goals. Through this work, LEAD and the City of Seattle will: - - Establish a common understanding of the intended impact of LEAD for participants, community, and criminal justice; Agree on processes and protocols—particularly related to criminal justice engagement with LEAD, police referral protocols, and participant progress through the program; Surface best practices in case management, behavioral health treatment, and diversion for individuals with characteristics of LEAD participants; and Generate a set of performance metrics that reflect the goals of the program, can be captured and reported consistently, and can be used for measuring the achievement of program goals. To accomplish this, we have designed a fast-paced eight-week engagement, consisting of two phases of work, described below in greater detail. As our work unfolds, we would continue to shape our approach to ensure the goals of the project are aligned with and responsive to the needs of the City. Approach Phase 1: Gather facts and generate preliminary findings (Weeks 1-5) Kick-off To begin, we would hold a kick-off phone call with the designated team from the Mayor’s Office to understand the City’s most pressing interests and questions for this project. To ensure our engagement fulfills the City’s needs and interests, we would further refine the scope of our work with LEAD to be tailored to the preferred focus questions surfaced during this conversation. Following this discussion, we would develop a detailed work plan to guide our engagement, including the focus questions shared with us during the kick-off call, and any other key research questions to inform our fact-gathering. We would then hold initial calls with LEAD leadership and the Seattle Innovation and Performance team to introduce ourselves, share the goals of the engagement, and confirm our plan for working together. Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A In-depth review of program materials After quickly developing a work plan and confirming common objectives, we would work closely with the Innovation and Performance team to submit a request to LEAD for a compendium of documentation related to the program, aimed at understanding LEAD’s current and proposed data elements for relevance to performance outcome measures. Materials and data collected from LEAD would illustrate the details and effects of LEAD’s programming, the evolution of its service offerings, and the ways in which those offerings contribute to its stated goals and theory of change. We imagine that central program materials to review would include items such as: - Any existing data captured by LEAD staff related to program performance or administration; - Any existing program evaluation, including the University of Washington’s 2017 nonrandomized controlled evaluation titled “Seattle’s Law Enforcement Assisted Diversion (LEAD): Program Effects on Recidivism Outcomes”; - Documents related to the program model or program design, which could include descriptions of the theory of change, program goals, a logic model, etc.; - Descriptions of programmatic offerings, including levels or tiers of services provided, eligibility criteria for clients, case management touch points, etc.; and - Any training materials or tools used by LEAD staff or service providers. The specific research questions which would guide our review of this material would be defined in collaboration with the Mayor’s Office and the Innovation and Performance team, though we imagine would include such questions as: - What practices are in place for LEAD to address the housing needs of participants who are experiencing homelessness, and to what effect? - What is the spectrum of services available to LEAD participants, and how is the volume of active participants distributed across this spectrum? o How is a case defined as “active”? o What is the average length of program enrollment? o How many contacts do LEAD representatives have with program participants during the pendency of a case? - How many referrals to outside programs does LEAD make, by service type? - How are program goals communicated and tracked across programs? National landscape review and research Concurrently, we would begin a thorough review of similar diversion programs nationwide, benchmarking their performance to understand and identify best practices that could inform the Seattle LEAD program and positively impact program and participant outcomes. Through a review of available literature and interviews with service providers, experts in the field, and city officials responsible for procuring and overseeing similar diversion programs, we would seek to: - Compare LEAD’s case management process with best practice case management and behavioral health treatment modalities; Supporting LEAD in assessing impact 3 Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A - - Compare LEAD as a diversion program to other kinds of programs aimed at reducing criminal justice system involvement, including, but not limited to, court diversion programs; Identify the most effective methods for data collection and analysis to monitor recidivism rates; Review the effectiveness of police referral protocols and practices, specifically with an eye toward diversion methods vs. social contact; and Surface key challenges faced by diversion programs nationwide and effective strategies to mitigate common issue areas. With this information in hand, we would begin to organize our emerging findings on the details and effects of LEAD’s programming and theory of change into a logic model framework to illustrate LEAD’s best practices and service offerings, specifically outlining: Program inputs Examples: Administrative costs, staffing, budget allocations Program outputs Examples: Number of people served, number of referrals made Program outcomes Examples: Decline in arrests, decline in number of people experiencing street homelessness Program impact Examples: Drop in crime, reduced justicesystem involvement Site visit Equipped with a greater understanding of the program model and performance, we would work with the Mayor’s Office and the Innovation and Performance Team to design a site visit to drill down further on answering and refining the focus questions, and to populate the logic model framework in more detail with qualitative insights from key partners working to implement LEAD on the ground. In addition to the review of substantive materials, we would prepare for the site visit by developing a list of city agencies, community-based organizations, and other involved stakeholders who we would like to speak with. With the benefit of in-person time, we would seek to supplement our understanding from written materials with individual interviews with LEAD staff, providers (e.g., REACH), city and county agencies (e.g., SDP, CAO, KCPO, court systems, etc.), and other partners, as appropriate. Our goal in these conversations would be to understand more deeply what practices are in place to: consistently implement and document program activities, capture data on the impact of LEAD’s work, track performance on an ongoing basis, and report out to partners on the impact of this work. We would also supplement our structured interviews with staff and partners with on-site observations of the day-to-day work of the program. These observations would provide insight into matters that cannot be understood through document review and structured conversations alone; most significantly, they would allow us to more fully understand in what areas, if any, practices on the ground diverge from written program models and protocols. Supporting LEAD in assessing impact 4 Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A Deliverable: To conclude the site visit, we would hold a working session with Mayor Durkan and her team with two primary goals: 1. To share preliminary findings anchored in our focus questions, including findings from our best practices research and preliminary insights from our assessment of Seattle’s LEAD program, and get early feedback and reactions from Mayor Durkan and her team on the direction of the work—including where further research or exploration might be needed; and 2. To confirm expectations for the final report, including its format, scope, content, and intended audience. This discussion would equip our team to more effectively shape the next phase of our work in order to develop, iterate on, and revise a final deliverable aligned with the needs of your office. Phase 2: Develop and refine recommendations (Weeks 5-8) Over the final four weeks of the engagement, we would move through an iterative process of developing and refining recommendations for the LEAD program to ensure its service offerings align with its stated theory of change. At this stage, as we revise findings and the format of the recommendations with the Mayor’s Office, we would also conduct additional research to resolve outstanding questions surfaced during the first phase of work. Using the strong fact base established through document review and the site visit, we would review, analyze, and synthesize our preliminary findings to confirm LEAD’s theory of change, crystallizing its trauma-informed approach to providing supportive services and diverting people from the criminal justice system. We would also develop a set of recommendations related to establishing methods for capturing and understanding impact over time. Working closely with the City’s Innovation and Performance team, we would surface and codify proposed measures and data collection approaches to evaluate LEAD’s progress towards its stated goals. Our recommendations would be aimed at ensuring alignment between the program’s operations and theory of change, including: - A proposed method for capturing and routinely communicating relevant and clearly defined performance metrics on an ongoing basis; - Adjustments to program operations to be more consistent with LEAD’s theory of change, foundational goals, and best practices; - New protocols or policies to improve data collection and reporting; - New routines or ways of working to support the theory of change and inform future program development; and - Implications for capacity and resources. We would hold weekly calls—or videoconferences, if preferred—with the team at the Mayor’s Office during this phase of the work as we iterate together and refine the form of our final report. At least one working draft of the recommendations will be shared with the Mayor’s Office for review and comments before the findings and recommendations are finalized. Supporting LEAD in assessing impact 5 Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A Deliverable: Final report to inform LEAD program development and performance measurement practices, addressing each of the research areas and questions outlined in the previous phases of work. We would return to Seattle to present our findings and recommendations in person to the Mayor. Timeline and deliverables Phase 1 2 3 Week 4 5 6 7 8 1. Gather facts and generate preliminary findings - Detailed work plan, including key milestones and engagement goals - List of research questions to guide in-depth assessment of program materials - Facilitation guides for interviews and group conversations with program staff - Documentation of key takeaways and insights from interviews and research - Presentation of preliminary findings and core practices and data points 2. Develop and refine recommendations - Operational implications of preliminary findings - Additional research to resolve outstanding questions - Assessment of LEAD’s service offerings and their alignment with its programmatic theory of change - Demonstrated evidence base for proposed performance metrics and data collection routines - Final recommendations to inform LEAD program development and funding allocations - In-person presentation of final recommendations to the Mayor Proposed budget Phase Total Phase 1: Gather facts (weeks 1-5, including travel to Seattle) $51,700 Phase 2: Develop and refine recommendations (weeks 5-8, including travel to Seattle) $34,300 Total $86,000 Supporting LEAD in assessing impact 6 Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A About Bennett Midland Bennett Midland is a management consulting firm that works exclusively in the civic sector. We support city agencies, non-profit organizations, and philanthropies to tackle pressing problems and design programs and services that have an impact on the people they serve. We have worked across sectors on topics including public safety, child welfare, community development, and homelessness. The firm’s partners have also been strategic advisors to senior government leaders in New York City and beyond. Confidentiality and trust We hold our clients’ trust in the strictest of confidence. As a firm, we work with a broad cross-section of organizations in the civic sector—government agencies, not-for-profits, and foundations. Some work on the same issues, but hold different policy views; some receive or are applying for funding from others. In any case where Bennett Midland may hold a possible conflict, we immediately discuss it with our partners and agree on an appropriate way to move forward. We routinely sign confidentiality agreements so that we are able to review sensitive internal material and engage in frank, direct conversations with staff and the clients they serve. Relevant project experience Bennett Midland has supported program design and assessment for agencies and organizations in a range of circumstances. Our clients come to us at various stages in the process of program design—from planning and research to implementation—and we adapt our approach to meet the specific demands of the situation. Below are a few relevant examples from our recent work. MacArthur Foundation: Criminal Justice Strategy 1 The John D. and Catherine T. MacArthur Foundation, among the largest private foundations in the country, engaged Bennett Midland to help design and deliver a national initiative—the Safety and Justice Challenge—aimed at reducing over-incarceration by changing the way jurisdictions think about and use jails. Bennett Midland worked with MacArthur’s Justice Team from the earliest stages of planning to help shape this new investment. We conducted research and analysis about trends to help the team weigh possible strategic directions and relative opportunities for impact. By compiling facts from a range of sources—data and outcome trends, perspectives of experts, and goals and accomplishments of other initiatives operating in the same space—we helped the Foundation identify pivotal decision points in a logical succession, organize and facilitate productive debate, and document key decisions. Five years after the launch of the Challenge, the Foundation engaged Bennett Midland again to design and facilitate a comprehensive and collaborative process that would provide 1 Note: Bennett Midland has never had any involvement in the process to consider or award the MacArthur Fellowship (“Genius”) grants, nor any contact with the MacArthur team that manages the Fellowship process. Supporting LEAD in assessing impact 7 Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A answers to critical questions about the initiative and yield a vision for future direction. Bennett Midland led a dedicated “Strategy Review Committee” representing a range of perspectives—including trusted program advisers, experts in both criminal justice and communications, thought partners and researchers, etc.—through a rigorous process over the course of six months to interrogate the initiative’s underlying theory of change, model, assumptions, progress, outcomes and impact, and the meaningfulness of its contributions to date. At the conclusion of the review, the Foundation will present a clear vision for the future, rooted firmly in the strengths, assets, and accomplishments of the initiative to date, and include a road map to the Foundation’s most important goals for the next five years of their work. FPWA For almost a century, FPWA (formerly the Federation of Protestant Welfare Agencies) has advocated on behalf of New York City’s most vulnerable residents and supported the efforts of human services and faith-based organizations that are working to improve the lives of low-income New Yorkers. Building on this rich history, FPWA recently set a new strategic direction that emphasizes economic security. FPWA engaged Bennett Midland to support its planning efforts around its new strategic direction, with particular attention to how the organization would define, achieve, and track systemic and durable change in New York City. Specifically, we partnered with FPWA to: - Develop a succinct, measurable theory of change to guide organizational decisionmaking about opportunities to pursue and galvanize stakeholders; and - Develop and integrate a new and ongoing process to ensure that current and future programs and policies align with the theory of change. This work unfolded over the course of seven months with guidance from a planning committee made up of FPWA leadership and Board members and with the input and participation of FPWA staff, member organizations, and peers. In close collaboration with FPWA, Bennett Midland facilitated a planning process that involved thorough research and analysis, regular touchpoints with the Planning Committee and FPWA leadership team, and targeted decision-making. Improving residential services to meet the needs of New York City’s young people New York City’s Administration for Children’s Services (ACS) is committed to enhancing the continuum of foster care and residential program services to appropriately meet young people’s needs. Despite significant investment in these programs, ACS did not have access to the full set of information necessary to understand the extent to which the programs were fulfilling their goals. In order to effectively plan for needed adjustments and enhancements to their continuum of programs, ACS sought to understand the likely level of need for the various service types. To help answer this question, ACS engaged Bennett Midland to conduct a qualitative analysis of a robust sample of recent cases, and to calculate from this “start of the case” data the likely distribution of service needs. Supporting LEAD in assessing impact 8 Supporting LEAD in assessing impact - BENNETT MIDLAND LLC / CBO201909 / Exhibit A Bennett Midland worked closely with a working group, comprised of representatives from various departments within ACS (including the Divisions of Child Protection, Family Permanency Services, and Policy, Planning and Measurement). The goals of our analysis were twofold: (1) to develop an informed estimate of the population expected to be best served in residential care in the coming years, and (2) to understand the types of service needs of this population. Using a methodology designed in collaboration with the working group, Bennett Midland conducted the needs analysis on four hundred sample cases to determine the required capacity of residential placement slots and the distribution of slots by specialized need category. Our analysis informed the agency’s procurement strategy for residential care services and was used to shape the scope of work and requirements for service providers. We would be happy to provide additional information about our experience and our firm or revise our approach based on your feedback. We are delighted by the prospect of working together and look forward to hearing from you. Supporting LEAD in assessing impact 9