Denver Day Works Pilot Year 1 Evaluation November 2017 PUHL If?; Center en Network Seienee DENVER Executive Summary What is Denver Day Works? The Denver Day Works (DDW) Pilot is a supported work program designed to provide a low- to no-barrier work experience for people throughout the city who are experiencing homelessness, while also connecting participants to supportive services such as food, shelter, and other necessities. The initial pilot program (from Nov. 1, 2016 to Oct. 31, 2017) is funded through a partnership between Denver's Road Home, Denver Public Works, Denver Parks & Recreation, and Denver Human Services. Bayaud, a nonprofit community partner, is contracted to administer the program on behalf of the City. Who can participate? Individuals interested in working on a Denver Day Works site (at a city park, Public Works site, nonprofit or private organization) are initially assessed for job readiness by the outreach team. Those who are not yet job-ready are connected to resources to help them establish their job-readiness and will be connected to a work site as soon as feasible. Those job-ready are referred to a job site and provided with information relating to location, contact person, proper attire, etc. How does it work? Participants are given the option of working a full-day or half-day shift. Denver Day Works participants are paid at the end of each shift and offered access to financial planning services at no cost to participants. Participants at city job sites are paid based on the City's ordinance guidelines. If a participant does not have access to identification, they are offered the same work experience as others and instead are provided other supportive services, including a non-monetary compensation. Initial background checks are not required, though may be required for longer-term employment following their time with Denver Day Works depending on the position. About this evaluation. The Center on Network Science evaluation process utilizes a Systems Framework that focuses on the boundaries, perspectives, and inter-relationships among the many actors and activities in DDW’s system. A Systems Framework approach includes perspectives from all members of the system and is typically summarized both as the parts of the system and how they interact. This allows the inquiry and focus of the evaluation to be on where the strengths and weaknesses in the system lie. Denver Day Works is a comprehensive program that includes several types of stakeholders and activities that interact together in different ways to make the program work. These include the public, government agencies, the contracted administrator, employers, community partners, participants, and others. In this case, we included the voices of multiple perspectives (including the City, contractor, participants, work site supervisors, and employers) and broke those perspectives down into three phases: Entry into the DDW Program; DDW Program Implementation; and DDW Long-Term Employment Placement. We framed the analysis in a systems perspective, broadly, to identify:  Where is the system strong?  Where is there room for opportunity and growth? 2 The report organizes the collected data within these three program phases to provide a comprehensive story of the program’s strengths and challenges, which are framed around the program’s potential sustainability and growth. Methods. Taking a systems approach to this evaluation required obtaining data from multiple perspectives in a variety of ways. To do this, the Center on Network Science used three separate data sources. The first was existing program data provided to the evaluation team by the contractor, Bayaud Enterprises. The second was interviews with stakeholders from different parts of the system: program administrators, program participants, and employers. The third was a social network analysis survey of the partners involved with DDW, using the PARTNER (Program to Analyze Record and Track Networks to Enhance Relationships) tool. Each of these methods are described in detail within the report. Program Data. Several goals (listed below) were established at the start of the program’s pilot phase. The program’s data demonstrates DDW’s success at meeting all but two of its stated goals.    OUTREACH: Engage with at least 300 people.   PERMANENT WORK: Connect at least 49 of the participants to permanent work. PARTICIPATE IN PROGRAM: Connect 150 to a work experience through Denver Day Works. RETENTION IN DDW: Retain at least 70 of the participants for more than one day of work at either city or established contract sites. RETENTION IN JOB: Retain at least 30 individuals connected to permanent work for at least 90 days. Ideally, some of the permanent jobs will be with city agencies. The only program goal not met is participant placement in permanent work inside city Agencies. The goal was to place 20 individuals within city positions, yet only 12 participants are currently situated in permanent jobs. Social Network Analysis Survey. The PARTNER survey was sent to 33 organizations and yielded a 78% response rate. The list of “partners” was identified with assistance from key informants at city agencies and Bayaud. Partners were identified as any organization that is connected to the Denver Day Works program. A total of 318 partnerships were described by the 26 survey respondents. A “partnership” is defined by any two organizations and their interactions described in the survey. Overall 43% of all partnerships reported they were either created through DDW or strengthened by the program – evidence of systems building happening across partnerships in the network. 3 Interview Data. Data was collected via 17 interviews with Denver Day Works administration (both city and Bayaud representatives), worksite supervisors, employers, and program participants. Findings. Phase 1: Entry into the Denver Day Works Program Strengths: Overall strengths of entry into the program included: 1) unique low-barrier work opportunities; and 2) successful program outreach and increasing demand. Opportunities for Improvement: Program entry challenges included: 1) the need for increased diversity in the program; and 2) overall limited capacity to access all the people on the waitlist, or expand current offerings. Phase 2: Denver Day Works Program Implementation Strengths: Overall strengths of DDW program implementation included: 1) providing initial work opportunities and job readiness support for participants; 2) earning money and program convenience; 3) integrating wrap around support services; 4) increasing self-sufficiency; 5) providing a support system and empowering participants; and 6) commitment from the City, increased positive reputation for the City, and improved understanding towards people experiencing homelessness. Opportunities for Improvement: Program implementation challenges included: 1) the high levels of resources and increased funding needed to scale the program; and 2) the difficulty of providing certain services (e.g. mental health support). Phase 3: Denver Day Works Long-Term Employment Placement Strengths: Overall strengths of this phase included: 1) reaching participant placement goals; 2) the quality and outcomes of hosting employment hiring events; 3) offering individualized employment support; 4) providing advantages to employers; and 5) meeting overall retention rates. Opportunities for Improvement: Long-term employment challenges included: 1) a need for additional placements of program participants into jobs within the City; 2) transportation and housing resource limits; and 3) the challenging nature of retention. Recommendations. High Level Program Recommendations The following section offers an overview of three high level areas of program recommendations. A. Increase the program budget to support expanding the program’s number of work days and diversity of worksites. The DDW program is viewed as successful on many fronts. All stakeholders agreed the program was meeting most of its goals and providing a new and innovative way to offer day work to people experiencing homelessness in Denver. The success of this program has led to an increased demand for services, both in number of slots, and in the types and number of worksites available. Nearly all participants in this evaluation agreed that the City should consider scaling the program to all five workdays. Participants and program staff emphasized specifically the need to grow the diversity of work options (for example to include clerical options) and the types of worksite available for DDW 4 participants. This recommendation would require a scaling of the budget for the program. Program staff indicated that increasing the size and number of worksites would scale the budget proportionately, with actual costs increasing at the same rate. B. Develop an employee-supported transition component between day labor and fulltime employment. Many respondents discussed the challenges participants faced in transitioning from a once a week job to a full time commitment. Several of the participants do not have transportation, a way to clean themselves or their clothing, or a place to store their stuff day-to-day. Others are suffering from mental, behavioral, and other health issues. There was support from participants in this evaluation to develop a more intentional transitional process. Ideally, a work crew cohort accompanied by a dedicated staff person to provide direct and individualized support for participants as they transition into permanent employment. Experience has illustrated that this approach can result in great long-term employment placement outcomes. Costs for this type of program, serving 5-12 participants, have been estimated by Bayaud as $261,698. While the City is indeed the largest single employer to hire Denver Day Works participants, it falls short of its self-imposed program goal of hiring 20 participants. While it is unclear how the goal of 20 participants was selected, this is the baseline agreed upon for the project. To reach this goal, the City can focus on alternative and additional strategies for addressing this gap, such as identifying additional types of potential employment opportunities that exist with city agencies. Or the City may want to reexamine the 20 participant goal and consider how to align the expectation for placements with available positions that participants are eligible for and in comparison to the other potential employees (who hired less overall permanent placements than the City during this pilot). We do recommend the City consider what kinds of protocols and considerations are preferred to have successful graduates of the program transition into permanent employment within the city. C. Engage in a participatory strategy session to determine the system of governance for the DDW program. An area that remains unresolved for stakeholders is where the DDW program should reside at the City. There was no clear consensus on this issue, largely due to limited discussion about the future direction of the program, as its success becomes broader beyond the original goals. While the DDW program has demonstrated success in Denver Human Services, some interviewees suggested that the City have an indepth facilitated discussion to define the future goals of the program before considering any change to its current home. These include options to extend the day work opportunities and leverage the lowbarrier entry and immediate payments of the program as a way to address the needs of people experiencing homelessness; or to transition to a more systemic workforce development program that is focused on integrating a long-term system of wraparound services and supports, or both. The success of DDW within DHS suggests that placing it in another agency risks disrupting what is currently an effective, popular, integrated city program. We recommend that the City devote some time and resources to engage in a participatory strategy session to flesh out where the program should reside. If the program continues to expand its wraparound services for participants and engage in a more focused intention on permanent employment, it should consider which City agency is the appropriate home for oversight of the program. 5 More program specific recommendations include: Phase 1 Recommendations: Entry into the Denver Day Works Program 1) Administrative adjustments, including enhancements to the entry process through additional staffing. 2) Pre-screening candidates to match labor to participant compatibility. 3) Additional outreach and staff training that incorporates Trauma-Informed Care prior to engaging with DDW program participants. Phase 2 Recommendations: Denver Day Works Program Implementation 4) Increased budget/program funding to support additional program staff & worksite supervisors. 5) Increasing the number of workdays from three to five days during the week. 6) Increasing the number and types of worksites. Adding a variety of worksite job tasks will align participants with job market demand by improving their qualifications for a wider range of positions (other than only landscaping or janitorial/maintenance). 7) An increase in program staff and worksite supervisors. Additionally, two interviewees mentioned hiring a formal coordinator to communicate between worksites, program staff, and participants. From these suggestions, the evaluation team suggests hiring or designating an existing program staff to coordinate worksite logistics and provide direct supervision of worksite tasks. This will help alleviate the burden on city employees currently supervising worksites and allow Bayaud program staff to focus on providing case management to participants. 8) In addition to ongoing partnerships, continue to add additional support services including collaborating with food-assistance, housing partners, and other services to support participants outside of formal programming. Phase 3 Recommendations: Denver Day Works Long-Term Employment Placement 9) The provision of supportive employment structures to support participant workplace expectations and impacts. Suggestions included: a. Establishing a crew of participants which work together over a period of time (rather than new groups each day), to both establish a stronger support system between participants, as well as providing a more consistent job training experience. b. Providing continued case management as participants transition into full-time work environments. 10) Engage the City and other potential employers in conversations to address the barriers to hiring both inside and outside the City (for example: criminal backgrounds). In Summary. The multiple perspectives of the Denver Day Works system within this evaluation confirms widespread support for the Denver Day Works program. A multitude of program strengths were expressed across all three phases of the program. Opportunities for improvements were presented as suggestions to expand the types of program opportunities and offer a more direct whole person approach to the work, shifting somewhat from a defined day work program to a more comprehensive workforce development program. Increased funding and diversity in worksites were emphasized by all groups. Additional support for transitioning participants from day work to full-time employment appears to be a pain point that needs to be addressed to adequately meet the long-term goals for job retention. The only goal that was not met, and does not seem on course to be met by the end of the pilot, is permanent placement in city agencies. It was unclear to many stakeholders why the City is not more proactive in placing DDW participants in permanent city positions. This is an area that should be addressed in future planning and goal setting for the program. 6 Contents Executive Summary....................................................................................................................................... 2 About Denver Day Works............................................................................................................................ 10 Who can participate? .............................................................................................................................. 10 How does it work? .................................................................................................................................. 10 Denver Day Works Goals ........................................................................................................................ 10 About the DDW Evaluation ......................................................................................................................... 12 Systems Framework ............................................................................................................................ 12 Methods ...................................................................................................................................................... 13 Program Data. ......................................................................................................................................... 13 Interviews................................................................................................................................................ 13 Social Network Analysis Survey. ............................................................................................................. 14 Program Data Analysis ................................................................................................................................ 15 Denver Day Works by the Numbers ....................................................................................................... 15 Who are the DDW Participants? ......................................................................................................... 15 What are the outputs of the DDW program? ..................................................................................... 16 2016-2018 Denver Day Works Program Financials..................................................................................... 17 Social Network Analysis - PARTNER Survey Results .................................................................................... 18 Where Should the Program Reside Inside the City? .................................................................................. 23 Description of Stakeholders Included in the Interviews ......................................................................... 24 Phase 1: Entry into DDW Program .............................................................................................................. 25 Strengths ................................................................................................................................................. 25 Opportunities for Improvement ............................................................................................................. 25 Phase 2: DDW Program Implementation.................................................................................................... 26 Strengths ................................................................................................................................................. 26 The Role of Bayaud ................................................................................................................................. 30 Opportunities for Improvement ............................................................................................................. 32 Phase 3: DDW Long-Term Employment Placement.................................................................................... 34 Strengths ................................................................................................................................................. 34 Opportunities for Improvement ............................................................................................................. 36 Recommendations for Denver Day Works Program ................................................................................... 38 A. Increase the program budgets to support expanding the program’s number of work days and diversity of worksites. ......................................................................................................................... 39 B. Develop an employee-supported transition component between day labor and full-time employment. ....................................................................................................................................... 39 8 C. Engage in a participatory strategy session to determine the system of governance for the DDW program............................................................................................................................................... 39 Specific Program Recommendations ...................................................................................................... 40 Phase 1 Recommendations: Entry into Denver Day Works Program ................................................. 40 Phase 2 Recommendations: Denver Day Works Program Implementation ....................................... 40 IN SUMMARY…............................................................................................................................................ 41 Appendices.................................................................................................................................................. 42 Appendix 1. Bayaud Data Summary Report............................................................................................ 42 Appendix 2: PARTNER Denver Day Works Survey .................................................................................. 45 Appendix 3. Key Informant Interview Guide .......................................................................................... 52 Appendix 4. Interview Codes .................................................................................................................. 54 9 About Denver Day Works The Denver Day Works (DDW) Pilot is a supported work program designed to provide a low- to nobarrier work experience for people throughout the city who are experiencing homelessness, while also connecting participants to supportive services such as food, shelter, and other necessities. The initial pilot program (from Nov. 1, 2016 to Oct. 31, 2017) is funded through a partnership between Denver's Road Home, Denver Public Works, Denver Parks & Recreation, and Denver Human Services. Bayaud, a nonprofit community partner, is contracted to administer the program on behalf of the City. Who can participate? Individuals interested in working on a Denver Day Works site (at a city park, Public Works site, nonprofit or private organization) are initially assessed for job readiness by the outreach team. Those who are not yet job-ready are connected to resources to help them establish their job-readiness and will be connected to a work site as soon as feasible. Those who are job-ready are referred to a job site and provided with information relating to location, contact person, proper attire, etc. How does it work? Participants are given the option of working a full-day or half-day shift. Denver Day Works participants are paid at the end of each shift and offered access to financial planning services at no cost to participants. Participants at city job sites are paid based on the City's ordinance guidelines. If a participant does not have access to identification, they are offered the same work experience as others and instead are provided other supportive services, including a non-monetary compensation. Initial background checks are not required, though may be required for longer-term employment following their time with Denver Day Works depending on the position. Denver Day Works Goals Several goals were set at the start of the program’s pilot phase. These are listed below.    OUTREACH: Engage with at least 300 people.   PERMANENT WORK: Connect at least 49 of the participants to permanent work. PARTICIPATE IN PROGRAM: Connect 150 to a work experience through Denver Day Works. RETAIN IN DDW: Retain at least 70 of the participants for more than one day of work at either city or established contract sites. RETAIN IN JOB: Retain at least 30 individuals connected to permanent work for at least 90 days. Ideally, some of the permanent jobs will be with city agencies. 10 DENVER DAY WORKS PILOT – CONCEPTS TO BE TESTED Overall Program:         PARTICIPATION IN PROGRAM: Are homeless individuals willing and able to engage in a day’s labor at city worksites? How many individuals? DAY LABOR-MUTUALLY BENEFICIAL TO PARTICIPANT & CITY: Can the City provide labor opportunities in a mutually-beneficial way? How many labor opportunities? ENGAGEMENT IN SERVICES: Is a day’s labor opportunity offered to homeless individuals unengaged from services an effective gateway to build connections to other services? COLLABORATE-COORDINATE WITH BAYAUD: How best should the City collaborate with an external nonprofit, including supervision governance? o Does this collaboration fit well with existing City work/workers? o How best can the City define and measure success? o In the above, what works and what doesn’t work? What may be missing? WORKSITE SUPERVISORS ACCLIMATE: Can City supervisors acclimatize to supervising new City employees from the DDW program, with their additional needs? PARTICIPANT NEEDS: What spectrum of client needs are being met by the pilot? Is it geared for the biggest opportunities? EMPLOYMENT SUPPORT: What balance between day labor, supported employment, and permanent employment meets participants’ needs for a successful transition to holding down a job? SUPPORT SERVICES: Why aren’t people engaging in public assistance, employment assistance, or retention services? By Program Manager:   CITY HIRING- PARTICIPANT EXPERIENCE ON WORKSITE-SKILLS: How might each City Department allow for 1% of their workforce to be replaced by DDW crews? FUNDING: How can this program best be funded in the future? Individual department contributions, program allocation, private contributions, social impact bond, etc. (Office of Budget Management, 2018 Budget Talks) Future Concepts to be Tested (Late Pilot or Post-Pilot):   EMPLOYMENT SERVICES: Can day-labor facilitate a referral to greater employment services (Workforce Center, community-based services, and direct employment)? SELF-SUFFICIENCY: Does no-strings day-labor supply a specific need on the path to selfsufficiency? This report is a summary of the findings from the first evaluation of the DDW Pilot program. In July 2017, the City of Denver contracted with the University of Colorado Denver’s Center on Network Science to conduct an evaluation of the DDW Pilot program. More detailed information can be found throughout a number of appendices. For more information on the Center on Network Science, go to www.center-networkscience.net or email cns@ucdenver.edu. 11 About the DDW Evaluation Systems Framework This evaluation utilizes a Systems Framework that focuses on the boundaries, perspectives, and interrelationships among the many actors and activities in DDW. A Systems Framework approach includes perspectives from all members of the system and is typically summarized both as the parts of the system and how those parts interact. This allows the inquiry and focus of the evaluation to be on where the strengths and weaknesses in the system lie. Denver Day Works is a comprehensive program that includes several types of stakeholders and activities that interact together in different ways to make the program work. These include the public, government agencies, the contracted administrator, employers, community partners, participants, and others. In this case, we included the voices of multiple perspectives (including the City, contractor, participants, work site supervisors, and employers) and broke those perspectives down into three phases: Entry into the Program; DDW Program Implementation; and DDW Long-Term Employment Placement. We framed the analysis in a systems perspective, broadly, to identify:   Where is the system strong? Where is there room for opportunity and growth? The report organizes the collected data within these three program phases to provide a comprehensive story of the program’s strengths and challenges, which are framed around the program’s potential sustainability and growth. 12 Methods Taking a systems approach to this evaluation required obtaining data from multiple perspectives in a variety of ways. To do this, the Center on Network Science (CNS) used three data sources. The first was existing program data provided to the evaluation team by the contractor, Bayaud Enterprises. The second was interviews with stakeholders from different parts of the system: program administrators, program participants, worksite supervisors, and employers. The third was a social network analysis survey of the partners involved with DDW, using the PARTNER (Program to Analyze Record and Track Networks to Enhance Relationships) tool. Each of these methods are described in detail below. Program Data Interviews Network Analysis Program Data. Bayaud Enterprises provided the evaluation team with data they have collected on Denver Day Works since the beginning of the program. At times in the report, a different total may appear than what has been reported on some of the quarterly reports completed by Bayaud due to the dynamic nature of the participants in the program (both in number of current participants and tenure in the program) and challenges with having complete data on all participants. The data received by the evaluation team included all available fiscal quarters of data on participant demographics including: education level, number of disabilities, ethnicity, race, primary language, gender, housing, offender status, citizenship, number of services used, number of assessments given, total trainings used, employment assignments, number of employers, current work status, and number of current jobs. Interviews. From July 2017 through August 2017, the evaluation team conducted a total of 17 in-person interviews with multiple stakeholders involved with Denver Day Works. The Denver Day Works Project Manager at Denver Human Services and Bayaud Enterprise staff helped the evaluation team identify individuals representing a variety of roles and perspectives related to the program. Interviews included staff from Bayaud Enterprises (2), Denver Human Services (2), Denver Parks and Recreation (2), Denver Public Works (1), Denver Central Library (2), a nonprofit employer (1), and current and former Denver Day Works participants (7). Interview questions were tailored for the interviewees’ role within Denver Day Works, however, each person was asked key questions about the program’s strengths, challenges, suggestions for improvements, and recommendations for scaling the program. Interviews ranged from 30 to 90 minutes in length and with interviewee permission, all but one interview, was audio recorded and transcribed. All interviews were coded using a constant comparative method and analyzed in Excel. Throughout the data collection and analysis process, the evaluation team met frequently to discuss emerging themes and codes, ensuring inter-rater reliability across coding analysis. 13 Social Network Analysis Survey. The evaluation team conducted a social network analysis (SNA) in August 2017 to assess the partnerships and types of activities that stakeholders in the network are currently engaged in related to DDW. To conduct the evaluation, the PARTNER (Program to Analyze, Record, and Track Networks to Enhance Relationships, www.partnertool.net) tool was used to conduct and analyze the survey data (see below for more detail on this tool and how it was used in this project). The PARTNER tool, housed and managed by the team at the Center on Network Science, includes a validated survey instrument, a data collection methodology, and an evaluation framework to guide analysis1. It has been used by over 2,500 cross-sector networks to analyze how partners are connected, how resources are exchanged, the levels of trust and perceived value among partners, and to link outcomes to the process of collaboration. In addition to the 19 validated PARTNER survey questions, CNS added 5 additional questions based on DDW project objectives. The types of questions included in the DDW PARTNER Survey are listed below in Table 1. A full list of questions is included in Appendix 2. Table 1: Denver Day Works PARTNER Survey Questions Demographic  Length of time involved in DDW Questions  Resources willing to contribute to DDW effort (organizational description): Questions About  Collective outcomes of the network the Network:  Perceived success/strengths of the network  Suggestions to improve the network Relational Questions: “From the list, select organizations with which you have an established relationship (either formal or informal) in the context of Denver Day Works.” Once each respondent selected their organizational partners, they were asked to answer the following questions: 1) Activities partners engage in together 2) How partnership started 3) Outcomes of the partnership 4) Extent of Value as: (a) Power/Influence, (b) Level of Involvement, (c) Resource Contribution. 5) Extent of Trust as: (a) Reliable, b) In Support of the Mission, and (c) Open to Discussion. Open-ended:  Please list one strength and one potential area of improvement  Please list other partners that work on homeless and day labor in Denver  What resources are necessary to achieve long-term sustainability  Did day labor programs result in greater access to employment services  Please list aspects of the Denver Day Works program that could use the most improvement Varda, Danielle M., Anita Chandra, Stefanie Stern, and Nicole Lurie (2008). “Core Dimensions of Connectivity in Public Health Collaboratives” Journal of Public Health Management and Practice, 14(5): E1-E7. 1 14 Program Data Analysis Denver Day Works by the Numbers Who are the DDW Participants? Female participants are more likely than male participants to access services (67% vs. 52%), access trainings (24% vs. 20%), and have a current job (29% vs. 25%). 25% Of participants that go through the Denver Day Works Program are placed with employment. Participants with a criminal background are more likely to have been exposed to services (93% vs 51%). Participants without a criminal background are more likely than those with a criminal background to access trainings (22% vs. 7%). Participants with disabilities are more likely than those without disabilities to access services (65% vs. 45%), access trainings (25% vs. 17%), and have current employment (28% vs. 22%). 15 What are the outputs of the DDW program? A review of quarterly reports submitted by Bayaud illustrated many of the outputs achieved by the DDW program. These results demonstrate the many ways the program collects data and tracks progress. These numbers show where DDW has met or exceeded their goals as of August 2017. Denver Day Works Outputs # of... Employer Participation Resources Provided 1st Qtr 2nd Qtr 3rd Qtr Bus tickets distributed (local day pass) Total: 1,146 Individuals who received additional support services through a needs assessment Total: 150 Gift cards distributed Total: 123 Monthly bus passes distributed Total: 92 Employers that were recruited to participate in program Total: 79 Employers that hired program participants Total: 50 Hours Contributed Individual Participation Individuals recruited for the program Total: 392 Individuals who participated in paid day work activity Total: 168 Individuals who were placed in permanent employment Total: 85 Individuals who participated in volunteer activities Total: 31 Individuals who succeeded meeting 90 day retention mark Total: 27 6 hr work days Documented wage subsidized 8-hour Total: 6,608 Documented volunteer 8-hour 6 hr work days Total: 654 0 200 400 600 800 1000 1200 1400 Number of outputs by category 16 2016-2018 Denver Day Works Program Financials *Projected Budget based on 3 workdays from Nov-Dec 2017 & expanding to 5 workdays from Jan-June 2018. The total projected budget for Nov 2016-June 2018 included figures from the 2016-17 original pilot year budget and Nov 2017-18 projected budget. 17 Social Network Analysis - PARTNER Survey Results The PARTNER survey was sent to 33 organizations and yielded a 78% response rate. The list of “partners” was identified with assistance from key informants at Bayaud and participating city agencies. Partners were identified as any organization connected to the Denver Day Works program. A total of 318 partnership were described by the 26 survey respondents. A “partnership” is defined by any two organizations and their interactions described in the survey. Who are the organizations that make up the DDW networks? The surveyed Denver Day Works network consists of diverse partners from a variety of sectors (see pie chart below). In the map below, the colorful squares each represent an organization that was identified as a partner within the DDW pilot program. The lines between them indicate a relationship. The DDW network is full of cross-sector relationships, with quite a large number of them very actively working in coordination with one another. 18 Partnership Outcomes: Respondents of the PARTNER survey reported on the “outcomes” of their partnerships with each other. About 1/3 of all respondents reported that their partnerships led to 1) program outcomes, 2) exchange of resources/services, and 3) placement/hiring DDW participants in a job, and 4) other partnerships. . When asked about the most important outcome of Denver Day Works thus far, nearly 1/3 of all respondents felt the most important outcome was implementing effective strategies and programming to increase access to employment opportunities, education, and services for homeless populations. What is the Most Important Outcome for Denver Day Works to Date? 19 Perceptions of Success: In the PARTNER survey, respondents were asked how successful DDW has been at reaching the goals of the pilot. Overwhelmingly, respondents felt the program was successful with three people even citing it has been a “complete success.” How successful has the Denver Day Works Pilot been at reaching its goals? What Contributed to the Success of DDW? 20 Level of Partnership Activities: Respondents also identified the level of activities that their partnerships entailed. Something unique about the DDW network is most of their activity (37%) is at the “integrated” level – which is the most intense and active. This is likely due to the high levels of service integration required to administer the program. These data demonstrate the level at which these partners in the DDW network already have strong, active relationships, which is a positive outcome of this network analysis since service coordination for participants is a primary goal. What kinds of activities does your relationship with this organization/department entail: (n=291) None N=1 Awareness of this organization Cooperative Activities: Involves exchanging information, attending meetings together, and offering resources to partners. Coordinated Activities: Include cooperative activities in addition to intentional efforts to enhance each other's capacity for the mutual benefit of programs. Integrated Activities: In addition to cooperative and coordinated activities, this is the act of using commonalities to create a unified center of knowledge and programming that supports work in related content areas. N=35 N=79 N=67 N=108 In the maps on the following page, we are able to see a bit more nuance in the relationships between partners. Map A highlights the partnerships where partners indicated their relationship consisted of “integrated” activities, such as sharing funding. Although working on integrated activities together may imply a higher level of partner engagement, integrated activities typically require more resources to maintain, and are not always necessary for “good” connectivity between partners. There is a high level of connectivity and activity in this network at the "integrated" level, including a relationship with one prospective partner. However, there tended to be more ties between agencies within the same sector, and fewer ties between agencies across sectors. Map A illustrates that building prospective partnership ties is missing in partnerships in the "integrated" level. The DDW team could use this information to consider where to invest in future potential partnerships that are more integrated (perhaps more resource intensive or committed). 21 Map A – Integrated Only Partnerships (4 Prospective Partners Not Connected at this level) Map B shows partners that indicated they interact at one of the lower “levels” – at a cooperative level. Generally, these types of ties are less resource intensive. The contrast between Map A and B show the level of intensity and resources that are required to maintain an integrated level of activity in this network, which is far more than other types of collaborative networks that may not be as focused on service integration. Map B – Cooperative Only Partnerships 22 Where Should the Program Reside Inside the City? A question that came up among a number of stakeholders was where the Denver Day Works program should reside inside the City. Since implementation, DDW has resided inside Denver Human Services (DHS), the agency responsible for providing assistance benefits and protection and prevention services to Denver residents. DHS is home to Denver’s Road Home, the City’s program to assist those experiencing homelessness. DHS also oversees two of the City’s workforce programs (Employment First and Colorado Works) and runs the Veterans Services Office, which has a significant focus on employment. DHS has successfully conceptualized, implemented, and administered the DDW program pilot, and has ongoing strategies for future program implementation. In the early implementation of DDW, the focus for the City was primarily on how to create a viable solution to the growing occurrence of those experiencing homelessness throughout Denver. The growing concern for encampments around the City, and the potential of DDW to provide a low-barrier entry into immediate income for the homeless, along with a number of other factors including experience administering other employment programs, has made DHS a good home for the program. As one interviewee mentioned, the initial problem DDW was addressing was how to support the immediate work and social service needs of this population. She stated, “This was not designed as a workforce development program; rather it was an intentional day work program. At the time it solved the question of how to find immediate resources for our growing homeless population, and therefore the focus of the program was on the homelessness issue, rather than a work issue.” As the program has progressed and with the integration of a wraparound services approach from Bayaud in place, the program has begun to transition to more of a workforce development program. One interviewee said that if the goal of the program is to address “how can we build a system of services around those experiencing homelessness that address their housing, work, health, and mental health needs” then we might want to take a close look at the viability of continued partnerships for the program such as in the Mayor’s Office of Economic Development. As an employment development organization, Bayaud has already implemented a number of other programs focused on this goal and is integrating previously used successful techniques into Denver Day Works. While the DDW program has demonstrated success in DHS, some interviewees suggested that the City have an in-depth facilitated discussion to define the future goals of the program before considering any change to its current home. These include options to extend the day work opportunities and leverage the low-barrier entry and immediate payments of the program as a way to address the needs of people experiencing homelessness and/or to transition to a more systemic workforce development program that is focused on integrating a long-term system of wraparound services and supports. The success of DDW within DHS suggests that placing it in another agency risks disrupting what is currently an effective, popular, integrated city program. 23 Interview Results Description of Stakeholders Included in the Interviews From July 2017 through August 2017, the evaluation team conducted a total of 17 in-person interviews with multiple stakeholders involved with the Denver Day Works. Interviews included staff from Bayaud Enterprises (2), Denver Human Services (2), Denver Parks and Recreation (2), Denver Public Works (1), Denver Central Library (2), a nonprofit employer (1), and current and former Denver Day Works participants (7) (see Appendix 4 for full interview data).      Bayaud Enterprises is the contracted nonprofit administering the day-to-day operations of Denver Day Works. Bayaud’s responsibilities include recruitment, screening, holding orientations, accounting, coordinating with worksites and employers, distributing participant payments, providing case management and support services navigation, employment placement and retention support, and tracking and reporting program outcomes. The two interviewed Bayaud staff are referred to as “program staff” or “Bayaud staff” in the interview analysis section. Denver Human Services (DHS) is the lead City contracting administrator and contact for Denver Day Works. DHS also provides a City Transportation Worker and bus to transport participants to/from worksites. The two interviewed DHS staff are referred to as “city staff” in the interview analysis. Denver Parks and Recreation, Denver Public Works, and Denver Central Public Library are both worksites and employers for DDW participants. One interviewed Denver Parks and Recreation staff member participated in the workgroup development process of DDW and is referred to as “city staff” in the interview analysis. The other interviewee from Denver Parks and Recreation along with the Denver Public Works and Denver Central Public Library staff members are referred to as “worksite supervisors/employers” in the interview analysis. Worksite supervisors/employers both oversee DDW on-site projects and participants hired as employees within their departments. The nonprofit employer interviewee, referred to as “employer” in the interview analysis, has participated in DDW employer hiring events and hired many DDW participants. The organization’s name is not disclosed for confidentiality purposes. A total of seven interviews were conducted with two current program participants and five former participants placed in jobs. Interviewed participants consisted of five males and two females. Employed former participants currently work at Bayaud Enterprises (1), Denver Public Works (1), Denver Central Public Library (2), and a nonprofit organization (1). All of these individuals are referred to as “participants” in the interview analysis section. These interviewed stakeholders are involved in multiple phases of the DDW program: entry into the program, program Implementation, and full-time employment placement. In the following section, we present the interview findings by each of these phases, starting with entry into the program. 24 Phase 1: Entry into DDW Program Strengths Unique Low-Barrier Work Opportunities One of the most unique aspects of Denver Day Works is offering a lowbarrier work opportunity for individuals experiencing homelessness. Both participants and partners expressed individuals experiencing homelessness have the opportunity to work because the program overcomes common employment barriers by not requiring an ID, social security card, or background check (n=5). One participant commented that not needing documentation “sounded too good to be true.” Another participant shared, “I was looking for a chance and they are hard to get, especially when you’ve got a record.” SUMMARY OF STRENGTHS AND OPPORTUNITIES FOR IMPROVEMENT IN PHASE 1 Successful Program Outreach and Increasing Demand Within its pilot year, the program has recruited 392 individuals experiencing homelessness, exceeding its goal of 300 (Bayaud, 2017Quarter 3 Report). These numbers indicate demand for the program and an opportunity for growth. Outreach tactics, according to interviewees (n=4), include media attention, referrals from city employees, and participant word of mouth. Onehundred percent of interviewed participants (n=7) stated they would recommend this program to others. Opportunities for Improvement Need for Increased Diversity Strengths Included:  Unique Low-Barrier Work Opportunities  Successful Program Outreach and Increasing Demand “I would love to see this program continue and expand!” -City Agency One interviewee mentioned how the City hopes to bring more diverse populations into the program (e.g. women, minorities, individuals with disabilities). This desire for more diversity was also voiced during participant feedback sessions held by Bayaud, where participants brought up the lack of women participation in the program and specific barriers that face women who do enter the program. “I have absolutely LOVED being a part of the Day Works team and feel it is so critical for the Denver community!!” -Bayaud Increasing Capacity to Enroll Participants in DDW Opportunities for Improvement:  Need for Increased Diversity  Limited Capacity (waitlist) While program outreach has been successful, the demand has exceeded capacity (n=3). Currently, there is a waitlist of approximately 70 individuals, which could mean wait times of 4-8 weeks to enter the program and its services (Bayaud, 2017Quarter 3 Report). To help alleviate the waitlist challenge, the program aims to have a quick and simple entry process that involves attending an orientation offered weekly at Bayaud, usually with 8-12 people, that starts connecting them to other Bayaud resources (n=2). “[We] need to get that person [experiencing homelessness] going as quickly as possible, engaged [in services] quickly. A waiting list does the opposite…by the time someone comes up on the waitlist their phone might be out, they might be gone.” –Program Staff 25 Phase 2: DDW Program Implementation Strengths Providing Initial Work Opportunities & Job Readiness Support Providing Initial Work Opportunities. In its pilot year, Denver Day Works has a goal to connect 150 individuals experiencing homelessness to a work experience and retain at least 70 participants for more than one workday. In the first nine months, the program is set to well exceed these goals, as they have engaged 148 participants in paid work experiences, and retaining 128 for more than one workday (Bayaud, 2017 Quarter 3 Report). Almost two-thirds of interviewees (n=11) indicated a significant program strength is simply providing an initial work opportunity for individuals experiencing homelessness to help them get back on their feet. One participant articulated a sentiment shared by other interviewees, “The program is like a spark plug to get [participants] back into the rhythm of working and have new possibilities open up.” Three participants expressed DDW is preferable to other day labor programs, which were said to be unreliable, involve long wait times, and offer less pay. Job Readiness Support. The program’s work experiences provide an opportunity to assess the job readiness of participants and begin finding them long-term employment (Bayaud, 2017- Quarter 2 Report). In addition to gaining recent work experience, participants receive training and transferable skills (n=5). Over the course of the program, 79% of participants have completed some type of training (Bayaud Data). Multiple participants (n=4) specifically mentioned completing a useful customer service training, which is offered through another Bayaud program, Denver’s Road to Work. As a result of attending this training, participants felt more energized and motivated by their employment prospects (Bayaud, 2017 Quarter 2 Report). Participants also show readiness for long-term employment by demonstrating a strong work ethic, taking responsibility, following directions, and being on time (n=1). During the workday, Bayaud Employment Specialists help participants with job searching, submitting applications, and updating their resume (n=6). Participants learn how to highlight their skills through their resumes and practice interview workshops (n=2). The majority of interviewed participants (n=6) reported the program made them feel more prepared for future jobs. SUMMARY OF STRENGTHS AND OPPORTUNITIES FOR IMPROVEMENT IN PHASE 2 Strengths:  Providing initial work opportunities & job readiness support  Earning money & program convenience  Integrating wrap around support services  Increasing self sufficiency  Providing a support system & empowering participants  Commitment from the City; Reputation to the City and Homelessness Opportunities for Improvement:  Need for increased funding to scale  High resources needed to implement program  Challenges of providing certain services (e.g. mental health) 26 Earning Money & Program Convenience Factors Earning Money. Another unique aspect of Denver Day Works is distributing immediate cash or an equivalent gift card after completing a workday (n=6). Receiving payment at the end of each day is an incentive for participants to join and remain involved with DDW. Additionally, this form of payment can accommodate participants without a bank account or ID. Convenience. Convenience is another programmatic factor which makes participation low-barrier (n=5). For example, Civic Center Park, the central meeting location for participants, is easily accessible from nearby shelters and public transportation (n=3). Another example is the low-commitment required to participate – a mechanism that can be prohibitive to some participants dealing with behavioral health, trauma, or other limiting factors to engage in full-time employment. One participant pointed out, “People that are part of the program may have commitment issues so doing DDW once a week makes it easy [for them to participate].” Integrating Wraparound Support Services Integrated Case Management. Over 80% of interviewees (n=14) identified providing wraparound support services as an important strength of the program. In addition to employment support, DDW participants are assigned a case manager who assesses their needs and connects them to public benefits and support services. Thus far, 133 participants have received assistance from Bayaud with support services such as SNAP, Medicaid, documentation retrieval, housing, and referrals to mental health and substance abuse resources (see Appendix 1 for list of the top services) (Bayaud Data). Bayaud’s approach integrates case management and service navigation into the workday in order to efficiently maximize their time and impact with participants. This integration serves as a reliable and effective entry point for connecting participants to needed assistance (n=2). As an entry point to services, the program extends its reach by linking waitlisted individuals to other Bayaud services after attending the DDW orientation (n=1). Quick & Direct Services Navigation. In comparison to other assistance programs, what sets DDW apart is faster and more direct access to support services and public benefits (n=5). Participants are accustomed to making appointments with long wait times, tedious application procedures, and feeling as if they are just part of the masses. The program’s dedicated on-site case management and benefits navigation staff can streamline processes and leverage personal connections to help participants more effectively. For example, while finding housing is a major issue facing DDW participants, eleven participants have secured permanent housing through working with Bayaud and DDW (Bayaud, 2017 Quarter 3 Report). Over half of the participants interviewed (n=5) mentioned how Bayaud helped them find housing more quickly than other programs. “All participants have to do is show up and [the program] wraps services around them. They get to work, get paid, have a bus ticket, navigation services, and a professional employment specialist… you do everything you possibly can when they are in front of you. Programs that are successful cannot just provide one piece; you need to look at the whole system.” – Program Staff 27 Supporting Participants’ Basic Needs. Interviewees spoke of DDW as a way to support participants’ basic needs such as having a safe place to go, getting meals during the workday, and receiving bus tickets to help with transportation (n=7). Additionally, Bayaud offers access to their food bank and laundry truck. The food bank has provided participants with a week’s worth of food 132 times since the program started. Many participants have used Bayaud’s mobile laundry truck and it has even served as a DDW worksite. In the near future, Bayaud plans to hire a DDW participant to work on the laundry truck (Bayaud, 2017 Quarter 3 Report). Increasing Participants’ Self-Sufficiency Almost two-thirds of interviewees thought Denver Day Works is providing participants a path to selfsufficiency (n=10). Some interviewees emphasized participants being able to sustain themselves by earning their own money rather than being given money. For a few participants, finding housing provided the needed stability to work a full-time job. Two participants emphasized how the program is helping to change and rebuild their lives. One worksite supervisor/employer observed the DDW participants hired by his department are “planning short-term life goals rather than day-to-day survival.” Stories of Volunteer Impact: Providing Home Cooked Meals - A Cost-Effective Idea. The DDW team, coordinated through Bayaud, launched a concerted effort to provide home cooked meals to participants working on the daily work crews. Programming design has all of the workers receiving breakfast and lunch during their workdays. In one period, on 11 different workdays, the participants received catered lunches cooked and delivered directly to the job sites (Public Works and a Denver Park) by volunteers in the community seeking to contribute to the program. This has been very successful and the goal is to have each workday catered by volunteers. Every day that food does not come out of the DDW budget because of volunteer intervention means that much more cash becomes available to pay participants a wage subsidy. Each workday a volunteer feeds our workers equals one additional full day’s pay for a DDW participant (Bayaud 2017-Quarter 3 Report). Providing a Support System & Empowering Participants Support System. The Denver Day Works program provides a critical support system for participants as they try to get their lives back on track. Over 80% (n=14) of interviewees indicated that caring and trusting relationships with program staff are a significant driver for participants feeling supported whether it was for job placement, emotional support, or finding services. Both participants and worksite supervisors/employers referred to the program’s high level of involvement, counseling, individualized support, and encouragement as vital for the success of participants and the overall program. One employer commented, “Bayaud is more engaged with participants than other programs we’ve been involved with. They really dig deep and build trust, connections, and continued support.” One participant shared, “Bayaud helps participants out the best way they can…if I didn’t have their help I wouldn’t have anything right now.” Accommodating of Participant Circumstances. Essential to participants feeling supported are program and worksite staff that are accommodating, understanding, and flexible towards the circumstances and challenges that individuals experiencing homelessness face (n=10). Nearly one- 28 quarter (n=4) of interviewees described the program as particularly accommodating for older individuals and those with a criminal background or injuries that may have hindered their employability. Continual Communication. Program staff foster a supportive environment through continual communication, check-ins, and monthly advisory councils where participants share their feedback (n=7). These communication touch points deliver invaluable information for Bayaud to use to guide, adjust, and develop the program. Additionally, two participants appreciated program staff’s responsiveness and follow-through on program suggestions and resolving issues between program participants. Empowering Participants. Being part of Denver Day Works has led to a sense of empowerment among several participants (n=7). Some participants felt empowered from working again, feeling useful, and displaying their capabilities. For other participants, it was the care, support, and positive reinforcement from program staff that made them feel confident, motivated, and inspired again. Others voiced how the program shifted participants from feeling hopeless to hopeful, in which “the more people with hope, the better.” Bayaud’s “Web of Connections.” Throughout the interviews, over half of the interviewees (n=10), including 100% of DDW participants, brought up Bayaud’s network of connections as an instrumental part of the support system provided to participants, specifically with housing and employers. One Bayaud staff member suggested their existing network of employers jumpstarted the program’s employment placement. One participant emphasized, “[The Bayaud staff] have an incredible network and when I get a full-time job I know it’ll be because of their connections.” Commitment to Denver Day Works City Commitment. Denver Day Works is an example of a local city government committing its resources, on multiple levels, to address homelessness (n=4). Administratively, city staff have been proactive, involved, and communicative with DDW program staff (n=1). Both a program staff and participant commented on the cross-sector collaboration involved with executing the program (n=2). Two interviewees explicitly expressed how the program made participants feel like the City actually cared about those experiencing homelessness (n=2). All participants (n=7) thought Denver Day Works was a good program for the City to provide. Additionally, two interviewees suggested high-level champions of the program, such as the mayor and agency directors, sends a message that the City is supportive and committed to the success of the program. “My staff knows [Denver Day Works] is a priority for me and they need to be on board…it’s important for me to set the standard and be a champion of the program. It has become an integral part of what we are doing on a day-to-day basis…Being involved has to be personal and sincere. As a city, we have to make sure there are people like me and others that are truly supportive of this program and it’s not just lip service.” – City Staff 29 Program Staff & Worksite Supervisors’ Commitment. Other interviewees brought up the importance of the personal commitment of program staff and worksite supervisors (n=5). A program staff and participant described needing people to have their hearts in the program in order to help participants and ensure the program’s success. Another participant observed, “The people involved really love what they are doing and will do anything can do to keep it going…often going above and beyond.” Participant Commitment. Some interviewees also thought participants must also be committed to succeeding in the program (n=4) and “making their own ladder of success.” One participant articulated, “What makes the program work is…participants being dedicated to doing the work, bettering themselves, and achieving their goals.” A worksite supervisor/employer also felt the program’s success relies on the participants, but links back to the program’s integral support system. The Role of Bayaud As an external nonprofit contracting with the City, Bayaud has effectively implemented Denver Day Works by leveraging its specific capabilities. The majority of program partners interviewed indicated Bayaud was doing a good job of managing the program (n=6). One city staff and two city worksite supervisors/employers believed Bayaud is better equipped than the City to administer a program working with the homeless population. For example, Bayaud has experience providing workday screening (i.e. for inebriation and mental health), support services, and counseling, which City worksite employees are not accustomed to handling. Additionally, Bayaud can administer the program’s low-barrier features, such as working with people that have a criminal background or distributing workday compensation to individuals without documentation. One participant even recognized Bayaud’s administrative capacity by noting, “Bayaud is smaller and less bureaucratic, unlike a government-operated city program. Bayaud is able to get more stuff done with the resources they are given.” However, two interviewees felt that a different nonprofit partner might be better at administering the program for the City. 30 Positive Publicity about Denver Day Works Positive media attention about Denver Day Works has an impact on the public’s perception of Denver’s commitment and credibility for addressing homelessness with city resources (n=3). One employer said, “The program is doing a great job with press coverage that informs the public and develops public buy-in to support its funding.” For one city staff, the media attention has generated inquiries from other cities and organizations about how to replicate Denver Day Works. Beneficial Impacts for the City Work Production. While Denver Day Works is primarily a day labor opportunity benefiting homeless individuals, it is also mutually beneficial for work production at city worksites. Almost 60% of interviewed city worksite supervisors/employers (n=4) recognized DDW participants are completing projects quicker than their department can due to limited staff and resources. One city worksite supervisor/employer said, “The strength that makes the program unique is participants are helping address the challenge of not having enough staff and resources to accomplish the work that needs to be done.” Financially Beneficial. City staff and worksite supervisors/employers (n=5) also indicated DDW is financially beneficial for their departments such as not having to hire private contractors and participants using existing equipment. City worksites contributed $50,000 for the program year and staff to support the worksite. Each worksite usually has 3-5 program participants working three days per week. All worksite supervisors/employers interviewed (n=4) thought they were getting a positive return on investment through the program. Societal Impacts of the Program Changing Perceptions of Homelessness. Almost 60% of interviewees acknowledged some type of systemic and/or societal impacts of Denver Day Works (n=10). Over half of these interviewees discussed how the program humanizes homelessness by changing the perceptions of the public and specifically worksite staff (n=5). For worksite staff, the program was an opportunity to dispel preconceived notions and foster greater understanding about individuals experiencing homelessness and their challenges. For participants, it was a chance to show others what they are capable of doing. One city staff, whose department is a worksite and employer, explained how the program, “humanizes homelessness for my staff that [are used to] seeing the negative side [of homelessness] instead of the human side. When the staff work with [DDW participants] they see them getting their lives back together, which makes the staff more aware of the challenges homeless people face.” Societal Contributions. More broadly, the program can change the public’s perception towards homelessness (n=2). One participant liked being able to work out in public so “society can see that homeless people can make a difference too.” Three interviewees commented on the reciprocity of the societal contributions stemming from the program. One participant noted the program is not only aiding homeless individuals, but also allowing participants to give back to society through cleaning and beautifying public spaces. Another participant shared, “the community thrives the more participants feel they are giving back when receiving help.” From an economic standpoint, a program staff spoke of participants shifting from having zero spending capacity to earning money that will go back into the local economy and community. On a systemic level, three participants shared the program is decreasing homelessness by getting participants off the streets and providing employment and economic stability. “Denver is on the cutting edge of addressing a problem like this. I get calls all the time about how are we doing this program. This is an indicator that [the city] is doing something right.” – City Staff 31 Opportunities for Improvement A number of challenges with implementing the program at different worksites were highlighted throughout the seventeen interviews conducted. The main challenges within Denver Day Works’ program implementation are discussed below. Funding On the programmatic side of Denver Day Works, the most frequently reported challenge was funding (n=11), which was mentioned by nearly half (47%) of all interviewees. In general, interviewees talked about how funding was a major barrier for expanding the program or better meeting the needs of participants. When referencing funding, interviewees mentioned a variety of needs, including: increasing the number of work days (n=6), increasing the capacity to allow for more participants per day (n=3), adding new worksites (n=2), funding for more worksite supervisors (n=1), and hiring more Bayaud or city program staff (n=1). Generally, interviewees from worksites mentioned funding as a challenge more than the other interviewees. While the program administrators from the City of Denver and Bayaud desired more funding to increase the program’s capacity (increasing the number of participants able to work each day and reducing the waitlist), interviewees from worksites were usually more focused on increasing the capacity of the program and their specific organization. For example, interviewees from worksites mentioned wanting increased staff, both at their own organization and at Denver Day Works, as well as more work days at their sites. Challenges Partnering with Diverse Agencies The second most frequently cited challenge for administering the program, mentioned by 29% of all interviewees, was the challenges that come with partnering with diverse agencies (n=8) (for example, aligning schedules, coordination of services, etc.) When asked about partnering with the Denver Day Works program specifically, interviewees cited logistics (such as coordinating schedules or transportation) as the most challenging aspect of working with the program (n=3). This being said, several interviewees mentioned that these logistical challenges were more prevalent towards the beginning of their partnership with Denver Day Works and have now been largely resolved over time. Further challenges regarding partnering with different agencies included overall schedule differences between Denver Day Works and the individual worksites (n=2), organizational differences and tension (n=2), and communication (n=1). Resource Intensive Nearly a third (29%) of interviewees discussed how Denver Day Works is a resource intensive program (n=7) in terms of time (n=4) or money (n=3) costs for participating worksites or the City. The most resource-intensive aspect of the program is staff time. From the perspective of Bayaud, staff time primarily revolves around providing wraparound case management, whereas at the worksites, they have to build staff capacity to provide appropriate supervision for the program. For example, as the program has grown, developing plans for worksite supervision have continued to evolve. Additional challenges mentioned by interviewees included having a long waitlist (n=3), developing the skills of participants (n=2), and including diverse populations (n=1). Interviewees representing the City and Program Staff discussed how having a long waitlist was a significant challenge because they are unable to get people into the program and engaged in services in a timely matter. Interviewees from worksites, however, were generally more concerned with having more time with participants in order to train them more effectively and allow them to develop skills. Worksite interviewees discussed how starting over from scratch with new participants every day or week limited their capacity to develop 32 participant skills and expand the variety of tasks that the participants could complete at the worksites. Lastly, one program administrator discussed how it has been a challenge finding appropriate work for a diverse population, particularly individuals with physical or mental disabilities. Separate from managing or administering the program, interviewees expressed several challenges that face organizational partners or individuals who are participating in the program. The most common challenges related to program participants are detailed below. Mental Health Interviewees, particularly those from Denver Day Works worksites, discussed how participants' mental health was a challenge for their participation or success in the program (n=4). The worksites talked about the need to have supervisors who are sensitive to the mental health needs of the population they are working with. One site supervisor stated, "Some have mental [health] issues and need more constant supervision. Therefore, when participants come they have to stay together as a group, [we cannot] separate them to be able to do more/different work." Personality Conflict Personality conflicts, or disputes between participants either with each other or with worksite staff, was the second most frequently mentioned challenge for participants at the worksites (n=4). These personality conflicts were mostly mentioned as a challenge by the participants themselves rather than program administrators or worksite supervisors. Typically, these disputes are interpersonal and quickly resolved with staff intervention. Accessing Support Services Three participants discussed how accessing support services (outside of Bayaud) during the program was challenging for them (n=3). Participants mentioned difficulties around finding affordable housing, particularly issues such as long waitlists, and finding and receiving accurate housing information. Many participants continue to struggle to find housing due to criminal backgrounds or blemishes on their rental history (Bayaud, 2017 Quarter 3 Report). Finally, issues connected to housing situations (n=2), substance abuse (n=2), and criminal background (n=1) were mentioned as challenges facing participants who are active in the Denver Day Works program. Interviewees mentioned specific issues connected to participants' housing situation were impacting their ability to participate effectively in the program, such as the inability for program staff to contact participants or not having a secure place for participants to store their belongings. Substance abuse was mentioned by two worksites as a challenge facing some of their participants, which impacts their ability to successfully complete the tasks at hand. Lastly, criminal background was mentioned by one worksite interviewee as a challenge for some participants who have "limited social skills" and a "jail mentality," which makes it difficult for them to interact with others during the program. A Need for Diversified Worksite Opportunities In the network analysis survey, partners were asked: Given the demand for this program, there is a need for more and diversified types of worksites. From your perspective, what are the factors/ mechanisms that need to be in place to make it feasible for employers to host worksites within the DDW program? 35% of respondents articulated that flexible and supportive worksites are needed to make it feasible for employers to host worksites within the DDW Program. 30% expressed that employer training and support is necessary, while 20% expressed participant employment services, and 15% expressed participant support services as necessary. Only 5% indicated that funding, open communication, and trial periods for participants is needed for employers to host worksites within the program. (DDW PARTNER SURVEY) 33 Phase 3: DDW Long-Term Employment Placement Strengths Successful Employment Placement Support Reaching Participant Placement Goals. The most cited strength of Denver Day Works is finding full-time employment for participants, which was mentioned by all interviewees (n=17). The program has recruited 79 employers to participate in DDW and 50 employers have actually hired participants. As of September 2017, the program has 62 participants working at 95 positions, with some participants having multiple jobs (Bayaud Data). Overall, 111 positions have been held over time by participants. When categorized, these positions are from 15 public agencies, 61 private agencies, and 28 nonprofits (some employment positions were listed as "unknown employer" and could not be categorized). The City of Denver aims to hire 20 DDW participants. At this time, Parks and Recreation (7), Denver Central Library (4), and Public Works (1) have hired 12 participants. Employment Hiring Events. Surpassing the overall job placement goal is due to the program’s integral employment placement assistance and support system. Interviewees mentioned one program strategy for finding jobs is employment hiring events during workdays, in which participants can interview and potentially be hired the same day by employers (n=8). Bayaud has hosted 11 hiring events featuring city agencies, nonprofits, and private companies (Bayaud, 2017 Quarter 3 Report). Two interviewees, both an employer and two participants, mentioned successful hiring from these events. Individualized Employment Support. The majority of participants (n=6) emphasized individualized employment support as an important aspect of the program. One participant shared, “The job placement services have helped a lot… [especially] to know I’m not alone and have a team supporting me and actively trying to help.” Another participant said, “[Program staff] have the knowledge and connections to work opportunities… [and] having someone there to support you is something you don’t really find in the real world.” At the city level, Denver Central Library has developed a more accessible application process specifically for Denver Day Works participants (n=1). Hiring Advantages for Employers. Denver Day Works provides employers with the unique advantages of pre-screening and observing potential hires. Over the course of the program, Bayaud pre-screens and recommends participants to employers that have the best chance to hold down a job (n=3). Two interviewees mentioned receiving good employee referrals through the program. Both city staff and worksite supervisors/employers identified a program benefit is observing how participants work, like an on the job interview, to find potential hires (n=5). This strategy assists the program’s goal of placing participants in city jobs. SUMMARY OF STRENGTHS AND OPPORTUNITIES FOR IMPROVEMENT IN PHASE 3 Strengths:  Reaching participant placement goals  Employment hiring events  Individualized employment support  Advantages to employers  Retention rates are meeting goals “The job placement services have helped a lot… [especially] to know I’m not alone and have a team supporting me and actively trying to help.” -Participant Opportunities for Improvement:  Few placements in city agencies  Transportation/ housing limit success  Retention is challenging 34 Employment Transition & Retention Support Strategies Retention in Full-Time Jobs. Once placed in employment positions, the program aims for 30 participants to retain their job for at least 90 consecutive days. As of August 2017, 27 participants have achieved this retention milestone and it is anticipated the program will reach 30 soon (Bayaud, 2017 Quarter 3 Report). Six interviewees felt the DDW participants were transitioning well to full-time employment. Two interviewees thought job retention for the program was above average in comparison to their experience with other similar programs. Resources Required for Sustainability For individuals experiencing homelessness, the job transition and retention phase is perhaps the most challenging. The following factors contributed to reaching DDW’s retention benchmark. In the network analysis survey, partners were asked: From your perspective, what resources are critical to achieve long-term sustainability of Denver Day Works for homeless individuals in Denver? Who or what is ultimately needed to sustain these resources? Understanding Employers. Employers that understand the  background and challenges of homeless individuals are a significant part of participants’ transition and retention to full-time employment (n=4). For example, one nonprofit employer’s liberal hiring policy has allowed them to hire ten participants, which is more than any other involved non-city employer. One program staff noted, “[This employer] has a clear understanding of the issues that may play out in the workplace and how to give people more chances to retain their employees. For example, they are concerned about their employees if they don’t show up instead of just firing them.” The interviewed employer confirmed, “We are willing to take a chance on people who might not look so good on paper…homelessness doesn’t scare us.” Prior Worksite Experience Resulting in Smoother Full-Time Employment Transition & Retention. Two worksite supervisors/employers who hired participants indicated prior work experience and familiarity of the job site made for an easier transition. One hired DDW participant pointed out the benefit of advance knowledge of what to expect and how DDW was like a practice run for his new job. Post-Program Communication & Support. The case management and support system participants experience during the program extends to the employment phase. A key aspect of the program’s retention strategy is maintaining relationships and communication with former participants by checking on their transition experience and support needs (n=5). A participant noted DDW’s distinction from temp agencies that “just send you off to the job and leave you alone. DDW is different because Bayaud consistently find out what they can         40% indicated that increased staff/capacity for case management and participant support services (education/skills, food, clothing, transportation assistance, health and mental health assistance) are required 35% indicated access to jobs (high paying, low skill) 30% indicated access to affordable housing 30% said increased general funding for program 20% indicated more community engagement 10% indicated funding from multiple sources and relationships/collaboration between partners 5% indicated flexibility for participants 5% said funding for business incentives 5% said continued media coverage is important for sustainability (DDW PARTNER SURVEY) 35 help you with, even if it is just talking to someone. This is a very supportive program and family oriented environment.” DDW program staff also make a point to have open lines of communication with employers to check on participants’ progress and address any issues that come up (n=4). One employer suggested the continued support of participants contributes to lower turnover. The majority of interviewed employers specifically spoke of the great relationship they had with Bayaud (n=5). Two city worksite supervisors/employers specifically noted Bayaud staff as the foundation of the program’s success. Work Standby to Earn Extra Money. Another transition support strategy mentioned by interviewees is former participants working with DDW on standby status (n=6). Each DDW work crew can hold 12 participants. If a current participant does not show up then a former participant can join for the day. This standby option provides former participants quick monetary support as they initially transition to bi-weekly paychecks in their new jobs. DDW Positive Employment Outcomes & Permanent Hiring Pipeline. One-hundred percent of the employers interviewed (n=6) indicated satisfaction with the DDW participants they hired. A few of the city worksite supervisors/employers described those hired as outstanding, remarkable, and model employees (n=3). However, these city hires are considered seasonal or on-call employees meaning that although they are working full-time, they are not permanent employees. Nevertheless, multiple city worksite supervisors/employers said if given the opportunity they would definitely hire these participants and others from DDW for permanent positions. Opportunities for Improvement From a programmatic standpoint, there were two main challenges regarding the employment placement phase of the program. The two programmatic challenges that were highlighted throughout the interviews included job placement (n=9) and employee attrition (n=5). Job Placement Almost 2/3 of all interviewees discussed at least one challenge related to job placement after participating in Denver Day Works. The most frequently cited challenge with job placement was the complexity of One interviewee stated, "The City needs to navigating the human resources and hiring look at why it’s taking so long to onboard systems, which was mentioned by 35% of people. Why has the city not hired more interviewees. This was the only challenge people? It’s tough to get people on in the mentioned by all of the groups City, so they place people in the private interviewed (city administration, Bayaud, sector." – Program Staff worksites, and participants). The complexity of the hiring processes is further exacerbated by several other challenges mentioned, such as background checks (n=4), lack of ID/documentation (n=3), physicals (n=2), lack of permanent address (n=2), and participants not having a bank account (n=1). Program administrators and staff alike reported difficulties specific to navigating the City's hiring system and that program participants may be excluded from even 36 applying for city jobs if they cannot meet these requirements. A city worksite supervisor/employer mentioned how they wanted to hire two individuals from Denver Day Works, but was unable to because of complex city hiring processes including background checks and physicals (a challenge due to less flexibility than non-governmental employers). Other significant challenges impacting the placement of participants in long-term employment with the City included lack of funding for new positions (n=4), lack of employer flexibility (n=2), and employer bias (n=3). Regarding perceptions of employer bias, one interviewee stated, “[Employers] would need to take a risk here or there and maybe say yes to someone on the borderline when previously they would not have. It’s a challenge in the private sector and with the City." Both city administrators and Bayaud staff expressed issues around whether Denver Day Works participants could have preferential hiring status (n=2). Bayaud reported receiving mixed signals about who to contact within the City's Human Resources department, what their criteria is, and whether these criteria can be flexible or not for Denver Day Works participants. One interviewee said, "Initially it felt that DDW participants would get more of a focus within city hiring processes, but it doesn’t seem that way anymore." Whereas, from City’s perspective, the City "can’t prioritize someone just from being in the program." Attrition Retaining past Denver Day Works participants as long-term employees was mentioned as a challenge by all five of the worksite supervisors/employers interviewed (n=5). Although all of the worksites/employers involved have successfully hired Denver Day Works participants as long-term employees, there has been attrition due to individual circumstances such as their lack of stable housing and interpersonal conflicts. Despite each worksite mentioning some level of attrition occurring, the worksite supervisors generally reported positive experiences with the program and that they would like to hire more Denver Day Works participants. One employer said, "If I had the opportunity to hire one of the former DDW participants, [I] would do it for sure. [He was] prompt, on time, laid back. If I had authority I would hire him." In addition to these programmatic or systemic challenges, which may inhibit participants from obtaining full-time employment, there were challenges related to the individual participants as well. The main challenges at the individual level included the ability to stay employed (n=11), housing (n=7), job placement (n=6) and transportation (n=3). Job Placement and Retention In regards to long-term job placement (n=6), challenges mentioned were having a criminal background (n=2), a gap in employment (n=2), mental health (n=1), and substance abuse (n=1). These factors are generally different (with the exception of one—mental health) than factors that interviewees reported as impacting participants' ability to retain a job. In terms of job retention (n=11), interviewees reported difficulty adjusting to full-time employment as the most significant barrier. This included adapting to working or being housed when the individual may have lived on the streets or been without work for a long period of time, as well as adjusting to being paid on a bi-weekly basis. Other factors reported as challenging for participants to maintain full-time employment consisted of mental health (n=2), personality conflicts (n=2), and the learning curve of starting a new job (n=1). 37 Housing Interviewees from Bayaud, the worksites, and the participants themselves each reported challenges associated with a participants' housing situation as a barrier to finding and keeping long-term employment. These issues revolved around things like losing IDs or other documents, not having a safe place to sleep or store their belongings, and difficulty with reaching the individual. When asked about the most challenging aspect for participants to retain a job, one interviewee stated, "This population will get jobs and will still sleep outside." In fact, one of the worksites hired a Denver Day Works participant full-time, but this individual was unable to keep the job because of his unstable housing situation. Transportation Transportation was brought up as a challenge for job placement by two participants and one worksite supervisor/employer. Clearly, being able to travel to and from a job would have a significant impact on whether participants can maintain employment. One participant summed it up as, "[I] couldn’t interview at places that were outside of the Denver metro area, and normally couldn’t commute to a job because I didn’t have a car." Recommendations for Denver Day Works Program Throughout the interview process of Denver Day Works (DDW), program and city staff, worksite supervisors/employers, and program participants each had clear recommendations relating to ongoing sustainability and program improvement. While some interviewees had specific suggestions for ideal changes respective to their own experiences or organizations, a few recommendations had directives for enhancing collaboration between agencies. Other comments concerned quality of expected participant outcomes and program modifications to maximize efficiency. Recommendations were collected and categorized by system phase (Entry into DDW, DDW Program Implementation, and Full-time Employment Placement) and compared for frequency and source of mention. Long Term Employment: In the network analysis survey, partners were asked the following open-ended question: Do you think Denver Day Works can result in greater access to employment services among homeless individuals? (e.g. Workforce Centers, community-based services, and direct employment). Please explain your answer. 85% of survey respondents answered yes, Denver Day Works can result in greater access to employment services among homeless individuals, while 5% answered no (participants are not job ready) and 10% answered N/A. Of those 85% that said yes, 39% further clarified that participant employment services is important, 17% said participants are better connected to the community, support services and having access to jobs/benefits, 11% indicated that there needs to be increased capacity to serve more homeless individuals and 6% indicated that increased collaboration and access to housing is important to greater access to employment services for homeless individuals. 28% did not clarify their answer further. (DDW PARTNER Survey) 38 High Level Program Recommendations The following section offers an overview of three high level areas of program recommendations. A. Increase the program budgets to support expanding the program’s number of work days and diversity of worksites. The DDW program is viewed as successful on many fronts. All stakeholders agreed the program was meeting most of its goals and providing a new and innovative way to offer day work to people experiencing homelessness in Denver. The success of this program has led to an increased demand for services, both in number of slots, and in the types and number of worksites available. Nearly all participants in this evaluation agreed that the City should consider scaling the program to all five workdays. Participants and program staff emphasized specifically the need to grow the diversity of work options (for example to include clerical options) and the types of worksite available for DDW participants. This recommendation would require a scaling of the budget for the program. Program staff indicated that increasing the size and number of worksites would scale the budget proportionately, with actual costs increasing at the same rate. B. Develop an employee-supported transition component between day labor and fulltime employment. Many respondents discussed the challenges participants faced in transitioning from a once a week job to a full time commitment. Several of the participants do not have transportation, a way to clean themselves or their clothing, or a place to store their stuff day-to-day. Others are suffering from mental, behavioral, and other health issues. There was support from participants in this evaluation to develop a more intentional transitional process. Ideally, a work crew cohort accompanied by a dedicated staff person to provide direct and individualized support for participants as they transition into permanent employment. Experience has illustrated that this approach can result in great long-term employment placement outcomes. Costs for this type of program, serving 5-12 participants, have been estimated by Bayaud as $261,698. While the City is indeed the largest single employer to hire Denver Day Works participants, it falls short of its self-imposed program goal of hiring 20 participants. While it is unclear how the goal of 20 participants was selected, this is the baseline agreed upon for the project. To reach this goal, the City can focus on alternative and additional strategies for addressing this gap, such as identifying additional types of employment opportunities that exist with the City. Or the City may want to reexamine the 20 participant goal and consider how to align the expectation for placements with available positions that participants are eligible for and in comparison to the other potential employers (who hired less overall permanent placements than the City during this pilot). We do recommend the City consider what kinds of protocols and considerations are preferred to have participants successfully transition into permanent employment within the City. C. Engage in a participatory strategy session to determine the system of governance for the DDW program. An area that remains unresolved for stakeholders is where the DDW program should reside at the City. There was no clear consensus on this issue, largely due to limited discussion about the future direction of the program, as its success becomes broader beyond the original goals. While the DDW program has demonstrated success in Denver Human Services, some interviewees suggested that the City have an indepth facilitated discussion to define the future goals of the program before considering any change to 39 its current home. These include options to extend the day work opportunities and leverage the lowbarrier entry and immediate payments of the program as a way to address the needs of people experiencing homelessness; or to transition to a more systemic workforce development program that is focused on integrating a long-term system of wraparound services and supports, or both. The success of DDW within DHS suggests that placing it in another agency risks disrupting what is currently an effective, popular, integrated city program. We recommend that the City devote some time and resources to engage in a participatory strategy session to flesh out where the program should reside. If the program continues to expand its wraparound services for participants and engage in a more focused intention on permanent employment, it should consider which City agency is the appropriate home for oversight of the program. Specific Program Recommendations Phase 1 Recommendations: Entry into Denver Day Works Program Some suggestions were given for program improvement prior to candidate entrance. In these formative stages, the most frequently mentioned recommendations related to: 1) Administrative adjustments, including enhancements to the entry process through additional staffing. 2) Pre-screening candidates to match labor to participant compatibility. 3) Additional outreach and staff training that incorporates Trauma-Informed Care prior to engaging with DDW program participants. These comments point to the situational necessity to address time-sensitive participant needs and improve the abilities to pair candidate readiness with variety and capacity of worksites. Phase 2 Recommendations: Denver Day Works Program Implementation Interview respondents had significantly more suggestions for the program implementation phase of DDW. Suggestions included: 4) Increased budget and program funding to support additional program staff and worksite supervisors. 5) Increasing the number of workdays from three to five days during the week. 6) Increasing the number and types of worksites. Adding a variety of worksite job tasks will align participants with job market demand by improving their qualifications for a wider range of positions (other than only landscaping or janitorial/maintenance). 7) An increase in program staff and worksite supervisors. Additionally, two interviewees mentioned hiring a formal coordinator to communicate between worksites, program staff, and participants. From these suggestions, the evaluation team suggests hiring or designating an existing program staff to coordinate worksite logistics and provide direct supervision of worksite tasks. This will help alleviate the burden on city employees currently supervising worksites and allow Bayaud program staff to focus on providing case management to participants. 8) In addition to ongoing partnerships, continue to add additional support services including collaborating with food-assistance, housing partners, and other services to support participants outside of formal programming. 40 Phase 3 Recommendations: Denver Day Works Long-Term Employment Placement Many of the recommendations made for improving the success of long-term employment placement emphasized the challenging transition between homelessness and full-time employment. Specific suggestions included: 9) The provision of supportive employment structures to support participant workplace expectations and impacts. Suggestions included: c. Establishing a crew of participants which work together over a period of time (rather than new groups each day), to both establish a stronger support system between participants, as well as providing a more consistent job training experience. d. Providing continued case management as participants transition into full-time work environments. 10) Engage the City and other potential employers in conversations to address the barriers to hiring both inside and outside the City (for example: criminal backgrounds). IN SUMMARY… The multiple perspectives of the Denver Day Works system within this evaluation confirms widespread support for the Denver Day Works program. A multitude of program strengths were expressed across all three phases of the program. Opportunities for improvements were presented as suggestions to expand the types of program opportunities and offer a more direct whole person approach to the work, shifting somewhat from a defined day work program to a more comprehensive workforce development program. Increased funding and diversity in worksites were emphasized by all groups. Additional support for transitioning participants from day work to full-time employment appears to be a pain point that needs to be addressed to adequately meet the long-term goals for job retention. The only goal that was not met, and does not seem on course to be met by the end of the pilot, is permanent placement in city agencies. It was unclear to many stakeholders why the City is not more proactive in placing DDW participants in permanent city positions. This is an area that should be addressed in future planning and goal setting for the program. 41 Appendices Appendix 1. Bayaud Data Summary Report Demographics As of August 2017, Bayaud Enterprises had information about 248 participants of the Denver Day Works Program. Below is the demographic information about the group. Race. The group is comprised of over half white participants (58.1%), followed by Black or African American participants (18.1%). All other groups make up less than 10% of the larger group. Race Category No Info White Black or African American Asian American Indian or Alaska Native Native Hawaiian or Other Pacific Islander Multiracial Other Total Count 6 144 45 3 19 5 20 6 248 Percent 2.4% 58.1% 18.1% 1.2% 7.7% 2.0% 8.1% 2.4% Gender. The majority of participants are male (76.2%), followed by females (19.8%), and other (0.4%). There was no information provided about the gender of 3.5% of participants. Education Level. There is a variety of education levels present within the group, with the majority of participants having a high school diploma or equivalency (35.1%), followed by those with some college (22.6%) and those who completed between 9th-12th grades but did not Education Level graduate (19.0%). Category Count Percent No Info 12 4.8% Disabilities. Almost half of the group th Less than 9 Grade 3 1.2% (44.4%) had disabilities recorded. 9th-12th Grade (nongrad) 47 19.0% High School Diploma or Equivalency 87 35.1% Primary Language. The overwhelming Some College 56 22.6% majority of the group’s participants speak Associate Degree 16 6.5% English as their primary language (94.8%). Bachelor’s Degree 15 6.0% Other languages provided were only Graduate Degree or Higher 4 1.6% represented by one or two participants Certification 7 2.8% for each, totaling 2% of participants who License 1 0.4% did not speak English as a primary Total 248 language. Information was not provided on primary language for 2.8% of participants. 42 Citizenship. Most of the participants are US Citizens (80.2%). Participants considered ‘lawfully in the US’ were the next largest group at only 1.2%, followed by refugees (0.8%) and resident aliens (0.8%). There was no information on citizenship for 16.9% of participants. Housing. The majority of participants are homeless (34.7%), followed by those living in shelters (13.7%), and those living with others (12.9%). Outcomes DDW Work Opportunities. Denver Day Works has provided 148 individuals experiencing homelessness with day work opportunities. 128 of those individuals (86.5%) have completed two or more day work opportunities. Housing Category Count No Info 56 Homeless 86 Shelter 34 Living with Others 32 Own 4 Rent 28 Other 8 Total 248 Percent 22.6% 34.7% 13.7% 12.9% 1.6% 11.3% 3.2% Employment Placements. Currently, one fourth of participants that go through the Denver Day Works Program are placed with employment (25.0%). Participants have 1-5 jobs over the course of their participation with DDW, with a mix between full-time and part-time employment. The majority of those with employment have one job (15.7%). Overall, 62 participants that are currently working have a total of 95 jobs. The employment placements vary in length from shorter, temporary employment to ongoing employment opportunities. Participants are mainly employed at nonprofit organizations (56.0%), followed by private agencies (25.7%) and public agencies (13.8%). Information on the employer was not provided for 4.6% of participants’ jobs. Leaving Employment. Of the total number of participants who have left employment positions (N=22), 82% quit voluntarily and 18% were terminated. Participants have primarily left voluntarily for personal reasons (45.5%), followed by those who left voluntarily to accept another job (27.3%). [Note: This does not include those who gained temporary employment and had that position end.] Reasons for Leaving Employment Category Terminated – Attendance Issues Terminated – Disciplinary Action Terminated – Reason Unknown Voluntary Quit – Accepted Another Job Voluntary Quit – Personal Reasons Voluntary Quit – Reasons Unknown Total Count 1 2 1 6 10 2 22 Percent 4.5% 9.1% 4.5% 27.3% 45.5% 9.1% Trainings Utilized. Bayaud Enterprises offers a variety of trainings, including Computer Skills Training, and Customer Service Training. Over three fourths of participants have received some type of training through Bayaud (79.0%). Services Utilized. Bayaud Enterprises offers a wide range of services to help Denver Day Works participants succeed in various aspects of their lives. About half of participants (53.6%) have utilized one or more services provided. 3,982 services have been received by participants of the Denver Day Works Program, with the majority of services being bus tickets (25.6%), food (not food bank or gift card) (25.3%), and using the computer lab (20.6%). 43 Outcomes by Attributes When looking at the outcomes by different demographic attributes, there were some interesting findings. The percentages given below are based on the number of participants from a demographic group that had that outcome, out of the total number of participants from that demographic group. The outcomes focused on below were accessing services, accessing trainings, and currently having employment. Gender and Outcomes. Female participants are more likely than male participants to access services (67% vs. 52%), access trainings (24% vs. 20%), and have a current job (29% vs. 25%). Criminal Backgrounds and Outcomes. Participants with a criminal background are more likely than those without a criminal background to access services (93% vs 51%). Participants without a criminal background are more likely than those with a criminal background to access trainings (22% vs. 7%). Disabilities and Outcomes. Participants with disabilities are more likely than those without disabilities to access services (65% vs. 45%), access trainings (25% vs. 17%), and have current employment (28% vs. 22%). 44 Appendix 2: PARTNER Denver Day Works Survey PARTNER Denver Day Works Survey Questions Introduction Thank you very much for taking this survey. The survey is part of an evaluation about the network of relationships that has been created because of the Denver Day Works pilot program. This includes understanding how information flows through the network and where we can identify gaps and overlaps in linkages. The participation of as many members of the network as possible is especially important in this type of research, and will help us both demonstrate the power of the network we have created and understand where to focus future strategies. Survey Structure In the first set of questions, you will provide information about your organization and your organization’s perspective on the Denver Day Works pilot program. The next set of questions will ask you to choose partners that you interact with on the program, and you will answer questions about each of those partners. Q# 1 2 3 4 Question Text Question Response Options Your organization should be listed below. If it is not, please return to the original email and click on that link. In this survey, we are hoping to learn more about Denver Day Works – progress and opportunities for growth. To help us, in a few sentences, please describe one strength and one area for improvement of Denver Day Works. How long has your organization been involved with Denver Day Works program (in months)? We are interested in resources that can be leveraged with partners in the community. Please indicate what your organization/ contributes, or can potentially contribute, to Denver Day Works (select all that apply). 1. Advocacy for the homeless population 2. Basic needs support (e.g. clothing, storage, facilities for showers and laundry) 3. Connections to community-based organizations 4. Connections to employers 5. Connections to high-level decision/policy makers 6. Connections to workforce development agencies 7. Connections to homeless programs, services, and shelters 8. Data Resources including data sets, collection and analysis 9. Day labor resources 45 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 5 What is the one resources that your organization is best positioned to contribute towards the mission of Denver Day Works? Employment assistance and job opportunities Expertise in homeless support services Expertise in workforce development Facilitation/Leadership Financial resources for homeless population Fiscal management (e.g. acting as fiscal agent) Food assistance Funding Housing assistance General information/ Feedback on the topic of homelessness In-Kind resources (e.g., meeting space) IT/web resources (e.g. server space, web site development, social media) Legal support and resources Media/public relations connections and support Mental/physical health and substance abuse services Paid staff to support the program Participant outreach and recruitment Strategic planning skills/expertise Support & commitment to engage in Denver Days Works Training and professional development opportunities Transportation Volunteers and volunteer staff Same as #4 46 6 To date, which outcomes would you attribute to the work of Denver Day Works? (select all that apply). 1. Access to long-term employment opportunities for homeless individuals 2. Developing job readiness among homeless individuals 3. Effective strategies to support availability and access to employment opportunities, education, and services for homeless population 4. Knowledge sharing about homeless support services among organizational stakeholders 5. Identification and recruitment of potential candidates for the program 6. Improved/increased long-term employment programs for the homeless population in Denver 7. Improved/increased knowledge and awareness of homelessness issues 8. Improved/increased self-sufficiency of homeless individuals 9. Improved/increased sharing of community resources and programs 10. Increased utilization of support services among homeless individuals 11. More coordinated approach to address the needs of local homeless community 12. Policy, law and/or regulation changes to support long-term employment opportunities for homeless population 13. Program development around hiring, employment services, and education for homeless individuals 14. Providing informational/educational materials to homelessness 15. Providing low-barrier work opportunities for homeless individuals 16. Successful transition and retention of homeless individuals within long-term employment opportunities 17. Retention of individuals participating in the program 18. Don't know 7 Which outcome has been the most important for Denver Day Works to date? Same as #6 8 How successful has Denver Day Works pilot been at reaching its goals of providing the homeless population in Denver with work experiences, connections to services, and Not Successful Somewhat Successful Successful Very Successful Completely Successful 47 preparing participants for long-term employment?” 9 10 11 What aspects of the Denver Day Works program have contributed to success? (Select all that apply) 1. Communication between sectors and participating organizations 2. Engagement of high-level leadership within participating organizations in promoting day labor programs 3. Improved/increased collaboration (e.g. through committees, boards, work groups, etc.) 4. Improved/increased relationships between partners 5. Improved/increased data sharing 6. Sharing knowledge and resources among partners 7. Shared advocacy or policy work among partners 8. Shared vision and commitment for strengthening the “system” for day labor in Denver 9. Commitment from the City of Denver 10. Commitment from the public and nonprofit sectors 11. Commitment from the private sector 12. None From the list, select organizations/programs/departments with which you have a formal or informal relationship In subsequent questions you will be asked about your relationships with these organizations/programs/departments in the context of the Denver Day Works program. What kinds of activities does your relationship with this organization/program/department entail [note: the responses increase in level of collaboration]? None Cooperative Activities: involves exchanging information, attending meetings together, and offering resources to partners (Example: Informs others of useful information) Coordinated Activities: Include cooperative activities in addition to intentional efforts to enhance each other's capacity for the mutual benefit of programs. (Example: Utilizing shared administrative processes for program implementation.) 48 Integrated Activities: In addition to cooperative and coordinated activities, this is the act of using commonalities to create a unified center of knowledge and programming that supports work in related content areas. (Example: Developing and utilizing shared priorities for funding effective prevention strategies. Funding pools may be combined.) 1. Through the Denver Day Works’ pilot program. 2. Our relationship was not developed through the Denver Day Works program, but our participation in DDW has strengthened our relationship. 3. We work in the same organization. 4. Through other community venues/work not Please describe how your relationship related to the DDW program. 12. with each of these partners was 5. This a mandated partnership for the program. developed [select all that apply] 6. Personal relationship outside of a work environment. We share resources such as shared space, funding, or other items. 7. We refer clients to one another. 8. Other 9. Don’t Know To what extent does this organization/program/department have power and influence to impact the overall mission of Denver Day Not at all Works? *Power/Influence: The A small amount 13 organization/program/department A fair amount holds a prominent position in the A great deal community by being powerful, having influence, success as a change agent, and showing leadership. What is this organization/program/department's level of involvement in the Denver Not at all Day Works program? *Level of A small amount 14 Involvement: The A fair amount organization/program/department is A great deal strongly committed and active in the partnership and gets things done. To what extent does this organization/program/department/s Not at all contribute resources to the Denver A small amount Day Works program? *Contributing A fair amount 15 Resources: The A great deal organization/program/department brings resources to the partnership like funding, information, or other resources. 49 16 17 18 19 How reliable is the organization/program/department? *Reliable: this organization/program/department is reliable in terms of following through on commitments. To what extent does the organization/program/department share a similar mission with the Denver Day Works’ mission and goals? *Mission Congruence: this organization/program/ department shares a common vision of the end goal of what working together should accomplish. How open to discussion is the organization/program/department? *Open to Discussion: this organization/program/department is willing to engage in frank, open and civil discussion (especially when disagreement exists). The organization/program/department is willing to consider a variety of viewpoints and talk together (rather than at each other). You are able to communicate with this organization/program/department in an open, trusting manner. This partnership has [select all that apply]: Not at all A small amount A fair amount A great deal Not at all A small amount A fair amount A great deal Not at all A small amount A fair amount A great deal 1. Been informative only (we only exchange information, knowledge about resources, etc.) 2. Improved my organization’s capacity 3. Led to advocacy work in Denver 4. Led to an exchange of resources/services 5. Led to collecting shared data 6. Led to improved services or supports for the homeless population in Denver 7. Led to increased awareness of services/resources 8. Led to hiring a Denver Day Works participant for a job. 9. Led to new funding relationships 10. Led to new client referrals 11. Led to new hiring/employment practices 12. Led to new program development 13. Led to new relationships with area employers 14. Led to new relationships with area shelters 15. Led to new relationships with homeless service providers 50 16. 17. 18. 19. 20 21. 22. 23. 24. Who are your other partners, not listed in this survey that you work most closely with in the Denver community around homeless and day labor? What resources are necessary (or critical) to achieve long-term sustainability of day labor programs for homeless individuals in Denver? Who or what is ultimately needed to sustain these resources? Do you think day labor programs can result in greater access to employment services among homeless individuals? (E.g. Workforce Centers, community-based services, and direct employment)? Please explain your answer. What aspects of the Denver Day Works program could use the most improvement? Please explain your answer. Do you have any questions or comments? Led to policy change Led to sharing of best practices Has not resulted in any systems change Has not resulted in any systems change, but we anticipate that it will Open-ended Open-ended Open-ended Open-ended Open-ended 51 Appendix 3. Key Informant Interview Guide Protocol for Key Informants Involved in Denver Day Works Thank you for talking with us today. To reiterate what we are doing, our project is taking a closer look at the Denver Day Works pilot program to see what has been working well and what can be improved. As the city thinks about the future of the program, we want to highlight successes, but also identify the pain points, factors that will need to be considered, and the ideas/feedback from people already engaged in the program. That is why we are talking to people like you, who are actively engaged in Denver Day Works. This information will help guide the future development of the program. (This can be conversational – no need to read this verbatim) Required (Consent Procedures): Before we get started, I want to assure you that your responses to these questions will be held in strict confidence, except as required by law. This research is being funded by Denver Human Services. The Center on Network Science at the University of Colorado Denver will use the information you provide for research purposes only. We will not disclose your identity or information that would identify you to anyone outside of the project without your permission, except as required by law. We will destroy all information that identifies you at the end of the study. Summary information from these interviews, together with material taken from public documents, will be presented; however, no specific individual or agency will be identified by name or affiliation in any reports. Findings from the study can be shared with all participants. Your participation in this discussion is completely voluntary. If you are uncomfortable with any question we can skip it. Do you have any questions about our confidentiality procedures before we begin? (If yes, respond to all questions. If no, proceed with discussion). Do we have your permission to record this interview? Name: Organization: Contact Phone Number: Contact Email: Interviewer: Interview Date & Time: 1. To start, we want to hear more about you, what you do, and how that relates to the Denver Day Works program. 52 2. Next we would like to hear about the background of the Denver Day Works program from your perspective. Can you tell us a little about the origins and current purpose of the pilot program? Probes- Did they cover:    Reasons for formation? Goals and activities (are these understood by key informants)? Comments on funding/sustainability of the program 3. What makes Denver Day Works unique from other programs addressing homelessness? Probe/Follow up:  What are the strengths of the program?  What has been most successful?  What has worked well (what has been the best strategies)? 4. What have been the “pain points” (challenges) of the program? Are there any challenges you or your organization have encountered while trying to implement this program? Probe/Follow up:  Why do you think that has been a challenge?  What could be done to help overcome those challenges?  Is there anything else that could be improved in the program? o Can you think of a solution to make this improvement? 5. We are exploring questions about scaling the program in the future. Do you think this program should be expanded? Probe/Follow up:  If yes, what do you think is necessary for expansion?  What kind of administrative structure is necessary to scale the program?  Can you talk about cost/benefits of the program? What are the costs? How do you think about the return on investment? 6. Are there specific partnerships that have been beneficial to implementing this program? If so, who and why? Probe/Follow up:  Who are your key partners? Who are key players in this program (including clients and providers)? How can these partnerships be strengthened to continue benefitting the future of this program? Are there potential new partnerships that should be developed? If so, with who? 7. Is there anyone else we should be talking to about this program? Thank you for your time today! Other Notes/General Reactions: 53 Appendix 4. Interview Codes Total # of People Interviewed: 17 (2 City Program Staff; 2 Bayaud Staff; 6 worksite and/or employer staff; 7 participants) Strengths Entry into DDW Program Codes % of responden ts (n=17) Counts of Who Said Main Code Count (in blue text) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) 1 1 1 2 6 1 2 11 2 3 Sub Code Count City Admin (n=2) For Participants Low-Barrier (e.g. no background check, documentation needed) 29% (n=5) 5 For Program Participant Outreach (reaching goal, demand, how being done) 35% (n=6) Total During DDW Program Codes % of responden ts (n=17) 3 1 5 Counts of Who Said Main Code Count (in blue text) Sub Code Count City Admin (n=2) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) For Participants Earning Money Convenience (central meeting location, easy commitment, quick entry process) 35% (n=6) 6 1 2 1 2 29% (n=5) 5 1 1 1 2 Wraparound Support Services Services Entry Point Case Management Integration Quicker-Direct Services Navigation Supporting Basic Needs (food, bus pass, storage, laundry, safe place) 82% (n=14) 29 2 1 1 9 1 2 3 3 5 1 1 7 2 3 1 4 54 Housing Assistance Self-sufficiency (earn own money, provide livelihood, stability) Empowerment (feel hopeful, motivated, useful, self-esteem boost, not alone, etc.) Providing Support System Supportive/ Trusting Relationships w/ Program Staff Continual Communication w/ Participants (advisory councils, check-ins, responsiveness) Accommodating of Participant Circumstances Being Understanding (from Bayaud/work sites) Flexibility of Worksite Supervisors Different than Traditional Day Labor Job Readiness Initial Opportunity towards Employment Transferrable Skills & Training Job Search Support (applying for jobs, resume/interviewing help) Connections (of program staff that help participants) 6 1 1 4 3 5 59% (n=10) 10 2 41% (n=7) 7 1 1 1 4 82% (n=14) 14 1 2 4 7 1 2 1 3 10 2 1 4 3 3 1 1 1 14 41% (n=7) 7 59% (n=10) 13 18% (n=3) 3 88% (n=15) 26 59% (n=10) 10 3 11 2 1 4 4 9 2 1 2 4 6 1 2 2 3 1 7 55 During DDW Program Codes % of responden ts (n=17) Counts of Who Said Main Code Count Sub Code Count City Admin (n=2) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) For Partners, Program, City Beneficial for City Work Production (DDW getting more work done or work can't be done due to limited staff/resources) Financially Beneficial (using DDW money not agency money, use existing equipment, less expensive than contractors, return on investment) Program Administration Capabilities (things Bayaud better equipped to do than city-worksite screening, pay cash, case management, handling no id/background check, experience w/ homeless) Commitment to Program City Commitment (involvement, resources, high level champions, collaboration) Partners Personal Commitment (program/worksite staff) Participant Commitment 29% (n=5) 9 35% (n=6) 6 41% (n=7) 12 4 1 3 5 1 4 1 4 1 1 1 1 5 1 2 2 3 1 4 1 2 56 Positive Publicity for the City (media attention, seen as model program) Societal/Systemic Benefits of Program Changing Perceptions/ Humanizing Homeless Individuals Societal-Economic Contributions Overall Decreasing Homelessness 18% (n=3) 3 65% (n=10) 11 3 5 1 3 1 2 1 2 1 171 3 117 22 29 Employment Placement Phase Codes 2 3 Total % of responden ts (n=17) 1 46 74 Counts of Who Said Main Code Count Sub Code Count City Admin (n=2) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) 2 2 3 For Participants Job Placement Support for Participants Employment Hiring Events Individualized Placement Support Successful Job Placement (for some within city agencies and quickly) Job Transition & Retention Support Smooth Transition/ Retention w/ FullTime Employment (esp. w/ prior experience at work sites) Post-program Relationships & Case Management Work Standby Option Increased Potential for Permanent Hiring 82% (n=14) 26 8 1 6 6 12 65% (n=11) 2 4 6 1 3 1 2 2 1 1 1 3 1 3 21 6 1 5 6 4 1 57 Employment Placement Phase Codes % of responden ts (n=17) Counts of Who Said Main Code Count Bayaud (n=2) Work Site/ Employer (n=6) 3 1 2 5 1 2 Sub Code Count City Admin (n=2) Participant (n=7) For Partner/ City Job Placement Advantages for Employers Employment PreScreening Observing-Working with Potential Hires DDW Participants are Great Employees Job Transition & Retention Support for Employers Having Accommodating Employers Good CommunicationResponsiveness 76% (n=13) 14 6 35% (n=6) Total 2 6 8 69 4 2 2 4 1 3 14 30 69 3 22 Challenges Entry into DDW Program Codes Diversity Counts of Who Said Entry % of respondents (n=17) Main Code Count 6% (n=1) 1 1 1 1 Total Sub Code Count City Admin (n=2) Bayaud (n=2) During DDW Program Codes Diversity in Participants Funding for Expansion Bayaud/City Program Staff Work Site/ Employer (n=6) Participant (n=7) Counts of Who Said % of respondents (n=17) Main Code Count 6% (n=1) 1 47% (n=8) 11 Sub Code Count City Admin (n=2) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) 1 1 1 58 Additional Worksites More Participants (Increased Capacity) Increased Work Days Worksite Supervisors 2 18% (n=3) 3 Resource Intensive Time 29% (n=5) 7 Mental Health Criminal Background Substance Abuse Personality Conflict Total 4 1 4 Money Schedule Differences (Worksites/DDW) Communication Logistics (Coordinating schedules, Transportation, etc.) Organizational Differences/ Tension Skill Development (Training, consistency, more time with participants) Housing Situation Issues (Difficult to reach, no place to store belongings, etc.) Accessing Support Services 2 29% (n=5) 1 2 1 1 1 Waitlist Difficulties Partnering 3 1 1 1 1 1 1 1 2 3 2 2 2 1 1 1 8 3 1 2 2 1 1 12% (n=2) 2 12% (n=2) 2 18% (n=3) 3 24% (n=4) 4 6% (n=1) 1 1 12% (n=2) 2 2 24% (n=4) 4 1 3 24 11 48 2 1 1 3 1 26 8 3 5 59 Employment Placement Phase Codes Job Placement Funding for Permanent City Positions Background Check ID/Documents Complexity of System (or slow system) DDW Preferential Hiring Bank Account Physical Counts of Who Said % of respondents (n=17) Main Code Count 71% (n=12) 29 Sub Code Count City Admin (n=2) 4 Bayaud (n=2) Work Site/ Employer (n=6) 1 3 4 3 1 1 1 2 2 6 1 1 2 2 1 1 1 2 2 Lack of Address Employer Bias Lack of Flexibility Participant (n=7) 2 1 2 2 3 2 2 1 1 1 Attrition Transportation Housing (as a general challenge-living in shelter, losing IDs, etc.) 29% (n=5) 5 5 18% (n=3) 3 1 2 41% (n=7) 7 3 2 Job Placement Criminal Background Mental Health Employment Gap 18% (n=3) 6 Substance Abuse Job Retention Adjustment (to full time employment) Mental Health Learning Curve 47% (n=8) 2 1 1 1 2 1 1 1 1 1 6 2 3 2 1 1 1 1 16 31 11 2 Personality Conflict Total 2 61 46 1 2 4 10 60 Recommendations Entry into DDW Program Codes Administrative Main Code Count Counts of Who Said Sub Code Count 8 City Admin (n=2) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) 2 1 3 2 Quick Entry Process 2 Pre-Screening 3 1 Coordinator 2 1 1 Outreach 4 1 3 Training 5 1 2 1 Support Services Partner Services, food stamps, housing assistance, etc. Increased Workdays Additional Worksites Administrative Field Supervisors Worksite Compatibility Program Staff Training Support Services Financial Support Main Code Count 2 1 1 5 3 Sub Code Count City Admin (n=2) Bayaud (n=2) Work Site/ Employer (n=6) Participant (n=7) 2 2 2 3 7 2 2 2 1 8 2 2 2 2 2 1 4 1 2 4 2 2 10 1 6 13 Counts of Who Said 1 9 1 2 4 2 4 1 1 1 1 4 1 2 1 2 6 Storage Units Training Total 1 19 9 15 Funding Schedule 2 1 11 During DDW Program Codes 1 3 Intake Assessment Total 2 34 1 1 1 1 29 9 10 13 7 61 Employment Placement Phase Codes Job Transition Employee Supported Employment Case-management Full-time positions Main Code Count Counts of Who Said Sub Code Count City Admin (n=2) Bayaud (n=2) 5 2 2 Work Site/ Employer (n=6) Participant (n=7) 18 4 1 1 2 1 5 2 Budget Increase Private Sector HR Intake policies Transportation 2 Total 25 1 2 14 2 1 1 3 1 1 1 1 5 6 3 62