Alternative Response Research in Missouri, Minnesota, and Virginia Findings in Six Areas Institute of Applied Research St. Louis, Missouri www.iarstl.org 1 Three Evaluation Studies „ „ „ Process and Impact Evaluation of The Missouri Family Assessment and Response System 1995 - 1998 „ Report: Missouri Family Assessment and Response Demonstration Evaluation Report, January 2000 (Digest) Five-Year Follow-up of the Statewide Missouri Implementation „ Report: Differential Response in Missouri after Five Years, February 2004 Minnesota Alternative Response Evaluation 2001-2004 „ Report: Minnesota Alternative Response, Final Report, November 2004 Reports available in PDF format on www.iarstl.org (papers and reports tab) „ Virginia Report is available at: http://www.dss.state.va.us/files/division/dfs/cps/reports/eval_drs.pdf. 2 Two Track Systems in all Three States Exit System Unsubstantiated Families NOT Appropriate for AR Traditional Investigations All reports accepted as potential Child Maltreatment Screening Track change possible Families Appropriate for AR Family Assessments Exit System Substantiated Investigation Formal Cases / Child Removals Accepts Voluntary Services Agency or community services / Formal or informal cases Declines voluntary services or no services needed Exit System Exit System 3 General Areas to be Considered „ Screening „ „ Child Safety „ „ Are services made available to more families under AR and do the types of services change? Recurrence of Maltreatment „ „ Does the non-adversarial approach of AR lead to improved participation and satisfaction of families? Services to Families „ „ Can child safety be maintained under Alternative Response (AR) at the same levels as in traditional investigations? Family Engagement „ „ What proportion of cases become AR and is screening consistent? Does AR lead to reduced abuse and neglect reports and reduced removal of children? Cost Effectiveness „ Is AR more or less cost effective over time? 4 Screening „ „ In Minnesota, Missouri, and Virginia, reports of child abuse and neglect were initially screened for either an Alternative Response (Family Assessment in Missouri and Virginia) or a Traditional Response, that is, a CPS investigation, based on allegation, history or situation. The proportion of reports screened for AR varied significantly from one local office to another in all 3 states: „ Minnesota average across counties -- 47% „ Missouri -- 71% „ Virginia average across counties -- 61% 5 Child Safety „ Based on information on cases provided by assessment workers and investigators: „ „ „ (Changes during the Initial Assessment or Case) No evidence was found that child safety was compromised under AR either in Missouri or Minnesota. Some evidence was found of relative improvement in child safety under AR. Responses to a Virginia survey showed that 51% of supervisors and 37% of workers believed AR probably or definitely increased child safety. Most of the others believed there was no change. 6 Missouri Safety Change: Based on Reviews of Sample Cases Positive changes (by last contact with family) Positive changes (during first 30 days) Pilot Comparison Regession or no known change (by last contact with family) Regession or no known change (during first 30 days) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1. Safety of children was not compromised by the demonstration. Moreover, there was evidence that child safety was improved in certain circumstances. 2. Children were made safer sooner. 7 Family Engagement Engagement of families was a factor in improved safety outcomes and increased services to families. „ „ „ Missouri: „ Cooperation increased; family flight decreased „ Family satisfaction improved „ Families’ sense of participation in decision making increased Minnesota: „ Family satisfaction improved „ Families’ sense of participation in decision making increased „ Families emotional responses were more positive under AR „ Workers rated families as more cooperative and less hostile Virginia: „ Workers and supervisors reported families often more willing to talk about problems and to accept services. 8 Missouri Family Cooperation: A Factor in Child Safety 100% 1. Cooperation between families and the child welfare agency increased. Family flight decreased. a. …because of the more positive and supportive orientation, or b. …because of earlier service contacts. 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Pilot Comparison Non-Cooperative Cooperative 9 Missouri Family Responses 14.7% Pilotassessment 14.3% comparison 38.5% 32.5% Pilotinvestigation 12.2% 6.9% pilot 45.4% Comparison 35.5% 0% A great deal 20% 40% Somewhat 60% A little 80% 100% Not at all Level of Involvement in Decision Making 0% 10% 20% 30% 40% Very satisfied Generally satisfied Generally dissatisfied Very dissatisfied 50% Satisfaction of Families with Children’s Services 1. Families felt they had greater involvement in decision making. 2. Families were more satisfied with the experience. 3. Workers responses reflected these differences. 10 Minnesota Family Responses 16.4% 7.0%8.4% Experimental 68.2% Experimental 58.0% p< 0.000 Control 45.1% 0% 20% A great deal Somewhat 6.2% 2.5% p< 0.000 22.2% 40% 33.3% 13.0% 60% A little 19.7% 80% Control 100% Not at all Level of Involvement in Decision Making 44.8% 0% 20% Very satisfied Generally dissatisfied 39.1% 40% 60% 9.3% 80% 6.8% 100% Generally satisfied Very dissatisfied Satisfaction of Families with Treatment by Worker 1. Families felt they had greater involvement in decision making. 2. Families were more satisfied with the way they were treated. 3. Workers responses reflected these differences. 11 Minnesota Family Emotional Response Differences in engagement and alienation were reflected in the reported emotional responses of families in Minnesota. „ „ Experimental families reported being relieved, hopeful, satisfied, helped, pleased, reassured and encouraged significantly more often. Control families reported being angry, afraid, irritated, dissatisfied, worried, negative, pessimistic and discouraged significantly more often. 12 Minnesota Worker Assessments of Cooperation „ As measured by the Minnesota SDM Family Risk Assessment, AR families had significantly better attitudes. „ „ „ Workers rated the primary caregiver as uncooperative for „ 44 percent of TR control families „ less then 2 percent of AR experimental families For TR control families, caregivers were rated as less motivated and as viewing the situation that led to the report less seriously than the agency. Workers rated families on cooperation as follows: „ „ The average levels of cooperation of families during the first visit and last visit were significantly greater for experimental families. Workers were more likely to report that control parents were hostile throughout the case. 13 Services to Families „ The Missouri demonstration was cost-neutral. „ „ „ In Minnesota, additional funds were available for the demonstration. „ „ No new funding was provided for services in Missouri. There was a greatly increased emphasis on linking families with community resources. Temporary funding was received from the McKnight foundation for the duration of the 20-county demonstration project only. In Virginia, no additional state funds were provided, but agencies sometimes obtained additional funds from local government, or applied for grants, or worked with community organizations to increase services. 14 Services to Families (continued) „ „ „ „ Services were provided to families earlier. This was evident in Missouri with its emphasis on services by assessment workers but was not measured in Minnesota. Families provided with post-assessment services increased in both states. Linkages of families to funded and unfunded community providers increased in both states. The types of services delivered to families changed in both states, with a shift toward family support services that would address financially-related needs. 15 Missouri Time to First Service 1. Services occurred in a more timely manner under the new approach. 16 Minnesota Proportion of Families with Service Case 15.1% Control Post-Assessment Services after Initial Report and Assessment 36.2% Experimental 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1. Over twice as many experimental families had a case-management workgroup opened (the condition for provided paid services) 17 Missouri Utilization of Community Resources Increased neighborhood organizations schools* alcohol/drug rehabilitation MR/DD providers youth organizations DFS--food stamps* DFS--welfare* health care provider* Job Service/Employment Security* pilot JTPA (employment and training) comparison legal services* support groups* child care/respite care/head start* community action agency domestic violence shelter *=statistically significant difference; p<.01 food pantry Overall there was a greater utilization of community resources in pilot areas. Most were not paid by CPS. church/religious organizations* recreational facilities (e.g. YMCA) neighbors/friends/extended family* 0% 5% 10% 15% 20% 25% 30% 18 Minnesota Utilization of Community Resources Increased school mental health provider support group A similar pattern was found in Minnesota neighbors/friends/extended family emergency food provider recreational facility (e.g. YMCA) neighborhood organization childcare provider/preschool provider/Head Start health care provider Experimental alcohol/drug rehabilitation agency or program Control community action agency job service/employment security legal services provider domestic violence shelter p=. 03 youth organization p=. 01 employment and training agency (JTPA, etc) church or religious organization MR/DD provider 0% 5% 10% 15% 20% 25% 30% 19 Missouri Services (Paid and Unpaid) Attempted for Families 1. More actions were taken in pilot areas to assist families in finding needed services. 2. Differences were small but statistically significant. Basic necessity services increased significantly 0.53 1. Basic necessity services 0.62 Pilot 0.15 2. Job related services Comparison 0.20 1.2 3. Counseling, instruction or crisis 1.28 1.87 All Services Combined 2.10 0 0.5 1 1.5 Average per family 2 2.5 20 Minnesota (Paid and Unpaid) Services to Families individual couns eling m arital/fam ily/group couns eling parenting clas s es help with bas ic hous ehold needs m ental health/ps ychiatric s ervices childcare/daycare s ervices help with rent or hous e paym ents em ergency food m edical or dental care dom es tic violence s ervices hous ing s ervices as s is tance with trans portation educational s ervices res pite care/cris is nurs ery Control Experimental TANF, SSI or food s tam ps legal s ervices as s is tance from s upport groups recreational s ervices hom em aker/hom e m anagem ent as s is tance as s is tance with em ploym ent vocational/s kill training fam ily pres ervation s ervices alcohol abus e treatm ent drug abus e treatm ent em ergency s helter dis ability s ervices independent living s ervices 0% 5% 10% 15% 20% 25% In Minnesota the change in types of services delivered was clearer. Family support services related to financial needs increased along with traditional counseling and therapeutic services. 21 Recurrence of Reports of Child Maltreatment „ „ „ Recurrence of child abuse and neglect (CA/N) could be measured indirectly: „ Families with new CA/N reports after final contact following the initial report. „ Families with later removal and placement of children. Report recurrence declined in both states under AR. Later removals of children declined in Minnesota but no corresponding evidence could be found in Missouri. 22 Missouri Child Abuse and Neglect Report Recurrence after Five Years (Original Demonstration Families) 4.5 Mean New Hotlines per Family 4.0 3.5 3.0 2.5 2.0 Pilot 1.5 Comparison 1.0 0.5 0.0 Low Moderate High CA/N Risk Level Very High 1. With five complete years of data pilot families continued to have lower rates of recurrence. 2. Difference by risk level dwarfed differences by pilot and comparison area 23 The Effects Service versus Approach on Recurrence of Child Maltreatment „ The experimental design in Minnesota permitted analysis of the relationship of services to families as well as the family-friendly approach of AR to reduction of later recurrence of reports. „ „ Services to families lowered recurrence. This might be expected but is a very difficult thing to prove outside the context of a controlled study. The increase in family support services addressing financially related needs may have been an important component of this effect both in Missouri and Minnesota. The non-adversarial and participatory approach to families reduced recurrence whether or not services were delivered. One possible explanation centers on family engagement, family cooperation, improved communication and participation. The exact mechanisms, however, are unknown and should be studied further. 24 The Costs of Alternative Response „ The Minnesota evaluation included a cost-effectiveness component. Data are still be collected but interim results have been positive. Costs related to case management and other services during the time the initial case was open were greater for AR than control cases. „ Costs for case management and other services following the closing of the initial case through the end of the follow-up period were LOWER for AR cases. On balance then, „ Total costs for case management and other services, both separately and combined, were less for AR cases than control cases. With the result that… „ Effectiveness: The mean cost per family of achieving the goal of recurrence avoidance with AR was $398 less than with the traditional approach. „ 25 The Costs of Alternative Response Total Cost Control $593 Period 1 Experimental $0.00 $1,132 $500.00 $1,538 $2,13 Period 2 $804 $1,93 $1,000.00 $1,500.00 $2,000.00 $2,500.00 Mean Cost Per Family