Evaluation Strategies for Wisconsin’s FoodShare
Employment and Training Program

Prepared for Wisconsin Department of Health Services Division of Medicaid Services

By
Nehemiah Chinavare
Kevin Dospoy
Chad Laurie

Workshop in Public Affairs
Spring 2018

 ©2018 Board of Regents of the University of Wisconsin System
All rights reserved.

For an online copy, see
http://www.lafollette.wisc.edu/outreach-public-service/workshops-in-public-affairs
publications@lafollette.wisc.edu

The Robert M. La Follette School of Public Affairs is a teaching and research department of the
University of Wisconsin–Madison. The school takes no stand on policy issues; opinions expressed in
these pages reflect the views of the authors.

The University of Wisconsin–Madison is an equal opportunity and affirmative-action educator and
employer. We promote excellence through diversity in all programs.

 Table of Contents
Acknowledgements ....................................................................................................................... 3
Foreword ........................................................................................................................................ 4
List of Tables ................................................................................................................................. 5
List of Figures ................................................................................................................................ 5
List of Abbreviations .................................................................................................................... 6
Executive Summary ...................................................................................................................... 7
Introduction ................................................................................................................................... 8
SNAP and FoodShare ................................................................................................................... 9
The FoodShare Employment and Training Program (FSET).................................................. 9
Work Requirements................................................................................................................................ 9
Lessons from Other SNAP E&T Evaluations .................................................................................... 11
Employment and Training Components............................................................................................. 11
Job Search and Job Search Training ................................................................................................... 11
Education ............................................................................................................................................ 12
Workfare ............................................................................................................................................. 12
Work Experience ................................................................................................................................ 12
On-the-Job Training ............................................................................................................................ 13
Self-Employment ................................................................................................................................ 13
Job Retention ...................................................................................................................................... 13
Summary of Components and Research............................................................................................. 13
State Comparison ................................................................................................................................ 14
Minnesota............................................................................................................................................ 14
Illinois ................................................................................................................................................. 14
Direct Comparison .............................................................................................................................. 14

FSET Vendors and Implementation ......................................................................................... 15
Evaluating the FSET Program .................................................................................................. 15
Overview of Current Data Sources and Collection Methods ............................................................ 15
FoodShare and FSET Administrative Data ......................................................................................... 15
Department of Workforce Development Wage Records .................................................................... 16
Regional FSET Vendors Records ....................................................................................................... 16
Regional FSET Vendor Caseworkers ................................................................................................. 16
FSET Participants ............................................................................................................................... 17
DHS Reporting Responsibilities .......................................................................................................... 17

Key Challenges of Evaluating FSET ......................................................................................... 18
Recommendations ....................................................................................................................... 20
Tier 3 Recommendations: Immediate changes requiring few additional resources ....................... 20
Monitoring vs. evaluating data............................................................................................................ 20
County participation and demographics ............................................................................................. 21
1

 Standardize utilization of public funds ............................................................................................... 21
Tier 2 Recommendations: Changes requiring some additional resources....................................... 22
Standardize tracking of FSET components ......................................................................................... 22
Collaborative focus groups ................................................................................................................. 22
Tier 1 Recommendations: Changes requiring significant additional resources ............................. 23
Explore options for streamlining data reporting processes ................................................................. 23
Data integrated evaluation approach ................................................................................................... 23

Legislative and Political Implications ....................................................................................... 24
Conclusions .................................................................................................................................. 25
Data Needs ............................................................................................................................................. 25
Future Research .................................................................................................................................... 25
Improved Metrics.................................................................................................................................. 25
Implications ........................................................................................................................................... 25

Works Cited ................................................................................................................................. 26
Appendix 1: FSET Components Research ............................................................................... 28
Work Requirements and Work Incentives ......................................................................................... 28
Job Search and Placement Supports ................................................................................................... 29
Job Search Assistance ......................................................................................................................... 29
Subsidized Employment ..................................................................................................................... 29
Education and Training ....................................................................................................................... 30
Case Management and Supportive Services ....................................................................................... 31
Sectoral Strategies................................................................................................................................. 32
Impacts ................................................................................................................................................... 32

Appendix 2: FSET Components ................................................................................................ 33
Appendix 3: FSET Vendor Map................................................................................................ 34
Appendix 4: Federal Nutrition Service Reporting Requirements .......................................... 35
Appendix 5: Survey..................................................................................................................... 36

2

 Acknowledgements
The authors would like to thank the Wisconsin Department of Health Services Division of Medicaid
Services for the opportunity to engage in this project on strategies for evaluating the effectiveness of the
FoodShare Employment and Training program. Thank you to Michele Dickinson, Stephanie Mabrey,
Jayne Wanless, Katie Vieira, and Mitchell Running of the Division of Medicaid Services for offering
their time and resources to help guide our work. Thank you to Professor Michael Collins for his continued
insight, support, and guidance through the progress of this report.

3

 Foreword
This report is the result of collaboration between the Robert M. La Follette School of Public
Affairs at the University of Wisconsin–Madison and the Wisconsin Department of Health
Services Division of Medicaid Services, a state agency. The objective of this project is to provide
graduate students at the La Follette School the opportunity to improve their policy analysis skills
while contributing to the capacity of partner organizations.
The La Follette School provides students with a rigorous two-year graduate program leading to a
master’s degree in public affairs. Students study policy analysis and public management, as well
as concentrated study in at least one policy area. The authors of this report are in their final
semester of their degree program and are enrolled in the course Public Affairs 869 Workshop in
Public Affairs at the University of Wisconsin–Madison. Although studying policy analysis is
important, there is no substitute for engaging actively in applied policy analysis as a means of
developing policy analysis skills. The Public Affairs 869 Workshop gives graduate students that
opportunity.
The Department of Health Services Division of Medicaid Services supports the state’s provision
of critical public supports by administering the systems that serve families in communities. The
Department has an ongoing focus on quality improvement, including through research
partnerships with the University of Wisconsin–Madison. The State of Wisconsin has a strong
history of working with La Follette School of Public Affairs students to perform rigorous
research. This report includes interviews, reviews of studies, and administrative research, with
the goal of helping the Department to improve the quality of support services for families
statewide.
I am grateful to the Department for partnering with the La Follette School on this project.
Department staff have been exceptionally generous with their time to support this project,
including collaborating on data, interviews and processes. The students have collectively
contributed hundreds of hours to this project, and in the process developed critical insights about
key programs. The La Follette School is grateful for their efforts and hope that this report proves
valuable for the Department and the State of Wisconsin to improve the outcomes of families.
J. Michael Collins
Professor of Public Affairs
May 2018
Madison, Wisconsin

4

 List of Tables
Table 1: Appropriate uses for FSET data sources....................................................................................... 20
Table 2: FSET Components ........................................................................................................................ 33
Table 3: Survey Methodology Matrix......................................................................................................... 36

List of Figures
Figure 1: Summary of Recommendation Tiers ............................................................................................. 7
Figure 2: FSET Application Process........................................................................................................... 10
Figure 3: DHS Data Collection for FNS Reporting Requirements ............................................................. 18
Figure 4: FSET Vendor Map ...................................................................................................................... 34

5

 List of Abbreviations

ABAWD:

Able-bodied adult without dependents

CSPED:

Child Support Noncustodial Parent Employment Demonstration

DHS:

Department of Health Services

DWD:

Department of Workforce Development

FPL:

Federal Poverty Level

FNS:

Food and Nutrition Service

FSET:

FoodShare Employment and Training

IM:

Income maintenance

IRP:

Institute for Research on Poverty

RFS:

Request for Services

SNAP E&T:

Supplemental Nutrition Assistance Program Employment and Training

SNAP:

Supplemental Nutrition Assistance Program

USDA:

United States Department of Agriculture

W2:

Wisconsin Works

WIOA:

Work Innovation and Opportunity Act

WDA:

Workforce Development Area

6

 Executive Summary
Employment and training programs play an increasingly important role in federal and state efforts to
transition more able-bodied working-age adults without dependents enrolled in safety net programs into the
labor force. Wisconsin is expanding the work requirements that certain working-age FoodShare beneficiaries
must satisfy to receive benefits. The changes include applying the work requirement to able-bodied workingage adults with dependents and increasing the number of hours participants must work or participate in an
employment and work training program in order to receive benefits. The success of these federal and state
efforts is largely dependent on the effectiveness of the employment training and placement services that they
provide.
As the State of Wisconsin implements these changes, the Department of Health Services (DHS) will
evaluate the training and employment program for FoodShare – the FoodShare Employment and Training
(FSET) program – to determine whether it improves employment and earnings outcomes and increases selfsufficiency.
This report provides an assessment of the programs offered in Wisconsin including an overview of
FSET, a review of the efficacy of FSET training activities, and an explanation of data sources, outcomes, and
measures. The report concludes with recommendations DHS can implement over the short, medium, and longterm to conduct a quality evaluation of FSET.
Currently, the work requirement applies to only able-bodied working-age adults without dependents,
also known as ABAWDs. FSET provides ABAWD FoodShare participants with opportunities to satisfy their
monthly work requirement. Research has found mixed results regarding the impact of employment and training
activities on participants’ earnings and employment outcomes. The review concludes that offering participants
comprehensive education and job-training services focused on in-demand occupations is the best way to ensure
participants find gainful long-term employment.
DHS data collection is labor-intensive and overly complex due to federal regulations, outdated data
storage, and reliance on third-party vendors to acquire earnings and employment data. Additionally, the
accuracy of data regarding participation in FSET activities is sometimes diminished by inconsistent data entry
practices among case managers. DHS must address these challenges to ensure it collects accurate earnings,
employment, and participation data with which to evaluate FSET.
This report identifies recommendations for addressing unreliable and insufficient data collection,
outdated data storage infrastructure, and methodology limitations. These recommendations vary in the time
and resources necessary to complete. This report organizes its recommendations into three tiers based on the
time, labor, and resources required to effectively implement. Figure 1 summarizes recommendations and tiers.

Figure 1: Summary of Recommendation Tiers

Tier 3

Tier 2

Short Term:

Medium Term:

Separate Monitoring
& Evaluation Data

Implement Uniform
Data Entry Processes

Tier 1
Long Term:
Redesign Database

We hope this report provides DHS with valuable insights as it continues planning its FSET evaluation
to determine the best way the program may serve enrollees transitioning to employment.

7

 Introduction
Over the past two decades, U.S. public policy shifted its focus on how to best aid low-income
people, largely replacing open-ended cash and in-kind assistance with conditional assistance dependent
on labor force participation paired with training programs to encourage employment. Wisconsin led the
way in implementing these changes, with the Wisconsin Works program being cited as a model for
federal welfare reform in 1996.
Since then, workforce and education training programs have become an invaluable resource to
enrollees of safety net programs. Participation in such programs allows enrollees to satisfy the work
requirement for assistance programs in lieu of employment and potentially improve skills, employment
prospects, and future earnings.
The federal government is encouraging states to transition more low-income adults into the labor
force by expanding work requirements to more “able-bodied” adults, increasing penalties for idleness,
and lengthening the hours required to satisfy a work requirement. Indeed, an executive order issued April
10, 2018, specifically called for changes to safety net programs that
“improve employment outcomes and economic independence (including by strengthening
existing work requirements for work-capable people and introducing new work requirements
when legally permissible)” (Exec. Order No. 13828)
As states experiment with improving labor force participation for beneficiary populations,
beneficiaries will increasingly rely on training programs to satisfy work requirements, continue receiving
public support, and find gainful employment. Therefore, it is important to determine whether such
programs actually improve employment outcomes for beneficiaries.
This report provides a set of recommendations for the Wisconsin Department of Health Services
(DHS) Division of Medicaid Services to assess its FoodShare Employment and Training (FSET)
program, including how activities may influence participant employment status, education outcomes,
earnings, and reliance on safety net programs.
In Wisconsin, FoodShare is the state’s Supplemental Nutrition Assistance Program (SNAP), a
federal program providing food assistance administered at the state level. Recent changes in the program
have expanded work requirements that certain participants must fulfill to receive benefits.
As of April 2015, able-bodied adults without dependents (also labeled as “ABAWDs”) must
satisfy a work requirement to continue receiving FoodShare benefits. ABAWD refers to working-age
adults ages 18 to 49 who do not have a dependent and are not defined as being unable to work due to
pregnancy, mental illness, homelessness, or participation in a drug/alcohol rehabilitation program. A
dependent is defined as either a child younger than 18 living in the household or a member of the
household who is unable to care for themselves. Qualifying dependents could include someone’s own
children, children of roommates or live-in partners, or an elderly or disabled family member. Able-bodied
adults with dependents are exempt from the requirement. If these participants do not satisfy the
requirement, FoodShare benefits become time-limited to three months of benefits in any three-year
period. ABAWD participants who are not employed can satisfy the requirement by completing 80 hours
of education or job-training activities each month.
Like many states, Wisconsin is proposing changes similar to those being discussed at the federal
level that would alter work requirements for major welfare programs. State agencies administering such
programs are developing systems and tools to assess the effectiveness of these policies. State agencies
also must determine if the benefits provided by education or job-training activities justify the costs to the
state. Given variations in the implementation of programs within and across states, having robust program
evaluation tools is important to understand the effects of FSET program participation.

8

 SNAP and FoodShare
The Federal Food Stamp Act of 1964 created the Food Stamps program, which was meant to
provide low-income people with fresh, healthy food. By 1974, the program had expanded to all 50 states.
In 2008, the Food Stamp program was renamed the Supplemental Nutritional Assistance Program
(SNAP). Similar to other federally mandated programs, SNAP is funded at the federal level, but
administered at the state level.
FoodShare is Wisconsin’s SNAP program. While the program is mostly administered at the local
level, the federal government funds benefit payments. Households with gross incomes below 200 percent
of the poverty line are eligible to receive benefits (Wisconsin Department of Health Services 2017).
Households with a blind, elderly, or disabled member with a gross income over 200 percent are still
eligible to receive FoodShare benefits if the net income is below 100 percent of the poverty line and the
person has less than $3,500 in assets (Wisconsin Department of Health Services 2017). In 2017,
FoodShare distributed $867 million in benefits to 925,047 recipients, including more than 386,000
children (Wisconsin Department of Health Services 2018).

The FoodShare Employment and Training Program (FSET)
The federal government requires every state to administer an employment and training program
for its SNAP recipients, referred to as a SNAP E&T program. States have some discretion in
administering this program regarding the services provided, populations served, methods of service
delivery, and whether people are required to participate to avoid sanctions. Wisconsin’s SNAP E&T
program is called the FoodShare Employment and Training (FSET) program. In 2015, the State of
Wisconsin implemented a work requirement for able-bodied adults without dependents – ABAWDs –
wishing to receive FoodShare benefits, and FSET became one of several options available for satisfying
that requirement.
Work Requirements
There are three ways to satisfy the FoodShare work requirement in Wisconsin:
1. Work at least 80 hours each month, or
2. Take part in a qualifying work program such as FSET, Wisconsin Works (W-2), or certain
programs under the Workforce Innovation and Opportunity Act (WIOA) at least 80 hours each
month, or
3. Work and take part in an eligible work program for a combined total of at least 80 hours each
month. (Wisconsin Department of Health Services 2017)
All beneficiaries subject to the work requirement receive a referral to FSET, but not all participate
in the program. Some elect to satisfy their work requirement another way. While this report focuses on
only FSET participants, crossover exists among other state programs. The FSET enrollment process is
relatively straightforward. See Figure 1. After a person applies for FoodShare, a case worker determines if
the work requirement is applicable. If so, the case worker refers the person to the FSET program.

9

 Figure 2: FSET Application Process
FoodShare Application

Approval/ABAWD1

Application
Determination

Status

Application
Approved
- Non-ABAWD
- Exempt

Application
Rejected

WIOA2 Programs

Application
Approved
ABAWD

Other Alternatives

Loss of
Non-ABAWD
or exempt
status

Non-Participant
Voluntary
Participation

FSET3

Referred to FSET
case manager
based on location

FSET Vendors by Region (R)
Forward
Service Corp.

ResCare

R1

R2

Workforce
Resource

R7

R8

R3

R4

Workforce
Connections

Wood Co.
Human Services

R5

R6

SW WI Workforce
Dane Co.
Dev. Board
Human Services

R9

R10

R11

Tribal Partners
Source: FSET Employee Handbook, 2018, Wisconsin Department of Health Services
1
ABAWD: Able-bodied adult without dependents
2
WIOA: Workforce Innovation and Opportunity Act
3
FSET: FoodShare Employment and Training

After a FoodShare enrollment office refers a participant to FSET, an FSET case manager contacts
the participant through mail or telephone to schedule an orientation appointment. During orientation, the
potential FSET participant learns about available training options. If the participant decides to enroll in
10

 FSET, the case manager conducts a personalized assessment to determine the person’s strengths, needs,
employment preferences, job readiness, educational attainment, and any existing employment barriers.
Following the assessment, the case manager assigns the participant to suitable employment and training
activities, or components, which are discussed in the following sections.

Lessons from Other SNAP E&T Evaluations
Before discussing the efficacy of individual program components meant to increase labor force
participation as a requirement for receiving public aid, the effect of overall programs should be
examined. In 1988, the federal Food and Nutrition Service (FNS) commissioned a study to analyze the
SNAP Employment and Training (E&T) program. The evaluation followed 12,000 non-exempt work
registrants who were assigned to randomized treatment and control groups in 53 program sites in 12 states
(Kogan, Paprocki and Diaz 2016). Originally, SNAP E&T programs focused on low-budget programs
that encouraged participants to find work quickly by requiring participation in job search and job search
training components. Such programs were capable of creating short-term increases in employment and
earnings, as well as reductions in reliance on public assistance. However, the results didn’t last much
longer than participation in the program. The study found no evidence that the SNAP E&T program
increased the likelihood of participants finding jobs. The study also failed to find any evidence that the
program had any significant effects on hourly wages, hours worked per week, or length of job retention
for those who did find employment.
No other comprehensive study was performed until 2009 when the Labor Market and Career
Information (LMCI) Division of the Texas Workforce Commission released findings from a fiveyear study of its SNAP E&T program. The study found that the number of participants working at the end
of the study was not significantly different from the percentage that had been employed at the beginning
of the study (47 percent and 49 percent, respectively). Average quarterly earnings for those who
participated in the study were about $1,400 higher after completing the program, but the study found that
the increased earnings were not enough to pull participants out of poverty (Kogan, Paprocki and Diaz
2016). More importantly, the applicability of this study is dubious at best because only 150 study
participants out of 22,000 had participated in anything other than job search assistance. The study may be
useful in determining the efficacy of job search assistance, but it does very little to examine the impacts of
the SNAP E&T program as a whole.

Employment and Training Components
Employment and training activities offered through FSET are organized into six components
aimed at improving skills and educational attainment while also satisfying work requirements. The
following section provides explanations of each component as well as a brief review of the relevant
literature. Additional information is in Appendix 1, and a more detailed list of FSET components is in
Appendix 2.
Job Search and Job Search Training
The job search and job search training component is designed to support participants who are
searching for employment, specifically those in need of general guidance with the job search process.
People who complete this component receive assistance in, “developing, practicing, and applying job
seeking skills with the goal of securing employment.” (Wisconsin Department of Health Services 2017)
Activities include:
•
•
•
•

Job development and placement
Participation in job clubs
Learning and practicing interview skills
Resume development assistance

11

 Research shows mixed results for the efficacy of job search assistance and job search training
programs, specifically in improving earnings and employment outcomes. Certain studies indicate that job
search activities increase short-term earnings but have muted impacts on long-term employment prospects
and earnings (Kogan, Paprocki and Diaz 2016). Research identifies placement support, which uses
employment agencies to find participants immediate employment, as one activity that affects short-term
earnings but has little-to-no effect on long-term outcomes (Autor and Houseman 2006). Some studies
show impacts on short-term outcomes while others show long-term benefits. Overall, research is
inconclusive on which job search activities have the greatest impact on earnings and employment.
Education
FSET provides education services to beneficiaries or financial assistance to beneficiaries
attending a post-secondary education institution. The increase in technical skills required for many jobs
has changed the program paradigm from an emphasis on general education to an emphasis on
transferable, hard skills. Activities categorized under the education component include:
•
•
•
•
•
•
•
•

Adult basic education
Customized skills training
Job skills training
Vocational literacy
English-as-a-second language
Higher education
High school equivalency diploma
Driver’s education

Measurements of education programs’ effect on earnings and self-sufficiency show mixed results.
D’Amico (2006) shows that classroom learning increases earnings and employment over the long-term if
the education focuses on job-specific skills in a highly sought-after field. Education activities not focused
on specific skills for high-demand occupations did not affect participant earnings or employment
outcomes. Additionally, Heinrich (2008) finds that attending educational programming lowers participant
earnings and employment due to the time spent in classrooms as opposed to time actually earning a wage.
The long-term effects of such programs may not make up for the opportunity cost of attending class.
Workfare
Workfare allows participants to build basic skills, employer references, and work history by
placing them in an unpaid position with a public or nonprofit employer. Only FSET participants subject to
the work requirement are allowed to take part in this component. FSET usually matches participants with
placements requiring little training. FSET establishes relationships with public and nonprofit employers to
provide placements, such as parks and sanitation departments, hospitals, and religious organizations. It is
forbidden to subsidize wages for participants in workfare with FSET funds.
Work Experience
Work experience offers participants opportunities for job-shadowing or short-term placement in a
work or training site. Ideally, work experience assignments expose participants to different employment
opportunities and ease transition to regular employment. This component includes transitional
employment and community service. FSET places participants with public or private employers. Work
experience activities must not prevent or replace regular employment. Additionally, work experience
participants are entitled to the same benefits and work conditions provided regular employees for
comparable work. FSET is statutorily required to oversee and evaluate workplaces to ensure they provide
quality placements for participants and comply with requirements mentioned above.
Transitional employment increases short-term earnings but has no impact on long-term outcomes.
Transitional employment also increases self-esteem and life stability, which improves self-sufficiency

12

 (Kirby 2002). However, research shows the most effective work experience programs connect
participants to regular, long-term employment opportunities (Hamilton 2012).
On-the-Job Training
A small number of FSET enrollees participate in on-the-job training, in which other allowable
programs subsidize wages of participants hired for regular, full-time positions with training opportunities
conducted by an employer. Employers may be public, nonprofit or private for-profit organizations. FSET
is prohibited from using its funding to subsidize participant wages. For participants enrolled in on-the-job
training, FSET requires wages to be subsidized by specific public programs, such as WOIA or W-2.
Evaluations by the U.S Department of Labor and independent researchers find “on-the-job
training is the most efficacious service for both adult women and men for whom this service activity was
recommended” (U.S. Department of Labor 2014) (D’Amico 2006). A review of previous evaluations of
on-the-job training demonstrations found on-the-job training creates lasting impacts on participants’
outcomes, including increased earnings, reduced reliance on public assistance programs and lower longterm poverty (Dutta-Gupta, et al. 2016). The review also identified programs with interventions longer
than 14 weeks have a high likelihood of producing increased earnings for participants, meaning benefit
duration could affect participant outcomes (Dutta-Gupta, et al. 2016). Programs with longer interventions
or complementary supports such as retention services may be likely to improve earnings, but more
research must be done to confirm this potential finding (Dutta-Gupta, et al. 2016).
Self-Employment
Participants enrolling in self-employment receive technical assistance for starting their own
businesses. This aid includes:
•
•
•
•

Creating a business plan
Locating financial resources, including low-cost loans and grants
Resolving credit problems
Developing successful marketing strategies

Self-employment services may be provided to participants for no more than 90 days after they no
longer qualify for FoodShare. Independent research analyzing the effectiveness of self-employment
strategies is lacking.
Job Retention
This component assists employed FSET participants with transitioning into the labor force.
Participants receive aid to maintain employment, meet workplace demands, and successfully meet
employer expectations. The specific activity provided depends on participant need. For example, under
this component, FSET is able pay for dry cleaning costs for a work uniform or provide transportation
services for participants who need to travel to work but do not have an automobile or other mode of
transportation. Participants are able to access these services for no more than 90 days after they first
secure employment, regardless of whether they are still eligible for FoodShare. Butler et al. (2012) posits
participants with barriers to employment – such as a criminal record – have particular trouble satisfying
workplace and employer demands, so retention services is likely a crucial determinant on whether these
participants maintain sustainable employment.

Summary of Components and Research
FSET procedures seek to ensure case managers spend time and effort to effectively match
participants with education and training activities based on their goals and preferences. This often entails
enrolling FSET participants into more than one component. According to DHS data, in May 2017, over
13

 70 percent of FSET participants enrolled in at least two components and 20 percent enrolled in three or
more components.
State Comparison
In addition to the literature review of basic SNAP E&T components, an analysis of surrounding
states is instructive. Examination of SNAP E&T practices within Illinois and Minnesota found similar
outcome expectations; however, implementation is hardly consistent.
Minnesota
Minnesota’s program offers many of the same services as Wisconsin’s, but implementation
varies. For example, while the FSET program is administered by regional vendors in Wisconsin,
Minnesota’s program operates at a county level. Similar services are offered, including job search
assistance, training, education, and support services. The intake procedure in Minnesota is different in that
each participant goes through an exhaustive interview in which rankings are assigned based on various
categories such as financial support, home life, and access to transportation. Ideally, participants are then
matched with programs to supplement the categories in which they were ranked lowest (Minnesota
Department of Human Services).
In addition, Minnesota’s program uses a form of enforcement in which financial penalties are
placed on participants who fail to meet certain requirements. Wisconsin and Minnesota share a
disqualification process. For example, when a person fails to attend orientation into the program, that
participant is subject to separation from the FSET program as well as the potential removal of SNAP
benefits.
Illinois
In 2017, the state of Illinois was under a federal waiver that allowed it to temporarily suspend the
mandatory work requirement for its SNAP participants. The waiver has not yet been approved for
2018. Despite the federal ABAWD waiver, 29 counties operated a mandatory SNAP E&T program.
ABAWDs in these counties risked losing their benefits if they failed to comply with the work requirement
(Illinois Hunger Coalition 2017).
The regular SNAP E&T program requirements in Illinois are similar to those in Wisconsin. However,
Illinois offers a spinoff program called EPIC, which is a program aimed at providing employment
opportunities, personalized services, individualized training, and career planning. EPIC operates in five
regions across Illinois as a collaboration between the Illinois Department of Human Services, the Illinois
Department of Commerce Job Training and Economic Development, and Illinois Businesses (Illinois
Department of Commerce 2017). No comprehensive analysis has been completed regarding the effects of
the program, therefore a direct comparison cannot be made.
Direct Comparison
While other states may use the same basic structure for SNAP E&T programs, the actual
implementation is quite different. In addition, the demographics between states are significantly
different so a direct comparison should not be drawn. Cost of living, transportation mechanisms, and
available public aid financed at the state level are inconsistent among states. The EPIC program in Illinois
may provide comprehensive data regarding the effects of collaboration between multiple organizations,
but to date, no such analysis has been completed. Due to the inconsistent method in which SNAP
programs are administered by each state, no direct comparison could be drawn from comparing the
implementation process between any group of states. Any analysis or recommendation should come from
within programs focused on populations in Wisconsin.

14

 FSET Vendors and Implementation
FSET program implementation is divided among 11 regions, corresponding to the state’s
Workforce Development Areas. Independent vendors provide employment and training services in each
region and are responsible for coordinating with employers, hiring caseworkers, and collecting
administrative data. Vendors range from for-profit companies with experience operating similar welfare
programs to nonprofit workforce development boards and county governments. Appendix 3 includes a
map of vendors assigned to each development region. Vendors offer activities based on the opportunities
and resources available in the region where they operate.

Evaluating the FSET Program
This section provides an overview of data sources in greater detail, including the processes used
by DHS and outside vendors to collect and access data, the purpose of each data collection activity, and
difficulties inherent to data access and utilization.

Overview of Current Data Sources and Collection Methods
DHS collects data from various sources to meet its statutory reporting responsibilities, monitor
policy adherence, and evaluate participant outcomes. These sources can be divided into five broad
categories: The FoodShare and FSET administrative databases, the Wisconsin Department of Workforce
Development’s (DWD) wage records database, FSET regional vendor records, FSET regional vendor
caseworkers, and FSET participants.
FoodShare and FSET Administrative Data
Administrative data for the FoodShare and FSET programs are stored within separate online
systems. The FoodShare data system, known as CARES Worker Web (CWW), contains various
information regarding each FoodShare participant, including demographics, the number of adults and
children in each household, and the monthly benefit received by each participant. This information is
entered by county enrollment/benefit staff when a person applies to participate in the program. DHS staff
have the ability to view individual and aggregate level participation records within CWW, while FSET
vendor staff members are able to view only individual level records. The FSET data system, known as the
FSET Tool, contains information such as the date a participant is referred to the program, activities the
person participates in to fulfil his or her work requirement, and the number of hours of participation a
person completes each month. This information is entered by county benefit/enrollment staff at the time a
person is referred to the program as well as by FSET regional vendor staff during a person’s participation
in the program. DHS staff and regional FSET vendor staff have access to the FSET Tool.
The current data systems have two key limitations. First, data are collected as monthly snapshots
of program participation and are not integrated into an annual report within the database. For example,
one participant will have a unique entry in the database for each month that he or she has participated in
either FoodShare or FSET. Aggregating monthly data is complicated and time consuming. Second, the
FoodShare and FSET data systems were created independently and at a time when it wasn’t necessary to
integrate data from the two programs for evaluation purposes. As a result, it is very difficult for DHS staff
to produce reports using participant data from both programs. However, to satisfy its federal reporting
responsibilities, DHS must be able to match participant characteristics data in the FoodShare data system
with data regarding participation in employment and training activities in the FSET data system. DHS
does not possess the necessary personnel and resources to create annual data from monthly snapshots or
to integrate data from the FoodShare and FSET data systems. Therefore, the department relies on Deloitte
Consulting, a third-party data management firm, to perform these services. To obtain curated data from
Deloitte, DHS must submit a request for services (RFS). The cost of each request depends on the
personnel time required to fulfill it. Currently, the DHS submits two separate requests each year. The first
15

 RFS concerns data related to participation in the FSET program, including the number of people referred
to program, the number of participants, participants’ demographic characteristics, and the types of
employment and training activities in which the participants engage. The second RFS concerns necessary
information to match FSET participant data to employment and income data collected by the Wisconsin
Department of Workforce Development (DWD).
These administrative data systems are among the best sources of information for evaluating the
FSET program due to the availability of information on all participants, including demographic
characteristics and records of program participation. Additionally, this information can be matched with
other agency databases using participants’ social security numbers. However, administrative data is not
without limitations. Participant data is entered into these databases by hundreds of different caseworkers
employed by seven different vendors in eleven regions throughout the state. As a result, there is often a
lack of uniformity in how the data is entered. For example, a caseworker in one region may record a
computer literacy class as an education activity while a caseworker in another region may enter it as a job
training activity. This reduces the overall accuracy of the data and must be considered when utilizing it
for evaluation purposes.
Department of Workforce Development Wage Records
The federal government requires DHS to report the employment status and earnings of FSET
participants in the second and fourth quarters after program completion. DHS obtains administrative data
managed by the Institute for Research on Poverty (IRP) at the University of Wisconsin-Madison. IRP has
access to a wide variety of state and federal agency databases including wage records from the Wisconsin
Unemployment Insurance Program. DHS contracts with the IRP to obtain data regarding the number of
and the median quarterly earnings of FSET participants who are in unsubsidized employment. DHS
provides IRP with participant data based on reports from Deloitte Consulting. IRP matches participant
records to employment data to produce a report for DHS containing employment and income data for
FSET participants. DHS generates this report annually to satisfy federal requirements.
Regional FSET Vendors Records

To monitor policy compliance and identify service-provision trends, DHS reviews records
maintained by each regional FSET vendor (Wisconsin Department of Health Services 2017). Vendor
invoices are reviewed on a monthly basis to detect questionable spending or notable trends that need to be
addressed with corrective action. Such invoices may be used for monitoring as well as evaluation. DHS
staff visit each regional FSET vendor and at least one tribal partner once per year to evaluate whether
practices at each site reflect standard operating procedures. Prior to each site visit, DHS reviews a sample
of referral, enrollment, and disenrollment case records from each regional FSET vendor. This allows DHS
staff to identify potential concerns before conducting each site visit. Additionally, the sample is useful for
monitoring purposes. However, the information should not be used in a rigorous evaluation because it
represents only a small sample of the overall information available. Each regional FSET vendor is also
required to submit quarterly reports to DHS describing participant activities and employer partnerships
within their region. These reports are useful for monitoring policy compliance by regional FSET vendors
but should not be used in a rigorous evaluation because they contain primarily qualitative information that
is difficult to compare across regions.
Regional FSET Vendor Caseworkers
DHS gathers information from FSET vendor caseworkers to assess trends in service provision
and policy compliance (Wisconsin Department of Health Services 2017). Prior to each annual vendor site
visit, the DHS distributes an online survey to each caseworker employed by that particular vendor. The
survey contains 27 questions covering a variety of topics including the respondent’s caseload, trends
relating to the size of that caseload, the employment and training activities being provided to participants,
and the supportive services available to participants. When utilizing information collected through this
survey, DHS should be aware of the response rate. In 2018, FSET vendors distributed the caseworker
16

 survey to 163 individuals. However, vendors collected only 49 responses, resulting in a response rate of
approximately 30 percent. Therefore, the responses collected do not necessarily reflect the views and
experiences of all FSET caseworkers.
During each onsite visit, DHS staff meet with caseworkers to discuss service delivery trends,
coordination with income maintenance agencies, and best practices for that FSET region. DHS uses
information collected during these discussions to create recommendations for process improvement.
Information gathered from individual FSET caseworkers is useful for monitoring purposes but should not
be used in a rigorous evaluation because it may not represent the views and experiences of all FSET
caseworkers employed by a particular vendor.
FSET Participants
DHS gathers information directly from FSET participants to assess their experiences in the
program, the barriers faced in finding and maintaining employment, and the services they do or do not
receive through the program (Wisconsin Department of Health Services 2017). Before each annual
vendor site visit, DHS distributes an online survey via email to FSET participants enrolled at that time by
that particular vendor. The survey contains 30 questions covering various topics, including history of
program participation, employment status, resources needed to find or maintain employment, and the
types of employment and training activities the participant is engaging in through FSET. Most of the
questions in the survey provide DHS with information regarding whether each FSET vendor is
implementing standard operating procedures. In 2018, DHS added several questions to collect
information that could be used to evaluate the program’s effectiveness.
DHS distributes the survey to only those participants who have a valid email address on file. In
2018, FSET distributed the survey to 5,569 FSET participants, which is roughly half of the average
number of participants enrolled at any given time during an average year. Of the 5,569 participants who
received the survey, only 282 completed it, resulting in a response rate of approximately 5 percent of
participants who received the survey and less than 3 percent of total FSET participants at any given time.
Due to these low response rates, the participant survey may be a useful tool for monitoring policy
compliance, but the data collected is most likely not representative of the entire FSET participant
population. Therefore, it should not be used for evaluation purposes without significant improvements in
response rates.
During each regional vendor site visit, DHS staff meet with participants to discuss their
interactions with FSET staff, the services offered to them, and the barriers they face when trying to find
and maintain a job. DHS uses information collected during these discussions to create processimprovement recommendations for each regional vendor. Information gathered from individual FSET
participants is useful for monitoring purposes but should not be used in a rigorous evaluation because it
may not represent the views and experiences of all FSET participants being served by a particular vendor.

DHS Reporting Responsibilities
Under Section 4022(a)(2) of the Agricultural Act of 2014, states must report data regarding
participant outcomes following participation in SNAP E&T programs, such as Wisconsin’s FSET
program, to the FNS. In accordance with this rule, DHS must submit an annual report to the FNS
containing a variety of information, including participant demographics, the number and percentage of
participants who completed each type of employment or training activity, and median earnings of former
participants. See Appendix 4 for a detailed list of the data required by the FNS. Annual reports are due to
FNS no later than January 1 following the close of the federal fiscal year. Figure 1 illustrates the process
by which the DHS collects the required data.

17

 Figure 3: DHS Data Collection for FNS Reporting Requirements
Department of
Health Services
(DHS)

Administrative data
regarding all FoodShare
and FoodShare
Employment and
Training (FSET)
participants is collected
in a DHS database.

Deloitte
Consulting
DHS contracts
with Deloitte
Consulting to
curate FoodShare
and FSET
administrative
data.

Department of Health Services (DHS)

Institute for Research on Poverty (IRP)
DHS provides the IRP with FSET
participant data. The IRP matches this
data with earnings records collected by
the Department of Workforce
Development and provides matched
data to DHS.

Federal
Nutrition
Service (FNS)
DHS uses curated
data from Deloitte
Consulting and
matched data from
IRP to create its
annual report for the
FNS.

Source: State of Wisconsin FoodShare Employment & Training Program Plan: Federal Fiscal Year 2018, 2017,
Wisconsin Department of Health Services

Key Challenges of Evaluating FSET
DHS would like to be able to evaluate the effectiveness of the FSET program based on five
participant outcome goals:
1. Increased participant educational attainment. This is measured using the number of general
education development (GED) credentials, certifications, etc., earned by participants.
2. Increased rate of employment among participants. This is measured using the number and
percentage of former participants in unsubsidized employment during the second and fourth
quarter after completing their involvement in FSET.
3. Increased longevity of employment by participants. This is measured using the number and
percentage of participants in unsubsidized employment during the second and fourth quarter
after completing their involvement in FSET.
4. Increased participant wages over time. This is measured using the median quarterly earnings
of all participants during the second and fourth quarter after completing their involvement in
FSET.
5. Decreased reliance on other public assistance programs by participants.
DHS faces three significant challenges to measuring achievement of these goals: significant
heterogeneity in service delivery, the inherent limitations of its pre-post evaluation structure, and the lack
of an appropriate comparison group. Seven unique vendors administer the FSET program by providing
services in 11 regions throughout Wisconsin. Each vendor is responsible for providing services related to
the six component categories. However, the services provided within these categories differ based on a
variety of factors, including the region in which the services are provided, the vendor providing the
services, and the specific location of the facility where the participant receives services. For example, 15
unique activities can be used to meet the educational and vocational training component. Additionally,
FSET participants engage in numerous activities across multiple component categories, creating countless
combinations of services. The significant heterogeneity of services and service combinations provided to
participants makes evaluation extremely challenging because the standards of effectiveness applied to one
activity may not be relevant to other activities. Also, the fact that FSET participants often engage in more
than one activity necessitates an examination to determine whether certain combinations of activities are
more effective than others.
18

 The second challenge of evaluating the FSET program is the inadequacy of the pre-post
evaluation structure that is often used to study the impacts of other SNAP E&T programs (Rosenbaum
and Bolen 2016). This approach calls for collecting baseline data regarding key outcomes when a person
begins participation in the program and comparing that data to participant outcomes after completion. A
pre-post evaluation structure cannot prove that a certain intervention caused a certain outcome because it
cannot isolate the impact of the treatment – in this case participating in a SNAP E&T program – from the
impacts of other aspects of a person’s life. Changes in employment status, education, and earnings may be
caused by participation in the program or they may be caused by other factors, including maturation,
secular trends, and regression to the mean (Rosenbaum and Bolen 2016). Maturation refers to the
possibility that changes in behavior and circumstances can simply be the result of the passage of time and
would have occurred with or without the treatment. Secular trends include overall unemployment rates
and the strength of the economy. These factors are independent of participation in a SNAP E&T program
but can have significant impacts on participant outcomes. Regression to the mean refers to the fact that a
baseline observation may be an extreme point relative to a person’s overall trajectory and that an
individual is unlikely to remain at that extreme point for an extended period. For example, people tend to
rely on nutrition assistance programs due to unexpected circumstances such as sudden loss of
employment (Rosenbaum and Bolen 2016). Therefore, increases in wages after participation in a SNAP
E&T program may simply reflect the fact that an individual has found new employment and has returned
to his or her regular income trajectory.
To overcome the inadequacies of the pre-post evaluation approach, researchers often compare
program participants to a control group, which is a group of people who are very similar to the participant
group except for the fact they have not participated in the program. However, there is currently no
suitable control group in which employment, education, and earnings outcomes can be compared to the
outcomes of FSET participants. People who are referred to FSET but do not enroll in the program share
many characteristics with people who complete the program. However, comparing the outcomes of these
two groups ignores the possibility that an individual’s reasons for not participating in the program may be
linked to the very outcomes that are being evaluated. This makes it very difficult to establish a causal link
between participation in FSET and improved employment, education, and earnings outcomes.
Every state is federally required to offer employment and training opportunities for people in
Supplemental Nutrition Assistance Programs. However, these programs, as well as job opportunities and
economic conditions, vary widely from state to state. The variance among states makes direct
comparisons of participant outcomes across state lines inherently difficult. Therefore, the most accurate
comparison group would be one found in Wisconsin. This would seem to suggest that people who
participate in other job training programs in Wisconsin, such as those funded by the Workforce
Innovations and Opportunity Act (WIOA), represent another potential comparison group. However,
ABAWD FoodShare recipients are permitted to utilize a combination of FSET and WIOA programs to
fulfil work requirements. The participant cross-over between the two programs makes it difficult to
isolate the impact of either program.
All current and future endeavors to evaluate the FSET program’s effectiveness should attempt to
address the concerns raised in this section. Otherwise, these challenges will make the true impacts of
program participation very difficult to determine.

19

 Recommendations
DHS faces numerous challenges as it attempts to evaluate the FSET program’s effectiveness. No
single solution will resolve all challenges and many of them can be addressed only through a significant
commitment of time and resources. In this section, we present three tiers of recommended changes that
require varying levels of commitment. Tier three recommendations can be implemented in the near future
without significant additional resources. Tier two recommendations require intermediate levels of
additional time and resources. Tier one contains long-term recommendations that require significant
additional resources.
Tier 3 Recommendations: Immediate changes requiring few additional resources
The following recommendations can be implemented in the near-future without significant
additional resources. These recommendations are not mutually exclusive. Each one addresses a unique
evaluation challenge and can be implemented simultaneously with the other recommendations.
Monitoring vs. evaluating data
DHS collects data from various sources to evaluate the FSET program’s effectiveness and
monitor policy compliance. It is important to ensure that DHS staff properly utilize each data source
based on the extent to which the data is generalizable to the program as a whole. For example, a reliable
data source that contains information on all or most of the participants in the program would provide very
useful information in a rigorous evaluation of FSET because conclusions drawn from that data would
apply to all or most of the program’s participants. This information would also be useful for monitoring
policy compliance. On the other hand, a reliable data source that contains information on a small nonrandom sample of the participant population may not provide useful information for a rigorous evaluation
because it cannot necessarily be generalizable to the population as a whole. However, such a data source
may still provide information that would allow DHS to monitor whether regional FSET vendors and
department staff are complying with the appropriate policies and procedures. Table 2 shows the
appropriate use for each FSET data source.
Table 1: Appropriate uses for FSET data sources
Data Source

Use

Monitoring
Evaluation
FoodShare and FSET Administrative Data
√
√
DWD Wage Records
√
√
Vendor Records: invoices
√
√
Vendor Records: vendor reports
√
FSET Caseworker survey
√
FSET Interviews with caseworkers
√
FSET Participant survey
√
FSET Interviews with participants
√
Source: State of Wisconsin FoodShare Employment & Training Program Plan: Federal Fiscal Year 2018, 2017,
Wisconsin Department of Health Services

Data from the FoodShare participant database, the FSET participant database, and the DWD
wage records database is suitable for use in a rigorous evaluation as well as for monitoring policy
compliance because both contain individual-level information regarding all FSET participants. Therefore,
conclusions based on these sources are generalizable to the FSET program as a whole. Monthly invoices
submitted by vendors are suitable for use in a rigorous evaluation to compare the billing practices of each
vendor. Quarterly reports submitted by vendors are suitable for monitoring policy compliance but should
20

 not be used in a rigorous evaluation because the information they contain is qualitative in nature and is
not easily comparable across regions. Data collected using the current participant and caseworker surveys
is not suitable for use in a rigorous evaluation because each survey receives very low response rates in
comparison to the total numbers of participants and caseworkers. Therefore, the data is not generalizable
to the overall populations of FSET participants and caseworkers. However, it is still useful for monitoring
policy compliance. For a further discussion of potential survey requirements and techniques, see
Appendix 5.
Face-to-face interviews conducted with small numbers of caseworkers and participants present a
similar situation. Information collected during these interviews is suitable for monitoring policy
compliance but should not be used in a rigorous evaluation because it is not generalizable to the overall
populations of FSET caseworkers and participants. Differentiating between data sources that are suitable
for monitoring and data sources that are suitable for evaluation will ensure that the conclusions of an
FSET evaluation will be generalizable to the program as a whole.
County participation and demographics
Determining the extent to which external factors influence FSET participant outcomes is crucial
for isolating the impacts of program participation. To understand how regional differences in
unemployment influence participant outcomes, DHS should compare the number of FSET referrals and
enrollments per county, average county-level participant earnings, and employment outcomes to countylevel unemployment data provided by the U.S. Bureau of Labor Statistics. Comparing FSET enrollment
and referral rates for each county in Wisconsin to the unemployment rate in that county may shed light on
whether people living in counties with higher levels of unemployment are more likely to participate in the
program than people living in counties with lower levels of unemployment. Comparing county-level
unemployment rates to the average employment and earnings outcomes of FSET participants in each
county may allow the DHS to determine whether participants living in counties with higher levels of
unemployment are less likely to have positive employment and earnings outcomes on average than
participants living in counties with lower levels of unemployment. Determining the influence of regionallevel factors, like unemployment, on FSET participation and employment-related outcomes will help
DHS isolate the impact of the program itself.
Standardize utilization of public funds
DHS is interested in determining whether participation in FSET has any impact on participants’
reliance on other public assistance programs. However, DHS does not have a clearly defined set of
criteria for measuring the program impact. To establish such standards, DHS would need to first
determine which public programs are most often utilized by FSET participants, acquire data regarding
each program, and then match the data to the FSET participant database. However, most public assistance
programs are available to only seniors, the disabled, and families with children; ABAWDs receiving
FoodShare benefits are not eligible for most programs. Reliance on Medicaid, known in Wisconsin as
BadgerCare, could potentially be used to measure reliance on public assistance programs because
ABAWDs may be eligible depending on their gross income level. However, the single adult eligibility
threshold for FoodShare is 200 percent of the federal poverty level (FPL) (McAtee 2018), while the single
adult eligibility threshold for BadgerCare is 100 percent of the FPL (Wisconsin Department of Health
Services 2017). The difference between FoodShare and BadgerCare suggests that using reliance on
BadgerCare as a measure of reliance on all public assistance programs would fail to provide data on
FSET participants between 100 and 200 percent of the FPL. To estimate the impact of FSET participating
on an individual’s reliance on other public assistance programs, DHS should establish clear criteria for
measuring reliance on such programs. These criteria should include a list of public assistance programs
that meet this definition and are commonly used by FSET participants.

21

 Tier 2 Recommendations: Changes requiring some additional resources
Implementing the following recommendations would require intermediate levels of additional
time and resources. These recommendations are not mutually exclusive. Each one addresses a unique
evaluation challenge and can be implemented simultaneously with the other recommendations.
Standardize tracking of FSET components
Although there are six major categories of services, known as the six core components, the ways
in which specific employment and training activities are reported vary from vendor to vendor. For
example, a particular activity may be reported under the education component in one region but under a
different component in another region. This variation makes it difficult to match specific activities to
specific participant outcomes. We recommend that the DHS establish a set of standards for determining
how employment and training activities are reported by FSET vendors. These standards could be
incorporated into future procurement contracts with regional FSET vendors. Standardizing how activities
are reported will make it easier to evaluate the effectiveness of individual activities, components, and the
program as a whole. In conjunction with further caseworker training, such a standardization process
would likely decrease administrative data errors.
Collaborative focus groups
To provide and properly evaluate effective employment and training services, DHS must
understand the barriers to employment facing FSET participants. DHS collects information regarding
these barriers using a standardized assessment issued to participants at program intake. The assessment
includes questions related to work history, legal issues, housing, and transportation and is valuable in
determining the most useful services for a particular participant. However, the assessment neither enables
caseworkers to fully understand how barriers impact a participant’s ability to secure employment nor
allows DHS to learn of employment barriers outside the specifics measured in the questionnaire. Gaining
a deeper understanding of employment barriers is vital to the DHS’s ability to provide and evaluate
appropriate services.
We recommend that DHS work with the Institute for Research on Poverty (IRP) at UW–Madison
or other appropriate partners to conduct in-depth focus groups with FSET participants. These focus
groups would allow DHS to gain a deeper understanding of the employment barriers and how those
barriers impact potential employment. This model will provide a variety of benefits over the current
survey-based approach, including the ability to ask follow-up questions, pursue topics as they arise
instead of simply going through a list of questions with multiple-choice answers, purposefully recruit a
random sample of focus group participants, and purposefully attempt to include people who drop out of
the program. The success of any focus group relies heavily on the experience and preparation of the
person facilitating the process. We believe working with appropriate partners, including IRP, will allow
DHS to take advantage of outside skills and experience. The specific topics discussed during each focus
group should include circumstances that led participants to apply for FoodShare, perceived barriers to
employment, reasons for ending participation before completing the program if applicable, and participant
experiences at each stage of FSET participation. Gaining a deeper understanding of these topics will
provide insights for improving service delivery and assessment practices.
To promote participation in focus groups, we recommend that DHS conduct a separate session in
each region to minimize travel by participants. In addition, we advise DHS to set aside resources to
provide local transportation and monetary incentives to program participants. To ensure the validity of
collected information, we recommend potential focus group participants be selected randomly from the
current FSET participant pool in each region. Each regional session should consist of no more than 10 to
15 people to ensure that all focus group participants are engaged in the process.

22

 Tier 1 Recommendations: Changes requiring significant additional resources
Implementing the following recommendations would require significant additional resources.
These recommendations are not mutually exclusive. Each one addresses a unique evaluation challenge
and can implemented simultaneously with the other recommendations.
Explore options for streamlining data reporting processes
Currently, all significant attempts to evaluate the effectiveness of the FSET program are impeded
by the outdated design of the administrative data systems. FoodShare and FSET data are collected and
stored in separate data systems and can be merged for evaluation only by engaging the services of a data
management firm. This makes evaluating the program extremely difficult, expensive, and time
consuming. To address these issues, we recommend that DHS work with a database management
consultant to explore options for streamlining the data reporting process. Potential solutions may include
the creation of a new unified data system or a software improvement that automates data integration. Any
proposed solution should provide the ability to create annual reports from monthly snapshots and
integrate FoodShare and FSET participation data in a single report without engaging the services of a data
management firm. Achieving these goals would likely require a significant up-front investment. However,
it would give DHS greater access to and control over its own data, which would streamline the evaluation
process and eliminate the need for securing the services of a data management firm on an annual basis.
Data integrated evaluation approach
Given the lack of a viable control group with which to construct an evaluation model, the best
available model for evaluating FSET is a rigorous, resource intensive survey approach. IRP, which the
DHS currently contracts to provide employment and earnings data on FSET participants, is currently in
the process of conducting a rigorous multi-year evaluation of a five-year pilot program known as the
Child Support Noncustodial Parent Employment Demonstration (CSPED). CSPED is an initiative of the
Office of Child Support Enforcement in the U.S. Department of Health and Human Services’
Administration for Children and Families. The program aims to increase child support payment
compliance by providing noncustodial parents with a variety of services including, integrated case
management, employment-oriented services, and parenting classes (Meyer, et al. 2015). In collaboration
with Mathematica Policy Research, IRP is conducting an in-depth evaluation of the implementation and
impacts of the CSEPD program. The evaluation design calls for an integrated approach utilizing
administrative data, data from various state and federal databases, responses to a survey of individual
caseworkers, and responses to a rigorous, resource-intensive survey of CSPED participants.
We recommend that DHS work with IRP or other appropriate partners to implement a rigorous
data-integrated evaluation of FSET based on IRP’s ongoing evaluation of the CSPED program. This
approach would require significant planning and coordination between the DHS and IRP, however, this
section will provide several key recommendations regarding the evaluation’s structure. The dataintegrated FSET evaluation model would incorporate administrative data from FoodShare and FSET, data
from other state agency databases provided by IRP, implementation insights from interviews and surveys
with participants and caseworkers, and data from a rigorous resource-intensive multi-platform participant
survey.
The participant survey would be the center-piece of this evaluation model. However, it would be
much different from DHS’s current participant survey. Survey participants would be randomly selected
from current FSET participants, participants who have completed participation within the previous two
quarters, participants who dropped out during the previous two quarters, and FoodShare applicants who
were referred to FSET during the previous two quarters but did not enroll. The number of participants
selected from each category would be determined by the percentage of total enrollees that fall into each
category.
Ensuring a meaningful response rate will require a significant investment of time and money.
Generally speaking, FSET participants are a difficult population to survey because they tend to be more
23

 likely than the rest of population to lack stable housing and a mailing address, regular access to an email
account, or regular access to a phone. As a result, securing a meaningful response rate will require a
resource-intensive multi-wave survey approach. During wave one, the evaluation team would attempt to
administer the survey via email. During wave two, the evaluation team would attempt to administer the
survey to non-responders via postal mail or telephone. During wave three, the evaluation team would
attempt to administer the survey to non-responders by visiting each person’s last known residence.
Monetary incentives such as gift cards for groceries or gas could be used to help to increase response rates
(Singer and Ye 2013).
The total number of participants selected as subjects of this survey would depend on the resources
available and the desired reliability of the results. Collecting responses from a greater percentage of the
total participant population will result in more reliable results. For example, all else being equal,
collecting responses from 500 randomly selected members of a total population of 10,000 people would
allow the evaluators to say with 95 percent confidence that a particular value is within approximately 4
percentage points of the reported value. In comparison, all else being equal, collecting responses from
1,000 randomly selected members of a total population of 10,000 individuals would allow the evaluators
to say with 95 percent confidence that a particular value is within approximately 3 percentage points of
the reported value.
The survey should include questions pertaining to at least the following three topics. First,
participant outcomes, including: employment, earnings, and reliance on public assistance programs.
Second, barriers to employment, including: access to transportation, adequate skills and experience, and
interactions with the criminal justice system. Third, resources that were useful in securing and
maintaining employment, if applicable. Using a rigorous survey of a randomly selected sample of the
FSET population to answer questions related to these three topics will allow DHS to conduct the most
methodologically sound evaluation of FSET outcomes that is currently possible in the absence of a viable
control group.

Legislative and Political Implications
The nature of statewide policy is constantly changing, specifically when requirements and
funding levels are mandated, at least in part, by the federal government. Various state legislatures have
applied for work requirement waivers, and others – including Wisconsin – have actually increased the
number of work hours required to qualify for SNAP benefits. The Wisconsin Legislature recently voted to
make the following changes to the FSET program:
• Implement a performance-based payment structure for regional FSET vendors
• Increase the work requirement for ABAWD participants to 30 hours per week
• Establish drug-testing requirements for ABAWDs who intend to participate in FSET
• Mandate that able-bodied adult FoodShare beneficiaries with children participate in employment
or training activities for 30 hours per week starting in 2019
It is important to note that the first two changes above will require a federal waiver before
implementation. These changes would significantly alter the FSET program’s implementation and
evaluation. For example, to implement a performance-based payment structure for regional FSET
vendors, DHS would need to consider the impact of whether job placements are permanent or temporary
and whether childless adults and adults with children face different barriers to employment. Although
changes to FSET will significantly impact the implementation of the program, the recommendations of
this report remain unaltered because the proposed legislative changes do not alter the underlying need to
streamline and restructure the data collection and evaluation process. Performance-based vendor
payments, increased work requirements, and mandatory drug-testing will impact the FSET program in
different ways. However, the need to accurately determine which FSET component is most effective
remains consistent. Regardless of political or legislative actions regarding the requirements for FSET, we
24

 urge DHS to move forward with the recommended changes to the data collection process and the FSET
program.

Conclusions
Data Needs
DHS does not currently possess the data or evaluation framework needed to make causal claims
regarding the FSET program’s impacts on participant employment and earnings at a program level or at
an individual component level. Current data practices make it difficult for DHS to gather sufficient
information to conduct an effective evaluation. DHS relies on third-party contractors to create major
reports on FSET participant earnings and employment outcomes. An outdated data infrastructure makes it
labor-intensive to combine administrative FoodShare and FSET data. Standardizing entry practices for
case managers reporting FSET participation data would make that data more reliable to use in an
evaluation. Overhauling the data system, as recommended in tier 1, requires significant time, money, and
labor investments, but would make it easier for DHS staff to access FoodShare and FSET administrative
data.
Future Research
Conducting a randomized control trial experiment, or another form of evaluation involving a
control group, would be the ideal way to determine the existence of a causal link between FSET program
participation and particular participant outcomes. Unfortunately, no suitable control group is currently
available. Absent a random experiment, focus groups could be a first step to identifying a causal link
between FSET and participant outcomes. Past research shows employment training programs’
effectiveness is determined partially by how well it addresses specific employment barriers facing the
populations it serves. Conducting focus groups of FSET participants will identify employment barriers
facing FSET participants and how well the program’s activities address those barriers.
Improved Metrics
In the absence of a randomized control trial, implementing a rigorous evaluation framework, like
the one described in Tier 1 of the recommendations section, and incorporating additional data would
greatly improve DHS’s ability to evaluate the FSET program’s effectiveness. Researchers currently use a
combination of federal and state administrative data and resource-intensive participant surveys to evaluate
CSPED. DHS could implement a similar approach for FSET. Implementing such an approach requires
significant long-term investments and collaboration with other actors, including IRP and the UW Survey
Center. This approach allows DHS to collect data on participants’ skills, past employment experience,
and barriers to employment. The insights gained by implementing appropriate evaluation measures will
increase policy-makers’ abilities to make informed decisions about the FoodShare program’s future.
Implications
Potential policy changes, such as the implementation of a performance-based payment structure
for FSET vendors, will not be feasible without reliable evaluation metrics. These metrics are needed to
determine what information provides the best predictors of success, to set appropriate benchmarks linked
to compensation levels, and to assess whether vendors are meeting these standards. Implementing a
performance-based payment structure without first establishing these tools would greatly impair DHS’s
ability to create appropriate incentives for FSET vendors and accurately monitor participant outcomes.
This example illustrates the importance of designing and implementing appropriate evaluation procedures
before enacting major policy changes to existing programs.

25

 Works Cited
Autor, David, and Susan Houseman. 2006. "Does Temporary Agency Employment Offer a Way out of
Poverty?" Focus (IRP) 24 (3): 1-7.
Butler, David, Julianna Alson, Dan Bloom, Victoria Deitch, Aaron Hill, JoAnn Hsueh, Erin Jacobs, Sue
Kim, Reanin McRoberts, and Cindy Redcross. 2012. What Strategies Work for the Hard-toEMploy? Final Results of the Hard-to-Employ Demonstration and Evaluation Project and
Selected Sites from the Employment Retention and ASdvancement Project. New York: MDRC.
Card, David, Jochen Kluve, and Andrea Weber. 2010. Active Labor Market Policy Evaluations: A MetaAnalysis. Working Paper, Cambridge: National Bureau of Economic Research.
Center for Innovation in Research and Teaching. 2018. "Modes of Survey Delivery." Grand Canyon
University.
Center for Innovation In Research and Teaching. 2018. Modes of Survey Delivery. Grand Canyon
University. Accessed March 2018.
Cook, Steven. 2017. "Memo: Timing of Data Availability for FSET Federal Reporting Requirements ."
April 7.
D'Amico, Ron. 2006. What's Known About the Effects of Publicly-FUnded Employment and Training
Programs. Oakland: Social Policy Research Associates.
Dechausay, Nadine, Cynthia Miller, and Victoria Quiroz-Becerra. 2014. Implementing a Conditional
Cash Transfer Program in Two American Cities: Early Lessons from Family Rewards 2.0. New
York: MRDC.
Decker, Paul T, Robert B Olsen, and Lance Freeman. 2000. Assisting Unemployment Insurance
Claimants: The Long-term Impacts of the Job Search Assistance Demonstration. Princeton:
Mathematica Policy Research, Incorporated.
Dutta-Gupta, Indivar, Kali Grant, Matthew Eckel, and Peter Edelman. 2016. Lessons Learned from 40
Years of Subsidized Employment Programs. Washington D.C.: Georgetown Center on Poverty
and Inequality.
Exec. Order No. 13828, 3 C.F.R. 07874. 2018. https://s3.amazonaws.com/publicinspection.federalregister.gov/2018-07874.pdf.
Hamilton, Gayle. 2012. Improving Employment and Earnings for TANF Recipients. Washington D.C.:
Urban Institute.
Heinrich, Carolyn J. 2013. Targeting Workforce Development Programs: WHo Should Receive What
Services? And How Much? Workforce Development Needs in the Global Economy: US and Cross
National Perspectives. College Park: School of Public Policy, University of Maryland.
Heinrich, Carolyn J, Peter Mueser, K Troske, and Jacob M Benus. 2008. Workforce Investment Act NonExperimental Net Impact Evaluation. Bethesda: IMPAQ International.
Herbst, Chris M, and Erdal Tekin. 2011. "Do Child Care Subsidies Influence Single Mothers' Decision to
Invest in Human Capital?" Economics of Education Review 30 (5): 901-12.
Holzer, Harry J, and Karin Martinson. 2006. "Can We Improve Job Retention and Advancement among
Low-Income Working Parents?" Focus (2): 31-38.
Illinois Department of Commerce. 2017. Epic. Accessed 2018.
https://www.illinois.gov/dceo/WorkforceDevelopment/ProgramsAndResources/Pages/JTEDSNAP.aspx.
Illinois Hunger Coalition. 2017. SNAP ABAWD Time Limits in Illinois. Illinois Hunger Coalition.
King, Christopher T. 2004. The Effectiveness of Publicaly Financed Training in the United States:
Implications for WIA and Related Programs. Kalamazoo: Upjohn Institute for Employment
Research.
Kirby, Gretchen G, Heather Hill, LaDonna Pavetti, Jon Jacobson, Michelle Derr, and Pamela Winston.
2002. Transitional Jobs: Stepping Stones to Unssubsidized Employment. Princeton: Mathematica
Policy Research, Incorporated.
26

 Klerman, Jacob, Robin Koralek, Ashley Miller, and Katherine Wen. 2012. Job Search Assistance
Programs - A Review of Literature. Bethesda: Aby Associates.
Kogan, Deborah, Anne Paprocki, and Hannah Diaz. 2016. Supplemental Nutrition Assistance Program
(SNAP) EMployment and Training (E&T) Best Practices Study: Final Report. Office of Policy
Support, Food and Nutrition Service, Alexandria: U.S. Department of Agriculture.
Lee, Bong Joo, Robert George, Mairead Reidy, J Lee Kreader, Annie Georges, Robert L Wagmiller Jr,
Jane Stavely, David Stevens, and Anne Dryden Witte. 2004. Child Care Subsidy Use and
Employment Outcomes of TANF Mothers During the Early Years of Welfare Reform: A ThreeState Study. University of Chicago, Chicago: Chapin Hall Center for Children .
Maguire, Sheila, Joshua Freely, Carol Clymer, Maureen Conway, and Deena Schwartz. 2010. Tuning in
to Local Labor Markets: Findings from the Sectoral Employment Impact Study. Philadelphia:
Public/Private Ventures.
McAtee, Rebecca. 2018. 2018 Federal Poverty Level Changes for ForwardHealth Programs. Wisconsin
Department of Health Services Bureau of Enrollment Policy and Systems Division of Medicaid
Services.
Meyer, Daniel, Robert Wood, Diane Paulsell, Jennifer Noyes, Rebekah Selekman, Lisa Klein Vogel,
Samina Sattar, and Benjamin Nerad. 2015. Helping Noncustodial Parents Support Their
Children: Early Implementation Findings from the Child Support Noncustodial Parent
Employment Demonstration (CSPED) Evaluation. Institute for Research on Poverty.
Minnesota Department of Human Services. October. SNAP E&T Manual. 2015 1. Accessed 2018.
http://www.dhs.state.mn.us/main/idcplg?IdcService=GET_DYNAMIC_CONVERSION&Revisi
onSelectionMethod=LatestReleased&dDocName=SNAP_0503.
Office of Quality Improvement. 2010. Survey Fundamentals: A Gudie to Designing and Implementing
Surveys. University of Wisconsin Survey Center, Madison: UW Madison.
Puma, Michael J, and Nancy R Burstein. 1994. "The National Evaluation of the Food Stamp Employment
and Training Program." Journal of Policy Nalysis and Management 2 (13): 311-30.
Rangaranjan, Anu, Alicia Meckstroth, and Tim Novak. 1998. The Effectiveness of the Postemployment
Services Demonstration: Preliminary Findings. Princeton: Mathematica Policy Research,
Incorporated.
Rosenbaum, Dorothy, and Ed Bolen. 2016. SNAP Reports Present Misleading Findings on Impact of
Three-Month Time Limit. Center on Budget and Policy Priorities.
Singer, Eleanor, and Cong Ye. 2013. "The use and effects of incentives in surveys." The ANNALS of the
American Academy of Political and Social Science 112-141.
U.S. Department of Labor. 2014. "What Works in Job Training: A Synthesis of the Evidence."
Washington D.C.: U.S. Department of Labor, July.
U.S. House. 2014. "HR 2642." Agricultural Act of 2014. Washington, D.C.: Government Printing Office.
University of Wisconsin Survey Center. 2010. "Survey Fundamentals: A Guide to Designing and
Implementing Surveys." Office of Quality Improvement, University of Wisconsin Madison,
Madison.
Wisconsin Department of Health Services . 2017. FSET Handbook.
Wisconsin Department of Health Services. 2017. "FoodShare Wisconsin - Monthly Income Limits."
dhs.wisconsin.gov. October 2. Accessed April 14, 2018.
https://www.dhs.wisconsin.gov/foodshare/fpl.htm.
Wisconsin Department of Health Services. 2017. "State of Wisconsin FoodShare Employment and
Training Program Plan - Federal Fiscal Year 2018." Department of Health Services, Madison.

27

 Appendix 1: FSET Components Research
Due to the overall lack of consistent data analysis at the national level, individual states have been
implementing pilot SNAP E&T programs. Within these programs, numerous components have been
utilized. Not all components of employment training programs are equally effective because numerous
variables can impact the people participating in specific programs. The U.S. Department of Agriculture
(USDA) studied the following components: work requirements and work incentives, job search assistance
and placement supports, subsidized employment, education and training, case management and
supportive services, sectoral strategies, and collaboration with other programs to provide comprehensive
services to targeted participants. While studying each component, the term effective must be defined.
Each program and component is discussed as being successful or effective, but to analyze results,
variables must be assigned to each component to determine what constitutes effective. The USDA used
three primary variables to measure the effectiveness of each strategy – increased employment and
earnings among participants, reduced public benefits dependency, and reduced government expenditures
(D'Amico 2006). The USDA studied the impact and effectiveness of these strategies using multiple peerreviewed sources. Summaries of each component are below.

Work Requirements and Work Incentives
Some states have implemented workforce development programs as a requirement to receive
defined benefits. Some of these programs require a specific number of hours worked. A program member
participating in the labor force may receive additional financial benefits as a reward for work
participation. Work requirements and work incentives are separate components within SNAP E&T
programs.
Many of the programs studies by the USDA occurred during the first decade of true U.S. welfare
reform after legislation was passed in 1996. Within the programs examined, two main mandatory
activities were present. Job search only activities and education and training only activities were the most
common mandatory components of the work requirements section of the programs studies. Many people
participated in both job search and education and training activities. Rigorous studies have shown that
participants who were assigned to a job search activity first were likely to gain employment sooner. The
study also found that while participants who received education and training activities were more likely to
catch up over time in obtaining employment, they did not appear to receive higher wages or earnings as a
result of having received education. Each program intervention was more effective in connecting people
to employment than no program at all (Hamilton 2012).
Work incentives provide financial benefits for work efforts. The most common incentive is the
Earned Income Tac Credit (EITC), which offers all low- to moderate-income workers an offset to federal
payroll and income taxes. Numerous states have established their own EITCs to supplement the federal
credit. Research indicates that the EITC increases work activity but does not necessarily have a significant
effect on low-income families’ total earned income (Kogan, Paprocki and Diaz 2016). Work incentives
have been shown to produce the strongest increases in employment and earnings when combined with job
services.
Financial rewards differ from work incentives in that they incentivize certain behaviors instead
of distributing cash benefits for employment. The impact of financial incentives on employment outcomes
appears to be reliant on the employment status of the participant at the beginning of the intervention.
People working full time at the beginning of the study were more likely to participate in targeted activities
such as children’s education and health, while people working part time were less likely to participate,
and those who were unemployed were significantly less likely to earn supplements (Dechausay, Miller
and Quiroz-Becerra 2014).
Both work incentives and financial rewards are most impactful for those who do not have barriers
to employment. Studies have shown that common reasons for non-participation include having a
28

 disability, being afraid of leaving part-time employment to look for full-time work, dependent care, and
concern about the level of public assistance available (Klerman, et al. 2012).

Job Search and Placement Supports
Job search and placement supports share the goal of quickly connecting participants with
employment, without any emphasis on long-term earnings.
Job Search Assistance
Job search assistance programs are meant to connect participants with available employment as
quickly as possible. Two main functions of job search assistance are enforcement and training. Job
search enforcement-based programs quickly find participants employment by placing sanctions on people
who fail to secure employment, such as reduced benefits. Not surprisingly, enforcement-based programs
are successful in reducing benefit receipt, but not necessarily in increasing earnings (Klerman, et al.
2012). Training-based programs aim to broaden search skills in addition to promoting preemployment skills, such as resume writing and interviewing skills.
All forms of job search assistance have been shown to reduce Unemployment Insurance benefits
in the short-term (Decker, Olsen and Freeman 2000). One study has shown that treatment was more
effective at promoting reemployment to people who had marketable skills and less successful at
reemploying people with preexisting barriers to employment (Decker, Olsen and Freeman 2000).
Overall, research on the effectiveness of job search assistance programs has produced mixed
results. Some research suggests that these programs produce short-term earnings but are unlikely to
produce long-term results, while other studies have shown no statistically significant differences in
participant earnings or job retention. Some studies have even shown that job search programs focusing on
employment search consistent with a participant’s existing experience are just as effective – or ineffective
– as those programs that focus on finding any employment as quickly as possible. Due to the mixed
results from numerous studies, the effectiveness of job search assistance programs has not been accurately
determined.
Placement support relies on employment agencies (temp agencies) connecting participants with
temporary work meant to provide rapid employment and increase marketable skills. These programs often
assume that participants will be able to make the transition from temporary worker to full-time
employment with the selected firm. Again, studies have shown that “temp agencies” are successful at
increasing short-term employment and immediate earnings, but fail to increase long-term earnings and
even long-term employment (Kogan, Paprocki and Diaz 2016).
Programs utilizing “temp agencies” often assume that individual companies use temporary
staffing to search for full-time employees. Instead, companies can use “temp agencies’ to fill short-term
project work or instead of hiring full time employees. As such, low-wage workers relying on temporary
work are continuously susceptible to recurring periods of unemployment.
Subsidized Employment
Subsidized employment programs link participants to employment positions that are financially
supported by government or nonprofit workforce development programs. The purpose of subsidized
employment is to improve future employability and marketable skills, thereby increasing long-term
earnings in the unsubsidized labor market. The USDA categorizes two forms of subsidized employment –
transitional employment and on-the-job training (OJT).
According to the USDA, most transitional job programs share the same basic components:
placement in a temporary wage-paying job, support services with some form of case management, and
job placement services to help the participant find a permanent job (Kogan, Paprocki and Diaz 2016).
Similar to participants who utilized “temp agencies,” transitional jobs can produce significant short-term
impacts on earnings, but over time, the impacts decrease. Some studies note that when subsidized
employment programs actively attempt to connect participants to regular, stable employment, long-term
29

 results can remain positive (Hamilton 2012). Subsidized employment is most effective when linked with
occupational training in growing sectors of the economy, such as healthcare or technology-related fields,
or vocational training and apprenticeships.
Numerous studies have shown that even if long-term effects on earnings are not visible,
transitional jobs can produce an increase in soft skills such as self-esteem, life stability, and exposure to
new opportunities – which are difficult to measure in any quantifiable way (Kirby, et al. 2002).
OJT is a form of subsidized employment in which an employer agrees to hire and train a
participant while receiving a wage subsidy. According to the U.S. Department of Labor, evaluations of
OJT programs have found that they lead to positive employment and earnings outcomes (U.S. Department
of Labor 2014). D’Amico found that “OJT is the most efficacious service for both adult women and men
for whom this service activity was recommended” (D'Amico 2006).
Research shows that while long-term welfare recipients assigned to classroom training
experienced little-to-no impact on long-term earnings, participants assigned to OJT experienced
substantial impacts in most years and a 12 percent earnings gain over the study period. Adult women saw
the most benefit from OJT intervention (King 2004).
OJT programs are not without drawbacks. Indeed, criticisms of such programs highlight the
worst-case scenario, in which the program is merely a payment from the government to a private firm to
secure a job for a program participant, without significant oversight into the quality of training that
occurs.

Education and Training
Education and training is one of the broadest categories that encompasses many forms of
intervention, including adult education, employer-driven occupational skills, vocational certificates,
community college courses, and higher-level degrees. Due to the amount of time and money associated
with associates and bachelor’s degrees, some workforce development programs have instead focused on
vocational education, certificates, and occupational skills.
Traditional education had been used prior to welfare reform during the 1990s. Since then, due to
a changing economy, employers have been more focused on hiring people with specific skills, instead of
those who are “ready to be trained.” The increase in technical skills required for many jobs has changed
the program paradigm from an emphasis on general education to an emphasis on transferable, hard skills.
D’Amico found that traditional classroom education alone does not improve employment or earnings
outcomes for participants in mandatory work programs or for people from lower-income households who
lack work experience (D'Amico 2006).
The USDA classifies occupational skills training into five separate components, but the specific
applicability to Wisconsin suggests that four categories are most useful, (1) short-term classroom-based
training, (2) online courses, (3) community college programs, and (4) apprenticeship programs. Data
surrounding these components show mixed results. Each component is examined below.
Classroom training can be provided by public or private institutions or third-party training
vendors. Research on classroom-based occupational training suggests that long-term benefits in terms of
increased employment and earnings are highly dependent on the employment sector in which the training
occurs. D’Amico’s study suggests that classroom training can be effective if the instruction focuses on
job-specific skills in highly sought-after fields (D'Amico 2006). Not surprisingly, studies have shown that
earnings decrease while the participant is enrolled in classroom training, as the hours worked necessarily
decline, but 18 to 24 months after program entry, earnings begin to increase (Heinrich, Mueser, et al.
2008).
Online courses must be addressed due to the increase in technology, and therefore increased
usage. Online courses appeal to participants who wish to continue working or who require increased
flexibility. Very few comprehensive studies have been performed regarding the long-term effects of
online learning as a component of SNAP E&T requirements. As such, a recommendation cannot be made
due to the lack of evidence supporting the efficacy of online education.
30

 Consistent with other forms of education and training components, community college
certificates can increase long-term earnings if earned in high-demand employment sectors. Few rigorous
studies have been performed to measure the effectiveness of for-credit occupational certificates, therefore
a recommendation cannot be made. While most reports indicate that completion of in-demand certificates
that align with employers’ needs may increase long-term earnings, rigorous causal studies have yet to be
completed (Kogan, Paprocki and Diaz 2016).
The U.S. Department of Labor defines the Registered Apprenticeship program as
“comprehensive, formal, workplace-based training supplemented by classroom training developed
through a close public-private partnership of government, firms, unions, and training institutions” (U.S.
Department of Labor 2014). Essentially, an apprenticeship is a connection between a person and an
employer that provides a blend of basic skills as well as educational and occupational training.
Apprenticeship programs are relatively less common in the United States than in Europe. They
exist mostly in trade or vocational employment, such as the plumbing, electrical, or construction
sectors. In Europe, apprenticeships are much more common, and indeed, several studies have shown
positive employment outcomes due to how well the programs address the labor market needs of specific
employers (Heinrich 2013).
While studies of European apprenticeship programs cannot be extrapolated and applied to
Wisconsin, the underlying results are similar to U.S.-based evaluations that show OJT and industryspecific training can yield long-term benefits.

Case Management and Supportive Services
Case managers, or the frontline employees who interact with SNAP E&T participants, often have
discretion when deciding which programs and components to apply to which participant. Case managers
also approve funds for support services, such as childcare and transportation. Therefore, any study that
aims to analyze the effectiveness of support services must also account for disparities between case
managers.
Working conditions of case managers can impact the assignment and delivery of services. Indeed,
one study found negative effects on employment and earnings for program participants based on case
management (Rangaranjan, Meckstroth and Novak 1998). Another study found that implementation of
case management by front-line employees was unsatisfactory due to the very large caseload assigned to
each caseworker (Holzer and Martinson 2006). The impact of case managers, like employees in other
industries, can be a function of workload and employee motivation. Individual case workers with an
overabundance of cases may move quickly to assign people to temp agencies just to complete a shortterm metric.
According to the USDA, childcare is an allowable supportive service expense for SNAP E&T,
but because of the high costs, relatively few states offer reimbursement of dependent-care expenses
(Kogan, Paprocki and Diaz 2016). In fiscal year 2013, only 10 states spent SNAP E&T money on
dependent care.
Consistent with most studies of SNAP E&T effectiveness, results depend on subsidy
recipients. Few experimental studies have been performed, but based on analysis of literature for existing
participants, childcare subsidies may be associated with an increase in earnings. Numerous nonexperimental studies have found a childcare subsidy may increase self-sufficiency of a single mother
living at the federal poverty level by 3.6 percent, increase the likelihood that a single mother enrolls in
courses at a school or university by 13 percentage points, and decrease the probability of ending
employment over the study period by 43 percent (in Illinois) (Herbst and Tekin 2011) (Lee, et al. 2004).
While these studies seem to indicate childcare programs are beneficial, no experimental studies have been
performed that could create a causal relationship between childcare subsidies and long-term benefits.
Transportation services can be anything from gas subsidies, rail cards, or bus passes provided
through SNAP E&T programs. In fiscal year 2013, 48 states provided “transportation and other”
31

 supportive services to program participants (Kogan, Paprocki and Diaz 2016). A frequent issue with
program participants is employment location relative to the person’s home. Employers may be spread
across suburbs while participants are in more urban areas. Public transportation outside urban
areas is often lacking, therefore the lack of transportation is often seen as a barrier to employment.
Despite the popularity of transportation as a funded, or partially funded component of SNAP E&T
programs, very few detailed analyses have been performed. A study by D’Amico found that access to
transportation to suburban areas did not increase employment or income for participants (D'Amico 2006).
Due to the lack of causal, rigorous literature concerning transportation supportive services, a
recommendation cannot be made.

Sectoral Strategies
Sectoral strategies focus on preparing people for work in growing, dynamic, and demanding
industries. Specific industries require more and more employees as sectors expand. Sectoral strategies are
collaborative and cooperative by nature. They involve various organizational entities, including unions,
community colleges, and nonprofit entities that work together to fill the needs of employers and program
participants. Sectoral strategies involve multiple components of SNAP E&T programs such as
occupational training, education, and job search assistance.
The Sectoral Employment Impact Study completed by Public/Private Ventures developed
an experiment to examine three programs across the United States, all of which trained low-income
people for employment in expanding sectors of their regional economies, ranging from computer
technology to medical billing. The study confirmed that over a two-year period after random assignment,
program participants were more likely to be employed than members of the control group, with
participants working an average of 1.3 months more over the course of a two-year period, while also
earning 18 percent more on average (Maguire, et al. 2010). These results show that sectoral strategies can
be very beneficial to low-income workers. Successful strategies involve a high level of cooperation and
collaboration among private companies, nonprofit organizations, and the government.

Impacts
Studies have shown that adult women benefit from SNAP E&T programs more than any other
demographic (Card, Kluve and Weber 2010) (D'Amico 2006). Female participants, as well as long-term
unemployed people tend to see larger program effects than other groups. According to one study, most of
the impacts of intervention programs for women were positive and sizeable (Card, Kluve and Weber
2010). Conversely, effects of program intervention on men are less noticeable, and positive gains are
more likely to decay after a few years, and ultimately disappear after around six years (D'Amico 2006).
Participants with barriers to employment, such as criminal records or health conditions, are more
difficult to measure. Indeed, such barriers can be a source of difficulty in obtaining permanent
employment. Studies have shown that of the programs specifically targeted to participants with barriers to
employment, few have proven successful in the long-term. The successful programs focused on providing
transitional, subsidized jobs, or a combination of unpaid work experience, job placement, and education
(Butler, et al. 2012). In this meta-study, Butler et al. noted that sustaining employment, rather than
gaining initial employment, may be the true barrier for many people. These findings suggest that retention
services may be especially important for those with barriers to employment.

32

 Appendix 2: FSET Components
Below is a matrix displaying each component of Wisconsin’s FSET program, as well as various
subcomponents that can be used to satisfy work requirements.
Table 2: FSET Components
Qualifying Component
Job Retention

FSET Activity
Job retention services

Job Search

Employment search
Career planning
Occupational testing
Workfare job search
Career advancement

Self-Employment

Self-employment

Educational And
Vocational Training

Literacy skills
Part-time student
Post-secondary ED. (2 years or less)
General education development
Job skills training
Job readiness/motivation
Adult basic education
English-as-a-second-language (ESL)
Vocational Adult Basic Education (ABE)

(Wisconsin Department of Health Services 2017)

Driver's education
Full-time student
Vocational literacy
Customized skills training
High sSchool equivalency diploma
Vocational ESL
Work Experience

W-2 (Co-enrollment)
Transitional employment (Co-enrollment)
Community service
Trial job (Co-enrollment)
Work experience/FSET
On-the-job training

Work Fare

Workfare

33

 Appendix 3: FSET Vendor Map
For purposes of the FSET program, Wisconsin is divided into 11 regions served by seven vendors. The
map in Figure 3 depicts the geographic boundaries.

Figure 4: FSET Vendor Map

34

 Appendix 4: Federal Nutrition Service Reporting Requirements
The Wisconsin Department of Health Services is required to report the following data regarding
the FoodShare Employment and Training program to the Federal Nutrition Service (FNS) under
Section 4022(a)(2) of the Agricultural Act of 2014 (U.S. House. 2014):
•
•
•
•
•

Participant characteristics at enrollment, including whether the participant is a
voluntary or mandatory participant, education level, able-bodied adult without
dependents (ABAWD) status, primary language, gender, and age.
Certain ABAWD-specific data points such as the average number of people who are
offered positions and participate in work programs, as well as the impacts of the
three-month time limit and 15% exemption policies on ABAWDs.
The number and percentage of participants who completed a training, educational,
work experience, or on-the-job training component.
The number and percentage of FSET participants and former participants who are in
unsubsidized employment during the second and fourth quarters after completion of
FSET participation.
The median quarterly earnings (based on total quarterly earnings data) of all E&T
participants and former participants who are in unsubsidized employment during the
second quarter after completion of participation in FSET.

This data must be submitted to the FNS no later than January 1 following the close of the federal
fiscal year (October 1 through September 30).

35

 Appendix 5: Survey
Administering a survey to program participants is a costly endeavor. The methods and
characteristics of the survey should be analyzed to determine the best form of delivery, collection, and
utilization (Center for Innovation in Research and Teaching 2018) (University of Wisconsin Survey
Center 2010). See below.
Table 3: Survey Methodology Matrix

Face-toFace

Advantages

Disadvantages

Tier

Highest cooperation
High quality responses
Follow-up questions

Most expensive and time
consuming
Subject to interviewer bias
No anonymity

Tier 1

Better response rates than mail
or internet
Must be kept short
Language barriers
Shorter data collection time and
Telephone
less expensive than face-to-face Not all participants are reachable
by phones
Can use software to coach
interviewer

Mail

Internet

Lower costs
Lower response rates
Smaller administrative team
Most likely need an incentive
Access to broad population
Slower data collection
Participants don’t need computer
Must have up-to-date addresses
skills
Not every respondent has a
Lowest cost
computer
Reduces time
Getting a representative sample
Preprogram software for
may be difficult
question flow
Need valid email addresses for
Easy data collection
each participant

Tier 2

Tier 2

Tier 3

Face-to-face interviews are, perhaps, the best form of data collection. However, they also are the
most time consuming and expensive. Surveys administered via the internet are the most widely available,
but they also are subject to the most barriers to participation. People must be technologically inclined,
have access to a computer, and preferably access to high-speed internet. Response rates are usually
highest for face-to-face interviews, but the rate is subject to influence by the interviewing organization.
Incentives may be offered for survey participation, provided such incentives are not large enough to
distort responses or the sample size. There is no agreed-upon minimum response rate, but the more
responses received, the more likely a statistically significant conclusion can be drawn (University of
Wisconsin Survey Center 2010). The UW Survey Center normally receives a 60 percent to 70 percent
response rate to mailed surveys and a 30 percent to 40 percent response rate for online surveys.

36