This is an exciting time for workforce development policy. There is a growing body of evidence around what works to ensure that people have the skills to succeed in the workforce, which can help put an end to the era of “train and pray,” as a former U.S. Secretary of Labor once put it. Independent evaluators at research organizations have assessed dozens of employment and training interventions and identified a number of effective evidence-based, results-driven models, many of which involve sector-based approaches. Successful interventions and strategies have achieved statistically significant and meaningful impacts on skills and education gains, reduced recidivism, employment rates, and wages, among others. The U.S. Department of Labor’s Clearinghouse for Labor Evaluation and Research (CLEAR) has collaborated with the U.S. Department of Health and Human Services’ Employment Strategies for Low-Income Adults Evidence Review (ESER) to systematically review evidence on workforce development for low-income adults. Reviews provide user-friendly summaries of outcomes and the strength of the evidence. Additionally, expanding on outcomes-based contracting tools developed and used by a growing number of local Workforce Development Boards (WBDs), the Workforce Innovation and Opportunity Act (WIOA) Pay-for-Performance (PfP) provisions allow workforce boards to link funding directly to priority outcomes and build evidence in real-time.
The Federal Government runs more than 40 workforce development programs spread across 14 agencies with annual spending of approximately $17 billion. Only a comprehensive system of services, from cradle to career, will achieve the outcomes needed for healthy, stable communities, and states should develop and implement combined workforce plans which coordinate their local, state, and federal workforce funds, including Supplemental Nutrition Assistance Program Employment & Training (SNAP E&T), Temporary Assistance for Needy Families (TANF), Perkins Career and Technical Education (CTE), the Social Services Block Grant, the Community Services Block Grants, and the Community Development Block Grant, to achieve the greatest impact.
WHAT THIS ROADMAP OFFERS
This roadmap provides guidance for state and local workforce development officials on how to improve outcomes for job seekers and employers by investing their federal, state, and local workforce dollars in evidence-based, results-driven solutions. Additionally, this roadmap describes strategies for prioritizing evidence-based, results-driven solutions when allocating WIOA funds and provides examples of how state and local government officials across the country have used these strategies in workforce development and related areas. And last but not least, this roadmap provides guidance on how to talk about efforts to drive taxpayer dollars toward evidence-based, results-driven workforce development solutions.
We Recommend These 7 Strategies
STRATEGIES FOR CONTRACTS AND GRANTS
1. Link funds directly to priority outcomes through performance-based contracts.
2. Offer flexible award sizes, time periods, and performance measures.
3. Award preference points to providers offering models with high or moderate causal evidence.
4. Set aside funding for contracts that pay directly for outcomes and providers offering models with high or moderate causal evidence.
STRATEGIES FOR BUILDING AND SUPPORTING EVIDENCE
5. Build evidence by promoting, conducting, and financing external evaluations.
6. Highlight effective strategies in state and local WIOA plans.
7. Establish a policy of transparency about funding and program performance.
You can contact Celeste Richie, Vice President of Workforce Development, at Celeste@results4america.org, to connect with Results for America for support in implementing these strategies.
Strategies for Contracts and Grants
1. Link funds directly to priority outcomes through performance-based contracts
- Include outcomes-based payments in contracts using performance-based contracting authority in the Federal Acquisition Regulations, which cover all federal contracts, as well as WIOA’s Pay for Performance (PfP) authority.
- Boost the capacity of localities that are interested in outcomes contracting, by using state WIOA funds for outcomes-oriented technical assistance and evaluations.
Rationale: Outcomes-based contracts help focus government and providers on what really matters — delivering results for the people they serve. By contrast, traditional cost-reimbursement contracts pay the same amount for the services provided, regardless of whether they actually help people gain skills, get jobs, or earn higher wages. Performance-based contracts help to build evidence and improve programs in an ongoing way. Providers are afforded the flexibility to serve participants in customizable ways, less hindered by prescriptive cost-reimbursement contract terms. Government partners build more muscle around managing contracts with real-time data while having the assurance of making full payment only if priority outcomes are met.
By realigning funding to reward quality over quantity, outcomes-based contracting is also an important tool in improving equity throughout the workforce development system. In addition to standard WIOA performance metrics, state and local workforce boards have the opportunity to collaborate with their communities to identify priorities and refine performance measures to focus on achieving positive impact for populations that have been historically under-served or where culturally competent services are not available. Traditional workforce metrics often focus on short-term results resulting in churning job seekers quickly in and out of low-quality or poor-fit jobs and incentivizing serving individuals who face fewer barriers to employment. Performance measures should be disaggregated and examined through an equity lens to ensure that there are not unintended consequences, such as serving those participants who are easiest to serve, often called “creaming.” Some WDBs, for example, offer higher payments for achieving outcomes for harder to reach populations and longer timelines when working with populations that may need basic education and job training.
Outcomes-based contracts often appeal to providers offering models with high or moderate causal evidence as defined by the U.S. Department of Labor (DOL), because they are confident that they can deliver results and have the research to prove it. At the same time, this approach still holds them accountable for delivering those results. Offering outcomes-based contracts also can attract additional providers, particularly those who rely on comprehensive evidence-based models where the focus is on achieving results, instead of engaging in the time-intensive cost reimbursement process.
The U.S. Department of Labor has actively promoted strengthening outcomes-based approaches, and in the final 2016 WIOA regulations DOL encouraged local areas “to refocus these traditional performance-based contracts to place an emphasis on the contractor achieving outcomes like participants obtaining and retaining good jobs, rather than outputs like the number of people served.” Pew’s Results First Initiative emphasized the need to examine “whether a program meets the needs of a particular community or has been proved effective in achieving the desired outcomes,” in its recent guidance on “How Evidence Can Inform Contracting for State, Local Governments.”
Some state and local Workforce Development Boards (WDBs) may want to consider using the PfP authority created in 2014, which DOL has described as “one of several innovative strategies WIOA adopts to place a higher emphasis on performance outcomes and provider accountability, drive better results, and incorporate rigorous evaluation and evidence-based practice into the delivery of workforce services.” A type of outcomes-contracting, PfP allows 10% of local WIOA funds to be used for outcomes payments, and the funds do not expire, allowing for longer-term outcomes. Even if a provider fails to reach the outcomes and receive payment, the WDB can retain the unexpended funds by reallocating them to another PfP-related activity. U.S. DOL’s Employment and Training Administration released a Pay-for-Performance template to help WDBs think through the components necessary for this type of outcomes contract and is working on additional guidance to help states properly account for no-year funds. State auditors and financial officers should be part of the planning team as well when beginning a PfP project.
States can make it possible for local boards to enter into these innovative arrangements by offering to support the development of key elements, such as a “rate card” that shows how much to pay for each outcome and providing access to state administrative data to support long-term outcomes measurement. WIOA expressly permits states to use their share of federal WIOA funds for technical assistance to local boards on PfP contracting, as well as to cover evaluation costs. States can also use non-federal funds to incentivize local boards to implement PfP contracting. Local WDBs will also need support from their state workforce agencies, state workforce boards, and governor’s offices in adjusting performance measures to reflect the time and cost necessary to achieve higher-bar outcomes or to focus on populations that had been previously underserved. The ability to renegotiate performance measures traditionally happens every one to two years, so it is important to start these conversations early-on in the process.
Outcomes-based contracts (not using WIOA Pay for Performance authority)
- San Francisco’s Office of Economic and Workforce Development currently offers WIOA job centers a 10% performance-based payment to encourage placement of individuals with barriers to employment into jobs paying above the minimum wage.
- The Memphis Workforce Investment Network used WIOA funds in 2018 to award an outcomes-based contract for transitional jobs services to the Center for Employment Opportunities (CEO), which offers an evidence-based model for reentry employment.
- New York State has used outcomes-based contracts in its SNAP E&T program for many years. Most recently, the state issued an RFP in 2016 specifying a series of payments linked to such milestones as educational gains, program completion, job entry, and job retention.
- The 2017 Ohio State WIOA Plan highlights the Ohio Department of Higher Education’s use of a “performance-based funding formula” for approved Workforce Readiness Education providers.
Outcomes-based contracts (using WIOA Pay-for-Performance Authority)
- The Northern Virginia Workforce Development Board’s SkillSource Group launched a contract in 2018, providing bonus payments, through WIOA PfP, to providers who achieve certain milestones in programs serving foster care and justice-involved young adults.
- The San Diego Workforce Partnership used the PfP authority to reward improved employment, education, and recidivism outcomes by providers serving justice-involved young adults.
- Nevada’s Governor’s Office of Workforce Innovation released a 2018 Work-Based Learning Pay-For-Performance Grants RFP that combined evidence-based strategy requirements with PfP.
2. Offer flexible award sizes, time periods, and performance measures
- Be flexible on award size, so that providers offering evidence-based workforce development models have sufficient funding to finance the effective services they provide.
- Award contracts to evidence-based providers for as long a time period as possible and offer extension opportunities to providers meeting interim outcome goals.
- Provide greater flexibility on performance measures, looking beyond near-term employment and earnings to other outcomes of interest.
Rationale: Being flexible on award size can reduce the barriers to entry for evidence-based providers newly seeking to serve a community. Knowing that there will be adequate funding available on a per-participant basis and in the aggregate can make it possible for them to establish the necessary staffing and other program infrastructure (including the cost of program evaluation, where applicable) in a new location. Achieving long-term, high-bar outcomes can also be more expensive, so the award must be adequate to allow these providers to offer more comprehensive and customized services. Where total funding is not flexible or is uncertain, reducing the total participants served may be necessary to achieving the desired outcomes.
When determining contract length, workforce boards and agencies should consider length of service delivery, follow-up, and evaluation. Making contracts available for longer time periods will be more workable for providers offering evidence-based models, which often produce their most impressive results over a multi-year time horizon. Providers also have time to respond to program performance data and course correct as needed. Extending contracts beyond service delivery and payments allows for continued engagement over a longer follow-up period as outcome data is collected.
Flexibility on performance measures is also important. Accountability for outcomes is critical, but workforce boards and agencies should consider a broader range of outcomes than the narrow WIOA measures when assessing the benefits of funding an evidence-based model. State and local boards are free to use additional performance measures, and under WIOA, state performance targets must be statistically adjusted to account for serving harder to serve participants. Additionally, states are required to consider local participant characteristics and economic conditions in setting local targets. State and local workforce officials should feel free to focus on the people who most need services and measure their outcomes in a comprehensive way, over a longer period of time. While non-traditional, useful workforce measures might include reduced recidivism or stable housing.
- The New Orleans Workforce Development Board’s 2019 RFP leverages a combination of cost-reimbursement and bonus payments to achieve outcomes for opportunity youth.
- The Pennsylvania Department of Corrections currently gives providers that meet recidivism prevention goals a 1% increase in their payment rate while providers that fail to meet targets for two consecutive years can have their contracts terminated.
- The Texas Workforce Commission currently focuses on customer outcomes instead of program processes by collaboratively selecting performance measures with each local board to focus on creating integrated services.
3. Award preference points to providers offering strategies or models with high or moderate causal evidence
- Restructure RFPs to give preference to proposals that rely on strategies and models with high or moderate causal evidence, as defined by the U.S. Department of Labor, by awarding additional points when scoring the proposals.
Rationale: State and local workforce agencies and boards have a simple but powerful mechanism to ensure that providers with high or moderate causal evidence have the funds they need to scale their efforts — using competitive procurement or grantmaking processes to incentivize the adoption of models with evidence of effectiveness. For example, WDBs can give preference points to those applicants who propose strategies with high or moderate causal evidence of achieving significant gains for priority outcomes. This type of incentive encourages evidence-based providers to pursue the opportunity to serve a particular community, since the preference points make it more likely they will succeed in securing funding. Second, these mechanisms signal to existing providers in a community that they need to pay closer attention to the research on effective models. Some provider communities may need additional guidance and training in how to leverage evidence-based solutions.
While there is limited evidence in many areas of workforce development, one helpful resource for assessing the strength of existing evidence behind a proposed approach is the Clearinghouse for Labor Evaluation and Review (CLEAR) database maintained by the Chief Evaluation Officer at the U.S. Department of Labor. CLEAR assigns one of three evidence ratings to the studies in its database:
(1) High Causal Evidence, which “means there is strong evidence that the effects estimated … are solely attributable to the intervention being examined” and currently only includes “well-implemented randomized controlled trials”;
(2) Moderate Causal Evidence, which “means there is evidence that the effects estimated in the study are attributable at least in part to the intervention being examined”; and
(3) Low Causal Evidence, which “means there is little evidence that the effects estimated in the study are attributable to the intervention being examined, and other factors are likely to have contributed to the results.”
- The U.S. Department of Labor awarded a competitive Pay-for-Success (PFS) grant in 2013 to scale the Center for Employment Opportunities (CEO) evidence-based transitional jobs program.
- The Corporation for National and Community Service has reformed the application process for its AmeriCorps State and National Grants Program, and currently awards up to 16 points out of 100 to applicants based on their evidence of effectiveness.
- The U.S. Department of Education requires applicants for all five of its largest competitive grant programs to use evidence-based practices, uses a common evidence framework to make these decisions, and promotes constant use of evidence-based practices through their What Works Clearinghouse.
- The federal Juvenile Justice Reform Act of 2018 requires states making subgrants through the competitive Youth Promise Grant to give priority to applicants using evidence-based strategies.
4. Set aside funding for contracts that pay directly for outcomes and providers offering models with high or moderate causal evidence
- Set aside a percentage of the local WIOA budget for use only with evidence-based strategies and outcomes-based contracts.
- Use a share of the governor’s 15% set-aside in WIOA, or other resources, to jumpstart investment in evidence-based models.
Rationale: State and local workforce development agencies and boards can help make sure that dollars are invested in priority outcomes and workforce development strategies with high or moderate causal evidence by dedicating funds to this purpose. The U.S. Department of Labor’s CLEAR database is a useful resource for assessing the level of evidence behind a proposed intervention.
States can play a leadership role by using some of their 15% discretionary set-aside in WIOA or other resources to jumpstart investment in models with high or moderate causal evidence levels and incentivize local boards to follow suit. For example, states could provide matching funds, or offer to pay for the first year of a contract with an evidence-based provider, with the local board picking up the costs thereafter.
- The U.S. Department of Labor currently requires a phased evidence approach for Reemployment Services and Eligibility Assessment (RESEA) Grants, encouraging the use of evidence-based strategies where they exist and requiring evaluations to build evidence in places where needed.
- The Oregon Department of Corrections, the Oregon Youth Authority, the Oregon Youth Development Division, and the Oregon Health Authority are currently required to spend at least 75 percent of their state funding on evidence-based programs.
- The Nevada Department of Education re-designed its federal Every Student Succeeds Act Title I school improvement sub-granting process to award funds to local school districts based on the extent to which their plan to improve their lowest performing schools called for investments in evidence-based solutions.
- Substance Abuse and Mental Health Services Administration’s Mental Health Block Grant includes a 10 percent set aside to support evidence-based programs that address the needs of individuals with serious mental illness, including psychotic disorders.
Strategies for Building and Supporting Evidence
5. Build evidence by promoting and financing external evaluations of models without high or moderate causal evidence
- Require workforce development providers to participate in an external evaluation, if asked, as a condition of receiving WIOA funding.
- Allocate a share of WIOA funds to support external evaluations of workforce development models that do not yet have high or moderate levels of evidence.
Rationale: Another way to support evidence-based policy is to help build the evidence about what works. This could be a good first step in smaller communities that have limited programming to offer their residents and want to learn more about the effectiveness of the services they are currently providing. Building rigorous evaluation into RFPs and contracts can help existing providers learn and improve.
At a minimum, state and local workforce agencies and boards can require providers to participate in an external evaluation as a condition of receiving WIOA funding. Even if funding for that evaluation is not available right away, this requirement sends a signal about the importance of evidence and gets the provider thinking about what it would take to undergo a rigorous evaluation.
State and local agencies and boards should go further, however, and use a share of their WIOA funds to finance evaluations for core programming. Indeed, Section 116(e) of WIOA specifically calls for states and local boards to “conduct ongoing evaluations … in order to promote, establish, implement, and utilize methods for continuously improving core program activities in order to achieve high-level performance within, and high-level outcomes from, the workforce development system.” Additionally, by investing in State Longitudinal Data Systems and enabling data sharing with local workforce agencies states can support low-cost evaluations using administrative data.
- The U.S. Department of Labor includes rigorous evaluation requirements in its five largest competitive grant programs and requires all grantees to agree to participate, if selected, in an evaluation.
- Arnold Ventures provides grants on a regular basis to local and state governments to evaluate targeted social services programs including workforce development programs. Here’s a list of their recent low-cost evaluations using administrative data, including reemployment services, sector-based training programs, and cognitive behavioral therapy in conjunction with job search services.
6. Highlight effective strategies in state and local plans
- Include language in the state WIOA plan that encourages local WDBs and all other public agencies that manage workforce funds to employ these effective strategies for building and acting on evidence in contracting, allocating funds, and designing RFPs detailed above.
- Develop a Combined State Plan that links WIOA funding, programming, and outcomes with other federal programs and funding streams, including SNAP E&T, TANF, and Perkins CTE, among others.
- Highlight evidence-based, results-driven workforce development strategies that could help achieve state and local goals as part of the next state or local WIOA plan update.
Rationale: The WIOA state plan provides direction to the entire state about how to approach workforce development policy, which makes it an ideal place to present and promote specific strategies for pursuing evidence-based solutions. It puts all of these ideas in front of local WDBs, as well as other agencies that manage workforce funding, and gives them concrete suggestions for how to improve outcomes for the residents they serve. Combined plans allow states to create seamless systems of workforce services and leverage federal and state dollars to create the most impact. For example, TANF funding could be used for initial job search services with WIOA dollars coming into play when more extensive training is needed.
States can play a leadership role in advancing evidence-based policy by identifying results-driven strategies for their local workforce boards to consider. States can help local agencies and boards stay on top of research and practice developments by providing information efficiently through the state plan, explaining the data and evidence behind the most successful strategies. This tactic is also consistent with the official guidance from the U.S. Department of Labor, which encourages states to reflect in their plans “a robust analysis” of the state’s “use of evidence-based practices.” Moreover, because WIOA allows states to submit combined plans across workforce funding streams, the influence of these signals can extend to other programs, such as TANF, SNAP E&T, and Career and Technical Education. Similarly, WDBs can signal in their local plans to the community of providers that future RFPs will preference evidence-based strategies and outcomes-based contracts.
Employers can be a helpful resource in identifying models with high or moderate causal evidence (as defined by DOL in the CLEAR database) that might work well in a particular state or locality, as can labor unions and other advocates for workers. A good place to start would be with employers who operate across state lines and have been partnering with a provider offering an evidence-based model in another area.
- Pennsylvania’s 2018 State WIOA Plan identified the Center for Employment Opportunities as an example of “innovative employment and training programs that achieve successful employment and recidivism outcomes.”
- 2018 State WIOA Plans in California, Hawaii, Ohio, and Pennsylvania have referenced exploring or using the PfP provisions in WIOA to improve outcomes.
- Colorado’s 2018 State WIOA Plan identifies six specific strategies as part of a long-term plan to develop an evidence-based workforce development system.
- The Denver Workforce Development Board’s 2016-2020 Strategic Action Plan included commitments to “develop methodology for incorporating pay for performance elements in workforce service provider contracts,” and to “review and approve payment of pay for performance elements of workforce service provider contracts.”
- District of Columbia’s 2018 WIOA Plan set a goal of directing funding to evidence-based services, accompanied by a metric for the “percentage of evidence-based programs in use in the system.”
- At least a dozen states currently have combined workforce plans which describe how they are coordinating their workforce efforts funded by WIOA, TANF, Perkins CTE, SNAP E&T, and/or other state and federal workforce funding streams to improve coordination and achieve greater impact.
- Employers and Chambers of Commerce are useful partners in identifying effective program models.
7. Establish a policy of transparency about funding and program performance
- Make information about workforce funding easily available to the public online.
- Make workforce program performance information easily available to the public and link data across public service systems.
Rationale: There are two types of transparency that state and local workforce agencies and boards can offer to help drive resources to providers of evidence-based models.
First, making information about the flow of workforce funds — agency/board budgets, plans, RFIs, RFPs, and awarded contracts — easily available to the public online would help providers and employers in and outside the community identify potential opportunities to fund evidence-based models. Sharing RFP and contract language also allows for sister agencies to easily adapt and revise their own procurement documents without reinventing the wheel each time.
States and localities should explore methods for compiling information across the multiple workforce development streams, so it is easier for the public to get a sense of what federal, state, and local funding is available and how it is being used.
Second, making accurate, timely information about workforce program performance easily available to the public online will help individuals seeking services select the program that will work best for them, and thereby drive more resources toward the higher-performing, evidence-based programs. Performance data should be presented in a user-friendly manner and broken down by training provider and program of study. This level of transparency is already required by WIOA regulations, but this requirement has not yet been fully implemented or enforced by the U.S. Department of Labor.
Flow of funds
- The State of Tennessee currently provides detailed, searchable information about payments by agency to individual vendors through the “Checkbook” feature of its Transparent Tennessee online database.
- The Rhode Island Department of Children, Youth, and Families issued an RFI in 2017 prior to recompeting a set of neglect & abuse prevention contracts, which included specific evidence-oriented questions.
Washington State’s Workforce Training and Education Coordinating Board currently operates a “Career Bridge” system that provides public information about hundreds of education and training programs, some of which are WIOA funded.
How to Talk About Evidence-based Policymaking in Workforce Development
We invest in workforce development not to fund programs, but to improve lives. Too often that message gets lost, which can make it a struggle to secure adequate funding despite the vast need. That message can also get lost when workforce officials try to do the hard work of adjusting performance targets to serve fewer people to achieve long-term, high-bar outcomes, or shifting funds away from underperforming providers who have been operating programs in a community for a long time.
One way to build support for investing in evidence-based models is to focus on the broad set of outcomes they can deliver. The best programs don’t just churn people back into a job in the short term. They open up new opportunities that can change a person’s life — to get onto a career pathway, to stay out of prison, to go to college, to support a healthy and happy family. Those are outcomes policymakers want to invest in and community members value, even if it means some changes in who gets funding.
Evidence-based, results-driven workforce development programs can also accomplish multiple goals.
- Governors interested in reducing recidivism should look not just within the criminal justice system for solutions, but also explore evidence-based workforce models that can deliver those results.
- Governors seeking to reduce hunger can conduct a SNAP outreach campaign to unemployed workers, signing people up not only for food assistance but also for evidence-based workforce services financed by SNAP E&T.
- State and local public agencies looking to hire skilled workers — whether in information technology or infrastructure-related projects — can partner with their state and local workforce agencies and boards to offer evidence-based training.
- A mayor or county executive looking to attract an employer as part of an economic development strategy should consider investing WIOA funds in evidence-based workforce development programs, which will help assure the employer that residents will be well-prepared for these new jobs.
Evidence-based workforce development programs have the power not only to change individual lives, but also to improve outcomes for an entire community.
Appendix A: Where to Learn More About Evidence-based Practices
Here are additional resources for state and local workforce agency and board officials interested in exploring how to direct resources toward evidence-based strategies.
- The Clearinghouse for Labor Evaluation and Research (CLEAR) is a database of program evaluations maintained by the U.S. Department of Labor (DOL). CLEAR also defines the three levels of evidence of effectiveness used by DOL.
- MDRC — a leader in the program evaluation field — has summarized the evidence on what works in workforce development in a short brief called “The Power of Career- and Employer-Focused Training and Education.”
- The National Association of State Workforce Agencies’ (NASWA) “Evidence-Building Capacity in State Workforce Agencies” report summarizes the research and analysis capacity of 41 member state workforce agencies as of 2017.
- Results for America’s “2018 Federal Standard of Excellence: Use of Evidence in Five Largest Competitive Grant Programs” provides detailed descriptions of how leading federal agencies are prioritizing evidence of effectiveness in competitive grant programs.
- Results for America’s “2018 Federal Standard of Excellence: Use of Evidence in Five Largest Non-Competitive Grant Programs” provides detailed descriptions of how leading federal agencies are prioritizing evidence of effectiveness in formula grant programs.
- Results for America’s “2018 Invest In What Works State Standard of Excellence” sets a national standard for how state governments can consistently and effectively use data and evidence in budget, policy, and management decisions to achieve better outcomes for their residents.
Appendix B: Federal, State, and Local Examples of Evidence- and Outcomes-based Workforce Efforts
Here are examples of federal, state and local efforts pushing for evidence-based strategies and outcomes-based contracting and grants in workforce and related programs.
- The U.S. Department of Labor awarded a competitive Pay-for-Success (PFS) grant with a maximum payout of $21.6 million to scale the Center for Employment Opportunities (CEO) evidence-based transitional jobs program. The outcome payments are linked to the number of jail/prison bed-days, engagement in transitional job, and increases in employment.
- The U.S. Department of Labor’s 2019 guidance on Unemployment Insurance (UI) Reemployment Services and Eligibility Assessment (RESEA) Grants, includes a new requirement for a tiered-evidence approach, encouraging “the Department to fund and states to use evidence-based strategies where they exist and to conduct evaluations and build evidence in places where needed. The goal is to ensure that each state employs RESEA interventions and service delivery strategies that, based on rigorous evaluations, improve employment outcomes and reduce benefit duration or that the intervention or service delivery strategy are being evaluated to determine their effectiveness in achieving these goals” (Section 306 of the Social Security Act).
- The U.S. Department of Labor includes rigorous evaluation requirements in its five largest competitive grant programs, involving: (1) full participation in a national evaluation as a condition of grant receipt, and/or (2) an independent third-party local or grantee evaluation with priority incentives for rigorous designs.
- The Corporation for National and Community Service (CNCS) has reformed the application process for its AmeriCorps State and National Grants Program, and currently sets aside up to 16 points out of 100 to be awarded to applicants based on their evidence of effectiveness. Over time, this preference has raised the share of funding going to projects with strong evidence, such as the Minnesota Reading Corps, whose model has been shown to improve reading scores. A randomized controlled trial evaluation of the Minnesota Reading Corps found that the average kindergarten student with an AmeriCorps tutor performed twice as well as students without one. It also found that tutors helped the average first grade student perform 26 percent better than the expected level for on-track students. This commitment to evidence-based policymaking has continued across Administrations; CNCS’s CEO, newly appointed in 2018, identified “prioritiz[ing] evidence-based interventions” as a major goal for the agency. Indeed, in its latest AmeriCorps competition, CNCS added a funding priority for applicants who proposed using one of 13 pre-approved evidence-based education interventions.
- The U.S. Department of Education requires applicants for all five of its largest competitive grant programs to use evidence-based practices, uses a common evidence framework to make these decisions, and promotes constant use of evidence-based practices through a robust What Works Clearinghouse. For example, in 2017, the Department used a competitive preference priority for projects based on moderate evidence of effectiveness for state and partnership grants (approximately $70 million in new awards) in the GEAR UP program, which seeks to increase the number of low-income students prepared to enter and succeed in postsecondary education.
- The federal Juvenile Justice Reform Act of 2018 requires states making subgrants through the competitive Youth Promise Grant program for delinquency prevention to “give priority to eligible entities that demonstrate ability in … employing evidence-based prevention strategies,” among other factors. It also requires state formula funds, totaling $60 million in FY19, to be used “with priority in funding given to entities meeting the criteria for evidence-based or promising programs.”
- For several years now, including in the most recent appropriations bill, Congress has set aside 10% of funding in the Mental Health Block Grant administered by the Substance Abuse and Mental Health Services Administration (SAMHSA) to support evidence-based programs that address the needs of individuals with serious mental illness, including psychotic disorders.
Employers and Chambers of Commerce
- Several non-profits operating evidence-based models have strong partnerships with employers located in more than one state, including Year Up, Per Scholas, the Center for Employment Opportunities, and Urban Alliance. Companies that have experience with these models are often eager to spread the news about their effectiveness. For example, a Bank of America executive has called Year Up “a strategic source of talent for Bank of America” and “a role model for preparing opportunity youth for first time professional employment.”
- The U.S. Chamber of Commerce also highlighted Year Up‘s evaluation results for its members and featured the Urban Alliance at public events.
- In 2016, the following states had combined workforce plans across WIOA, Perkins CTE, SNAP E&T, and/or TANF:
- Arkansas: TANF, SNAP E&T
- Maryland: Perkins CTE, TANF, SNAP E&T
- Minnesota: Perkins CTE, TANF, SNAP E&T
- Mississippi: TANF
- Ohio Perkins: CTE
- Pennsylvania: Perkins CTE
- Rhode Island: TANF
- Tennessee: TANF, SNAP E&T
- Washington: TANF, SNAP E&T
- Virginia: Perkins CTE, TANF, SNAP E&T
- The San Diego Workforce Partnership (SDWP) used the PfP authority to reward improved employment, education, and recidivism outcomes by providers serving justice-involved young adults. Additionally, SDWP is the first workforce board to leverage income sharing agreements which offer access to education without student loans, interest rates and decades of repayments. A fund fronts the cost of a student’s education, which the student pays back in payments based on a percentage of income over a set period of time, but only if the student begins earning more than $40,000 per year.
- San Francisco’s Office of Economic and Workforce Development restructured its contracts for WIOA-funded American Job Centers, with help from the Government Performance Lab at Harvard’s Kennedy School, to offer a 10% performance-based payment to encourage placement of individuals with barriers to employment into jobs paying above the minimum wage. The agency is now considering adding performance incentives to other provider contracts. The language in the RFP explains that the agency “will provide a base cost-reimbursement amount for start-up, operating expenses, training and supportive services. 90% of the grant budget will be offered as the base reimbursement amount. The remaining 10% of the grant amount will be set aside for performance-based grant achievements. Post-procurement, OEWD will negotiate with the grantee(s) the detailed amounts allocated for each of the performance-based grant achievements and the performance metrics that will trigger the incentive payments.”
- While Colorado acknowledged in their 2018 WIOA state plan that “there are a limited number of evidence-based practices in the field of workforce development, and the ones that currently exist may be difficult to implement with fidelity in Colorado,” they committed to a long-term plan to develop an evidence-based workforce development system. Strategies include: “1) Review existing program evaluation tools and processes; 2) Work with partner agencies to determine whether additional resources are required, and develop additional methods to evaluate existing data; 3) Develop policies and guidance on how to identify and evaluate potential promising practices, and use evaluation as part of process improvement; 4) Require the use of logic models and outcome measures as part of the development of new projects and proposals; 5) Share information regarding promising practices identified through these activities with the appropriate state or federal entity for further evaluation to determine whether it might be an evidence-based practice; and 6) Work with the Colorado Workforce Development Council, local workforce development boards and state partner agencies to develop and use quasi-research methods to evaluate the effectiveness of core programs and new initiatives.”
- The Denver Workforce Development Board’s 2016-2020 Strategic Action Plan included commitments to “develop methodology for incorporating pay for performance elements in workforce service provider contracts,” and to “review and approve payment of pay for performance elements of workforce service provider contracts.” As a public document, this plan serves a similar purpose in signaling to the marketplace that providers should ready themselves to compete for PfP contracts.
District of Columbia
- The 2018 state WIOA plan submitted by the Mayor of the District of Columbia set a goal of directing funding to evidence-based services, accompanied by a metric for the “percentage of evidence-based programs in use in the system.” While the plan does not specifically name the models, it sets in motion a process for identifying evidence-based programs.
- The Pasco-Hernando Workforce Board in Florida is taking a step towards outcomes-based contracting in a way that allows all parties to adjust to the learning curve. They issued an RFP that combined cost-reimbursement with “a significant portion of the funds (a minimum of 10%) withheld until achievement of measurable performance outcomes has been documented.”
- Florida’s PK-20 Education Data Warehouse provides public access to data linked across PK-12, university, and workforce outcomes. This comprehensive longitudinal data system supports the State’s WIOA plan commitments to evidence-based practices and continuous improvement.
- A number of state plans have referenced exploring or using the PfP provisions in WIOA to improve outcomes, including 2018 plans in California, Hawaii, Ohio, and Pennsylvania. In Hawaii, for example, the state has pledged to provide “technical assistance to local areas that are implementing WIOA Pay-for-Performance contract strategies and conducting evaluations of such strategies,” subject to funding availability.
- The New Orleans Workforce Development Board partnered with the New Orleans Business Alliance to support the expansion of workforce services and employer engagement through private investments focused on outcomes outside of the traditional WIOA performance metrics. Their 2019 RFP leverages a combination of cost-reimbursement and bonus payments to achieve outcomes for opportunity youth such as continuous employment for 6 months with no more than 2 job changes, no arrests resulting in convictions, and at least $400 in personal savings per participant by program end.
- Nevada’s Governor’s Office of Workforce Innovation (OWINN) used the Governor’s WIOA set-aside funding to release a 2018 Work-Based Learning Pay-For-Performance Grants RFP that combined evidence-based strategy requirements with PfP. Nonprofits or employers were eligible to apply for the funding to implement specific work-based learning programs that have been shown to be impactful, including paid internships/co-operatives, on-the-job-learning, and pre-apprenticeship and registered apprenticeship programs. OWINN included PfP as a way to “develop stronger contracts linked to outcomes and ensure more effective use of tax dollars.” The contract will allow for a base level of funding depending on the difficulty of the population being served, and the remainder of the funds would only be paid out once agreed-upon outcomes are reached.
- The Nevada Department of Education re-designed its federal Every Student Succeeds Act (ESSA) Title I school improvement sub-granting process to award funds to local school districts based on the extent to which their plan to improve their lowest performing schools called for investments in evidence-based solutions. More specifically, the new NV State Education Agency Title I grant application requires school districts to include proposed interventions backed by Tier 1, 2 or 3 evidence, as defined by ESSA. As of January 2018, 100% of the state’s federal school improvement funds are being awarded to local school districts for investments in evidence-based solutions, up from just 15% three years ago.
- The State of New York has used outcomes-based contracts in its SNAP E&T program for many years. Most recently, the state issued an RFP in 2016 specifying a series of payments linked to such milestones as educational gains, program completion, job entry, and job retention. The RFP offered 5-year contracts for up to $400,000 annually per organization for this program, which it calls the SNAP Employment and Training Venture Program. A mix of public and private providers currently offer services across the state, including two evidence-based model providers, the Center for Employment Opportunities and Per Scholas, who are among several non-profits serving New York City.
- Capital Area Workforce Development in Raleigh, NC released a full table of PfP measures and associated payments in their 2018 RFP. For adults and dislocated workers, for example, a provider would be paid based on the number of participants who find employment each quarter divided by the agreed upon target. For example, if the provider achieves only 75% of their goal, they only receive 75% of their performance payment.
- In 2017, Ohio’s State WIOA Plan highlighted the Ohio Department of Higher Education (ODHE) Aspire’s use of a “performance-based funding formula” for approved Workforce Readiness Education providers. Grants are awarded in a three year cycle, and “applicants must provide statistical evidence of program effectiveness for the prior three years related to successful student outcomes of educational gains, obtainment of high school equivalence diploma, and transitions to postsecondary education and training and employment. Performance is measured annually and funding is increased or decreased based on performance against established benchmarks.
- Their plan also highlights their efforts to share and link data, recognizing that “evaluation and research projects on activities under the WIOA core programs first requires access to relevant data.” Specifically, Ohio leveraged a grant from the Workforce Data Quality Initiative and the creation of the Ohio Longitudinal Data Archive to support “policy analysis, research and program transparency (dashboards).”
- An Oregon law states that the Oregon Department of Corrections, the Oregon Youth Authority, the Oregon Youth Development Division, and “the part of the Oregon Health Authority that deals with mental health and addiction issues” shall “spend at least 75 percent of state moneys that the agency receives for programs on evidence-based programs.” Oregon law defines an evidence-based program as “a program that: (a) Incorporates significant and relevant practices based on scientifically based research; and (b) Is cost-effective,” and further defines scientifically based research as “research that obtains reliable and valid knowledge by: (a) Employing systematic, empirical methods that draw on observation or experiment; (b) Involving rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; (c) Relying on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations and across studies by the same or different investigators; and (d) Utilizing randomized controlled trials when possible and appropriate.”
- The 2018 state WIOA plan submitted by the governor of Pennsylvania identified the Center for Employment Opportunities as an example of “innovative employment and training programs that achieve successful employment and recidivism outcomes,” noting that CEO had recently begun providing services in Philadelphia. The plan went on to explain that “[i]n 2012, CEO released results on its model program via a random assignment evaluation and achieved a statistically significant 22 percent reduction in recidivism for high risk individuals as well as increased employment placements.” This language in the state plan has encouraged other localities to take a look at CEO’s model.
- The Pennsylvania Department of Corrections began using performance-based contracting in 2013, setting performance targets for its community corrections program. Providers that meet recidivism prevention goals receive a 1% increase in their rate while providers that fail to meet targets for two consecutive years can have their contracts terminated. In 2014–2015, the program’s recidivism rate dropped by 11.3%.
- The Rhode Island Department of Children, Youth, and Families issued a 2017 RFI prior to recompeting a set of neglect & abuse prevention contracts, which included specific evidence-oriented questions such as “How can the [agency] use data and other methods to track the implementation and effectiveness of prevention efforts?” and how should the agency “ensure model fidelity (if appropriate) and hold [providers] accountable for results?”
- The Memphis Workforce Investment Network used WIOA funds to award an outcomes-based contract for transitional jobs services to the Center for Employment Opportunities (CEO), which offers an evidence-based model for reentry employment. The 2018 contract, based on an RFP issued in 2017, pays for meeting each of three specific thresholds: (1) participant enrolled, entered into the system, and confirmed to be eligible (payment: 40% of the $10,000 per participant cost, up to a total of $1,000,000); (2) participant completed transitional employment (40%); and (3) participant successfully placed in unsubsidized employment, as proven with two pay stubs (20%). The contractor is required to invoice on a monthly basis for participants meeting the thresholds. Memphis and CEO will then be able to track outcomes for these participants for at least four quarters using WIOA data and better understand the link between the outcomes payments and longer-term impacts.
- The state of Tennessee provides detailed, searchable information about payments by agency to individual vendors through the “Checkbook” feature of its Transparent Tennessee online database. The public can see the payments made each month by an agency — including the state Department of Labor & Workforce Development — listed by purpose, vendor name, and amount.
- In an effort to focus on customer outcomes instead of program processes, the Texas’ Workforce Commission (TWC) successfully “obtained a performance measure waiver” from DOL that gave them the flexibility to collaboratively select performance measures with each local board that focus on creating integrated services and breaking down silos. Additionally, The Workforce Information System of Texas (TWIST) links data across workforce funding streams for intake, eligibility determination, assessment, service tracking, and reporting of TWC-administered programs, such as subsidized child care, Supplemental Nutrition Assistance Program (SNAP) Employment and Training, TANF/Choices employment services, and the Workforce Innovation and Opportunity Act.
- With this support from the State, Austin’s Workforce Solutions Capital Area is working to design and execute “an outcomes rate card that targets educational and employment outcomes in Austin and Travis County.” The rate card will include a menu of priority outcomes and associated values/performance payments, allowing for standardized outcomes-based contracting. Services will focus on residents of low-to-moderate income to improve limited education and language skills, credential completion, job placement, retention, and earnings.
- A law enacted in 2017 in Utah set aside dedicated funds for a new “Employability to Careers” program financed through a results-based contract, targeted at unemployed or underemployed adults without a high school diploma who are receiving public assistance. The law aims to fund a “program that will successfully achieve performance outcome measures that are cost effective and will result in cost savings or increased tax revenue to the state.”
- The Northern Virginia Workforce Development Board’s SkillSource Group was the first in the nation to use the WIOA PfP authority. The agency launched a contract in 2018 providing bonus payments for achieving certain milestones in programs serving foster care and justice-involved young adults. The provider was awarded a $1.28 million annual contract to provide WIOA out-of-school youth services, including up to four bonus payments for each program participant, one for each successful outcome, for a total potential pool of $150,000 in bonus payments over three years. The initial payments are linked to WIOA measures, though the agency has said it hopes to expand to track longer-term outcomes, including recidivism and public assistance use.
- In 2017, the City of Seattle passed an ordinance requiring the city’s Human Services Department to use results-based contracting when making its $105 million in annual investments in human services programs. A press release from the Mayor’s office described his interest in offering “incentive pay for performance, which provides increased compensation when planned outcomes are met or exceeded.”
- Among the states providing access to information about program performance is Washington state, whose Workforce Training and Education Coordinating Board operates a “Career Bridge” system that provides public information about hundreds of education and training programs, some of which are WIOA funded. Expanding this effort to include performance dashboards on basic employment services, disaggregated by demographics, would strengthen the ability of local areas to create real-time continuous learning cycles to improve services and achieve priority outcomes.