Want to know what is working in government? Subscribe to our new Substack – “The Department of What Works”

The Workforce Evidence-Based Spending Guide

Strategy 1: Define Evidence of Effectiveness

2. Prioritize Evidence 3. Link Payments to Outcomes 4. Use Active Contract Management 5. Build Evidence Through Evaluations 6. Braid, Blend, or Sequence Funding

Why It Matters

Clearly defining “evidence of effectiveness” is a key step in including evidence standards in government contract and grant applications. By determining what counts as evidence, workforce development practitioners can evaluate and score proposals against those standards and set clear expectations for applicants.

In a broad sense, evidence of effectiveness refers to all the evaluations, analyses, and other work showing how well a program or policy worked in achieving its goals. To make procurement and grant-making more evidence-based, workforce agencies will need a specific evidence framework, with corresponding equity considerations, that fits their unique contexts.

 

The Role of Evidence Definitions

Evidence definitions allow governments to more clearly and effectively shift dollars to what works and deliver better results for all through their grants, contracts, budgets and purchasing. RFA’s updated evidence framework encourages government decision-makers to consider the full body of evidence for a policy, program, or practice. This includes taking into account the local context and how it affects implementation and results, and focusing on why, how and for whom specific programs work. These definitions can also help governments incorporate valuable input from the community and build a culture of ongoing learning and improvement. By using evidence definitions, government decision-makers can direct public funding to solutions that are most likely to work in their communities.

 

Overview

An evidence framework or continuum is a tool to categorize practices, policies, and programs based on the number and types of rigorous evaluations that demonstrate policy-relevant results. These frameworks are not only useful for steering funds toward programs that are effective but can also help identify programs where more information is needed and where to invest funds in new evaluations.

Evidence frameworks generally contain the following:

  1. A set of categories to reflect the rigor, quantity, and outcomes of an evaluation. Categories may be designated numerically or with descriptive titles such as high, medium, low or promising, theory-based, and mixed effects.
  2. A definition for each category, usually including some quantitative indicators such as the number of evaluations, studies, or peer-reviewed reports. May also include timelines such as 1 year or 5 years of outcomes, as well as considerations for data disaggregation to understand equity within subpopulations.
  3. A designation for new or emerging programs that may not yet have evidence but are important to the work of the agency.

Frameworks may also include supplementary documents such as instructions for category selection to be used by respondents or outcomes requirements that direct applicants as to what data to collect, track, and report and the specific methodology to use.

Evidence-Based Program

A program with either impact or implementation evidence that is relevant, credible, and has an informed rationale

Evidence-Building Program

A program that has an informed rationale and is undergoing an impact evaluation or implementation evaluation that is relevant and credible

RFA’s Evidence Definitions

Building on the strong foundation of evidence definitions already in use, Results for America is sharing new definitions of “evidence-based” and “evidence-building” that help pave the way for governments to more clearly and effectively shift dollars to what works and deliver better results for all.

Learn more

Examples

Embedding Equity

 

Equity Considerations for Evidence Frameworks

Using evidence and data without an equity lens can perpetuate disparities despite our best intentions. Simply delivering more services to historically underserved populations does not necessarily address the ongoing gap in available evidence about what works within these populations. To use an equity lens within an evidence framework necessitates building evidence and investing in programs that work for specific populations, including those that are historically underserved. While the explicit incorporation of an equity lens into the development and use of a framework is newer in its application, it is no less critical to ensuring that high quality solutions not only have the necessary data to demonstrate that they are successful, but also that data is disaggregated to address disparities in outcomes. Below are three ways workforce development agencies can build equity into their evidence frameworks.

Prioritize research and evaluation

Prioritize and fund research and evaluation approaches that empower workers and communities to be full partners in the research, from designing the research and interpreting data to shaping how findings are used. Funding evidence building through impact and implementation evaluations can help to reduce the gap between evidence that is currently available and more comprehensive evidence that will enable workforce agencies to understand what works, for whom, and under what circumstances.

Community-Based Participatory Action Research (PAR) is an umbrella term for research approaches that engage community members as full partners at every stage of the research process, including shaping research questions and empowering community members to collect and analyze data. PAR disrupts historical power imbalances between researchers and communities, generating better research and knowledge. This approach recognizes that deep expertise lies within each community, making research done without community input incomplete and inaccurate. The Conservation Law Foundation’s PAR Field Guide includes guidance on setting up a PAR project, developing research questions and tools, and collecting and analyzing data. The Healthy Neighborhoods Study is a PAR project examining the impact of gentrification on the health of residents in nine communities in the metropolitan Boston area. Residents and community groups drive the study in partnership with researchers and regional planners.

The Urban Institute’s Community Voice and Power Sharing Guidebook offers practical advice on partnership building, community advisory boards, community-engaged survey development, and youth engagement. Urban also has a guide for holding Data Walks, in which program administrators and service providers empower program participants and community members by sharing program outcome data or research findings with them. Data Walks give community members an opportunity to ground the data in their lived experiences, shedding light on how and why programs and services are serving their community well or not. The guide includes detailed descriptions of how the Urban Institute used Data Walks to share data with public housing residents in Chicago, Illinois and Portland, Oregon as part of a demonstration project testing innovative wraparound services for those residents. The Urban Institute team has also used a Data Walk in Washington, DC to gather community input on the design of its Promoting Adolescent Sexual Health and Safety project.

Compensate community input

Embed equity in the evidence frameworks by seeking out and compensating the input of those who will be most impacted by programs, such as jobseekers (particularly jobseekers from historically marginalized groups), neighborhood groups in historically disadvantaged areas, community-based organizations, and community-based training providers.

Getting Started