Resources for Defining and Prioritizing Evidence of Effectiveness in State Grant Programs

Select a definition of evidence

Definitions of evidence of effectiveness allow policymakers and those who deliver services to have a shared understanding of which interventions have qualifying evidence.  

If your state has already defined evidence, you can use that definition in your agency. If not, you can adopt or adapt a framework established by another state, such as:

• Colorado has an Evidence Continuum that categorizes interventions in five categories (Proven, Evidence-Informed, Theory-Informed, Opinion Based and Evaluation Investment). 
• Minnesota uses a framework with six categories (proven effective, promising, theory based, mixed effects, no effect and proven harmful).
• Tennessee’s Evidence Framework provides standardized language to classify programs based on the level of available evidence supporting the program.

These frameworks are structured to make it easier to use research clearinghouses to assign an evidence category to a given intervention. The Results First Clearinghouse Database contains ratings for more than 3,000 programs drawn from nine prominent national clearinghouses. If your evidence definition is compatible with these tools, then you (and your potential grantees) will be able to efficiently assign ratings to programs featured in this tool. Results for America’s Economic Mobility Catalog is a useful tool for finding case studies and tips for successful implementation for about 200 interventions that facilitate economic mobility. 

The federal government has also established definitions of evidence of effectiveness, including:

• Evidence Provisions of the Every Student Succeeds Act (ESSA), which includes four levels of evidence, with a focus on improving student outcomes.
• The Department of the Treasury’s American Rescue Plan Reporting and Compliance Guidance, which defines three tiers of evidence (strong, moderate and preliminary).
• Americorps defines evidence for the AmeriCorps State and National grant program using four tiers of evidence (strong, moderate, preliminary and pre-preliminary evidence).

Select a strategy for prioritizing evidence 

The two most common approaches to prioritize funding for evidence-based practices are to establish an evidence requirement and to allocate points that reward proposals that define and prioritize evidence.

The strategy you select will be influenced by a number of factors, including:

• The extent to which relevant evidence-based interventions exist for the goals your program is seeking to achieve
• The importance placed on the use of proven practices versus creating the opportunity for innovation and evidence building
• The degree of uniformity desired in service availability across the communities that will be served
• Capacity of potential applicants to deliver various interventions

Evidence Requirement
Your grant program can specify that grant funds must be used for interventions that are evidence-based by designating specific interventions or by establishing criteria. For example:

• Juvenile Justice in Michigan. To promote and preserve community-based placement for youth involved in the juvenile justice system, the Michigan Department of Health and Human Services invited applications to provide Evidence-Based Services for Youth in the Juvenile Justice System. The program exclusively funds interventions that use one of the three following evidence-based models: Multisystemic Therapy (MST), Multisystemic Therapy for Problem Sexual Behaviors (MST-PSB) and Functional Family Therapy (FFT).
• Child Welfare in Georgia. To prevent child abuse and neglect, the Georgia Department of Human Services invited applications for its Promoting Safe and Stable Families Program. Grantees must deliver interventions with evidence of effectiveness that meet the standards of the California Evidence-Based Clearinghouse (CEBC).
• Mental Health in Minnesota. To help adults with serious mental illness achieve permanent housing, the Minnesota Department of Human Services invited applications for grantees to Provide Housing with Supports for Adults with Serious Mental Illness. Interventions must have evidence of effectiveness from at least one impact evaluation that uses an experimental or quasi-experimental design. The State provides training to grantees on delivering evidence-based services and requires grantees to monitor and report fidelity to the evidence-based model. The selection criteria for grant awards consider applicants’ capacity to serve underserved and minority populations in a culturally responsive manner.

Prioritize evidence through selection criteria
Your grant program can allocate points that reward proposals that define and prioritize evidence. For example:

• Education in Nevada. To improve educational outcomes, the Nevada Department of Education prioritized evidence-based practices in multiple grant programs through  a rubric that awarded points for proposals that include evidence-based interventions. Up to 10 points were awarded based on the proportion of the proposed activities that met evidence criteria associated with the Every Student Success Act (ESSA).
• Workforce in Texas. To prepare youth for employment in construction occupations, the Texas Workforce Commission invited applications for its Building Construction Trades program. In the process of scoring applications for grant funding, up to 10 bonus awards points were awarded based on the level of causal evidence, as defined by the Texas Workforce Commission’s Evidence Based Grantmaking Framework. The program provides guidance to help applicants determine which of five tiers their proposed services would fall under. This tiered structure allows applications without evidence of effectiveness to receive funding while they evaluate impact and build evidence.
• Workforce in Pennsylvania. To help meet industry workforce needs, the Pennsylvania Department of Labor and Industry invited applications for Industry Partnerships grants. In scoring applications for grant funding, five points are awarded based on evidence of effectiveness and five points are awarded for plans for evidence building. The Request for Proposals provided a statewide definition of evidence, with four tiers, including strong evidence, moderate evidence, preliminary evidence and pre-preliminary evidence. The Request for Proposals also referred to the Department of Labor’s CLEAR definition of strong or moderate study design.

Please contact us at [email protected] if you have questions or if you are interested in receiving assistance using evidence of effectiveness in your state grant program.