Strategy 2: Prioritize Evidence

• The Texas Workforce Commission (TWC) began using its Evidence Framework to prioritize evidence of effectiveness when awarding $1.5 million in training grants for the 2021 Building and Construction Trades Program, which prepares youth for employment in construction occupations. TWC provided up to 10 points out of 110 to applicants based on the causal evidence supporting each application, and up to 15% of grant payments were tied to specified outputs and outcomes, including participation completion, receipt of industry-based certification, and employment. TWC has continued to leverage this framework across other grant programs including the Texas Talent Connection Program, Perkins Equitable Access and Opportunity Program, and additional rounds of Building and Construction Trades grants (2023, 2023 Addendum).
• Pennsylvaniaʼs Department of Labor and Industry developed an evidence continuum and leverages it when awarding grants and providing technical assistance to providers.
• The U.S. Department of Labor’s Reentry Employment Opportunities Program supports reentry into the community for individuals who have been involved in the criminal legal system. The department prioritizes evidence of effectiveness in this program by awarding up to 10 points out of 100 based on the quality of an applicant’s project design, including evidence of effectiveness for the proposed project.
• These RFA templates and examples can help your agency prioritize evidence in grants and contracts.

Equity Considerations for Prioritizing Evidence

As some populations have been historically underserved or excluded from service receipt and evaluation initiatives, existing evaluation results and datasets may not be disaggregated and may lack the necessary information to determine if the intervention will be equally successful for all populations. While it isn’t always possible to alter or expand completed work, this risk can be reduced going forward by requiring respondents to describe how they would adapt their approach to ensure specific populations are served, explicitly requiring data disaggregation, building in sufficient budget and time to allow for the collection of larger datasets that are more representative, and including focus populations as a requirement for evaluation efforts.

Organizations led by people with marginalized identities, and those with an equity focus, may not have traditionally been recipients of government funding and may require additional investments to increase their capacity to take on more rigorous evidence-based work. To increase the pool of diverse service providers in the area, and tap into unique lived experiences, localized relationships, and practical knowledge, workforce agencies can:

• Analyze historical awards to unearth gaps in partnership with Black, Indigenous, and People of Color (BIPOC), people with disabilities, and gender and sexual minority led or focused organizations.
• Consider providing community forums before issuing an RFP to understand the needs and concerns of these organizations.
• Make technical assistance available early in the process.
• Encourage partnership between organizations to address potential experience gaps around evidence and data.

Considerations for Deciding How to Prioritize Evidence

Every workforce agency is unique, so the way they prioritize evidence of effectiveness must be tailored to their specific context. Agencies will want to consider the following:

The extent to which existing evidence-based approaches can achieve the program’s goals. If there are many proven approaches that have been effective for a service population, it may be appropriate to require providers to use one of those approaches. If there are few proven approaches for a service population, awarding preference points for providers that do use evidence-based approaches may be more appropriate and gives providers greater flexibility. You can search for evidence-based workforce development approaches in Results for America’s Economic Mobility Catalog, the U.S. Department of Labor’s CLEAR, and the Results First Clearinghouse from Pew Charitable Trusts and Pennsylvania State University.

• The importance of using proven approaches versus innovating and discovering what works. Requiring or encouraging providers to use a proven approach can lead to better outcomes for a service population, and provide greater insights into differentiated results through disaggregation. However, an agency may also want to give providers flexibility to test out new approaches and evaluate them over the grant or contract period. Creating flexibility for innovation is particularly critical in emergent fields such as the application of new technology or the delivery of services to sub populations not previously supported or tracked.
• The degree of consistency desired in service availability across communities. If an agency is looking to provide consistent services across its workforce area, it might require the use of a standardized set of evidence-based approaches. If consistency is not a high priority, an agency might award preference points to providers that use evidence-based approaches, giving providers more flexibility in the approach they choose.
• The capacity of potential applicants to deliver services through a variety of approaches. Evidence-based approaches require a level of rigor and expertise that often varies by organization. For example, smaller organizations that have not previously received government funds may need additional technical assistance or require investment in internal capacity to facilitate necessary tracking, reporting and evaluation. Carefully consider the provider’s current state and, if necessary, explore how evidence-based requirements can be ramped up over time as capabilities are built. If the target providers already have a high capacity to successfully implement evidence-based approaches, requiring the use of a specific approach may be seamless.

Communicating Changes to Applicants

Shifting toward evidence requirements or preference points may mark a change from the previous grant or contract processes, and workforce agencies should communicate this new approach to applicants. Agencies can explain their reasoning in a problem or goal statement in the RFP, including why the use of evidence is a priority for the agency, what problem the agency is trying to solve using evidence-based approaches, how the implementation of evidence-based approaches can improve equity and shed light on gaps, and how the use of evidence will create progress toward the agency’s goals. Problem and goal statements can provide applicants with context, making it easier to understand the agency’s intentions and expectations. For additional guidance on drafting RFPs and problem statements, see Harvard Government Performance Lab’s Guidebook for Crafting Results-Driven RFPs. Communications can also be coupled with training sessions, or more personalized technical assistance, to build the capacity of the local provider or grantee community. Agencies can consider taking it a step further by providing an opportunity for potential respondents to provide input to a draft RFP before it is published in order to proactively gauge respondents’ understanding of evidence-based approaches and capacity to deploy them.

How to Get Started

1. Assess your prior award history. Examine the types of organizations that have received grants or contracts, the criteria or points allocations used in making the decisions, and the outreach approach used to get the word out to diverse communities.
2. Determine which procurement or grant you will use as your starting point.
3. Consider the available body of evidence based on the procurement or grant.
   1. Has the work been happening for some time allowing for more data and evidence?
   2. Is there likely to be disaggregated data available based on the scope of prior related work?
   3. Are there a variety of different organizations that deliver the service?
   4. Are there BIPOC-led organizations that are currently involved in the work or is there a gap?
   5. To what degree do you want to diversify providers as well as increase the evidence base?
4. Decide whether requiring evidence or incentivizing evidence is a better fit for your organization and your intervention. Make sure to base this decision on the information collected. For example:
   1. If the concept is well known and there are multiple providers in the space, consider drafting a set of requirements for evidenced-based approaches in your RFP. This can include requiring a certain number or level of past intervention outcomes, such as disaggregated data, requiring the performance of an evaluation, or establishing a minimum portion of the budget for use in collecting, analyzing and evaluating data.
   2. If the concept is more emergent or there is interest in seeing a more diverse set of approaches and/or respondents, consider using preference points to incentivize the use of evidence-based approaches. Keep in mind that the point value should be sufficient for it to be meaningful (e.g., 10-20%) but should not overshadow other key evaluation criteria such as staffing, budget or prior experience.
5. Draft the requirements or preference points and coordinate with your procurement personnel around potential impacts and considerations before finalizing. Download this template for sample language to use.
6. Finalize the RFP, including the requirement or preference points, and hold information sessions for potential respondents to ensure they understand what is expected and the appropriate avenues for questions if needed.
7. Hold training for the evaluators to make sure they are familiar with the new requirements/preference points before initiating their review.

Want more templates?

We have a full list of ready-to-use templates and language for each of the strategies

Learn More