Moneyball for Head Start

USING DATA, EVIDENCE, AND EVALUATION TO IMPROVE OUTCOMES FOR CHILDREN AND FAMILIES

Executive Summary

Head Start is a valuable federal program that improves the lives of our nation’s most vulnerable children and their families. Research shows that Head Start programs improve children’s learning at school entry and have a positive impact on long-term life outcomes. Yet research also suggests that Head Start could have a stronger impact on children’s early learning, school, and life outcomes. The key question is, how can policymakers and practitioners maximize outcomes for Head Start children and their families?

This paper — the product of a combined effort of Results for America, Bellwether Education Partners, the National Head Start Association, and the Volcker Alliance — outlines a vision for a continuous improvement approach that uses data, evidence, and evaluation to improve outcomes at all levels of the Head Start program.

At the local grantee level, all Head Start grantees need systems of data collection and analysis that support data-informed, evidence-based continuous improvement, leading to better results for children and families. Head Start grantees collect and report data on a variety of outcomes, but effectively using this information to improve quality and outcomes requires a high level of intentionality, planning, and expertise in analyzing, interpreting, and acting on data. The best Head Start grantees already do this, but making data-informed continuous improvement the rule, rather than the exception, will require cultural shifts and increased capacity across all grantees. Federal policymakers can support these efforts by reviewing and revising reporting requirements, changing requirements that limit grantees’ staffing flexibility, and allocating existing training and technical assistance funds to help build grantees’ capacity to analyze and act on data. Groups of local grantees and researchers should also work together in Networked Learning Communities to share and analyze data to improve grantee practices and child outcomes.

At the federal oversight level, the Office of Head Start (OHS), within the Administration for Children and Families of the U.S. Department of Health and Human Services, needs a valid accountability and performance measurement system that allows federal officials to detect trends and patterns in performance; identify, learn from, and disseminate lessons from trends and patterns in performance; differentiate oversight, support, and interventions on the basis of grantee performance; and support continuous improvement across Head Start as a whole. Head Start grantees are already accountable through the Aligned Monitoring System and the Designation Renewal System. But supporting continuous improvement in Head Start will require measuring grantee quality with more results-based, robust, and fair systems that use valid, reliable, and accurate ways of measuring child outcomes; family outcomes; and indicators of program quality, safety, health, and compliance. Establishing such systems will require an iterative process that engages a variety of federal agencies, researchers, practitioners, and the philanthropic sector to identify the outcomes or components of program performance that are most important, develop ways to measure those outcomes or performance indicators, and pilot and renew measurement tools. Once new tools or measures are ready to be implemented at scale, OHS should give grantees time to become familiar with and buy into the new systems before attaching consequences. OHS should publicly report data on common measures of program performance using transparent, interactive data reports that enable stakeholders to access data on common program performance indicators and compare and analyze data across subsets of grantees and the program as a whole. OHS should also use these data to differentiate grantees by levels of performance — high-performing, adequately performing, and low-performing — across multiple domains.

At the research and evaluation level, federal policymakers and the philanthropic sector need to support research that builds the knowledge base of what works in Head Start and informs changes in program design and policies. Funding for Head Start research, demonstration, and evaluation should be increased from less than 0.25 percent of total appropriations to 1 percent, and should focus on activities that are most likely to result in building knowledge that local grantees can use to improve their quality and outcomes, specifically:

1. identifying and understanding effective practices of high-performing grantees;
2. supporting measured trials of new, scalable practices and approaches to improve child and family outcomes; and
3. developing high-quality assessment tools and measures that provide valid and reliable information on child and family outcomes, program quality, and other key domains of performance.

Local grantees bear the primary responsibility for integrating data-informed practice into their work. Other actors, such as the philanthropic sector, universities and other research institutions, and the private sector, can also play a crucial role in building grantee capacity or supporting the development, evaluation, and dissemination of promising practices. But federal policies play a major role in facilitating, incentivizing, or deterring local grantees’ ability to shift culture and build capacity to support continuous learning and improvement. And as a result, changes in federal policies are needed.

Federal Policy Recommendations

The necessary changes — both in grantee practice and in federal oversight — cannot happen overnight. Fully realizing the vision outlined in this report will require a multiyear commitment to research and cycles of experimentation to address outstanding technical and measurement challenges. But there are steps that Congress and the administration can take now to both advance this vision in the near term and support the research needed to fully realize it in the future. These steps include:

1. Congress and the Secretary of Health and Human Services should make data- informed, continuous improvement a key priority in any legislative or regulatory policy action on Head Start.
2. The Office of Head Start and the philanthropic sector should invest in building grantee capacity to use data to improve performance.
3. Federal policymakers should initiate an iterative process to develop robust, common performance indicators for Head Start and should engage researchers, the philanthropic sector, and Head Start grantees as partners in this process.
4. Federal research agencies should work with researchers and the philanthropic sector to support the development of solid, trusted metrics of Head Start child outcomes, family outcomes, and program capacity.
5. The Office of Head Start should provide transparent, interactive, public reporting on grantee performance.
6. Once the Office of Head Start has developed a sufficiently robust system to measure grantee performance, it should use this system to differentiate grantee performance in order to identify high-performing grantees and learn from and scale their effective practices; support improvement in adequately performing grantees; intervene in low-performing programs; and, when necessary, defund or require them to compete for grants.
7. The Office of Head Start should continue, learn from, and build on efforts to make program monitoring more performance focused and less compliance oriented.
8. The Secretary of Health and Human Services should implement a robust research agenda for Head Start, and Congress should increase the cap on Head Start research, demonstration, and evaluation spending from $20 million to 1 percent of total appropriations.
9. Congress should authorize the Secretary of Health and Human Services to grant additional exibility to allow cohorts of programs working with researchers to pilot new approaches to serving children and families.

Through these steps, federal policymakers can support Head Start grantees in using data, evidence, and evaluation to strengthen their practice and improve results for children and families.