This report is designed to help deepen the agency’s understanding of the most effective interventions it has funded and its knowledge base on issues involved in scaling them. This information will also help inform the agency’s interest in identifying the intervention components that are critical for an intervention’s effectiveness.

Goals:
The goal of this report is to identify AmeriCorps-funded interventions with evidence of effectiveness. This project defines evidence of effectiveness as rigorous research that has shown favorable impacts on the majority of the targeted outcomes for people receiving the intervention services. 

Research Questions:
The research questions are:

  • What interventions consistently demonstrate evidence of effectiveness for their targeted outcomes? 
    • Which AmeriCorps-funded interventions demonstrate evidence of effectiveness on participants’ outcomes? 
    • Among interventions with evidence of effectiveness, what are their characteristics? 
  • What intervention characteristics demonstrate evidence of effectiveness at a scale that suggests readiness for replication more widely? 
    • How effective are AmeriCorps interventions with evidence of effectiveness? 
    • What characteristics of the interventions might be associated with the estimated impacts? 

Findings
This report found that:

  • 39 percent of interventions implemented by grantees given a strong or moderate evidence rating in a systematic evidence review by independent contractors and reviewers for AmeriCorps (other than Mathematica) met the evidence of effectiveness standards developed for this project. 
  • Among the assessed criteria, the primary reasons that studies did not meet the Mathematica standards were:
    • A lack of internal validity that gives confidence that the intervention, and not other factors, impacted intervention participants (46 percent)
    • Results that were not consistently favorable (44 percent) 
    • A lack of a comparison group (40 percent)
  • The 32 interventions that met the Mathematica standards fell primarily in the education focus area (56 percent), with 28 percent aligning with the economic opportunity focus area and 16 percent falling under the healthy futures focus area. These 32 interventions were predominately implemented in urban settings (97 percent), although 16 percent were implemented in rural areas. About half (47 percent) were implemented in more than one region of the country, and more than one-quarter (29 percent) had AmeriCorps members involved in delivering services or supporting the delivery.
  • Participants in the AmeriCorps-funded interventions meeting the Mathematica effectiveness standards had better outcomes than 61 percent of participants who did not receive the intervention. This represents an 11 percentage point increase in beneficial outcomes. The size of this impact varied by focus area. 
  • Interventions in the education and healthy futures focus areas increased beneficial outcomes for intervention participants by 9 percentage points, compared to those not receiving the intervention. 
  • For interventions in the economic opportunity focus area, the impact reached 14 percentage points. The size of the impacts varied by AmeriCorps’ priority objective. 
  • Interventions aligning with the employability, K–12 success, postsecondary education support, school readiness, and supportive family environments priority objectives had statistically significant positive impacts. 
  • Impacts did not vary by geographic region, or the use of AmeriCorps members, which suggests that interventions might be equally effective across communities and that AmeriCorps members are as effective as other types of personnel at delivering or supporting the delivery of services. 
    • Impacts also did not vary by intervention funding year, which suggests that AmeriCorps has year-to-year consistency in identifying interventions supported by rigorous evidence of effectiveness.

For more information, download the report.

Scaling Evidence-Based Models (SEBM) Project

 The Office of Research and Evaluation (ORE) initiated the Scaling Evidence-Based Models project to support the scaling of effective interventions. This report is part of ORE’s Scaling Evidence-Based Models project, which includes additional resources that contribute to the study and application of scaling effective interventions. Below are additional scaling resources:

Full report

Full Report

Further information

Program/Intervention
Best Practice Dissemination, AmeriCorps Research Guidance to Scale Programming
Implementing Organization
AmeriCorps Office of Research and Evaluation Commissioned Report
AmeriCorps Program(s)
AmeriCorps State and National
Social Innovation Fund

Office of Research and Evaluation
Focus Population(s)/Community(s)
Opportunity Youth
Schools
Nonprofits
Tribes
Veterans and Military Families
Rural
Suburban
Urban
Low-income
Outcome Category
Disaster assistance provided
Financial literacy
Housing
Employment
School readiness
K-12 success
Post-secondary educational support
Energy efficiency
At-risk ecosystems
Awareness of environmental issues
Green jobs
Aging in place
Obesity and food
Access to care
Veterans and military families served
Nonprofit development
Improving AmeriCorps
Other
Study Type(s)
Review or Meta-analysis
Study Design(s)
Non-experimental
Researcher/Evaluator
Mathematica
Published Year
2021