Research rigor

• Evidence of Effectiveness in AmeriCorps-Funded InterventionsThis report summarizes three years of assessments Mathematica conducted of AmeriCorps’ State and National and Social Innovation Fund grants to identify which AmeriCorps-funded interventions had demonstrated evidence of effectiveness, defined by industry standards for evaluation rigor and objectivity.
• What Makes for a Well-Designed, Well-Implemented Impact StudyThis guide helps organizations ensure their evaluators produce high quality impact studies that provide evidence about whether an intervention is effective and whether it is worth scaling to reach more people.
• What Works Clearinghouse Standards Brief: AttritionThis brief discusses the causes of attrition, or the loss of sample members from a random-assignment study, and why it matters.
• What Works Clearinghouse Standards Brief: Confounding FactorsThis brief discusses confounding factors. When present in an experimental or quasi-experimental study, a confounding factor makes it impossible to tell whether a difference in participant outcomes is due to the intervention or to the confounding factor.
• Baseline Equivalence: What It Is and Why It Is NeededThis guide helps organizations work with researchers to design an impact study with baseline equivalence—that is, a study with two groups of people that are identical to each other from a statistical perspective, except that one group—a treatment group—can receive the intervention, whereas the other group—a comparison group—cannot.

Intervention effectiveness

• U.S. Department of Health and Human Services Criteria for Evidence-Based ModelsThis web page explains the criteria that the Home Visiting Evidence of Effectiveness clearinghouse uses to determine whether an intervention is evidence based.
• Using the What Works Clearinghouse to Find Every Student Succeeds Act Tiers of EvidenceThis guide helps organizations ensure their evaluators produce high quality impact studies that provide evidence about whether an intervention is effective and whether it is worth scaling to reach more people.
• What Works Clearinghouse Standards Brief: AttritionThis brief discusses the causes of attrition, or the loss of sample members from a random-assignment study, and why it matters.
• What Works Clearinghouse Standards Brief: Confounding FactorsThis brief discusses confounding factors. When present in an experimental or quasi-experimental study, a confounding factor makes it impossible to tell whether a difference in participant outcomes is due to the intervention or to the confounding factor.
• Baseline Equivalence: What It Is and Why It Is NeededThis guide helps organizations work with researchers to design an impact study with baseline equivalence—that is, a study with two groups of people that are identical to each other from a statistical perspective, except that one group—a treatment group—can receive the intervention, whereas the other group—a comparison group—cannot.

Intervention relevance

• TRANSFER Conversation Guide This table, included in a journal article, lists several factors for organizations to consider when deciding if externally produced research is generalizable to their organization’s or intervention’s own context.
• SUPPORT Tools for Evidence-Informed Health Policymaking 9: Assessing the Applicability of the Findings of a Systematic Review (subscription may be required)This tool provides a list of questions that organizations can answer to decide if externally produced research is generalizable to their organization’s or intervention’s own context.

Research clearinghouses

Use these research clearinghouses to find research on the evidence of effectiveness of a variety of interventions.

• Blueprints for Healthy Youth DevelopmentReviews studies of family-, school-, and community-based programs intended to promote positive behaviors among youth.
• California Evidence-Based Clearinghouse for Child WelfareReviews research on a range of programs and practices related to child welfare.
• Clearinghouse for Labor Evaluation and ResearchReviews studies of labor programs to inform decisions about employment-related policies and programs.
• Home Visiting Evidence of EffectivenessReviews research on home visiting models that serve pregnant women or families with children up to kindergarten age.
• Pathways to WorkReviews studies of interventions designed to help job seekers with low incomes succeed in the labor market.
• Results FirstCentralizes information from nine national clearinghouses on the effectiveness of social policy programs.
• Social Programs that WorkOffers information about evidence of effectiveness for interventions across a range of policy areas.
• What Works ClearinghouseReviews the existing research on educational programs, products, practices, and policies to help educators make evidence-based decisions.

For information on additional research clearinghouses, see AmeriCorps’ “Clearinghouses and Evidence Reviews for Social Benefit Programs”

Define intervention

• How to Fully Describe an InterventionThis guide helps organizations thoroughly describe (1) the intervention model as it was designed, (2) the intervention as it is implemented, and (3) the parts of the intervention and the population that were evaluated and their effectiveness.
• Logic Model Development GuideThis guide helps organizations construct or strengthen the logic model for their interventions.
• Designing Community InterventionsThis online toolkit guides organizations on how to design and fully define an intervention.

Lay evaluation groundwork

• Impact Evaluability Assessment ToolThis tool helps organizations determine their readiness to engage in a rigorous impact evaluation of their interventions.
• How to Develop the Right Research Questions for Program EvaluationThis presentation helps organization personnel understand the importance of research questions, the basic steps for developing research questions, and ways to write research questions for different types of evaluation designs.

Prepare for evaluation

• Budgeting for EvaluationThis presentation helps organization personnel understand why budgeting for evaluation is important, how costs vary by evaluation type, the components of an evaluation budget, and how to approach developing an evaluation budget.
• How to Manage an External EvaluationThis presentation guides organization personnel on working with external evaluators to conduct evaluations of interventions.
• Making the Most of DataThis guide highlights issues for organizations to consider when using, collecting, and managing their data.

Conduct evaluation and assess effectiveness

• Basic Steps in Conducting an EvaluationThis presentation describes how to plan for an evaluation by covering the primary components of an evaluation plan, data collection and analysis approaches, and ways to disseminate and use findings to improve interventions.
• Evaluation Plan Guidance: A Step-by-Step Guide to Designing a Rigorous EvaluationThis guide helps organizations design a plan for a rigorous evaluation. Components of such a plan would include a theory of change, research questions, specifications for data collection and analysis, and a reporting strategy.
• What Makes for a Well-Designed, Well-Implemented Impact StudyThis guide helps organizations ensure their evaluators produce high quality impact studies that provide evidence about whether an intervention is effective and whether it is worth scaling to reach more people.
• Key Items to Get Right When Conducting Randomized Controlled Trials of Social ProgramsThis guide includes a checklist to help organizations identify items that should be present to ensure a randomized controlled trial produces valid findings about an intervention’s effectiveness.
• Selecting an Appropriate Design for the EvaluationThis resource discusses external validity, as well as other considerations organizations should take when selecting an evaluation design.
• What Works Clearinghouse Standards Brief: AttritionThis brief discusses the causes of attrition, or the loss of sample members from a random-assignment study, and why it matters.
• What Works Clearinghouse Standards Brief: Confounding FactorsThis brief discusses confounding factors. When present in an experimental or quasi-experimental study, a confounding factor makes it impossible to tell whether a difference in participant outcomes is due to the intervention or due to the confounding factor.
• Baseline Equivalence: What It Is and Why It Is NeededThis guide helps organizations work with researchers to design an impact study with baseline equivalence—that is, a study with two groups of people that are identical to each other from a statistical perspective, except that one group—a treatment group—can receive the intervention, whereas the other group—a comparison group—cannot.

Well-specified intervention and well-defined target population

• How to Fully Describe an Intervention This guide helps organizations thoroughly describe (1) the intervention model as it was designed, (2) the intervention as it is implemented, and (3) the parts of the intervention and the population that were evaluated and their effectiveness.
• Logic Model Development Guide This guide helps organizations construct or strengthen the logic model for their interventions.
• Designing Community Interventions This online toolkit guides organizations on how to design and fully define an intervention.

Implementation supports

• Organizational Capacity Assessment Tool Organizations can use this AmeriCorps tool to gather additional information on organizational strengths, clarify perceptions of different stakeholders, and plan strategies to enhance capacity in identified areas.
• Improving Organizational Management and Development This online toolkit gives guidance for enhancing governance at organizations and improving communication among personnel and stakeholders.
• How to Structure Implementation Supports This guide helps organizations set up implementation supports in a way that facilitates implementing an intervention with fidelity.
• Continuous Quality Improvement This web page describes continuous quality improvement (CQI) and gives guidance on how to implement CQI processes. It also offers a real-word example of a CQI process used by a child welfare team in California.

Enabling context

• Build Organizational Capacity to Implement an Intervention This guide describes the specific components of the implementation infrastructure and the enabling context that an organization needs to implement a planned or current intervention.
• Organizational Capacity Assessment Tool Organizations can use this AmeriCorps tool to gather additional information on organizational strengths, clarify perceptions of different stakeholders, and plan strategies to enhance capacity in identified areas.

Implementation infrastructure

• Build Organizational Capacity to Implement an Intervention This guide describes the specific components of the implementation infrastructure and the enabling context that an organization needs to implement a planned or current intervention.
• Organizational Capacity Assessment Tool Organizations can use this AmeriCorps tool to gather additional information on organizational strengths, clarify perceptions of different stakeholders, and plan strategies to enhance capacity in identified areas.
• Improving Organizational Management and Development This online toolkit gives guidance for enhancing governance at organizations and improving communication among personnel and stakeholders.

Additional resources

• Scaling Evidence-Based Interventions: Insights from the Experiences of Three Grantees This report discusses the approaches that AmeriCorps grantees took while scaling, and their experiences during scaling. Although the report focuses on specific grantees, its insights are broadly applicable to a range of interventions and organizations.
• Planned Scaling Activities of AmeriCorps-Funded Organizations This report further discusses the five conditions necessary to successfully scale an intervention.
• General Advice on Evidence-Based Programs This report compiles lessons learned and advice related to implementation of evidence-based interventions. Lessons stem from the experiences of organizations implementing teen pregnancy prevention programs but can be useful for a variety of practitioners.
• Active Implementation Hub This repository, hosted by the National Implementation Research Network, contains online modules, lessons, courses, and other resources to help organizations implement and scale interventions.
• How to Scale Up Effective Programs Serving Children, Youth, and Families This brief synthesizes practices to help scale effective interventions from across the literature.