Keystone Smiles - AmeriCorps Evaluation Report

Keystone SMILES AmeriCorps recruits, trains, develops, and engages AmeriCorps members to provide:

• academic support with host site school districts and alternative education and mentoring with community organizations, 
• community strengthening service to host sites, and 
• member and community training. 

During this program cycle, they provided structured programs to offer academic and social/behavior tutoring support within 60 schools and a wide array of service-learning support services from 12 community-based host sites across 12 counties in western and northwestern Pennsylvania.

Study Goal:
The goal of the study is to understand the effects of the targeted academic assistance activities provided by AmeriCorps members in the schools they serve (the impact of the program). The evaluation used a quasi-experimental design (QED) along with periodic interviews with host site supervisors. The QED compared 4th-grade mathematics test scores of schools with and without AmeriCorps members. Treatment and comparison schools were matched by performance quartiles from school-level achievement and demographic data.

Research Question:

• Do students in schools who receive targeted services from Keystone SMILES AmeriCorps members achieve better mathematics scores on the Pennsylvania System of School Assessment (PSSA) standardized test than students in schools with similar academic and socioeconomic backgrounds who do not receive services?

Findings:
The evaluation found the following:

• Eight of the twelve (67%) treatment schools involved in the evaluation scored better than the comparison schools on at least three of the four measurements: Year to Year Average Growth for Total Number of Students, Year to Year Average Growth for Historically Underperforming Students, Average Growth 2015-2019 for Total Number of Students, and Average Growth 2015-2019 for Historically Underperforming Students. 
• Based on the quantitative data, there is evidence that receiving services from Keystone SMILES AmeriCorps results in higher test scores on the 4th-grade PSSA mathematics assessment. 
• The themes that emerged from the host site interviews indicate that schools heavily depend on the Keystone SMILES AmeriCorps program to provide a quality educational experience for all students, especially those students needing extra support.

For more information, download:

Full Report