ServeMinnesota: MAVRIC

The Minnesota Assessment of Vocabulary for Reading Improvement and Comprehension (MAVRIC) program is a project of ServeMinnesota. MAVRIC is delivered through existing Reading Corps infrastructure and focuses on improving student vocabulary outcomes. The program’s theory of change is premised on two functional elements:

• Data-driven decision-making
• Optimizing vocabulary interventions

Study Goals

The goal of study is to identify the impact on vocabulary improvement for at-risk prekindergarten, kindergarten, and first grade students who participated in MAVRIC compared to similar students who did not receive MAVRIC interventions. An implementation evaluation also assessed program fidelity and exposure/dosage.

Research Questions

The study included both impact and implementation research questions. The research questions include:

• Impact Questions:
  • What is the impact on vocabulary improvement for at-risk prekindergarten, kindergarten, and 1st grade students who participate in MAVRIC compared to similar students who do not receive MAVRIC interventions?
  • How much improvement in vocabulary skills is observed for prekindergarten, kindergarten, and 1st grade students who participate in MAVRIC?
  • What number and percentage of prekindergarten, kindergarten, and 1st grade students who participate in MAVRIC are no longer considered at-risk for poor vocabulary outcomes after participation in MAVRIC?
  • What is the performance for participating prekindergarten, kindergarten, and 1st grade students on vocabulary progress assessments?
• Implementation Questions:
  • To what degree are MAVRIC interventions for prekindergarten, kindergarten, and 1st grade students implemented as intended?
  • To what degree are MAVRIC assessments of prekindergarten, kindergarten, and 1st grade students administered as intended?
  • What is the overall inter-scorer reliability for a randomly selected subsample of MAVRIC assessments administered to prekindergarten, kindergarten, and 1st grade student?
  • What is the average, standard deviation, and range for number of minutes of MAVRIC intervention?

Findings

The report found the following:

• Exploratory impact evaluation results indicated all students, including treatment and control, made comparable gains across pretest and posttest data collection periods. These results were consistent with the confirmatory impact evaluation results showing no statistically significant impact of MAVRIC on student vocabulary skills.
• Consistent with prior evaluation findings, implementation evaluation results showed all aspects of implementation (with the exception of prekindergarten intervention delivery) were implemented with high fidelity.
• Although the confirmatory impact evaluation results were unexpected, several factors—including select methodological limitations—were potentially related to the nonsignificant findings. The evaluation team identified three primary considerations:
  • The potential for different vocabulary assessments to better measure differential growth between MAVRIC participants and non-participants;
  • The possible benefit of providing additional time in intervention for MAVRIC participants;
  • An opportunity to address implementation factors (e.g., student group size; tutor training; material refinements) that are plausibly related to vocabulary outcomes.
• Investigating and addressing these changes will be the focus of development and evaluation work in Year 5 (2017-18 school year) of the MAVRIC project.

For more information, download the full report and report brief