How to use the RE-AIM framework to evaluate public health interventions – Healthcare Economist

The RE-AIM framework developed by Glasgow et al. (1999) has five evaluation components.

• Reach: share of people from a given population who participate in a program and their characteristics. For instance, organizational reach with stakeholders could be indirect, intended or direct. Reach is the number of participants in the program divided by the size of the target population.
• Effectiveness: positive and negative program outcomes. These are typically measured as the share of participants who experience given positive and negative program outcomes. In addition to patient clinical outcomes, other types of outcomes include (i) behavioral outcomes for participants (e.g., smoking cessation, eating patterns, physical activity), (ii) outcomes for staff who deliver an intervention (approaching patients, delivering prompts and counseling, making follow-up calls), and (iii) outcomes for the payers and purchasers who support the intervention (adopting an intervention, changing policies). Quality of life should also be considered such as patient functioning, mental health, and consumer satisfaction.
• Adoption: percent of possible settings (e.g., organizations) and staff that have agreed to participate in the program. One could think of this as organizational buy-in. Adoption is usually assessed by direct observation or structured interviews or surveys.
• Implementation: the extent to which the program was delivered as intended and its cost. Implementation can be evaluated at the individual and organization levels. At the individual level, one can measure participant follow-through or “adherence”; at the organizational level, one can measure the share of staff members who implement the program as intended.
• Maintenance: maintenance of the primary outcomes (>6 months) for individuals; sustainability of the delivered programs for organizations. For individuals, relapse or discontinuation is a challenge and should be evaluated; at the organizational level, one should measure the extent to which a health promotion practice or policy becomes routine and part of the everyday culture and norms of an organization.

A summary of the 5 dimensions in table form is below.

Each of the 5 dimensions listed above is represented on a 0 to 1 (or 0% to 100%) scale. Part of the reason for the name is that an intervention’s impact is a function of it’s reach and effectiveness (Impact = Reach × Efficacy) as outlined by Abrams et al. (1996). Additionally, Effectiveness = Implementation × Efficacy. An intervention may be efficacious, but it is only effective if that intervention readily translates to the real-world. However, interventions may have a large impact in short-term research-based environments but may not be sustainable. The “AIM” component aims to more completely characterize the public health impact of an intervention based on organizational factors. RE-AIM has been used to evaluate a variety of interventions such as:

• Behavioral outcomes (Dzewaltowski et al. 2004, Kessler et al. 2013)
• Weight loss programs (Akers et al. 2010, Kahwati et al. 2011)
• Nutrition (Dunton et al. 2009)
• Injury prevention (Finch 2012, Li et al. 2008)
• Physical activity (Dunton et al. 2009, Estabrooks et al. 2008)

A key question is whether RE-AIM can be used not only for individual interventions, but also for multi-faceted interventions that target a number of different stakeholders (e.g., providers, educators, others). A paper by Sweet et al. 2014 note that RE-AIM could be applied to to multi-faceted intervention in one of three ways:

1. Activity specific approach. Under this approach, each activity undertaken would be evaluated based on the RE-AIM dimension it targeted. “For instance, awareness-raising strategies would target reach or adoption, physical activity promotion strategies might be used to improve effectiveness and maintenance, and provider-training strategies could be identified as methods to improve adoption and implementation of evidence-based guidelines.”
2. Additive approach. In this methodology, information would be collected across the initiatives and then amalgamated to inform each RE-AIM dimension. For instance, reach could be evaluated based across all activities (e.g., mailers, website visits, participant enrollment). This approach allows for a good capture of overall impact but not how any specific activity moved the needle on a given RE-AIM dimension.
3. Hybrid approach. In this approach, researchers decide pragmatically which strategies deserve a more thorough RE-AIM evaluation and which are best evaluated using one or two RE-AIM dimensions. The decision is often determined ‘pragmatically’ based on the perspectives of the stakeholders. Swett et al. argues that this approach “can speed the integration of research, policy and practice…[and] also reduce burden on participants and partner organizations…”

The Sweet al. 2014 paper applies the RE-AIM multifaceted framework to the SCI Action Canada, which was an initiative whose mission was to ‘develop and mobilize strategies to inform, teach and enable people living with SCI to initiate and maintain a physically active lifestyle’. The specific metrics used as part of their application of RE-AIM to the SCI Action Canada intervention are shown below.

You can read more about how they implemented this approach in the full paper here.