Skip

The Logic Model: The Foundation to Implement, Study, and Refine Patient-Centered Medical Home Models

March 2013
AHRQ Publication No. 13-0029-EF
Prepared For:
Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, 540 Gaither Road, Rockville, MD 20850, http://www.ahrq.gov/

Prepared By: Dana Petersen, Ph.D. (dpetersen@mathematica-mpr.com), Erin F Taylor, Ph.D., and Deborah Peikes, Ph.D. (Mathematica Policy Research).

Abstract

This brief focuses on using logic models to evaluate patient-centered medical home (PCMH) models. It is part of a series commissioned by the Agency for Healthcare Research and Quality (AHRQ) and developed by Mathematica Policy Research under contract, with input from other nationally recognized thought leaders in research methods and PCMH models. The series is designed to expand the toolbox of methods used to evaluate and refine PCMH models. The PCMH is a primary care approach that aims to improve quality, cost, and patient and provider experience. PCMH models emphasize patient-centered, comprehensive, coordinated, accessible care, and a systematic focus on quality and safety.

Suggested Citation

Petersen D, Taylor EF, Peikes D. Logic Models: The Foundation to Implement, Study, and Refine Patient-Centered Medical Home Models. Rockville, MD: Agency for Healthcare Research and Quality, February 2013. AHRQ Publication No. 13-0029-EF.

This brief and companion briefs in this series are available for download from pcmh.ahrq.gov.

I. The Logic Model

A logic model—also known as a program model, theory of change, or theory of action—is a graphic illustration of how a program or intervention is expected to produce desired outcomes. It shows the relationships among the inputs and resources available to create and deliver an intervention, the activities the intervention offers, and the expected results. A useful logic model does the following:

  1. Identifies the intermediate and ultimate outcomes of the intervention and the pathways through which intervention activities produce those outcomes.
  2. Shows the interrelationships among intervention components.
  3. Recognizes the influence of external contextual factors on the intervention’s ability to produce results.
  4. Helps guide program developers, implementers, and evaluators.

In this brief, we discuss how logic models can guide evaluations of PCMH models. A logic model is not only a useful evaluation tool, but also a valuable planning tool that forms the foundation for monitoring implementation. A useful logic model answers the following questions:

  1. What problem is the intervention trying to solve, and what outcomes represent success?
  2. What activities and supports are required to achieve these outcomes?
  3. What inputs and resources are needed to deliver these activities and supports?

The resulting logic model shows the links in a chain of reasoning about “what causes what” in the pathway toward the desired outcomes. Understanding the underlying logic of the intervention from start to finish allows evaluators to select measurable indicators to be used in implementation and impact analyses, including measures of whether the intervention provided sufficient resources, successfully implemented key intervention activities and delivered services as planned, and attained the outcomes of interest.

Given the complex and multifaceted nature of PCMH interventions, both implementers and evaluators are likely to benefit from a well-conceived logic model. Moreover, logic models are most accurate and effective as evaluation tools when evaluators work directly with program staff to develop the models and gain a deep understanding of exactly what the intervention is attempting to achieve, and how.

Recognizing that PCMH models can vary in their components and are more complex in reality than logic models can convey, Figure 1 provides an example of a basic and overarching logic model for a PCMH intervention. 1 As shown in the figure, a PCMH intervention depends on a variety of inputs including available funding, staff capacity, time, health information technology (IT), training and technical assistance, and the availability of practice- and patient-level data generated by payers. We use the AHRQ definition of the PCMH to specify the PCMH components, which include comprehensive care, patient-centeredness, coordinated care, accessible care, and a commitment to providing safe, high quality care (see www.pcmh.ahrq.gov). Specific interventions may operationalize these components in different ways, and these approaches may be further adapted to each practice’s unique context.

The logic model depicts selected activities related to each of the PCMH components. For example, activities related to accessible services could include developing new modes of patient communication, offering translation services, expanding office hours, and offering after-hours assistance. These activities are intended to lead to various outputs and outcomes. The evaluation measures whether they actually do, and assesses whether the inputs and resources were sufficient and intervention activities were fully implemented with fidelity to the intervention. If the intervention was implemented fully, the evaluation also tests the program theory linking a well-implemented intervention to improved outcomes.

Ideally, the PCMH intervention affects several ultimate outcomes, as shown in the far right of the figure. In this example, we use the three-part aim, as well as improved provider experience, to define the ultimate outcomes—although a PCMH intervention may tailor efforts to focus more directly on one of these or on a different outcome. Additionally, the model shows that there are multiple contextual and external factors related to the specific practice environments and the overall health care environment that could affect PCMH implementation and its ability to achieve outcomes.

back to top

II. Uses of the Logic Model

Those interested in the PCMH are keenly aware of the value of evidence about whether different models of care improve patient outcomes and reduce costs, and researchers have used different forms of logic models to describe particular medical home interventions and guide their evaluation efforts. A number of evaluators have developed logic models for various medical home initiatives in the published literature. 2

Figure 1. Basic PCMH Logic Model

Figure 1. Basic PCMH Logic Model

Below we discuss several uses of logic models.

Facilitate understanding of complex interventions. Logic models help evaluators (and program implementers) better understand complex interventions and the mechanisms through which they work. The PCMH model is a complex intervention with multiple interacting components that function on multiple levels (e.g., practice, patient, and community). A logic model can show evaluators and stakeholders at a glance what activities the intervention is providing and what the intervention intends to achieve, emphasizing the link between the two. A collaborative approach in which evaluators and implementers jointly develop a logic model is often quite useful, and sometimes uncovers disagreement or at least varying perspectives among implementers and evaluators.

Guide the development of measures. Logic models can serve as a guide for the development of measures of critical intervention inputs, processes, and outcomes. They provide a logical and theorybased structure for identifying measurable and evaluable changes (Livingood et al., 2007). By laying out the relevant components and inner workings of an intervention (the proverbial “black box”), a logic model serves as a road map for data collection—aiding in decisions about the key aspects of the intervention that merit evaluation and ensuring that evaluators identify indicators of all elements critical to the intervention theory. Logic models help evaluators identify the critical questions to be answered and guide evaluation priorities and allocation of resources. Designing data collection to align with a logic model allows evaluators to examine and test the intervention logic and provides a plausible explanation for the hypothesized causal mechanisms if desired outcomes are attained. If an intervention does not achieve desired outcomes, an evaluation firmly based on the intervention’s logic model will help reveal why. For example, tracking outputs can help evaluators determine whether ineffectiveness is the result of (1) insufficient inputs or other implementation challenges, or (2) other issues such as unavoidable external factors or incorrect logic (if the intervention was implemented with fidelity but did not have the intended effects).

Clarify goals and conceptual gaps. A logic model can help intervention planners reach consensus about their goals and uncover gaps in the intervention logic. Considering these issues at the outset of intervention development enables planners to further specify or modify resources and activities before full-scale implementation. For example, planners might address questions such as: Does a PCMH intervention to increase access contain sufficient communication with patients about newly available after-hours care so they begin to use this care? Does an intervention to improve pre-planning of chronic care visits remind practice staff in advance of the scheduled visit to review the patient’s record and order needed tests?

Track progress and changing needs. A logic model provides a critical framework for evaluators (and implementers) to monitor operations and track how the intervention evolves over time. Tracking indicators for each step in the logic model helps determine whether resources are sufficient and whether activities are being implemented according to plan. This process identifies areas for program refinement, mid-course corrections, and/or technical assistance to support ongoing implementation. For example, the logic model may posit that providers will use health IT to document patient interactions during office visits while evaluation data may show that some providers are unable to do so effectively because they do not fully understand how to use the new software—thus highlighting a need for additional provider training.

back to top

III. Advantages

Below we describe several advantages to using logic models in evaluating PCMH interventions.

Maintain focus on intervention process and context. Using a logic model holds evaluators accountable for looking at both the PCMH intervention process and its outcomes, while also revealing data needs. In the absence of a logic model, evaluators may be tempted to design data collection systems that focus largely, if not solely, on outcomes, neglecting implementation analyses that measure the processes and intermediate outcomes needed to achieve these outcomes, or not fully accounting for the context in which the intervention takes place.

Provide overview of complex processes. Logic models provide coherence across multiple, often interdependent, components and help evaluators balance brevity and complexity in data collection and analysis. Although it may seem overwhelming to fully specify all relevant linkages within a PCMH intervention, articulating a logic model, or series of logic models, can help evaluators distill the project down to its core pieces. To deliver each of the five components of a PCMH, practices may undertake a complex set of activities with specific short-term outcomes that collectively have a favorable effect on outcomes. For example, enhancing access may include developing systems to enable providers to use email to communicate with patients, extending office hours, providing translation services, and having a physician and a nurse on call after hours. In cases like these, the evaluator may consider developing one overarching model that shows the key components of the intervention (Figure 1) and supplemental models for specific parts of the intervention that benefit from additional elaboration (in this case introduction of email visits) (Figure 2). Logic models can help guide systematic approaches to evaluation that balance the need to limit data collection efforts for efficiency’s sake, yet capture the complexity of medical home constructs (for example, see Livingood et al., 2007).

Figure 2. Logic Model of Specific PCMH Strategy Related to Accessible Services

Figure 2. Logic Model of Specific PCMH Strategy Related to Accessible Services

Highlight systems change and interactions. Logic models are well suited for better understanding systems change involved with complex interventions. PCMH evaluations must not only capture details of how each component is implemented, but also document how these components interact with one another over time to affect desired outcomes. This can be done by using a set of logic models as described above—specific models for intervention components and an overarching model that describes the key relationships and interactions among all components—thus forming the overall system. For additional information on the application of logic models to health systems research, see Alexander and Hearld (2012) and Crabtree et al. (2011).

Adapt to interventions of any scale. The use of logic models is not restricted to larger-scale, rigorous impact studies. They are effective when used in evaluations of single practice interventions, pilot projects with a small number of practices, and case studies. Creating a logic model to examine new investments may also help uncover gaps in the logic of transformation and assess whether the intervention has a reasonable chance of improving desired outcomes.

back to top

IV. Limitations

As with any research tool, there are some challenges with using logic models.

Uses simplified format. Logic models aim for simplicity rather than accurately capturing every detail of an intervention. Many interventions, especially PCMH interventions, are too interactive, recursive, and complex to be fully depicted in a logic model, or even in a series of logic models. The goal is to develop a logic model that balances brevity and complexity, distilling an intervention to its key components and relationships without oversimplifying connections.

Involves balancing depth and detail. Establishing appropriate boundaries on logic models can be difficult. This is especially true when trying to maintain simplicity in the graphic illustration. Deciding what level of detail to include on intervention activities is challenging and there is often a tension between focusing more narrowly on facets of the intervention, and specifying all the external forces that might affect outcomes (such as the practice setting and the health care environment). The challenge is to include enough depth to identify the context and key components of the model, without losing the ability to understand the intervention as a whole.

Requires mid-course review. Some mistakenly consider logic models static, missing the evolving nature of interventions when implemented. Delivery models change over time in response to refinements made by the implementers, formative feedback from evaluators, and external influences such as changing insurance policies, payment approaches, and national, regional, and local contexts. It is important for implementers and evaluators alike to regularly review and update logic models to ensure that they reflect innovation and current conceptions of the PCMH intervention.

May inhibit creative thinking. Logic models are sometimes used too rigidly, which can stifle innovative thinking and adaptive management. Interventions are often not as linear or contained as logic models portray, and it is important for implementers and evaluators to remain open minded and flexible so that they recognize unforeseen dynamics and outcomes when they arise. When applied to evaluations, an inflexible use of logic models can lead to data collection methodologies that miss unintended consequences or outcomes. For example, if evaluators were to stick rigidly to Figure 2 above, they might not capture information on unintended consequences of introducing patientprovider email communication systems, such as staff burnout due to additional time spent checking and responding to email, reduced practice revenue if providers are not reimbursed for time spent emailing and fewer patients come in for office visits, and medical errors if the absence of an in-person visit leads the provider to miss some key information about the patient’s condition.

Must be combined with other tools. A strong logic model is just one of many tools needed in a strong evaluation. Logic models alone do not establish causality, although they are an extremely useful and complementary tool that some evaluators overlook.

back to top

V. Conclusion

Logic models that illustrate how an intervention is expected to produce desired outcomes are not only useful evaluation tools, but also valuable planning tools that form the foundation for monitoring implementation. Logic models help evaluators and implementers to better understand complex interventions such as the PCMH and the mechanisms through which they work. Together with other tools, a strong logic model can guide evaluation design, data collection, and analysis, and serve as a useful framework for interpreting results.

back to top

VI. References

  1. Agency for Healthcare Research and Quality, Patient Centered Medical Home Resource Center. Defining the PCMH. www.pcmh.ahrq.gov/page/defining-pcmh
  2. Alexander JA, Hearld LR. Methods and metrics challenges of delivery-system research. Implement Sci 2012;12:7–15.
  3. Crabtree BF, Chase SM, Wise CG, et al. Evaluation of patient centered medical home transformation initiatives. Med Care 2011; 49(1):10–6.
  4. Livingood WC, Winterbauer NL, McCaskill Q, Wood D. Evaluating medical home constructs for children with special needs: integrating theory and logic models. Fam Community Health 2007;30(5):E1–15.
back to top

VII. Resources

General Guides for Developing a Logic Model

  1. Innovation Network, Inc. Logic model workbook. www.innonet.org/client_docs/File/logic_model_ workbook.pdf
  2. Jackson B. Designing projects and project evaluations using the logical framework approach. International Union for the Conservation of Nature and Natural Resources (IUCN) Monitoring and Evaluation Initiative, The World Conservation Union, 1997. cmsdata.iucn.org/downloads/ logframepaper3.pdf
  3. McCawley PF. The logic model for program planning and evaluation. University of Idaho Extension. http://www.uiweb.uidaho.edu/extension/LogicModel.pdf.
  4. University of Kansas. Community tool box. ctb.ku.edu/en/tablecontents/sub_section_main_1877.aspx
  5. University of Wisconsin-Cooperative Extension. Program development and evaluation. http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html. (Accessed August 2012.)
  6. WK Kellogg Foundation. Logic model development guide. Battle Creek, MI: WK Kellogg Foundation, 2004. http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx

Examples of Logic Models for the PCMH

  1. Agency for Healthcare Research and Quality. IMPaCT (Infrastructure for Maintaining Primary Care Transformation). www.ahrq.gov/research/impactaw.htm
  2. Alexander JA, Hearld LR. Methods and metrics challenges of delivery-system research. Implement Sci 2012;12:7–15.
  3. Bitton A. Evaluating clinical quality in the patient centered medical home. Presentation at National Medical Home Summit; 2011 Mar 14; Philadelphia (PA). www.ehcca.com/presentations/medhomesummit3/bitton_ms1_2.pdf
  4. Crabtree BF, Chase SM, Wise CG, et al. Evaluation of patient centered medical home transformation initiatives. Med Care 2011; 49(1):10–6.
  5. Huang ES. Will the patient-centered medical home improve efficiency and reduce costs of care? A measurement and research agenda. University of Chicago, PCMH Evaluators Collaborative Efficiency Workgroup, 2010. http://www.commonwealthfund.org/~/media/Files/Publications/Other/2010/Nov/huang.pdf
  6. Livingood WC, Winterbauer NL, McCaskill Q, Wood D. Evaluating medical home constructs for children with special needs: integrating theory and logic models. Fam Community Health 2007;30(5):E1–15.
back to top

Footnotes

  1. 1. Logic models can vary in their complexity and take many different forms, including flowcharts, tables, pictures, and diagrams, and can include different components. For additional information on logic model uses and development, see the “General Guides for Developing a Logic Model” section in the Resources at the end of this document.
  2. 2. See “Examples of Logic Models for the PCMH” articles in the Resources section.