The Milbank Memorial Fund is an endowed operating foundation that publishes The Milbank Quarterly, commissions projects, and convenes state health policy decision makers on issues they identify as important to population health.
We focus on a number of topic areas identified by state health policy leaders as important to population health.
The Center for Evidence-based Policy at Oregon Health & Science University is a national leader in evidence-based decision making and policy design.
Keep up with news and updates from the Milbank Memorial Fund. Get the latest from thought leaders, including Christopher F. Koller, president of the Fund.
We publish The Milbank Quarterly, as well as reports and issues briefs on topics important to population health.
September 2012 (Volume 90)
September 2012 | Trisha Greenhalgh, Fraser Macfarlane, Catherine Barton-Sweeney, Fran Woodard
Context: The long-term sustainability of whole-system change programs is rarely studied, and when it is, it is inevitably undertaken in a shifting context, thereby raising epistemological and methodological questions. This article describes a transferable methodology that was developed to guide the evaluation of a three-year follow-up of a large health care change program in London, which took place during a period of economic turbulence and rapid policy change.
Methods: Using a mixed-method organizational case study design, we studied three services (stroke, kidney, and sexual health) across primary and secondary care. Each had received £5 million (US$7.8 million) in modernization funding in 2004. In 2010/2011, we gathered data on the services and compared them with data from 2004 to 2008. The new data set contained quantitative statistics (access, process, and outcome metrics), qualitative interviews with staff and patients, documents, and field notes. Our data analysis was informed by two complementary models of sustainability: intervention-focused (guided by the question, What, if anything, of the original program has been sustained?) and system-dynamic (guided by the question, How and why did change unfold as it did in this complex system?).
Findings: Some but not all services introduced in the original transformation effort of 2004–2008 were still running; others had ceased or been altered substantially to accommodate contextual changes (e.g., in case mix, commissioning priorities, or national policies). Key cultural changes (e.g., quality improvement, patient centeredness) largely persisted, and innovative ideas and practices had spread elsewhere. To draw causal links between the original program and current activities and outcomes, it was necessary to weave a narrative thread with multiple intervening influences. In particular, against a background of continuous change in the local health system, the sustainability of the original vision and capacity for quality improvement was strongly influenced by (1) stakeholders’ conflicting and changing interpretations of the targeted health need; (2) changes in how the quality cycle was implemented and monitored; and (3) conflicts in stakeholders’ values and what each stood to gain or lose.
Conclusions: The sustainability of whole-system change embodies a tension between the persistence of past practice and the adaptation to a changing context. Although the intervention-focused question, What has persisted from the original program? (addressed via a conventional logic model), may be appropriate, evaluators should qualify their findings by also considering the system-dynamic question, What has changed, and why? (addressed by producing a meaningful narrative).
Author(s): Trisha Greenhalgh, Fraser Macfarlane, Catherine Barton-Sweeney, and Fran Woodard
Keywords: whole-system transformation, sustainability, complexity, evaluation, health care reform, organizational innovation
Read on Wiley Online Library
Read on JSTOR
Volume 90, Issue 3 (pages 516–547)
Published in 2012
What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program
Off the Hamster Wheel? Qualitative Evaluation of a Payment-Linked Patient-Centered Medical Home (PCMH) Pilot