Large-System Transformation in Health Care: A Realist Review

Large-System Transformation in Health Care: A Realist Review

ALLAN BEST , 1 TRI SHA GREENHALGH, 2

STEVEN LEWI S , 3 JESS I E E . S AU L , 4 S I M O N CARROLL , 5 and J E N N I F E R BI T Z 1

1InSource Research Group; 2Barts and the London School of Medicine and Dentistry; 3Access Consulting Ltd.; 4North American Research and Analysis Inc.; 5University of Victoria

Context: An evidence base that addresses issues of complexity and context is urgently needed for large-system transformation (LST) and health care reform. Fundamental conceptual and methodological challenges also must be addressed. The Saskatchewan Ministry of Health in Canada requested a six-month synthesis project to guide four major policy development and strat- egy initiatives focused on patient- and family-centered care, primary health care renewal, quality improvement, and surgical wait lists. The aims of the review were to analyze examples of successful and less successful transformation initiatives, to synthesize knowledge of the underlying mechanisms, to clarify the role of government, and to outline options for evaluation.

Methods: We used realist review, whose working assumption is that a particular intervention triggers particular mechanisms of change. Mechanisms may be more or less effective in producing their intended outcomes, depending on their interaction with various contextual factors. We explain the variations in outcome as the interplay between context and mechanisms. We nested this analytic approach in a macro framing of complex adaptive systems (CAS).

Findings: Our rapid realist review identified five “simple rules” of LST that were likely to enhance the success of the target initiatives: (1) blend des- ignated leadership with distributed leadership; (2) establish feedback loops; (3) attend to history; (4) engage physicians; and (5) include patients and fami- lies. These principles play out differently in different contexts affecting human

Address correspondence to: Allan Best, InSource Research Group, 6975 Marine Drive, West Vancouver, BC, Canada V7W 2T4 (email: allan.best@in-source.ca).

The Milbank Quarterly, Vol. 90, No. 3, 2012 (pp. 421–456) c© 2012 Milbank Memorial Fund. Published by Wiley Periodicals Inc.

421

THE

MILBANK QUARTERLY A MULTIDISCIPLINARY JOURNAL OF POPULATION HEALTH AND HEALTH POLICY

422 A. Best et al.

behavior (and thereby contributing to change) through a wide range of different mechanisms.

Conclusions: Realist review methodology can be applied in combination with a complex system lens on published literature to produce a knowledge syn- thesis that informs a prospective change effort in large-system transformation. A collaborative process engaging both research producers and research users contributes to local applications of universal principles and mid-range theories, as well as to a more robust knowledge base for applied research. We con- clude with suggestions for the future development of synthesis and evaluation methods.

Keywords: health policy, health care reform, organizational innovation, com- plex adaptive systems, realist evaluation, realist review.

Background

most of the published literature on change in health care describes relatively small-scale initiatives typically carried out by a single health care organization or service. An evidence base thus is urgently needed for large-system transformation (LST), as there is no agreed-on definition of LST in the literature. In this article we offer our working definition:

Large-system transformations in health care are interventions aimed at coordinated, systemwide change affecting multiple organizations and care providers, with the goal of significant improvements in the efficiency of health care delivery, the quality of patient care, and population-level patient outcomes.

The relatively sparse literature on LST highlights the crucial in- fluence of political and institutional context. For example, Carolyn Tuohy compared the process of large-scale changes in health care in the United States, Canada, the United Kingdom, and the Netherlands and demonstrated different institutional logics (respectively, “mosaic,” “incremental,” “big bang,” and “blueprint”) in these different political contexts and health systems (Tuohy 1999). While we acknowledge the significance of political analyses in informing LST efforts in health care, we also need evidence for the social mechanisms by which transformative

Large-System Transformation in Health Care 423

efforts may achieve their impacts in different contexts, which we address here.

In 2001, the Institute of Medicine (IOM) produced a landmark report entitled Crossing the Quality Chasm, which endorsed the idea that health care systems are complex adaptive systems (CAS) (IOM 2001). This report followed an important publication on CAS in 1998 (Zimmerman, Lindberg, and Plsek 1998) and emerged at the same time as an influential series of papers in the British Medical Journal (e.g., Plsek and Greenhalgh 2001, Plsek and Wilson 2001), which emphasized the value of a CAS lens to better understand how to improve and transform health systems.

These publications argued that although CAS are complex and un- predictable, they are amenable to guided transformation by applying simple rules that are sufficiently flexible to allow for adaptation, an important operating principle for potential agents of health system transformation. In contrast to many top-down LST efforts, a CAS ap- proach seeks to draw out and mobilize the natural creativity of health care professionals to adapt to circumstances and to evolve new and better ways of achieving quality (Lanham et al. 2009). To improve processes and outcomes, the key is to create positive conditions for change by supporting a work environment conducive to harnessing both relationships and the skills and capacities of individuals in the system.

The implications for planning are far-reaching. The agent of change must give up notions of “control” over the process of change and should avoid language that emphasizes “overcoming resistance” (Plsek and Wilson 2001; Sterman 2006). Instead, efforts should be directed to- ward iterative planning and practice cycles that build on an understand- ing that successful action is less about meeting targets and more about shifting the system’s behavior through generic guidance and steering mechanisms. Changing the principles by which people carry out their work is much more important than attaining a predefined target (which may have been arbitrary in the first place).

Implementing change in CAS requires constant monitoring and adap- tation to new contexts. Building in principles and resources that sup- port a learning environment (Senge 1990) allows organizations to take full advantage of local knowledge in generating continuous improve- ments. Similarly, evaluating change in LST, as informed by a CAS lens, means adopting appropriate goals and objectives, not overspecifying

424 A. Best et al.

multiple outcomes, and paying attention to positive movement in generic processes that support improvement.

All this means that the evidence base for achieving LST cannot take the form of hard and fast statements about “what works.” Rather, richly described case studies of LST efforts will lend themselves to (at best) making broad statements about “what tends to work, for whom, in what circumstances” and to explaining the fortunes of particular pro- grams as mechanisms in context. A well-matched approach to generating such statements and explanations in empirical studies is realist evalua- tion (Pawson and Tilley 1997). A realist review (the secondary research equivalent to realist evaluation) is an interpretive, theory-driven narra- tive summary that uses cross-case comparison to understand and explain how and why different outcomes have been observed in a sample of primary studies (Pawson et al. 2005). The working assumption behind realist review is that a particular intervention (or class of interventions) triggers particular mechanisms of change somewhat differently in dif- ferent contexts. In realism, it is mechanisms—defined as “underlying entities, processes, or [social] structures which operate in particular contexts to generate outcomes of interest” (Astbury and Leeuw 2010, 368)—that trigger change rather than interventions themselves. In other words, realist reviews focus on “families of mechanisms” rather than on “families of interventions.” An explanation of the interplay of context, mechanism, and outcomes is then sought. The reviewer constructs one or more “mid-range theories” (i.e., more fine-grained than “grand theories” but still open to flexible interpretation in different contexts) to account for the findings. A realist review methodology focuses on the types of interactions between local context and specific mechanisms of change that make up the foundation of CAS and, more specifically, LST for CAS. An international collaborative study to develop methodological guid- ance and reporting standards for realist review is ongoing (Greenhalgh et al. 2011).

A CAS perspective draws attention to the basic rules or principles of action of a system and its environmental parameters, and a realist perspective seeks to unpack and explore particular mechanisms and how they interact with the context. Ensuing policy recommendations avoid elaborate checklists or specific instructions for change. Rather, the recognition of complexity tied to a focus on “theories of change” allows (indeed, requires) researchers to begin by examining the local context and expressing findings as broad principles of action and contingent

Large-System Transformation in Health Care 425

approaches (“in situations like X, try Y and watch out for Z”). It has been argued that the conclusions from a realist review may be more helpful to policy than those of Cochrane-style systematic reviews and meta- analyses, which address a narrower set of questions framed in explicitly experimental terms in which context has been “controlled for” or reduced to a handful of predefined variables whose influence is assumed to be constant and predictable (“what is the impact of intervention X on outcome Y, and what is the influence of mediating variables M1, M2 and M3?”) (cf. Berwick 2008).

In this article we describe a realist review of approaches to large- system transformation (LST), taking account of the policy contexts in which they were undertaken. In the discussion, we return to more gen- eral theoretical and methodological issues and consider the extent to which the broad principles and contingent lessons identified by the realist approach proved useful to the policymakers who commissioned the review. We also look at the implications—both empirical (wider lessons for those seeking to implement or support LST) and method- ological (wider lessons for realist reviewers and those contemplating commissioning such reviews).

The KAST Project

The Knowledge to Action for System Transformation (KAST) project was designed to provide a rapid systematic review and synthesis of knowledge about LST for the provincial Saskatchewan Ministry of Health in Canada. The review was funded by the Canadian Institutes of Health Research (CIHR), under a pilot “Evidence on Tap” program and its “Expedited Knowledge Synthesis” mechanism (CIHR 2011). The Saskatchewan ministry requested the six-month synthesis project (April to September 2010) to guide four major policy development and strat- egy initiatives: patient- and family-centered care, primary health care improvement, “lean” management for health care, and shorter surgical wait times (the “Saskatchewan Surgical Initiative”).

In this rapid review for a specific policy sponsor, we defined a sys- tematic review as a review of the literature according to an explicit, rigorous, and transparent methodology rather than as an exhaustive and comprehensive summary of every paper ever published on the topic (Greenhalgh et al. 2004). We applied realist methodology (Pawson

426 A. Best et al.

2002a, 2002b; Pawson et al. 2005), a principle of which is to address policymakers’ needs in context. For example, within the geographic and political boundaries of the Saskatchewan health system (which nei- ther we nor the sponsors of the review were in a position to change), we sought to inform the structural, process, and policy changes needed to support change in each current initiative. We distinguished among research, theory, and practice knowledge (Best and Holmes 2010; Van de Ven 2007) and posited that a synthesis of all three kinds of knowl- edge would be necessary to make inferences about the factors influencing large-system transformation and how they might interact dynamically over time.

The four preliminary objectives for the synthesis were the following:

1. Identify a range of examples of LST that were more or less successful, and in those examples, determine the role of the provincial government, including its policy development and implementation.

2. Develop a deeper understanding of the mechanisms that con- tribute to success in LST and how these play out in different contexts.

3. Identify barriers and challenges to LST, and recommend what roles the government might play in addressing (or working around) them.

4. Identify options for monitoring and evaluating the processes and outcomes for LST.

Our research questions were the following:

1. What are the key mechanisms or social processes that influence or drive successful large-system transformation in the health care sector?

2. What are the contextual factors that have the most impact (pos- itive or negative) on large-system transformation efforts in the health care sector?

3. If there are identifiable “transition” points in large-system trans- formation efforts, how do the key mechanisms and contexts in- teract to produce these changes?

4. What is the role of government in large-system transformation efforts?

Large-System Transformation in Health Care 427

Methods

Framing the Problem with Policymakers’ Input

A preliminary step in realist review is dialogue among the research team members as well as with the intended user(s) of the review, to clarify its focus and prioritize questions. The steering committee, convened by the Saskatchewan Ministry of Health, was made up of senior representa- tives from relevant ministry divisions, regional health authorities, and a provincial quality council. The committee met three times with the research team and periodically by teleconference, and they all exchanged emails throughout the project. The ministry provided to the research team the principal (i.e., updated during the project’s life span) back- ground documents on the four strategic initiatives; they described and discussed the relevant context; and the committee added input to the draft high-level statements extracted and synthesized from the litera- ture to ensure clarity and accessibility of language and meaning for the diverse stakeholders for whom the output of the review was aimed.

There was broad agreement that even though there was much scope for improvement in Saskatchewan (care was, in general, far from patient or family centered; primary health care provision was variable and lim- ited in scope; the duplication of local and regional health systems was inefficient; and surgical waiting lists were long), there was also a high degree of inertia in the system and (perceived) limited motivation for change, which ministry staff members hoped to change. Thus, although the steering committee wanted us to answer the realist question “what works for whom under what circumstances?” they also needed specific recommendations for ways that the government (i.e., the Saskatchewan Ministry of Health) could work to effect, support, and sustain transfor- mation in the four areas previously identified as priorities.

This prioritization linked well with our chosen realist approach. Con- ventional change management research tends to focus on defining a set of abstracted variables and quantifying the (assumed) causal links between them—such as “top management support,” “dedicated budget,” and “training.” In contrast, the mechanisms that are the focus of realist review are considered to work either wholly or largely through the perceptions, reasoning, and actions of human actors. In other words, the mechanisms set out how the people on whose efforts LST depends actually use pro- gram resources such as top management support, financial resources, or

428 A. Best et al.

training to make the changes happen and sustain them—and how their efforts play out differently in different contexts (Astbury and Leeuw 2010).

Creating an Interpretive Dialogue

We knew that academic publications on LST were sparse and that many high-quality studies on this topic were situated in the gray literature. We also were aware that evidence on LST is complex and nuanced, that it is interpreted differently by different stakeholders, and that (hence) there are few, if any, a priori truths. For these epistemological reasons, we chose to undertake the review alongside a series of dialogues. In addition to the steering committee, we convened an expert panel and a consultation group.

The international expert panel was composed of eight leaders from Canada and the United Kingdom, whose expertise spanned systematic reviews, system transformation, and the four strategic topic areas we had been charged with informing (patient- and family-centered care, surgical wait lists, quality improvement, and primary health care re- newal). The research team maintained an interactive dialogue with these experts, mostly via email. Discussion among the experts was prompted at strategic points throughout the review with a view to gaining critical feedback on the research questions, the literature review methods, and the presentation and interpretation of findings.

In addition, toward the end of the review period, we became aware that many of the gaps that we had identified in the literature, partic- ularly the lack of granularity in published findings, might be filled in by the expertise of those currently or very recently involved in trans- formation efforts. To that end, a consultation group of forty-four in- ternational leaders participated in a short online survey in which they were asked to comment on the preliminary findings to help refine rec- ommendations for government action based on their knowledge and experiences.

Search Methods

Realist review recognizes the limitations of fixed search protocols and instead encourages iterative searching that begins with a broad

Large-System Transformation in Health Care 429

direction, is refined through progressive focusing, and responds flexibly to emerging findings (Pawson et al. 2005). The initial search protocol ad- dressed health services and macro-systems transformation in relation to health care reform, surgical initiatives, “lean” culture, patient-centered care, and primary health care renewal. Members of the research team and expert panel provided search terms from which the list of key terms for each concept was built (see table 1). The published literature was searched using these terms and the subject headings in the databases MEDLINE and EMBASE from 2000 to the present. Numerous iterative searches were performed in these databases, resulting in nearly 1,000 po- tential references. Two members of the research team reviewed the titles and abstracts for relevance based on broad inclusion criteria, including the use of a theory-driven approach to identifying the underlying mech- anisms that were driving change (particularly the ways in which human agency drew on program resources to achieve goals), using methods or descriptions that were consistent with a complex systems and/or real- ist perspective, a focus on whole or partial system transformation with lessons that could be applied on a macro level, the adaptability of the findings to a Saskatchewan context, a focus on the “why” and “how” of system transformation, and articles written by major authors in the field. From the nearly 1,000 references, 211 were selected for further review based on these criteria. Two of the team members again reviewed all 211 references, based on titles and abstracts, and assigned them to one or more of five categories (LST broadly construed, lean culture, patient- and family-centered care, primary health care renewal, and shorter surgical wait times), depending on the document’s scope and content. When the team members were unsure which, if any, mechanisms of change were evident in a particular paper, we sought advice from our wider research group (up to four of whom considered the paper and discussed the candi- date mechanisms). When two reviewers disagreed on whether to select a reference for full review, their disagreement was resolved by discussion. Of the 211 references considered, 114 were reviewed in their entirety.

Based on discussions among the team members, the expert panel, and the ministry, the depth and type of the searches evolved. We searched the references from papers in each of the five topical categories for other relevant papers and hand-searched six journals from health, busi- ness, and sociology dated 2000/2010 and known to publish papers on LST. We also reviewed the full texts of 64 papers from the six journals.