Anthony F. Lehman, Howard H. Goldman, Lisa B. Dixon,
and Rachel Churchill
(To request a bound copy of this report, click here.
To see a complete list of Milbank reports, click here.
When ordering, be sure to specify which report you want,
your name, mailing address, and phone number.)
Table of Contents
The purpose of this report is to inform policymakers about the significance of recent advances in evaluating evidence for allocating resources to and within public mental health programs. These advances are of particular importance because the public sector is the largest payer for services to persons with chronic mental illness.
The authors' central point is that the best evidence yields both good and bad news. The good news is that "many potentially available treatments and services" have been shown to produce "improved symptoms and functioning" in patients with severe, persistent mental illness. The bad is that "there are substantial gaps between what science tells us to do and what we do in actual practice, despite the significant investment of public resources."
The research about which the authors report is increasingly useful to policymakers and clinicians. Many scientists are working collaboratively, often across national boundaries, to collect, analyze, and synthesize evidence about interventions to prevent and treat illness. A recent Milbank Report describes this international scientific advance (Ray Moynihan, Evaluating Health Services: A Reporter Covers the Science of Research Synthesis).
This report on the significance for policy of evidence about mental health services began as a request from policymakers. Leaders in the legislative and executive branches of government in Maryland asked the Milbank Memorial Fund to collaborate in organizing a meeting to discuss the strengths and weaknesses of that state's public mental health programs. The Fund asked Anthony Lehman, Chair of the Department of Psychiatry at the University of Maryland School of Medicine, to prepare a background paper on evidence-based mental health treatments and services for this meeting. Lehman, a national leader in urging that the best available evidence should guide mental health services, invited his departmental colleagues Howard Goldman and Lisa Dixon to be coauthors. Because of the international reach of the science that underlies evidence-based practice, Lehman also invited Rachel Churchill, the London-based Coordinating Editor of the Cochrane Collaboration's Depression, Anxiety and Neurosis Review Group, to be a coauthor. The Cochrane Collaboration is the leading international organization in reviewing systematically evidence about health services.
The Fund invited policymakers, researchers, and clinicians to review this report in draft; they are listed in the Acknowledgments. As a result of both the authors' and reviewers' expertise, this report is a useful guide for understanding how science can inform policy for mental health services.
Daniel M. Fox
Samuel L. Milbank
The following persons reviewed this report in draft. They are listed in the positions they held at the time of their participation.
Bernard S. Arons, Senior Science Advisor to the Director, National Institute of Mental Health, Bethesda, Md.; Steve Baron, President, Baltimore Mental Health Systems; Tom Barrett, Director, Colorado Department of Human Services, and President, National Association of State Mental Health Program Directors; Yvonne Block, Acting Director, Mental Health, Manitoba Health, Canada; Sharon E. Carpinello, Executive Deputy Commissioner, New York State Office of Mental Health; Patricia Carter, Clinical Director, Children, Youth and Families, Missouri Department of Mental Health; Kevin Concannon, Director, Iowa Department of Human Services; John Conway, Member, Saskatchewan Health Quality Council, and Professor of Psychology Emeritus, University of Saskatchewan, Canada; Gene Davis, Member, Executive Appropriations Committee, Utah Senate; Adelaide C. Eckardt, Member, Joint Health Care Delivery and Financing Committee, Maryland House of Delegates; Susan Essock, Director, Division of Health Services Research, Mount Sinai School of Medicine, New York; Wayne Fenton, Deputy Director for Clinical Affairs, Division of Mental Disorders, National Institute of Mental Health, Bethesda, Md.; Junius Gonzales, Chief, Services Research and Clinical Epidemiology Branch, National Institute of Mental Health, Bethesda, Md.; Gail Gray, Director, Montana Department of Public Health and Human Services; Jessie C. Gruman, Executive Director, Center for the Advancement of Health, Washington, D.C.; William Hargreaves, Department of Psychiatry, University of California, San Francisco; Toni Nathaniel Harp, Chair, Appropriations Committee, Connecticut Senate, and Chair, Health Committee, National Conference of State Legislatures; Margaret Henbest, Minority Caucus Chair, Idaho House of Representatives; Brian Hepburn, Acting Executive Director, Mental Hygiene Administration, Maryland Department of Health and Mental Hygiene; Paula C. Hollinger, Chair, Education, Health and Environmental Affairs Committee, Maryland Senate; Pamela S. Hyde, Secretary, New Mexico Department of Human Services; Billy E. Jones, B. Jones Consulting Services, New York; Jeffrey C. Lerner, President and Chief Executive Officer, ECRI, Plymouth Meeting, Pa.; Joyce McRoberts, Deputy Director, Idaho Department of Health and Welfare; David Mechanic, Professor of Behavioral Science, Institute for Health, Health Care Policy and Aging Research, Rutgers University, New Brunswick, N.J.; Kenneth C. Montague, Jr., Secretary, Maryland Department of Juvenile Justice; Oscar Morgan, Senior Consultant, Healthcare Policy, National Mental Health Association, Alexandria, Va.; Karen Oliver, National Institute of Mental Health, Bethesda, Md.; Margaret Pennington, Commissioner, Kentucky Department for Mental Health and Mental Retardation, and Vice President, National Association of State Mental Health Program Directors; Sheila Peterson, Director, Fiscal Management Division, North Dakota Office of Management and Budget; Daryl C. Plevy, Acting Chief, Montgomery County Adult Mental Health and Substance Abuse Services, Rockville, Md.; Simon G. Powell, Principal Analyst, Office of Policy Analysis, Maryland Department of Legislative Services; Linda Raines, Executive Director, Mental Health Association of Maryland; Peggy Reeves, Member, Joint Budget and Appropriations Committee, Colorado Senate; Robert Rosenheck, Professor of Psychiatry, VA Administration, West Haven, Conn.; Steven S. Sharfstein, President and Chief Executive Officer, Sheppard and Enoch Pratt Hospital, Baltimore, Md.; Greg Simon, Investigator, Center for Health Studies, Group Health Cooperative, Seattle, Wash.; Richard Surles, Senior Vice President, Medical Information Technologies, Comprehensive NeuroScience, Inc., Pennington, N.J.; Leticia Van de Putte, Member, Education Committee, Texas Senate.
States continue to bear the major responsibility for mental health care for persons with serious and disabling mental illnesses, both through direct provision of services and through funding of private-sector care. Central to this mission is how best to allocate limited resources so as to gain maximum value for patients, families, and society. This briefing makes the following essential points, illustrated with examples of evidence-based practices:
- A substantial body of outcomes research supports the efficacy of a wide range of mental health treatments.
- Mental health services can be expected to provide evidence-based practices in order to yield good outcomes.
- Mental health service authorities and providers should be held accountable for providing services consistent with evidence-based practices.
- Measures of "program fidelity" have been developed that permit monitoring and accountability.
- Outcomes should be monitored regularly by clinicians as a part of good practice.
- The wide array of effective treatments should be available within a community, because even when treatments are equally effective on average, many of them are not equally effective for significant subgroups.
- Treatment choice and wide selection are essential in order to maximize treatment response and adherence to treatment.
- Access to evidence-based practices is necessary but not sufficient to ensure a quality mental health service system.
Glossary of Terms
Blinding: A research technique often used in clinical trials, such that the patient does not know, and often the researchers do not know, to what treatment condition the patient has been assigned. This reduces the risk of bias occasioned by patients' or researchers' knowledge about what treatment is actually being received, influencing response to treatments or interpretation of results. Once the study is completed, the "blind" is broken so that the results can be analyzed and interpreted.
Collaborative Care: In the context of this paper, refers to approaches to treating patients with mental illness in primary care settings through the collaboration of primary care personnel (doctors, nurses, etc.) and mental health specialists.
Data Extraction Form: A form used to summarize information from research studies in a standardized manner in order to compare results across studies.
Efficacy v. Effectiveness: "Efficacy" refers to how well a treatment works under carefully controlled research conditions. "Effectiveness" refers to how well the treatment works when applied in more typical clinical circumstances.
Home Rule: The principle or practice of self-government in the internal affairs of a dependent political unit. In the context of this paper, "home rule" refers to the increased importance of local (mental health) authorities in policy and decision making devolved from state government.
Meta-Analysis: A set of statistical techniques used to combine the results of multiple research studies in order to answer certain questions.
Negative Symptoms: Symptoms that occur in some persons with schizophrenia, including loss of emotional expression (flattened affect), poverty of speech, and loss of drive. They are referred to as "negative" because they are defined by loss of functions.
New Federalism: Refers to the recent national trend according to which certain governmental responsibilities and authority are being decentralized from the federal government to state governments. This is one aspect of a broader national tendency to devolve authority and responsibility to more decentralized levels of government.
Patient Attrition: This occurs when patients in a research study drop out of the study, for any reason, before the study is completed.
Positive Symptoms: Symptoms that occur in some persons with schizophrenia, including hallucinations, delusions, and disordered thinking. They are referred to as "positive" because they are experiences beyond the normal range of functioning.
Program Fidelity: The degree to which a treatment or program as it is delivered in usual practice remains true to the treatment as originally developed and evaluated in a research study.
Randomized Clinical Trial: A standard design for research studies evaluating alternative treatment strategies, in which subjects are assigned by chance (for example, by flip of a coin) to alternative treatments. Randomization is used to reduce the risk that results are inadvertently biased in favor of one treatment over anotherso-called treatment allocation bias (see below).
Treatment Allocation Bias: Refers to the undesirable situation in a clinical trial in which research subjects with different characteristics are assigned to different treatments, thus biasing the results toward a particular treatment. Randomization seeks to eliminate this bias.
Usual Community Care: In treatment studies that compare alternative treatment approaches, refers to the treatment usually provided in the absence of the research study. Typically, it is the comparison or "control" condition with which experimental treatments are compared.
The states have historically assumed substantial responsibility for the care of persons with severe and disabling mental illnesses across the life span. This responsibility manifests itself in the form of treatment programs directly operated by or contracted for by the states, as well as in the form of state Medicaid programs that support private-sector services. Many persons with severe mental illnesses rely upon these programs as a result of their disability, and the consequent poverty and unemployment to which they are vulnerable. Hence, the treatment of persons with severe mental illnesses represents a major investment of resources for states, as well as for other local government entities. It is essential that these resources be invested wisely so as to maximize value to patients, families, and society.
A significant approach to increasing the value gained from the expenditure of health care dollars is adoption of evidence-based practices: that is, the purchase of treatments and services that have been scientifically confirmed to improve outcomes. With respect to treatment for persons with severe mental illnesses, the news is both good and bad. The good news is that solid scientific evidence suggests that many potentially available treatments and services are efficacious; substantial gains in the form of improved symptoms and functioning are possible with the right treatment. The bad news is that there are substantial gaps between what science tells us to do and what we do in actual practice, despite the significant investment of public resources.1 These gaps exist for several reasons:
- The knowledge and skills of practitioners, as well as of state mental health authorities, lag substantially behind the evidence. Hence, practitioners and service systems often continue to provide some interventions that either are unsupported by evidence or have been shown to be ineffective.
- Policies related to the expenditure of public mental health dollars often do not hold practitioners or public mental health authorities accountable to provide evidence-based practices and to eliminate practices that do not help people. Monitoring program fidelity and outcomes is essential for ensuring this accountability.
- Public funding for mental health services in various instances is often inadequate, or is constrained in such ways as to make support for certain evidence-based practices difficult (for example, the awkward and multiple funding streams needed to support certain employment rehabilitation services), or is poorly invested for other reasons.
This report illustrates the substantial scientific basis for treatments and services for adults, children, and adolescents with severe mental disorders, with the aim of providing a foundation for policy discussions at various levels of government, especially states, geared to planning their investment of mental health care dollars to enhance the value of the services they fund or provide. The report is not intended to provide a comprehensive summary of evidence-based practices in mental health, but rather to provide instructive examples of such practices. Readers interested in a comprehensive summary of evidence-based practices are referred to Mental Health: A Report of the Surgeon General, 2 the National Guideline Clearinghouse of the Agency for Healthcare Research and Quality (AHRQ) recommendations,3 and professional clinical practice guidelines.4 In England the National Institute for Clinical Excellence (NICE)5 and in Scotland the Scottish Intercollegiate Guidelines Network (SIGN)6 produce clinical guidelines using evidence-based reviews. State-of-the-art evidence reviews are available in the Cochrane Library7 and in reviews commissioned by AHRQ from the Evidence-Based Practice Centers.8
The policy challenge is not only how to ensure that there are adequate resources to enable the provision of evidence-based practices, but how to hold practitioners and state mental health authorities accountable to ensure that these practices are delivered in an effective manner and that resources are not wasted on ineffective treatments and services. We begin with a brief overview of the methods for evaluating evidence-based practices; proceed with summaries of evidence-based practices for adults, children, and adolescents; and conclude with policy implications. Two general themes emerge:
- The most effective services combine optimal medication management with psychosocial interventions that provide the patient and the family with information about the illness, ongoing supports, and rehabilitation services.
- These evidence-based treatments provide a tangible and measurable road map for improving outcomes and accountability.
Methods of Evidence-Based Practices
Conclusions regarding evidence-based practices have been reached through the systematic review of thousands of studies combining new statistical techniques and the judgment of expert reviewers. Groups such as the Agency for Healthcare Research and Quality in the United States and the Cochrane Collaboration have established principles for determining the effectiveness of treatments. These include:
- Randomized clinical trials improve the validity of causal conclusions.
- Replication of results in multiple settings improves the validity of results for actual practice.
- Consistency of findings builds confidence.
- Evidence can be ranked in terms of validity, clinical confidence, and expert judgments.
Examples of evidence-based mental health practices are available in the Cochrane Library,9 in reviews commissioned by AHRQ from the Evidence-Based Practice Centers,10 and in professional associations' clinical practice guidelines.11 One of the most recent reference documents is Mental Health: A Report of the Surgeon General.12
The process for conducting systematic, evidence-based reviews includes:
- Statement of objectives and eligibility criteria
- Identification of (all) potentially eligible studies
- Application of eligibility criteria
- Use of unbiased procedures for extracting data
- Critical appraisal
- Assembly of the most complete data set feasible
- Analysis of the data set, using statistical syntheses and sensitivity analyses only if appropriate and possible
- Preparation of a structured report
A statistical set of methods used in the process of systematic review, called meta-analysis, has itself become a science in the last decade. New statistical techniques allow for combining data in ways that allow assessment of effect sizes and of the level of confidence in the replicability of the conclusions. In its most structured form, researchers conducting these systematic reviews rely upon the following:
- Design and pilot testing of data extraction forms so as to ensure the standardization of the data extracted from each study
- Selection of studies for review based upon explicit eligibility criteria and agreed upon by more than one reviewer
- Assessment of inter-rater reliability in extracting the data from studies
- When possible, blinding of observers to authors, institutions, and journals
- Data extraction and quality assessment by more than one observer
- Careful assessment of biases in studies, including treatment allocation biases, blinding, and handling of patient attrition
- Listing excluded studies, along with reasons for the exclusions
This methodology, as in other scientific endeavors, is designed to allow readers to evaluate the methods, and to allow others to try to replicate the findings, of the review.
There are two groups of mental health patients of particular concern as regards public-sector policy in the United States:
- Adults with severe and persistent mental disorders, including disorders of late life (e.g., schizophrenia, bipolar disorder, dementia)
- Children and adolescents with severe emotional disturbances (e.g., schizophrenia and other pervasive developmental disorders in addition to conduct disorder)
Patterns of governance, responsibility, accountability, and financing for mental health services have been changing:
- The public mental health system and public mental health authorities now serve most population groups.
- No one agency is accountable for all services and resources.
- The service system is fragmented, making it difficult to meet the multiple needs of each individual.
- Financing streams, especially federal funding sources, have changed significantly in the past decade with contraction and reduction of social service funding sources and expansion of Medicaid as a source of ever-widening treatment and support.
- Considerable devolution of responsibility has taken place at all levels of government, in accordance with the so-called new federalism and the trend toward increasing home rule.
Effective Treatments and Services
A range of effective treatments exists from which to choose in order to suit patient preferences and provider skills. The following discussion is intended to be illustrative, not exhaustive, of all evidence-based mental health treatments. In general, the following patterns emerge across various mental illnesses:
- Medications are effective for most of the illnesses discussed here, and although they often have side effects, they are superior to psychosocial treatments alone in severe cases of many conditions.
- Combined treatments (medication plus psychosocial interventions) often produce the best results.
- Some psychotherapies are empirically supported, particularly cognitive-behavioral and interpersonal psychotherapies, and have equal efficacy vis-à-vis medication in mild-to-moderate cases of many conditions.
- Other psychosocial treatments (e.g., family education) and services (e.g., assertive community treatment and supported employment for adults; multisystemic treatment for children with conduct disorder) provide advantages for some conditionsparticularly to promote rehabilitation and recovery in the most impaired individuals.
The following examples illustrate these general principles.
For Adults with Schizophrenia
At least five categories of treatments and services for adults with schizophrenia have substantial evidence bases:
- Antipsychotic medications
- Family education and support
- Illness-specific counseling
- Assertive Community Treatment
- Supported employment
Clearly, medications are the most extensively evaluated treatments for reducing hallucinations, delusions, and thought disorganization. The evidence for the efficacy of antipsychotic agents in reducing symptoms and symptom relapse is impressive. Figure 1 aggregates the results of randomized, placebo-controlled clinical trials to assess the impact of medication on annual relapse rates. Patients on placebo had an aggregated relapse rate of 70 percent, compared with only 23 percent for those on antipsychotic medication. The figure also shows that the projected annual relapse rates with these medications in usual practice are in the range of 40 to 50 percent.13 The reasons for lower effectiveness (i.e., in actual practice) as compared with efficacy (i.e., in research trials) are complex. Research studies often exclude patients who present with complicating problems, such as medical illnesses or noncompliance, and hence these studies may overestimate the effectiveness that can be achieved in routine practice where complicating problems are common. However, there are other practice factors that may influence effectiveness and that can be addressed to improve care, in particular, the use of unproven dosages in practice and the failure to attend to side effects that may lead to patient noncompliance and relapse. Similarly, more careful attention and intervention with respect to common comorbid conditions that affect outcomes, such as comorbid substance abuse, can enhance overall outcomes.
A number of psychosocial treatments, in combination with appropriate pharmacotherapy, have been found to provide benefits in the form of reducing symptoms and relapse. Schizophrenia is a chronic illness that challenges the coping skills of patients and their families; psychosocial treatments provide guidance and support to enhance coping.
Interventions to help families cope more effectively have been shown to improve outcomes. Interventions that educate families about schizophrenia, provide support, and offer training in effective problem solving and communication have been subjected to numerous randomized clinical trials.14 The data strongly and consistently support the value of such interventions in reducing symptom relapse, and there is some evidence that they contribute to improved patient functioning and family well-being. The randomized clinical trials have reported one-year relapse rates for patients receiving these family "psychoeducation" programs in combination with medication that are more than 50 percent lower than for patients receiving medication alone (see Figure 2). This relative reduction in relapse rates has persisted for at least two years in the one clinical trial that followed patients for that long.15 Furthermore, a recent study found psychoeducational programs using multiple family groups to be more effective and less expensive than individual family psychoeducational interventions for Caucasians, though not for African Americans.16 Other beneficial outcomes of family interventions that have been reported include reduced rates of hospital admission, reduced family burden, and improved patient-family relationships. Based upon the evidence, persons with schizophrenia and their families who have ongoing contact with each other should be offered a family intervention, the key elements of which are a duration of at least nine months, illness education, crisis intervention, emotional support, and training in how to cope with illness symptoms and related problems.
Similarly, there is evidence that illness-specific forms of psychotherapy for persons with schizophrenia improve outcomes. Controlled studies of cognitive behavioral psychotherapy (CBT) have reported benefits in reducing symptom severity among patients who do not respond fully to antipsychotic medications.17 CBT focuses on changing maladaptive behaviors, including certain patterns of thinking, by attempting to alter underlying cognitive distortions. For example, persons with paranoid delusions may habitually resort to distorted interpretations about their environment that in turn reinforce a sense of persecution; CBT is designed to help the person recognize such maladaptive thought habits and change them. Most studies of cognitive behavioral psychotherapy have been performed with individual CBT of at least several months' duration; in some studies, group CBT and/or therapy of a shorter duration have been used. In all of the studies, clinicians who provided CBT received specialized training in the approach.
Persons with schizophrenia who have residual psychotic symptoms despite receiving adequate pharmacotherapy may benefit, therefore, from cognitive behaviorally oriented psychotherapy. The key elements of this intervention include a shared understanding of the illness between the patient and therapist, identification of target symptoms, and the development of specific cognitive and behavioral strategies to cope with these symptoms. Figure 3 summarizes the results from recent studies comparing CBT with some control interventions in the case of schizophrenia patients with residual symptoms on medications who are living in the community.
The program of Assertive Community Treatment (ACT) is a specific model of community-based care. Its origin is an experiment in Madison, Wisconsin, in the late 1970s in which the multidisciplinary inpatient team of the state hospital was moved into the community.18 The experiment arose from the observation that as inpatient programs became more successful at discharging patients, a new problem of the "revolving door" arose owing to the inadequacy of community services. The researchers in essence "deinstitutionalized" the inpatient treatment team to create ACT. The team took with it all of the functions of an inpatient team: interdisciplinary teamwork, 24-hour/7-days-per-week coverage, comprehensive treatment planning, ongoing responsibility, staff continuity, and small caseloads. ACT is designed to treat patients who are at high risk for hospital readmission and who cannot be maintained by more usual community-based treatment.
Randomized trials comparing ACT to other community-based care programs have consistently shown that ACT substantially reduces inpatient utilization and promotes continuity of outpatient care.19 Patient satisfaction with this model is generally high, and family advocacy groups, such as the National Alliance for the Mentally Ill in the United States, strongly support its use and dissemination. Results are less consistent regarding the impacts of ACT on other outcomes, although at least some studies have shown enhancement of clinical status, functioning, and quality of life. Figure 4 summarizes the impressive results of ACT from over two dozen controlled trials.
Supported employment is an approach to improving vocational functioning among persons with various types of disabilities, including schizophrenia.20 "Supported employment" refers to an approach to vocational rehabilitation that emphasizes rapid placement of the patient into a competitive, not necessarily full-time, job based upon the patient's preferences and skills, and provision of ongoing supports and training to help the patient maintain employment. For this reason it is also referred to as a "place and train" model of vocational rehabilitation, in contrast to the "train and place" approach that is much more widely used by rehabilitation services and that has not been found to consistently help patients achieve competitive employment. The evidence-based supported employment programs that have been found effective incorporate the key elements of individualized job development, rapid placement emphasizing competitive employment, ongoing job supports, and integration of vocational and mental health services.
Randomized trials have consistently demonstrated the effectiveness of supported employment in helping persons with schizophrenia to achieve competitive employment (see Figure 5).21 Employment outcomes related to the duration of employment and the amount of earnings also favor supported employment over traditional vocational services, and there is no evidence that engagement in supported employment leads to stress, increased symptoms, or other negative outcomes.22 Evidence is inconsistent about the relationship between clinical and demographic variables and successful vocational performance. It is recommended, therefore, that any person with schizophrenia who expresses an interest in work should be offered supported employment.
For Adults with Mild to Moderate Depression
Depression is the most common mental illness, and it is among the leading causes of disability in established market economies.23 Cognitive behavioral and interpersonal psychotherapies have been found effective in reducing or eliminating the symptoms and sequelae of depression, as have a range of antidepressant medications.24 Figure 6 provides a composite view of the advantages of antidepressant medications over placebo for major depression and for dysthymia, a less acutely disabling but more chronic form of depression. As shown, patients are significantly more likely to improve with medication as compared with placebo.25
Because many patients with depression are treated in primary care settings, development of effective strategies in these settings is essential. Current research is examining a range of innovative strategies for collaborative care using mental health specialists in primary care-based disease-management strategies. These strategies involve two basic approaches. First is the approach that adds routine, systematic screening to identify primary care patients who are depressed, followed by a collaborative care approach that enhances the usual primary care treatment for depression.26 Often such patients otherwise go undiagnosed and untreated. Instead, a mental health specialist works collaboratively with the primary care team to provide evidence-based depression treatment.
A second approach adds collaborative, evidence-based mental health care to the primary care setting for patients already diagnosed with depression. Some of these approaches make efficient use of regular telephone consultations with patients to help guide treatment decisions.27 Such programs vary in their effectiveness in producing depression-free days and in the costs required to achieve these outcomes, but in general the innovative programs demonstrate gains vis-à-vis primary care services provided without these collaborative approaches. For example, the evaluation of one model of collaborative care using nonphysician mental health specialists shows that patients with depression treated with the collaborative care model in primary care settings experienced a significantly greater reduction in symptoms over a one-year period than did patients treated with usual primary care (see Figure 7).28
For Children with Conduct Disorders
Multisystemic treatment (MST) is a community-based, intensive, short-term (three to four months) home- and family-focused treatment approach for youths with severe emotional disorders.29 MST seeks to promote more socialized behavior and better long-term outcomes by intervening in multiple contextswith family, peer groups, school, and neighborhoodto enhance the capacity of parents and community organizations to support troubled youths. To date, eight randomized trials of MST with various youth subgroups have been published: Three studies focused on violent and chronic juvenile offenders, and one study each focused on substance-abusing juvenile offenders, youths presenting with psychiatric emergencies, youths with maltreating families, inner-city delinquents, and juvenile sex offenders. Studies consistently indicate that MST reduces long-term rates of rearrest by 25 to 70 percent and long-term rates of days in out-of-home placements by 47 to 64 percent; reduces psychiatric symptoms and substance abuse; and improves mainstream school attendance, family relations, and consumer satisfaction.30
As is the case for most conditions, accurate diagnosis is the key first step in appropriate treatment of attention deficit hyperactivity disorder (ADHD). Unfortunately, misdiagnosis, in the form of both underdiagnosis (particularly in poorer families) and overdiagnosis, is common. Stimulants and other newer medications are effective in controlling behaviors that cause children with this condition to otherwise have poor outcomes in school and in later life. Experience with these medications suggests that close monitoring and close contact with families are essential to good practice. Behavioral treatments are also effective with children, particularly those who prefer not to take medications or who do not respond to medications. In the landmark Multimodal Treatment for Children with Attention Deficit Hyperactivity Disorder (MTA) Study, children with ADHD were randomly assigned to four treatment protocols: medication management, behavioral treatment, combined medication and behavioral treatment, or usual community care (CC).31 As Figure 8 shows, medication management and combined medication and behavioral treatment were superior to usual community care or behavioral treatment alone in reducing hyperactive-impulsive symptoms. For some other outcomes, behavioral treatment was superior to usual community care.
For Other Mental Illnesses
The effectiveness of various other treatments for a range of other mental disorders has also been demonstrated for such conditions as severe mood disorders, bipolar disorders, anxiety disorders (such as panic disorder and obsessive-compulsive disorder), posttraumatic stress disorder, and borderline personality disorder.32
There are various dimensions to the gap between the development of scientifically validated treatments and actual practice. The one on which we have focused thus far is failure to adopt a given "practice." But it is also true that there are different levels of implementation, and too often interventions are not properly implemented in practice based upon how they were administered in the original scientific studies. A readily apparent example is the prescription of the appropriate medication, based upon scientific evidence of efficacy but using a dose not evaluated in the original clinical trials. The challenges are even greater when it comes to implementation of evidence-based psychosocial interventions that require significant changes in practitioner and service system behavior and structure.
Program fidelity measures have been developed that permit monitoring and accountability for several evidence-based psychosocial interventions, including assertive community treatment, supported employment, and treatment for patients dually diagnosed with both mental illness and substance abuse.33 Fidelity in implementing programs is key to both effectiveness and costs, as illustrated in Figure 9. As shown in this large study by the Department of Veterans Affairs of the implementation of an assertive community treatment program called Intensive Psychiatric Community Care (IPCC), the sites in which IPCC was implemented with good fidelity to the assertive community treatment model realized cost reductions and better outcomes. In contrast, at sites where program fidelity was not good, costs actually increased and outcomes were poorer. The lesson from a policy perspective is that the commitment of resources to implement evidence-based treatments must be accompanied by regular monitoring of program fidelity and associated outcomes.34
Further Issues, Services, and Policy Implications
The success of all human services depends heavily upon the adequate supply of a quality workforce. It should be clear from the information provided here that effective mental health systems that provide quality, evidence-based treatment must rely upon a well-trained and available workforce. Producing such a workforce is a daunting task. For the most part, mental health practitioners, in the appropriate circumstances and given the appropriate training and incentives, will be willing to adopt practices that improve outcomes. Too often, however, practitioners are overworked and underpaid, lack training in many of the evidence-based practices presented here, and lack incentives to change. Simply mandating changes in practice, then, is likely to fail. Practitioners need opportunities and incentives to learn new, evidence-based approaches, combined with the expectation that they will avail themselves of such opportunities and put into practice what they learn. It must be noted, too, that whereas substantial efforts are needed to address the performance of the existing workforce, comparable efforts are also necessary within the institutions of higher learning that are training the future workforce. As important as it is that these institutions emphasize competence in evidence-based practices, it is even more critical that they instill openness to adopting new evidence-based practices as knowledge evolves over practitioners' careers.
What about Services Lacking an Evidence Base?
Although evidence-based practices are necessary for a quality mental health system, they are not sufficient. This is a critical point. A distinction must be made between treatments and services that have been shown to be ineffective (or substantially less effective than other evidence-based alternatives) and those about which there is little or no systematic evidence either way. Clearly, the former are to be discouraged in favor of proven evidence-based alternatives. For many services, however, there is often little systematic outcome evidence upon which to base informed decisions. Practitioners and policymakers are left to evaluate the merit of these services based upon prevailing professional standards of practice, community needs, and other pragmatic factors.
Whereas services that are consuming substantial resources should be subjected to evaluations that can guide decisions about their value, a categorical approach to not funding services unless they have a systematic evidence base can do considerable harm, eliminating some essential services. Some services are of self-evident value, and investment in their systematic evaluation may provide little new information; it seems obvious, for example, that provision of shelter and food to homeless persons with severe mental illness has value. Other services may warrant continued support but with the expectation that some evaluation occur to ensure that resources are well spent.
Implications for Policymakers
There are a number of key points for policymakers from this body of scientific evidence:
- A substantial body of outcomes research supports the efficacy of a wide range of evidence-based mental health treatments.
- Mental health services should be expected to provide evidence-based practices in order to yield good outcomes.
- Existing service providers should be held accountable for providing services consistent with evidence-based practices.
- Funding and policy entities should be held accountable for creating an environment that enables service providers to deliver evidence-based practices.
- Measures of "program fidelity" have been developed that permit monitoring and accountability.
- Outcomes should be monitored regularly by clinicians as a part of good practice.
- Programs that achieve and maintain fidelity to evidence-based practices should be expected to achieve those expected outcomes.
- The wide array of effective treatments should be available within a community, because even when treatments are equally effective in general for the entire population, many of them are not equally effective for significant subgroups.
- Treatment choice and wide selection are essential in order to maximize treatment response and adherence to treatment.
- Federal sources of system support should incorporate and require evidence-based practices in considering approval of state and county Medicaid plans.
These recommendations, if implemented, could help balance the costs of care and treatment with the advantages to patients and the benefits to society. Most of the treatments and services described in this report are considered to be cost-effectiveand they have been implemented in settings in which care has been managed with reasonable cost.
1Institute of Medicine 2001; Lehman, Steinwachs, and the Survey Co-Investigators of the PORT Project 1998.
2U.S. Department of Health and Human Services 1999.
3http://www.guideline.gov/ (accessed March 15, 2004).
4American Psychiatric Association 1993, 1997.
5http://www.nice.org.uk (accessed March 15, 2004).
6http://www.sign.ac.uk (accessed March 15, 2004).
7http://www.cochrane.org (accessed March 15, 2004).
8http://www.ahcpr.gov/clinic/epcix.htm (accessed March 15, 2004).
9http://www.cochrane.org (accessed March 15, 2004).
10http://www.ahcpr.gov/clinic/epcix.htm (accessed March 15, 2004).
11American Psychiatric Association 1993, 1997.
12U.S. Department of Health and Human Services 1999.
13Dixon, Lehman, and Levine 1995.
14Dixon, Adams, and Lucksted 2000; Dixon and Lehman 1995.
15Hogarty et al. 1991.
16McFarlane et al. 1995.
18Stein and Test 1980.
19Mueser et al. 1998; Scott and Dixon 1995.
20Bond et al. 1999.
21Bond et al. 1997.
23Murray and Lopez 1997.
24Beck, Rush, and Shaw 1979; Frank et al. 1990; Frank et al. 1991.
25U.S. Department of Health and Human Services 1999.
26Lave and Schulberg 1999; Mynors-Wallis et al. 1995; Peveler et al. 1999.
27Hunkeler et al. 2000; Katon et al. 1995; Katon et al. 1996; Katon et al. 1999; Simon et al. 2000; Von Korff et al. 1998.
28Unutzer et al. 2002.
29Halliday-Boykins and Henggeler 2001.
30Halliday-Boykins and Henggeler 2001; Henggeler et al. 1998; Henggeler et al. 2002; Hoagwood et al. 1996; Schoenwald et al. 2003; U.S. Department of Health and Human Services 1999.
31The MTA Cooperative Group Moderators and Mediators of Treatment Response for Children with Attention-Deficit/Hyperactivity Disorder 1999.
32U.S. Department of Health and Human Services 1999.
33McGrew et al. 1994.
34Rosenheck et al. 1995.
American Psychiatric Association. 1993. Practice Guideline for Major Depressive Disorder in Adults. American Journal of Psychiatry 150 (suppl. 4): 126.
American Psychiatric Association. 1997. Practice Guideline for the Treatment of Patients with Schizophrenia. Washington, D.C.: American Psychiatric Association Press.
Beck, A.T., A.J. Rush, and B.F. Shaw. 1979. Cognitive Therapy of Depression. New York: Guilford Press.
Bond, G.R., L.L. Dietzen, J.H. McGrew, and L.D. Miller. 1995. Accelerating Entry into Supported Employment for Persons with Severe Psychiatric Disabilities. Rehabilitation Psychology 40:91111.
Bond, G., R.E. Drake, D. Becker, and K. Mueser. 1999. Effectiveness of Psychiatric Rehabilitation Approaches for Employment of People with Severe Mental Illness. Journal of Disability Policy Studies 10:1852.
Bond, G., R. Drake, K.T. Mueser, and D. Becker. 1997. An Update on Supported Employment for People with Severe Mental Illness. Psychiatric Services 48:33546.
Chandler, D., J. Meisel, T. Hu, M. McGowen, and K. Madison. 1997. A Capitated Model for a Cross-Section of Severely Mentally Ill Clients: Employment Outcomes. Community Mental Health Journal 33:50116.
Dickerson, F.B. 2000. Cognitive Behavioral Psychotherapy for Schizophrenia: A Review of Recent Empirical Studies. Schizophrenia Research 43:7190.
Dixon, L., C. Adams, and A. Lucksted. 2000. Update on Family Psychoeducation for Schizophrenia. Schizophrenia Bulletin 26:520.
Dixon, L., and A. Lehman. 1995. Family Interventions for Schizophrenia. Schizophrenia Bulletin 21:63143.
Dixon, L., A. Lehman, and J. Levine. 1995. Conventional Antipsychotic Medications for Schizophrenia. Schizophrenia Bulletin 21:56778.
Drake, R.E., G.J. McHugo, R.R. Bebout, D.R. Becker, M. Harris, G.R. Bond, and E. Quimby. 1999. A Randomized Clinical Trial of Supported Employment for Inner-City Patients with Severe Mental Disorders. Archives of General Psychiatry 56:62733.
Drake, R.E., G.J. McHugo, D.R. Becker, W.A. Anthony, and R.E. Clark. 1996. The New Hampshire Study of Supported Employment for People with Severe Mental Illness: Vocational Outcomes. Journal of Consulting and Clinical Psychology 64:3919.
Frank, E., D.J. Kupfer, J.M. Perel, C. Cornes, D.B. Jarrett, A.G. Mallinger, M.E. Thase, A.B. McEachran, and V.J. Grochoconski. 1990. Three-Year Outcomes for Maintenance Therapies in Recurrent Depression. Archives of General Psychiatry 47:10939.
Frank, E., D.J. Kupfer, E.F. Wagner, A.B. McEachran, and C. Cornes. 1991. Efficacy of Interpersonal Psychotherapy as a Maintenance Treatment for Recurrent Depression. Archives of General Psychiatry 48:10539.
Gervey, R., and J.R. Bedell. 1994. Supported Employment Invocation Rehabilitation. In Psychological Assessment and Treatment of Persons with Severe Mental Disorders, edited by J.R. Bedell, 13963. Washington, D.C.: Taylor and Francis.
Halliday-Boykins, C.A., and S.W. Henggeler. 2001. Multisystemic Therapy: Theory, Research and Practice. In Balancing Family-Centered Services and Child Well-Being, edited by E. Walton, P.A. Sandau-Beckler, and M. Mannes, 32035. New York: Columbia University Press.
Henggeler, S.W., S.K. Schoenwald, C.M. Borduin, M.D. Rowland, and P.B. Cunningham. 1998. Multisystemic Treatment of Antisocial Behavior in Children and Adolescents. New York: Guilford Press.
Henggeler, S.W., S.K. Schoenwald, J.G. Liao, E.J. Letourneau, and D.L. Edwards. 2002. Transporting Efficacious Treatments to Field Settings: The Link between Supervisory Practices and Therapist Fidelity in MST Programs. Journal of Clinical Child Psychology 31(2):15567.
Hoagwood, K., P.S. Jensen, T. Petti, and B.J. Burns. 1996. Outcomes of Mental Health Care for Children and Adolescents, I. A Comprehensive Conceptual Model. Journal of the American Academy of Child and Adolescent Psychiatry 35:105563.
Hogarty, G.E., C.M. Anderson, D.J. Reiss, S.J. Kornblith, D.P. Greenwald, R.F. Ulrich, and M. Carter. 1991. Family Psychoeducation, Social Skills Training, and Maintenance Chemotherapy in the Aftercare Treatment of Schizophrenia: Two-Year Effects of a Controlled Study on Relapse and Adjustment. Archives of General Psychiatry 48:3407.
Hunkeler, E.M., J.F. Meresman, W.A. Hargreaves, B. Fireman, W.H. Berman, A.J. Kirsch, J. Groebe, S.W. Hurt, P. Braden, M. Getzell, P.A. Feigenbaum, T. Peng, and M. Salzer. 2000. Efficacy of Nurse Telehealth Care and Peer Support in Augmenting Treatment of Depression in Primary Care. Archives of Family Medicine 9:7008.
Institute of Medicine. 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: National Academy Press.
Katon, W.J., P. Robinson, M. Von Korff, E.H.B. Lin, T. Bush, E.J. Ludman, G. Simon, and E. Walker. 1996. A Multifaceted Intervention to Improve Treatment of Depression in Primary Care. Archives of General Psychiatry 53:92432.
Katon, W., M. Von Korff, E. Lin, G.E. Simon, E. Walker, J. Unutzer, G. Simon, J. Russo, and E. Ludman. 1999. Stepped Collaborative Care for Primary Care Patients with Persistent Symptoms of Depression: A Randomized Trial. Archives of General Psychiatry 56:110915.
Katon, W.J., M. Von Korff, E.H.B. Lin, E.A. Walker, G.E.M. Simon, T. Bush, P. Robinson, and J. Russo. 1995. Collaborative Management to Achieve Treatment Guidelines: Impact on Depression in Primary Care. Journal of the American Medical Association 273:102631.
Lave, J.R., and H.C. Schulberg. 1999. Integrating Cost-Effectiveness Analyses within Clinical Trials of Treatment for Major Depression in Primary Care Practice. In Cost-Effectiveness of Psychotherapy: A Guide for Practitioners, Researchers, and Policy-Makers, edited by N.E. Miller and K.M. Magruder, 7584. New York: Oxford University Press.
Lehman, A.F. 1995. Vocational Rehabilitation in Schizophrenia. Schizophrenia Bulletin 21:64556.
Lehman, A.F., R. Goldberg, L.B. Dixon, S. McNary, L. Postrado, A. Hackman, and K. McDonnell. 2002. Improving Employment Outcomes for Persons with Severe Mental Illness. Archives of General Psychiatry 59:16572.
Lehman, A.F., D.S. Steinwachs, and the Survey Co-Investigators of the PORT Project. 1998. Patterns of Usual Care for Schizophrenia: Initial Results from the Schizophrenia Patient Outcomes Research Team (PORT) Client Survey. Schizophrenia Bulletin 24:1120.
McFarlane, W.R., R.A. Dushay, S.M. Deakins, P. Stastny, E.P. Lukens, J. Toran, and B. Link. 2000. Employment Outcomes in Family-Aided Assertive Community Treatment. American Journal of Orthopsychiatry 70:20314.
McFarlane, W.R., E. Lukens, B. Link, R. Dushay, S.A. Deakins, M. Newmark, E.J. Dunne, B. Horen, and J. Toran. 1995. Multiple Family Group and Psychoeducation in the Treatment of Schizophrenia. Archives of General Psychiatry 52:67987.
McGrew, J.H., G.R. Bond, L. Dietzen, and M. Saylers. 1994. Measuring the Fidelity of Implementation of a Mental Health Program Model. Journal of Consulting and Clinical Psychology 62:6708.
The MTA Cooperative Group Moderators and Mediators of Treatment Response for Children with Attention-Deficit/Hyperactivity Disorder. 1999. Archives of General Psychiatry 56:108896.
Mueser, K.T., G.R. Bond, R.E. Drake, and S.G. Resnick. 1998. Models of Community Care for Severe Mental Illness: A Review of Research on Case Management. Schizophrenia Bulletin 42:3774.
Mueser, K.T., R.E. Clark, M. Haines, R.E. Drake, G.J. McHugo, G.R. Bond, S.M. Essock, D.R. Becker, R. Wolfe, and K. Swain. Forthcoming, 2004. The Hartford Study of Supported Employment for Persons with Severe Mental Illnesses. Journal of Consulting and Clinical Psychology.
Murray, C.J.L., and A.D. Lopez. 1997. Alternative Projections of Mortality and Disability by Cause, 19902020; Global Burden of Disease Study. Lancet 349:14981504.
Mynors-Wallis, L.M., D.H. Gath, A.R. Lloyd-Thomas, and D. Tomlinson. 1995. Randomized Controlled Trial Comparing Problem Solving Treatment with Amitriptyline and Placebo for Major Depression in Primary Care. BMJ 310:4415.
Peveler, R., C. George, A.L. Kinmonth, M. Campbell, and C. Thompson. 1999. Effect of Antidepressant Drug Counseling and Information Leaflets on Adherence to Drug Treatment in Primary Care: Randomized Controlled Trial. BMJ 319:6125.
Rosenheck, R.A., M. Neale, P. Leaf, R. Milstein, and L. Frisman. 1995. Multisite Experimental Cost Study of Intensive Psychiatric Community Care. Schizophrenia Bulletin 21:12940.
Schoenwald, S.K., A.J. Sheidow, E.J. Letourneau, and J.G. Liao. 2003. Transportability of Multisystemic Therapy: Evidence for Multi-Level Influences. Mental Health Services Research 5(4):22339.
Scott, J., and L. Dixon. 1995. Assertive Community Treatment and Case Management. Schizophrenia Bulletin 21:65768.
Simon, G.E., M. Von Korff, C. Rutter, and E.H. Wagner. 2000. Randomized Trial of Monitoring, Feedback, and Management of Care by Telephone to Improve Treatment of Depression in Primary Care. BMJ 320:5504.
Stein, L.I., and M.A. Test. 1980. Alternative to Mental Hospital Treatment, I. Conceptual Model, Treatment Program, and Clinical Evaluation. Archives of General Psychiatry 37:3927.
Unutzer, J., W. Katon, C. Callahan, J.W. Williams, Jr., E. Hunkeler, L. Harpole, M. Hoffing, R.D. Della Penna, P.H. Noel, E.H. Lin, P.A. Arean, M.T. Hegel, L. Tang, T.R. Belin, S. Oishi, C. Langston, for the IMPACT Investigators. 2002. Collaborative Care Management of Late-Life Depression in the Primary Care Setting: A Randomized Controlled Trial. Journal of the American Medical Association 288:283645.
U.S. Department of Health and Human Services. 1999. Mental Health: A Report of the Surgeon General. Rockville, Md.: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health.
Von Korff, M., W. Katon, T. Bush, E.H. Lin, G.E. Simon, K. Saunders, E. Ludman, E. Walker, J. Unutzer. 1998. Treatment Costs, Cost Offset, and Cost-Effectiveness of Collaborative Management of Depression. Psychosomatic Medicine 60:1439.
DOI # 10.1599/2004Lehman
http://www.ahcpr.gov/clinic/epcix.htm (accessed March 15, 2004).
http://www.cochrane.org (accessed March 15, 2004).
http://www.guideline.gov/ (accessed March 15, 2004).
http://www.nice.org.uk (accessed March 15, 2004).
http://www.sign.ac.uk (accessed March 15, 2004).
Anthony F. Lehman, M.D., M.S.P.H., is Professor and Chair of the Department of Psychiatry at the University of Maryland School of Medicine. His clinical, teaching, and research activities have focused on serving persons with severe and persistent mental illnesses and examining outcomes, the impacts of substance use disorders, the effectiveness of Assertive Community Treatment programs, supported employment, and interventions for families. In 1998 the National Alliance for the Mentally Ill presented Lehman with its Distinguished Service Award. He recently chaired the Work Group updating the American Psychiatric Association's Practice Guideline for the Treatment of Patients with Schizophrenia.
Howard H. Goldman, M.D., Ph.D., is Professor of Psychiatry at the University of Maryland School of Medicine. Goldman served as the Senior Scientific Editor of the Surgeon General's Report on Mental Health from 1997 to 1999, for which he was awarded the Surgeon General's Medallion. During 2002 and 2003 he was a consultant on evidence-based practices for the President's New Freedom Commission on Mental Health. He and Robert Drake edited Evidence-Based Practices in Mental Health for the American Psychiatric Association in 2003. In 2002 Goldman was elected to the Institute of Medicine of the National Academy of Sciences.
Lisa B. Dixon, M.D., is a Professor of Psychiatry at the University of Maryland School of Medicine. She serves as Director of the Division of Services Research in the School's Department of Psychiatry. Dixon is also the Associate Director for Research of the Veterans Administration Mental Illness Research, Education and Clinical Center (MIRECC) in VISN 5, the Capitol Health Care Network. Dixon is an active researcher with grants from the National Institute of Mental Health, National Institute for Drug Abuse, the Veterans Administration, and numerous foundations. Her research activities have focused on improving the health outcomes of persons with severe mental illnesses and their families.
Rachel Churchill, Ph.D., M.Sc., is a Lecturer in Psychiatric Epidemiology in the Health Services Research Department at the Institute of Psychiatry in London. Churchill is active in both teaching and research, and her interests include evaluating interventions for a range of mental health problems, policy research and mental health legislation, and evidence-based psychiatry. She was instrumental in both setting up the Cochrane Depression, Anxiety and Neurosis Collaborative Review Group (CCDAN) and in establishing a specialist trials register in the field. She is now Coordinating Editor of CCDAN.
(To request a bound copy of this report, click here.
To see a complete list of Milbank reports, click here.
Be sure to specify which report you want,
your name, mailing address, and phone number.)
Milbank Memorial Fund
645 Madison Avenue
New York, NY 10022
(c) 2004 Milbank Memorial Fund
The Milbank Memorial Fund is an endowed operating foundation that engages in nonpartisan analysis, study, research, and communication on significant issues in health policy. In the Fund's own publications, in reports or books it publishes with other organizations, and in articles it commissions for publication by other organizations, the Fund endeavors to maintain the highest standards for accuracy and fairness. Opinions expressed by individual authors, however, do not necessarily reflect opinions or factual determinations of the Fund.
This file may be redistributed electronically or in print as long as it remains wholly intact, including this notice and copyright. The Fund will freely distribute this document in its original published form on request.
Printed in the United States of America.
Online producer: Stephanie Moe-Quiggle