A Framework for Evaluating Primary Care Investment

Despite a robust body of evidence demonstrating that strong primary care is foundational to a high-performing health system associated with improved population health, greater equity, and lower overall costs, primary care in the US suffers chronic underinvestment. The United States currently spends only 4 to 5 cents on primary care for every dollar spent on health care.

To enable recruitment and retention of clinicians and staff and better equip them to meet the public’s need for high-quality primary care, nearly 20 states have initiatives to measure and, in several states, increase primary care spending. The federal government has taken more limited actions to increase spending on primary care, most recently through changes in the Medicare Physician Fee Schedule.

As momentum builds for investing in primary care, it is critical to conduct systematic evaluations to understand whether these initiatives are accomplishing their objectives. In addition to measuring the overall success of policies, evaluations can inform opportunities for mid-course corrections and offer implementation lessons. To date, few evaluations have been performed of existing state primary care spending initiatives, and states in the process of adopting spending policies have not issued well-articulated plans for comprehensive evaluations. 

Although the Center for Medicare and Medicaid Innovation (CMMI) has conducted evaluations of its federal primary care payment demonstrations, these efforts have largely examined changes in payment methods rather than increases in the overall level of primary care spending. Moreover, the evaluations that have been conducted of both state and federal primary care initiatives have been criticized for the limited set of measures used and their emphasis on cost and quality outcomes. With few exceptions, evaluations have largely overlooked assessing whether more resources reached front-line practices.

In this post, members of the Primary Care Centers Round Table present a framework for evaluating initiatives to increase primary care spending. While it is particularly applicable to state primary care spending initiatives, the framework may also be informative for evaluations of federal primary care payment policies and of primary care investment initiatives by individual health plans and health care organizations.

Evaluation Framework

This figure presents the framework’s six sequential domains and one cross-cutting domain, with examples of measures.

View full-size Framework image.

The seven domains are as follows:

1. Is more being spent on primary care? The first domain assesses whether health plans are complying with regulations to increase spending. Measures assess if primary care spending as a percentage of total health plan medical spending is increasing, and whether that is reflected in increases in payment schedules for primary care services or other modes of payment.
   
2. If spending is increasing, are the dollars reaching front-line primary care practices? The flow of funding is complicated when primary care clinicians and practices are owned and operated by health systems and other large organizations, or under delegated payment models where a health plan issues payment to an intermediary such as an independent practice association or accountable care organization that compensates clinicians. There is a risk that enhanced plan payments for primary care will get “stuck” in the centralized machinery of large organizations and fail to be passed along to front-line practices for infrastructure building. Measures in this domain should include items such as practice-level revenues and the number of full-time clinicians and staff working at primary care practices.
   
3. If more funds are reaching primary care practices, are practices using the funds to implement the elements of advanced primary care? Many well-accepted conceptual models, including the UCSF Center for Excellence in Primary Care 10 Building Blocks of High Performing Primary Care and the Primary Care Investment Guide by the Harvard Center for Primary Care, delineate the key elements of advanced primary care. Evaluations should assess whether practices are implementing these elements such as patient empanelment, behavioral health integration, and community health workers to address social needs.
   
4. If practices are better resourced, are the core functions of primary care improving? The fourth domain assesses functional performance by examining the four core functions of primary care: access, continuity, comprehensiveness, and coordination. Research has documented the effects of these functions on outcome domains.
   
5. Are additional outcomes improving? Domains 5-7 address components of the “quintuple aim” for health reform that might be expected to improve over time as a result of strengthened primary care. Domain 5 includes disease-specific quality measures (e.g., cancer screening), patient experience, and joy in practice for people working in primary care.
   
6. Are costs that can be reduced by better primary care decreasing? Although supporters of higher primary care spending sometimes assert that this spending will produce a short-term financial return on investment by lowering total costs of care, we consider it unreasonable to expect a sector accounting for 5% of health spending to control the other 95% of costs. We therefore recommend not overemphasizing the cost domain in evaluations and limiting measures to those most directly influenced by better primary care, such as ambulatory care sensitive hospitalizations.
   
7. Is health equity increasing?
   Equity is a cross-cutting domain pertinent to all the other framework domains. If primary care spending policies are not intentional about enhancing equity, there is a risk that they may worsen inequities, given that certain practices and regions face greater barriers to accessing enhanced payment and implementing advanced primary care. Assessing health plan compliance with primary care spending should analyze whether spending patterns differ across plans based on type of plan (e.g., Medicaid vs commercial plans), demographics of the populations served by different plans, geography, and related features. Similarly, evaluations should assess whether implementation of the elements of advanced primary care and performance on the core functions of primary care vary according to the patient populations served by different practices.

The accompanying tables in the PDF appendix provide more information about measures for each domain and potential data sources. The measures in the table should be regarded as a menu and not an exhaustive or mandatory list.

Pragmatic Considerations

Evaluation designs must consider timing, resources available, readiness of stakeholders to contribute data, evaluation priorities for policy decision makers, and other issues. Below we highlight some of these considerations.

• Timing
  The best time to plan an evaluation is during the development of a primary care spend policy, so that the evaluation can commence at the inception of the policy intervention. Ideally, data should be measured at least annually and continue for the duration of the spending increase timeline (and several years beyond). That said, evaluations may be conducted with a retrospective component. Archival data are often available for measures using secondary data such as health plan claims.

• The logic of the logic model
  Although we present domains as following a logical sequence, evaluators should not interpret the framework too linearly. However, we do believe in the primacy of the first 3 domains, especially domains assessing practice-level measures, which are often overlooked in evaluations. Practice-level domains are critical, in part, because states have authority over only commercial insurers, Medicaid, and state employee health benefits. States do not have jurisdiction over federal programs such as Medicare or self-insured employer plans, and, in some cases, the spending target doesn’t apply to all the plan types for which they do have jurisdiction. As a result, an evaluation of a state policy applying to health plans covering only 10-15% of the population might find that despite these plans being fully compliant with increasing their spend, the policy has not resulted in an increase in a typical primary care practice’s total revenues sufficient to hire more staff and implement whole-practice advanced primary care transformation.

• Evaluation expense
  Substantial resources and technical expertise are required to conduct a systematic, multi-year evaluation measuring all the domains in our framework. Unfortunately, the practice-level data required for domains 2 and 3 are not currently routinely collected and are expensive to obtain. Measurement is less costly when measures derive from existing secondary data sources such as medical claims. Many, but not all, states have All Payer Claims Databases that can provide data for several domain measures. Some states also have standardized reporting of quality measures by plans and provider organizations.
  
  One approach to primary data collection for these domains would be to enroll a diverse sample of practices for sequential data collection over time, similar to sentinel networks for collecting practice level data on seasonal respiratory viruses. The sample should be selected to capture variation in key attributes, such as size, ownership, geography, and patients served, States with ongoing programs in primary care practice improvement facilitation might be able to use that infrastructure to assist with data collection for spending evaluations.
  

• Stakeholder collaboration
  Engaging diverse stakeholders in the evaluation planning and oversight process enhances the likelihood of evaluation feasibility and impact. In addition to the central role of the state agencies administering primary care spending regulations and the health plans subject to these regulations, involving state legislative champions may provide opportunities for appropriations for evaluation funding. State philanthropies, primary care professional societies, consumer groups, and academic institutions each bring important insights and expertise.

• Evaluations and Scorecards
  The 2021 NASEM report Implementing High-Quality Primary Care recommended developing a national scorecard to track the state of primary care in the nation. In response, the Milbank Memorial Fund and The Physicians Foundation has supported the Robert Graham Center in developing annual Health of US Primary Care scorecards. A few states have also created primary care scorecards for their jurisdictions (e.g., VA, MA, NY, CA). However, scorecards are not synonymous with systematic evaluations. Most scorecards provide a high-level impression of the overall climate for primary care. Scorecards alone cannot tell us whether a state’s investment is reaching practices and transforming care. Evaluations designed to generate rapid, actionable evidence from the ground up can close the feedback loop on the impact of primary care spending policies.

Conclusion

As more states move to adopt policies to increase primary care spending, it is critical to conduct systematic evaluations to understand whether these policies are achieving their objectives, and why or why not. We offer our framework as a comprehensive and logical approach to evaluation, emphasizing the importance of the practice-level domain. Strengthening the primary care workforce and infrastructure, and implementing advanced primary care practice elements, are critical short-to-medium term objectives for primary care spending. Success on more performance domains like outcomes largely depends on making progress on these key practice-level domains.