research health care system

HCSRN 2024 Annual Conference Award Winners

Paper of the Year

Lisa R. Miller-Matero, PhD, ABPP

Associate Scientist, Associate Director, Health Services Research Director, Psychology Internship Program Director, Health Psychology Fellowship Program

Suicide Attempts After Bariatric Surgery: Comparison to A Nonsurgical Cohort Of Individuals With Severe Obesity

Surgery for Obesity and Related Diseases (2023) 1-9

Mentor of the Year

Jordan Braciszewski , PhD

Associate Scientist and Associate Director for Training and Education

Investigator of the Year

Jennifer M. Boggs, PhD, MSW

Investigator

Results from a Hybrid Effectiveness-Implementation Trial to Improve Uptake of a Secure Firearm Storage Program in Pediatric Primary Care

HCSRN 2024 Annual Conference Poster Session Winners

Poster Session I Winner

Jana Hirschtick , PhD, MPH

Research Scientist

Estimating Underdiagnosis Using the Electronic Health Record: A Long COVID Case Study

Poster Session II Winner

Amandeep Mann-Grewal, MPH

Statistical Analyst

Self-Reported Survey Results about Experience of Virtual Reality Headset on Reducing Anxiety in Pediatric Urgent Care Clinic

HCSRN 30TH Anniversary Merchandise

Now is your chance to purchase HCSRN Merchandise!

Show your HCSRN pride with our water bottle, tote or zip jackets.Order today to receive your merch in time for the HCSRN Annual Conference. Production time is 7 – 10 days and is created on demand. The items feature the 30th anniversary logo recognizing this important milestone in HCSRN’s history.

Don’t Delay! Order Today!

2024 Marks HCSRN’s 30th Anniversary!

Watch this site for programs and events throughout the year!

Attend Our Monthly Scientific Webinars

Join us on the second Tuesday of each month at 4 pm ET

VDW Data Model

Member directory, privacy overview.

• Mission and objectives
• History and achievements
• Secretariat
• Celebrating over two decades of advancing and supporting health policy and systems research
• Working in partnership
• Supporting the generation of knowledge
• Strengthening capacities
• Embedded research
• Evidence synthesis
• Gender and intersectionality
• Primary health care
• Systems thinking
• Subscribe to our newsletter »
• All publications
• Core documents
• Project documents
• Teaching and training material »
• Feature stories
• Current calls »
• Archive of past calls »
• What we do /
• What is Health Policy and Systems Research (HPSR)?

Section navigation

• How we work
• Thematic areas of focus
• Implementation research platform

Health policy and systems research (HPSR) is an emerging field that seeks to understand and improve how societies organize themselves in achieving collective health goals, and how different actors interact in the policy and implementation processes to contribute to policy outcomes. By nature, it is inter-disciplinary, a blend of economics, sociology, anthropology, political science, public health and epidemiology that together draw a comprehensive picture of how health systems respond and adapt to health policies, and how health policies can shape − and be shaped by − health systems and the broader determinants of health.

Health policy and systems research can be employed at several points in the policy cycle, from getting an issue onto the policy agenda to evaluating and learning from implemented policies. In this way, HPSR is characterized not by any particular methodology, but the types of questions it addresses. It focuses primarily upon the more upstream aspects of health, organizations and policies, rather than clinical or preventive services or basic scientific research (for example into cell or molecular structures). It covers a wide range of questions − from financing to governance − and issues surrounding implementation of services and delivery of care in both the public and private sectors. It is a crucial policy analysis tool − of both policies and processes − including the role, interests and values of key actors at local, national and global levels.

The appropriate mix of disciplines to be used in HPSR depends largely on the nature of the research question being addressed. An evaluation of a health insurance scheme might draw upon economics to understand the financial consequences of the scheme and its impact upon demand for services, anthropology to understand various socio-cultural and organizational aspects as well as patterns of consumption, and epidemiology to understand its health consequences.

HPSR and the building blocks of a health system

HPSR can address any or several of the health systems building blocks (see graphic) and their ultimate objective to promote the coverage, quality, efficiency and equity of health systems. In doing so, it acknowledges the inherent connections and dynamics among the different building blocks in assessing and understanding how interventions might play out across them. It also seeks to unpack the behaviour, reactions, and interconnectedness of health systems and the people within those systems. The way HPSR conceptualizes and analyzes these interactions helps to illuminate not only what works, but for whom, and under what circumstances.

Linking health policy with health systems research

Why health policy and health systems research? Why are these two different domains fused into one? While seemingly separate − with health policy research principally studying how different actors interact in the policy and implementation processes and contribute to policy outcomes, and health systems research addressing questions such as the coverage, quality, efficiency and equity of health systems − the two have clear and multiple synergies:

• Health policies are subject to political processes that govern health systems. Understanding these processes is not only critical in the design of effective policies, but in the creation of evidence to inform those policies. Health policies and health systems are not separate entities: HPSR is a recognition that everything is connected.
• Understanding the processes and dynamics of health systems can directly inform policy- and decision-making.
• Active linkage and exchange between health system researchers, decision-makers and other research-users promotes evidence-informed policy and policy-informed research.
• A systems perspective is critical in evaluating and learning from implemented policies
• Removing both from their silos builds the capacities of key actors in health policy, in health systems research, and creates actors versed and able in both.

WHO Strategy on Health Policy and Systems Research

The WHO Strategy on Health Policy and Systems Research, Changing Mindsets, advocates for greater generation and use of research evidence in health policy...

Health Policy and Systems Research

Health Policy and Systems Research - A Methodology Reader is a collection of high quality papers that demonstrates the application of different health...

World Report on Health Policy and Systems Research

The report describes the evolution of the field and provides figures on the number of publications produced, funding trends and institutional capacity...

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

5 Critical Priorities for the U.S. Health Care System

• Marc Harrison

A guide to making health care more accessible, affordable, and effective.

The pandemic has starkly revealed the many shortcomings of the U.S. health care system — as well as the changes that must be implemented to make care more affordable, improve access, and do a better job of keeping people healthy. In this article, the CEO of Intermountain Healthcare describes five priorities to fix the system. They include: focus on prevention, not just treating sickness; tackle racial disparities; expand telehealth and in-home services; build integrated systems; and adopt value-based care.

Since early 2020, the dominating presence of the Covid-19 pandemic has redefined the future of health care in America. It has revealed five crucial priorities that together can make U.S. health care accessible, more affordable, and focused on keeping people healthy rather than simply treating them when they are sick.

• Marc Harrison , MD, is president and CEO of Salt Lake City-based Intermountain Healthcare.

Partner Center

• Fact sheets
• Facts in pictures

Publications

• Questions and answers
• Tools and toolkits
• HIV and AIDS
• Hypertension
• Mental disorders
• Top 10 causes of death
• All countries
• Eastern Mediterranean
• South-East Asia
• Western Pacific
• Data by country
• Country presence 
• Country strengthening 
• Country cooperation strategies 
• News releases
• Feature stories
• Press conferences
• Commentaries
• Photo library
• Afghanistan
• Cholera 
• Coronavirus disease (COVID-19)
• Greater Horn of Africa
• Israel and occupied Palestinian territory
• Disease Outbreak News
• Situation reports
• Weekly Epidemiological Record
• Surveillance
• Health emergency appeal
• International Health Regulations
• Independent Oversight and Advisory Committee
• Classifications
• Data collections
• Global Health Estimates
• Mortality Database
• Sustainable Development Goals
• Health Inequality Monitor
• Global Progress
• Data collection tools
• Global Health Observatory
• Insights and visualizations
• COVID excess deaths
• World Health Statistics
• Partnerships
• Committees and advisory groups
• Collaborating centres
• Technical teams
• Organizational structure
• Initiatives
• General Programme of Work
• WHO Academy
• Investment case
• WHO Foundation
• External audit
• Financial statements
• Internal audit and investigations 
• Programme Budget
• Results reports
• Governing bodies
• World Health Assembly
• Executive Board
• Member States Portal
• Health topics /

Research is indispensable for resolving public health challenges – whether it be tackling diseases of poverty, responding to rise of chronic diseases,  or ensuring that mothers have access to safe delivery practices.

Likewise, shared vulnerability to global threats, such as severe acute respiratory syndrome, Ebola virus disease, Zika virus and avian influenza has mobilized global research efforts in support of enhancing capacity for preparedness and response. Research is strengthening surveillance, rapid diagnostics and development of vaccines and medicines.

Public-private partnerships and other innovative mechanisms for research are concentrating on neglected diseases in order to stimulate the development of vaccines, drugs and diagnostics where market forces alone are insufficient.

Research for health spans 5 generic areas of activity:

• measuring the magnitude and distribution of the health problem;
• understanding the diverse causes or the determinants of the problem, whether they are due to biological, behavioural, social or environmental factors;
• developing solutions or interventions that will help to prevent or mitigate the problem;
• implementing or delivering solutions through policies and programmes; and
• evaluating the impact of these solutions on the level and distribution of the problem.

High-quality research is essential to fulfilling WHO’s mandate for the attainment by all peoples of the highest possible level of health. One of the Organization’s core functions is to set international norms, standards and guidelines, including setting international standards for research.

Under the “WHO strategy on research for health”, the Organization works to identify research priorities, and promote and conduct research with the following 4 goals:

• Capacity - build capacity to strengthen health research systems within Member States.
• Priorities - support the setting of research priorities that meet health needs particularly in low- and middle-income countries.
• Standards - develop an enabling environment for research through the creation of norms and standards for good research practice.
• Translation - ensure quality evidence is turned into affordable health technologies and evidence-informed policy.
• Prequalification of medicines by WHO
• Global Observatory on Health R&D
• Global Observatory on Health Research and Development
• Implementation research toolkit
• Ethics in implementation research: participant's guide
• Ethics in implementation research: facilitator's guide
• Ethics in epidemics, emergencies and disasters: Research, surveillance and patient care: WHO training manual
• WHA58.34 Ministerial Summit on Health Research
• WHA60.15 WHO's role and responsibilities in health research
• WHA63.21 WHO's role and responsibilities in health research
• EB115/30 Ministerial Summit on Health Research: report by the Secretariat
• Science division

WHO advisory group convenes its first meeting on responsible use of the life sciences in Geneva

Challenging harmful masculinities and engaging men and boys in sexual and reproductive health

Stakeholders convene in Uganda on responsible use of the life sciences

The Technical Advisory Group on the Responsible Use of the Life Sciences and Dual-Use Research meets for the first time

WHO Science Council meeting, Geneva, Switzerland, 30-31 January 2024: report

This is a visual summary of the meeting of the WHO Science Council which took place on 30 and 31 January 2024.

WHO Technical Advisory Group on the Responsible Use of the Life Sciences and Dual-Use Research (‎TAG-RULS...

The Technical Advisory Group on the Responsible Use of the Life Sciences and Dual-Use Research (TAG-RULS DUR) was established in November 2023 to provide...

Target product profile to detect "Dracunculus medinensis" presence in environmental samples 

Dracunculiasis, also known as Guinea-worm disease, is caused by infection with the parasitic nematode (the Guinea worm). In May 1986, the Thirty-ninth...

Target product profile to detect prepatent "Dracunculus medinensis" infections in animals

Dracunculiasis, also known as Guinea-worm disease, is caused by infection with the parasitic nematode Dracunculus medinensis (the Guinea worm). In May...

Coordinating R&D on antimicrobial resistance

Ensuring responsible use of life sciences research

Optimizing research and development processes for accelerated access to health products

Prioritizing diseases for research and development in emergency contexts

Promoting research on Buruli ulcer

Research in maternal, perinatal, and adolescent health

Undertaking health law research

Feature story

One year on, Global Observatory on Health R&D identifies striking gaps and inequalities

Video: Open access to health: WHO joins cOAlition S

Video: Multisectional research on sleeping sickness in Tanzania in the context of climate change

Related health topics

Clinical trials

Global health ethics

Health Laws

Intellectual property and trade

Related links

Research and Development Blueprint

WHO Collaborating Centres

R&D Blueprint for Action to Prevent Epidemics

International Clinical Trials Registry Platform

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Most Black Americans Believe U.S. Institutions Were Designed To Hold Black People Back

6. black americans and mistrust of the u.s. health care system and medical research, table of contents.

• In their own words: Quotes from our 2023 focus groups of Black Americans
• Most Black adults say they experience racial discrimination
• Black adults feel angry or undermined in the face of discrimination 
• Black adults say they must work more than everyone else to get ahead 
• Black Americans believe the criminal justice system was designed to hold them back
• Black adults and mistrust about policing and prisons 
• Many Black Americans believe the U.S. political system was designed to hold them back
• Black Americans, Black political leaders and mistrust of the U.S. political system
• Black Americans believe the economic system was designed to hold them back
• Mistrust of big businesses
• About half of Black Americans believe U.S. news media was designed to hold them back 
• Most Black adults say they encounter inaccurate news about Black people
• Some Black Americans believe the health care system was designed to hold them back
• Mistrust about medical research
• Mistrust of family-related government policy  
• Mistrust of government reproductive health policy
• Acknowledgments
• The American Trends Panel survey methodology
• Focus group methodology

Editorial note to readers

A version of this study was originally published on June 10. We previously used the term “ racial conspiracy theories ” as an editorial shorthand to describe a complex and mixed set of findings. By using these words, our reporting distorted rather than clarified the point of the study. Changes to this version include: an updated headline, new “explainer” paragraphs, some additional context and direct quotes from focus group participants.

Claudia Deane, Mark Hugo Lopez and Neha Sahgal contributed to the revision of this report.

Although the Tuskegee Syphilis Study is one of the best-known examples of race-based medical malpractice, there are others.

Throughout the 20th century, many Black women were subject to eugenics laws that forcibly sterilized them . In 1951, Henrietta Lacks’ cervical cells were harvested and studied without her knowledge or consent.

Today, some Black women specifically seek out Black obstetricians to avoid racial discrimination in medical care and improve their health outcomes. This history of mistrust provides the context for Black Americans’ beliefs about the health care system and medical research.

A 2022 Pew Research Center report found mixed results in how Black adults assessed their experiences with health care . While nearly half (47%) said health outcomes for Black people have improved over the last 20 years, sizable minorities said they have stayed the same (31%) or gotten worse (20%).

And the majority of Black Americans (55%) said they have had negative experiences with doctors , including having to speak up to get proper care and feeling like the pain they were experiencing was not taken seriously.

In the current survey, 51% of Black Americans say the U.S. health care system was designed to hold Black people back a great deal or fair amount. Another 28% say it was designed to hold Black people back some, and 19% say not too much or not at all.

Black adults differ significantly on this question by gender. Indeed, Black women (58%) are more likely than Black men (44%) to say the health care system was designed to hold Black people back. But Black women under 50 (61%) are much more likely to say this than older Black women (54%) and all men regardless of age (44%). These patterns are like those in the 2022 study , which found that Black women (particularly those under 50) were significantly more likely than Black men to report negative experiences in health care. This includes not having their women’s health concerns taken seriously.

While many Black adults say the U.S. health care system was designed to hold Black people back (51%), 78% say they have heard the idea that medical researchers experiment on Black people without their knowledge or consent. Only 19% say they have not heard about this at all.

When it comes to medical research, 55% of Black Americans believe nonconsensual experiments are being conducted on Black people today. Fewer say this is a thing of the past (30%) or that it never happened (10%).

Like their general belief that the U.S. health care system was designed to hold Black people back, Black women (57%) are slightly more likely than Black men (52%) to believe medical experimentation on Black people without their knowledge or consent is something that is happening today.

Black adults with some college (58%) or a high school diploma (55%) are more likely than those with a bachelor’s degree or higher (49%) to say medical experimentation on Black people without their knowledge or consent happens today.

Likewise, Black adults with lower incomes (60%) are the most likely among the income groups to agree. Black adults who live in the Midwest (60%) are more likely than those in the Northeast (52%) to say medical experimentation against Black people happens today. About half of Black adults in the South (54%) and the West (53%) say the same. Black adults in urban areas (59%) are more likely than those in the suburbs (51%) to say these types of experiments happen today, while 57% of those in rural areas agree.

Sign up for our weekly newsletter

Fresh data delivery Saturday mornings

Sign up for The Briefing

Weekly updates on the world of news & information

• Black Americans
• Criminal Justice
• Racial Bias & Discrimination
• Trust in Government

Most Black Americans Believe Racial Conspiracy Theories About U.S. Institutions

An early look at black voters’ views on biden, trump and election 2024, a look at black-owned businesses in the u.s., 8 facts about black americans and the news, black americans’ views on success in the u.s., most popular, report materials.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

© 2024 Pew Research Center

• Open access
• Published: 26 May 2022

Structural changes in the Russian health care system: do they match European trends?

• Sergey Shishkin   ORCID: orcid.org/0000-0002-0807-3277 1 ,
• Igor Sheiman   ORCID: orcid.org/0000-0002-5238-4187 2 ,
• Vasily Vlassov   ORCID: orcid.org/0000-0001-5203-549X 2 ,
• Elena Potapchik   ORCID: orcid.org/0000-0001-7004-3100 1 &
• Svetlana Sazhina   ORCID: orcid.org/0000-0002-2023-3384 1  

Health Economics Review volume  12 , Article number:  29 ( 2022 ) Cite this article

7566 Accesses

3 Citations

4 Altmetric

Metrics details

In the last two decades, health care systems (HCS) in the European countries have faced global challenges and have undergone structural changes with the focus on early disease prevention, strengthening primary care, changing the role of hospitals, etc. Russia has inherited the Semashko model from the USSR with dominance of inpatient care, and has been looking for the ways to improve the structure of service delivery. This paper compares the complex of structural changes in the Russian and the European HCS.

We address major developments in four main areas of medical care delivery: preventive activities, primary care, inpatient care, long-term care. Our focus is on the changes in the organizational structure and activities of health care providers, and in their interaction to improve service delivery. To describe the ongoing changes, we use both qualitative characteristics and quantitative indicators. We extracted the relevant data from the national and international databases and reports and calculated secondary estimates. We also used data from our survey of physicians and interviews with top managers in medical care system.

The main trends of structural changes in Russia HCS are similar to the changes in most EU countries. The prevention and the early detection of diseases have developed intensively. The reduction in hospital bed capacity and inpatient care utilization has been accompanied by a decrease in the average length of hospital stay. Russia has followed the European trend of service delivery concentration in hospital-physician complexes, while the increase in the average size of hospitals is even more substantial. However, distinctions in health care delivery organization in Russia are still significant. Changes in primary care are much less pronounced, the system remains hospital centered. Russia lags behind the European leaders in terms of horizontal ties between providers. The reasons for inadequate structural changes are rooted in the governance of service delivery.

The structural transformations must be intensified with the focus on strengthening primary care, further integration of care, and development of new organizational structures that mitigate the dependence on inpatient care.

In the last two decades, health care systems in the European Union countries have faced global challenges, including aging populations, a substantial rise in chronic and multiple diseases, the emergence of new medical and information technologies, and a growing citizen awareness of the role of a healthy lifestyle in disease prevention [ 1 ]. The responses of health systems to these challenges included structural changes in their organization with a focus on the promotion of healthy lifestyles and disease prevention, the growing scale of screening for early disease detection, strengthening primary care, changing the role of the hospitals, the development of chronic disease management programs, etc. [ 2 , 3 ]

Studies of these trends address mostly Western countries. Much less attention has been paid to the post-Soviet countries. In this paper, we study structural changes in the health care in Russia. Russian health care has inherited the Semashko model of health care organization. Its main distinction is state-centered financing, regulation, and provision of health care. The model has specific forms of provider organization, for example, outpatient clinics (polyclinics) with a large number of various specialists, the separation of care for adults and children, and large highly-specialized hospitals [ 4 ].

The Soviet and post-Soviet health systems have been underfunded. Public health funding in the 1990s dropped almost by one third in real terms [ 5 ]. The organization of medical care in the 1990s has not changed significantly relative to Soviet times, and the system has adapted through the reduction in the volume of services and increased payments by patients, frequently informal [ 6 ]. The surge in oil prices after 2000 allowed health funding to increase and while encouraged noticeable changes in service delivery.

The changes in the Russian health system have been discussed in the literature mostly focusing on specific sectors and health finance reforms [ 5 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. But these changes in different sectors were not analyzed together, from a single methodological position, as changes in the structural characteristics of the Russian health care system, i.e. the changes in the ratio of different types of medical care, in the structure of medical service providers, in functionalities and modes of their interaction.

The objective of this paper is to explore the entire complex of structural changes over the past two decades in comparison with European trends. What were the structural changes in European health care systems, what were they like in Russia, and how can their differences be explained?

Study design

We followed a six-step methodological framework. The first stage involved designation of the types of medical care and the types of structural changes for identification and comparison. We considered four main areas of medical care delivery: preventive activities, primary care, inpatient care, long-term care. We focused on three different dimensions of structural changes: i) changes in the organizational structure of medical service providers; ii) changes in the structure of their activities (in its types and in their coverage of the population / patients); iii) changes in the organization of interaction between different service providers.

The second stage consisted of identifying for each type of medical care the changes in these three dimensions in the last twenty years before the COVID-19 pandemic. We described the changes that met two criteria: 1) these changes are assessed in the OECD, WHO, and World Bank reviews, and other review publications on this topic as the most noteworthy characteristics of the development of European health care systems, and 2) they have spread in a large number of European countries.

The changes identified according to the formulated criteria cover not all dimensions of structural changes for each type of medical care. For preventive activities, there are changes in the types of activities and in their coverage of the population. In primary and inpatient care, there are changes in the organizational structure of service providers, in the structure of their activities, and in the organization of interaction with other providers. In long-term care, there are changes in the structure of developed activities and their coverage of the population.

To describe the ongoing changes, we use both qualitative characteristics and, if possible, quantitative indicators that highlight them to the greatest extent.

The third stage involved detection of structural changes in four main areas of medical care delivery in Russia. We used the results of our previous studies and conducted an additional search for data characterizing structural changes in health care, using new statistical data, evidence derived from our survey of physicians and interviews with top managers in medical care system.

On the fourth stage we compared the identified structural changes in European health care systems (HCS) with the changes taking place in Russian health care. We identified the presence or absence of similar types of structural changes and the differences between them. The fifth stage was the consideration of the driving forces of structural changes in the Russian health care system. The sixth stage included discussion of the reasons for the distinctions with European developments.

Data sources

To identify the main structural changes in medical care delivery during last twenty years we searched the literature addressing both European HCS and Russia in the all aspects of changes of health care system indicators, better classified by MeSH term “health care reform”. We searched MEDLINE using the query: (russia OR europ* OR “european union” OR semashko) AND health care reform [mh] AND 2000:2021[dp]). All 788 findings were checked manually and 86 were relevant. We also used sources snowballed from these reports and the grey literature related to Russian health care, including those in limited circulation, unpublished documents, memorandums, and presentations from our personal collections covering more than twenty years.

We also used data from an online survey of 999 primary care physicians (further – survey) conducted by the authors in April–May 2019. The respondents representing 82 out of 85 regions of the Russian Federation were asked about implementation of the national prophylactic medical examination program. We also interviewed four leading specialists of the national Ministry of Health on the criteria for the inclusion of the components into the program.

To identify the driving forces of structural changes in the Russian health care system, we used materials from 10 interviews on the issues of implementing state health care programs that we conducted in 2019 with current and former top-managers in the federal government and in five regional governments. We also used the grey literature as well as published reports.

We used statistical data from the international databases of OECD [ 18 ], WHO [ 19 ], World Bank [ 20 ], as well as the Russian sources — the Federal State Statistics Service [ 21 ] and the Russian Research Instuitute of Health [ 22 ]. The data was analyzed for the period from 2000 to the latest date with available data for both EU member states and Russia. To ensure the comparability of the composition of countries in different years, the analysis of the dynamics of some indicators was limited to EU 19 members, i.e. excluding Cyprus, Greece, Croatia, Bulgaria, Luxemburg, Malta, Netherland, Poland, and Romania. The averages for EU 19 estimates are based on population size-weighted averages. If the studied publications and databases did not contain the necessary indicators, we made our own estimates.

Each section of the paper contains a brief description of the main trends in the European countries, and then provides a comparative analysis of the corresponding changes in Russian health care. The comparison is followed by a discussion of the driving forces and the limitations of structural changes in Russia compared to the main European trends. We limited our analysis to the pre – COVID-19 pandemic years.

The development of preventive activities

European hcs.

Most of them have implemented health check-ups, and population and opportunistic screenings for the early detection of diseases. These activities are viewed as a way to improve outcomes by ensuring that health services can focus on diagnosing and treating disease earlier [ 23 ]. The population covered by screenings is high and growing. In Germany 81% of population between 50 and 74 years in 2014 had been tested for colorectal cancer at least once, in Austria 78%, France 60%, Great Britain 48% [ 24 ].

The impact of these activities on health outcomes depends on the selection of preventive services, as well as on their implementation in specific national contexts. The selection of preventive services is increasingly based on research into their potential impact on mortality and other health indicators, as well as their cost effectiveness, with some services being declined because of their inadequate input into health gains [ 25 ]. It is particularly important that screenings are focused on socially disadvantaged groups with the highest probability of disease identification and the expected benefits of their management. Therefore, screening programs are based on the evaluation of local needs. Physicians have discretion in the choice of patients for screenings, depending on their importance for specific groups of the population, and individual risks and preferences.

It is increasingly common for a screening program to include follow-up management of any detected illnesses, with the implication that policy makers design such programs as a set of interrelated preventive and curative activities [ 26 ].

The original Semashko model and the current legislation prioritize preventive activities, while their implementation has been limited by the chronic underfunding of the health system. In the 2000s, the priority of prevention campaigns was revitalized in the form of a national prophylactic medical examination program (Prophylactic Program, called Dispanserization) that is a set of health check-ups and screenings. The major expectation from this Prophylactic Program is the same as in European HCS [ 27 ].

To supplement the analysis of the Prophylactic Program, we analyzed the evidence base for the components of the program and interviewed leading specialists of the federal Ministry of Health on the criteria for the inclusion of the components into the program. We found that some screenings were not evidence based and effect on the population health and/or health of participants is small [ 28 ]. The screening package of the dispanserization was expanded and reduced couple of times, but still number of ineffective screenings are included in the package (electrocardiography (ECG) screening of healthy subjects, prostate specific antigen (PSA) screening of middle age and adult men, urinalysis and routine blood tests, mammography from age 40 etc.).

Primary care physicians play a major role in conducting screenings and check-ups as well as subsequent interventions. There are also public health units responsible exclusively for these preventive activities in big polyclinics. Polled in 2019, primary care physicians responded that in 11% of polyclinics check-ups are carried out in these departments only, and in 24% of primary care organizations the check-ups are conducted by district physicians as well as by staff of these preventive units.

Under the current Prophylactic Program, people over 40 are supposed to have a set of check-ups annually; those 18–39 every three years. Most children go through physicals only. The official estimates of the coverage of the eligible population in the Prophylactic Program are around 100% [ 29 ], while service providers are less optimistic. According to the survey, more than half of the respondents reported that this share was less than 60%, while 17.4% reported less than 20% [ 27 ].

An important shortcoming of the Prophylactic Program design and implementation is the gap between its major objective and the capacity of primary care. The shortage of primary care physicians does not allow the target groups to be provided with all preventive services. Physicians have to distort the service to their registered population and to underprovide the follow-up care of detected cases. The lack of a systematic approach, less focus on local conditions, and the lack of a professional autonomy of providers are the major distinctions between Russian prevention campaigns and similar activities in Europe.

The Prophylactic Program is built on the presumption that preventive activities should include the follow-up management of any detected conditions. There is some evidence, however, that this is not taking place: according to our survey, a half of primary care physicians are unaware of the results of check-ups and screenings. The reported coverage and quality of the follow-up management of identified cases are low: a half of the respondents indicate that less than 60% of patients with identified diseases become objects of the follow-up disease management. Only 7.7% of respondents indicate that a set of disease management services corresponds to a pattern of dispensary surveillance issued by the federal Ministry of Health. The majority reports that these requirements are met only for some patients or are not met at all.

Disease management of newly identified chronic and multiple cases is focused on process rather than outcome indicators. The information on the latter is very fragmented. According to our survey, a decrease in the number of disability days of chronic patients is reported by only 14% of physicians. More than a half of respondents are unaware of the number of emergency care visits and hospital admissions of their chronic patients.

Strengthening primary care

There is a trend of multi-disciplinary primary care practices or networks development and promotion of teamwork and providers coordination in response to the growing complexity of patients. In Spain, France, and the UK it is increasingly common for large general practices to serve more than 20,000 people and provide a wider spectrum of services than in traditional solo and group practices. These emerging extended practices include pharmacists, mental care professionals, dieticians, and sometimes 2–3 specialists [ 30 , 31 ]. The role of nurses is also expanding. Most advanced nurses independently see patients, provide immunizations, health promotion, routine checks for chronically ill patients in all EU member states [ 32 ]. Related to these extended practices is the growing concentration of primary care providers via mergers and reconfigurations that increase the size of the units. The major benefits are economies of scale and scope through staff sharing and better integrated care.

There is also a general trend to strengthen the links with the local community, social care and hospitals [ 32 ]. Primary care providers are increasingly involved in chronic disease management programs together with other professionals in and out of general practices. Links with hospitals are developing beyond simple referral systems [ 33 ].

The trend of multidisciplinary practices development has greatly affected Russian health care. However, this trend in Russia differs significantly from the European HCS. It began in the 1980s, when large numbers of specialists were employed by polyclinics, which are the major providers of both primary care and outpatient specialty care. Today, large urban polyclinics employ 15–20 categories of specialists, and polyclinics in small towns 3–5 categories. The generalist who serves for the catchment area (district doctors) is limited in the scope of services they provide. Multidisciplinary practices are built through employing new specialists, while in European countries mainly through nurses and other categories of staff. Specialists in Russian polyclinics do not supplement, but essentially replace district doctors: they accounted for 66% of visits in 2019. Footnote 1

The scope of district doctors’ services is limited: at least 30–40% of initial visits end with referrals to a specialist or to a hospital, while in Europe only 5–15% [ 35 , 36 ]. Gatekeeping is promoted, but district doctors are overloaded and not interested in expanding the scope of their services. Specialists in polyclinics have insufficient training and poorly equipped, e.g. urologists do not do ureteroscopy and ophthalmologists do not practice surgery.

Since the 1990s, some regions started replacing district doctors and pediatricians with general practitioners. But this initiative has not been supported by the federal Ministry of Health, therefore the institution of a general practitioner is not accepted throughout the country. Currently, the share of general practitioners in the total number of generalists serving a catchment area is only 15% (Fig.  1 ). The model of general practice is used only in some regions. The main part of the primary care in the country is provided by district doctors and pediatricians, whose task profile remains narrower than that of general practitioners. The division of primary care for children and adults is preserved. The family is not a whole object of medical care. This division is actively defended by Russian pediatricians with references to specific methods of managing child diseases.

Distribution of generalists in Russia by categories in 2000, 2019. Source: Calculated from RRIH [ 22 , 37 ]

The prevailing trend in all European HCS is to increase the role of nurses. In Russia, the participation of nurses in medical care is limited to fulfilling doctors’ prescriptions and performing ancillary functions.

The transformation of inpatient care

Due to increased costs, technological advances in diagnosis and treatment, there were changes in patterns of diseases and patients treated in hospitals. A substantial amount of inpatient care has been moved to outpatient settings with a respective decrease in bed capacity. This is an almost universal trend in European HCS [ 19 ].

Hospitals continue to be centers of high-tech care, which concentrate most difficult cases and intensify inpatient care with a corresponding decrease in the average length of stay. These changes have been promoted by the move to diagnostic related groups based payment systems and a growing integration with other sectors of service delivery.

In many European countries, most hospitals no longer act as discrete entities and have become units of hospital-physician systems which are multi-level complex adaptive structures [ 3 ]. A new function of hospital specialists is their involvement in chronic disease management in close collaboration with general practitioners, outpatient specialists, and rehabilitative and community care providers [ 38 ].

Over the past two decades the treatment of relatively simple cases and preoperative testing have gradually moved to day care wards and polyclinics. In annual health funding, the federal government sets decreasing targets of inpatient care which are obligatory and which regions use to plan their inpatient care. However, inpatient care discharges per 100 people have been almost stable (21.9 in 2000 and 22.4 in 2018) in contrast to the EU 19 members Footnote 2 (18.4 in 2000 and 16.9 in 2018) [ 18 ]. The pressure of decreasing targets resulted in a drop in the average length of hospital stays (Fig.  2 ) and the total bed-days per person (Fig.  3 ). These indicators, along with bed supply (Fig.  4 ), decreased even faster than in the EU.

Average length of stay in hospital in EU members and Russia (days). Note: Calculated for EU 19 member states (see Methods). The EU 19 average length of hospital stay estimates are calculated as the sum of the products of inpatient care discharges by the average length of stay for each country, weighted average by the total inpatient care discharges. Source: OECD Health Statistics [ 18 ]

Number of bed-days per person in the EU and Russia. Note: Calculated for EU 19 member states (see Methods). EU 19 estimates are calculated as the sum of the products of inpatient care discharges by the average length of stay for each country weighted by the total population. Source: OECD Health Statistics [ 18 ]

Hospital beds per 1000 people in the EU and Russia. Note: Calculated as the average for all EU 28 members weighted by the total population. Source: World Bank [ 20 ]

At the same time, the intensity of medical care processes in hospitals in Russia remains significantly lower than in European countries. An indicator of this is the gap in the number of hospital employees per 1000 discharged (Table  1 ).

Over the past 20 years, significant efforts have been made to deploy day wards, both in hospitals and polyclinics, to reduce the burden on hospitals. As a result, the proportion of patients treated in day wards in the total number of patients treated in hospitals increased from 7.6% in 2000 to 20.8% in 2016 [ 21 ]. However, there is fragmentary evidence that this figure is still noticeably lower than in Europe. The share of cataract surgery carried out as ambulatory cases varies in most European countries between 80 to 99% [ 24 ] but is negligible in Russia.

Despite these positive trends, the health system remains hospital centered. The number of bed-days per person remains nearly twice as high as the EU average (Fig. 3 ).

An important trend is the increasing concentration of hospitals. The number of hospitals halved between 2000 and 2018, mostly due to mergers, but also due to the closures of inadequately equipped hospitals. This process has led to an increase in the average size of hospitals from 156 beds in 2000 to 223 beds in 2018 [ 21 ]. This figure is higher today than in Western countries with large territories. The average hospital size in France was 130 beds in 2018 and in Germany 215 beds in 2017 [ 18 ]. In Russia, with its very low population density, the reduction in the number of small rural hospitals resulted in some accessibility problems.

At the same time, the incorporation of previously independent polyclinics into hospitals is under way. The proportion of independent polyclinics in the total number of polyclinics has decreased from 35% in 2000 to 19% in 2014 [ 36 ].

The development of long-term care

Over the last 20 years, most European countries have increasingly developed the public provision of long-term care. The number of nursing and elderly home beds per 100,000 people in the EU increased from 581.7 in 2000 to 748.3 in 2014 [ 19 ], although the pace of changes, the coverage of citizens in need of long-term care, and its organization and funding differ substantially across countries [ 39 ]. Many countries control costs by keeping people in their homes longer and shifting the responsibility for non-institutional forms of care to communities [ 40 ]. An expected outcome of investment in long-term care is the reduction of informal care utilization.

Compared to European HCS, long-term care is underdeveloped in Russia. The number of nursing care beds declined from 14.7 per 100,000 people in 2011 to 10.6 in 2019 [ 22 ]. The share of citizens over working age and people with disabilities receiving outpatient and inpatient care within the long-term care system in the total number of citizens over working age and people with disabilities in need of long-term care, was only 2.9% in 2019 [ 41 ].

In contrast to the European HCS, Russia has not built a strong long-term care sector with the capacity to reduce the workload of acute inpatient care settings. Hospitals have to keep some patients longer resulting in a relatively higher length of stay. Palliative care as another sector of the long-term care which started to develop only a few years ago.

Driving forces and tools of structural changes in the Russian health care system

These changes have been driven by the federal and regional governments. They use two main tools to manage structural changes: 1) setting health care targets for the entire country and for regions, and 2) implementing vertical health care programs.

Since 1998, the federal government has annually approved a program of benefit packages for health (the Program). It sets targets for the utilization of medical care for each sector of service delivery, as well as unit cost targets. The Program is designed to balance the volumes of care with the amount of public funding. The annual versions of the Program gradually reduced the targets for inpatient care to encourage a shift to outpatient care. The federal targets are used in regional health planning. In the first decade of using the Program, the changes in the actual volume of medical care were small, but in the second decade, pressure from the federal center on the regions increased, and the gap between the federal targets and the actual utilization of care has noticeably narrowed (Table  2 ).

The development of the legislation on the delimitation of responsibility between levels of government, carried out in the last two decades, has consistently strengthened the regional governments role in restructuring medical care delivery. In 2012, almost all resources of health care governance were transferred from the municipal to the regional level (including the governance of primary health care. During the period 2000–2019 the number of public hospitals has decreased by 2.2 times, the number of hospital beds by 1.5 times, polyclinics 1.3 times, feldsher-obstetric posts 1.3 times. Footnote 3

When oil prices increased, the federal government poured additional resources into vertical programs. They are administered by the federal Ministry of Health and regional governments. The major programs: the ‘Priority national health project’ (2004–2012), the Prophylactic Program (2008 – ongoing), and regional programs for the modernization of health care (2011–2013). All additional and some basic resources are earmarked in an attempt to develop the highest priority activities: preventive care, obstetric care, cardiovascular surgery, oncology, etc.

The role of the centralized administration of these priority programs is controversial. The federal government initiated them, provided regions with additional funding, and made the program’s targets a priority of health policy. According to interviews with federal and regional officials, the implementation of programs is heavily controlled by the federal government: practically all decisions on specific activities, target indicators and resource allocation are approved on the federal level. The Russian regions have low flexibility to respond to local needs such as variation in disease incidence, the capacity of health care, or vulnerable population groups.

Structural changes in the provision of inpatient care were prompted by the introduction of a diagnostic related groups based payment system in the early 2010s. This was initiated by the federal government and implemented with the participation of the World Bank experts. It makes more profitable for hospitals to reduce the duration of hospitalizations and to complicate the structure of inpatient treatment [ 44 ].

We found that despite significant differences in health care organization, some structural changes in Russia have followed the general European trends. A similar rise in the coverage of the population with screenings is underway in Russia. There is a clear tendency to replace some inpatient care with day care. The volume of inpatient care is reducing —mostly due to a significant decrease in the length of stays, while the rate of hospital admission remains relatively stable. As in the most European HCS, the concentration of medical organizations and the formation of large outpatient and inpatient complexes is developing.

However, there are some substantial differences: the development of prevention programs is relatively less focused on the most vulnerable target groups and on local needs; primary care specialization is much stronger than in European HCS; the role of first contact generalists is waning; the worldwide tendency of increasing the role of nurses is almost invisible in Russia; long-term care is starting to develop but is still at a very low level and palliative care is in its infancy; integration in the health system are much less pronounced—both at the level of individual medical organizations and between health sectors.

The reasons for these differences are rooted in the specific features of health governance in Russia.

The Semashko model, by virtue of its genesis, reproduces the state administration patterns of a planned economy. The main driving force of changes is the bureaucracy. Its managerial activities are guided by the mechanism described by J. Kornai: ‘postponement, putting out the fire, postponement’ [ 45 ]. The governance focuses on mobilizing and distributing available resources to solve or mitigate the most pressing problems - ‘fire fighting’. This is what determines the fragmentation of structural changes in Russian health care compared to structural changes in European countries.

Materials of interviews with heads of federal and regional health authorities suggest that in the existing governance system each of its levels must demonstrate the success of its activity exclusively to the higher levels of management. It is easier to achieve success when solving problems of optimizing the volume of medical care and the organizational structure of medical institutions, and much more difficult when solving problems of improving the efficiency of all elements of medical care system, which requires changes in their functionality and ways of interaction. It requires more financial resources and better management at all levels of health governance.

A number of deeply rooted limitations for carrying out structural transformations in Russian health care can be highlighted.

Firstly, the low capacity of primary care providers and to some extent the unwillingness of patients to replace inpatient care with outpatient treatment prevents a shift of patients from hospitals to polyclinics.

Secondly, a feature of the Russian health care system is the weak development of horizontal links between medical organizations related to different levels of medical care, and between medical workers within medical organizations working in different departments [ 36 , 46 , 47 ]. The interaction of different providers is carried out mostly through vertical channels. This is a serious obstacle to the development of horizontal integration [ 36 ].

Thirdly, democratic institutions for the development of health care are historically underdeveloped in Russia and this influences the choice of health policy priorities. According to interviews with heads of regional health authorities, the role of local communities is negligible, and the role of the medical community is marginal. Professional organizations are rarely involved in decision-making on health issues. The input of public councils to government bodies is largely imitative. Information about the activities of the system as a whole and of individual medical organizations is restricted for public use. This enables health authorities to focus on achievements in their reports, while hiding shortcomings. Feedback from patients, and society as a whole, is poorly expressed.

Conclusions

Russian health care, whose genetic basis was the Soviet Semashko model, after a difficult ‘survival’ period in the 1990s, underwent significant structural changes over the next two decades. To a large extent, the directions of these changes have coincided with European trends. The prevention and the early detection of diseases have developed intensively. The reduction in hospital bed capacity and inpatient care was accompanied by an intensification of inpatient treatment and a decrease in the average length of stay. Russia has followed the European trend of service delivery concentration in hospital-physician complexes, while the increase in the average size of hospitals is even more substantial. Structural changes in primary care are much less pronounced. The resources and competences of providers and the governance of primary care are still not enough to abolish the hospital-centered model of service delivery. Russia has intensively implemented vertical health care programs to develop the priority activities, but still significantly lags in the level of development of horizontal ties among services providers.

Specific structural changes in Russia are rooted in the organization and governance of service delivery. The interests of federal and regional bureaucracies, which act as the main drivers of changes, are pushing them to prioritize the changes in volumes of medical care and organizational structure of health care providers and not spend a lot of effort on improving their functionality and modes of interaction between providers of medical care. An important role is also played by the low capacity of primary care units to provide quality care.

To respond effectively to modern global challenges, reduce mortality, and improve the health of the population, structural transformations in Russian health care must be intensified with the focus on strengthening primary care, the further integration of care, and an accelerated development of new structures that mitigate the dependence on inpatient care.

Availability of data and materials

The data used and analysed during the current study are publicly available.

Calculated using data from [ 34 ].

See Methods.

Calculated using data from [ 21 ].

Abbreviations

European Union

Health Care System

Organization for Economic Co-operation and Development

World Health Organization

World Health Organization. Better Health for Europe: more equitable and sustainable. Copenhagen: WHO; 2019. https://apps.who.int/iris/bitstream/handle/10665/327427/9789289054225-eng.pdf . Accessed 9 Mar 2021.

Kringos D, Boerma W, Bourgueil Y, et al. The strength of primary care in Europe: an international comparative study. Br J Gen Pract. 2013;63(616):e742–50. https://doi.org/10.3399/bjgp13X674422 .

Article   PubMed   PubMed Central   Google Scholar  

McKee M, Merkur S, Edwards N, Nolte E, eds. The changing role of the Hospital in European Health Systems. European Observatory on Health Systems and Policies. Cambrige University Press; 2020. https://www.euro.who.int/__data/assets/pdf_file/0010/448048/Changing-role-of-hospitals-eng.pdf . Accessed 11 Feb 2021.

Davis C. Priority and the shortage model: the medical system in the socialist economy. In: Davis C, Charemza W, editors. Models of disequilibrium and shortage in centrally planned economies. London: Chapman and Hall; 1989. p. 427–59.

Chapter   Google Scholar  

Shishkin S. Russia’s health care system: difficult path of reform. In: Alexeev M, Weber S, editors. The Oxford handbook of the Russian economy. Oxford: Oxford University Press; 2013. p. 748–74.

Google Scholar  

Shishkin SV, Vlassov VV. Russia's healthcare system: in need of modernisation. BMJ. 2009;338:b2132. https://doi.org/10.1136/bmj.b2132 .

Rese A, Balabanova D, Danishevski K, McKee M, Sheaff R. Implementing general practice in Russia: getting beyond the first steps. BMJ. 2005;331(7510):204–7. https://doi.org/10.1136/bmj.331.7510.204 .

Human Rights in Patient Care. Stockholm Region Office in St.Petersburg. Russian Healthcare System Overview. St. Petersburg, 2010. 2010. https://health-rights.org/index.php/cop/item/russian-healthcare-system-overview . Accessed 29 Nov 2020.

Gordeev VS, Pavlova M, Groot W. Two decades of reforms. Appraisal of the financial reforms in the Russian public healthcare sector. Health Policy. 2011;102(2-3):270–7. https://doi.org/10.1016/j.healthpol.2010.08.009 .

Popovich L, Potapchik E, Shishkin S, Richardson E, Vacroux A, Mathivet B. Russian Federation: health system review. Health Syst Transit. 2011;13(7):1-190, xiii-xiv.

PubMed   Google Scholar  

Bochkareva VK. Health-care reform and ensuring equal access to free medical services. Probl Econ Transit. 2013;55(12);12-31. https://doi.org/10.2753/PET1061-1991551203 .

Sheiman I, Shevski V. Evaluation of health care delivery integration: the case of the Russian Federation. Health Policy. 2014. https://doi.org/10.1016/j.healthpol.2013.12.011 .

Kontsevaya A, Sabgaida T, Ivanova A, Leon DA, McKee M. How has the management of acute coronary syndrome changed in the Russian Federation during the last 10 years? Health Policy. 2017;121(12):1274-9. https://doi.org/10.1016/j.healthpol.2017.09.018 .

Tagaeva TO, Kazantseva LK. Public health and health Care in Russia. Probl Econ Transit. 2017;59(11-12):973-90. https://doi.org/10.1080/10611991.2017.1431488 .

Belova N. Health Care in Rural Areas. Sociol Res. 2018;57(2):123-35. https://doi.org/10.1080/10610154.2018.1577621 .

Sheiman I, Shishkin S, Shevsky V. The evolving Semashko model of primary health care: the case of the Russian Federation. Risk Manag Healthc Policy. 2018;11:209-20. https://doi.org/10.2147/RMHP.S168399 .

Somanathan A, Sheiman I, Salakhutdinova S, Buisman L. Universal health coverage in Russia: extending coverage for the poor in the post-soviet era. Universal health coverage study series. Washington, DC: Universal Health Coverage Study Series The World Bank Group; 2018. p. 37. https://openknowledge.worldbank.org/handle/10986/30026 . Accessed 10 Mar 2021.

OECD. Health Statistics. OECD; 2020. https://stats.oecd.org/Index.aspx?ThemeTreeId=9 . Accessed 01 Dec 2020.

World Health Organization (WHO). European Health for All database (HFA-DB). WHO; 2020. https://gateway.euro.who.int/ru/datasets/european-health-for-all-database/ . Accessed 29 Nov 2020.

World Bank. World Bank Open Data. The World Bank Group, 2020. https://data.worldbank.org/ .

Federal State Statistics Service website (Rosstat). Health care in Russia. Moscow: 2020. https://rosstat.gov.ru/folder/210/document/13218 . Accessed 02 Dec 2020.

Russian Research Institute for Health (RRIH). Health statistics. Moscow: 2020. https://mednet.ru/napravleniya/mediczinskaya-statistika . Accessed 29 Nov 2020.

International Agency for Research on Cancer. Cancer Screening in the European Union. Report on the implementation of the Council Recommendation on cancer screening. Lyon: European Commission; 2017. https://ec.europa.eu/health/sites/default/files/major_chronic_diseases/docs/2017_cancerscreening_2ndreportimplementation_en.pdf . Accessed 19 Dec 2020.

OECD. Health at a Glance 2019: OECD Indicators. Paris: OECD Publishing; 2019.

Book   Google Scholar  

McKee M, Rechel B. Screening. In: Rechel B and McKee M eds. Facets of public health in Europe. WHO Observatory for health systems and health reforms. Berkshire: Open University Press; 2014. https://www.euro.who.int/__data/assets/pdf_file/0003/271074/Facets-of-Public-Health-in-Europe.pdf . Accessed 21 Mar 2021.

Sagan A, McDaid D, Rajan S, Farrington J, McKee M. Screening: When is it appropriate and how can we get it right? European Observatory on health systems and health reforms. European Observatory Policy Brief. 35. Copenhagen, 2020. https://apps.who.int/iris/bitstream/handle/10665/330810/Policy-brief-35-1997-8073-eng.pdf . Accessed 20 Apr 2021.

Sheiman IM, Shishkin SV, Shevsky VI, Sazhina SV, Ponkratova OF. Regular medical check-ups in Russia: expectations and reality. Mir Rossii. 2021;30(4):6-29. https://doi.org/10.17323/1811-038X-2021-30-4-6-29 .

Rebrova OI, Fedyaeva VK, Omel'ianovskiĭ VV, Ilyin MA. Stage 1 screenings of periodical health check: a systematic review of long-term efficiency. Profilakticheskaya Meditsina. 2017;20(3):55-59. https://doi.org/10.17116/profmed201720355-59 .

Federal Fund of Mandatory Health Insurance. Report about the mandatory health insurance system activity in the Russian Federation in 2019. Mandatory Med Insur Russian Federat. 2020;4:22.

Dedeu T, Marti T. Multiprofile primary health care teams in Catalonia, Spain: a population-based effective model of service delivery. WHO; 2018. https://www.euro.who.int/__data/assets/pdf_file/0009/366498/HSS-NCDs-Policy-brief-cat.pdf . Accessed 15 Feb 2020.

Kumpunen S, Curry N, Farnworth M, Rosen R. Collaboration in general practice: surveys of GP practice and clinical commissioning groups. Nuffield trust, Royal College of General Practitioners; 2017. https://www.nuffieldtrust.org.uk/files/2017-10/collaboration-in-general-practice-2017-final.pdf . Accessed 20 Dec. 2020.

OECD. Realizing the full potential of primary care. Policy Brief. OECD; 2019. https://www.oecd.org/health/health-systems/OECD-Policy-Brief-Primary-Health-Care-May-2019.pdf . Accessed 11 Dec 2020.

Damiani G, Pascucci D, Sindoni A, et al. The bigger, the better? A systematic review on the impact of mergers on primary care organizations. Eur J Pub Health. 2021;31(2):244-52. https://doi.org/10.1093/eurpub/ckaa248 .

The Ministry of Health of the Russian Federation. The accounting form No. 30 of the Federal Statistical Surveillance for 2019 (raw data) (not in the public domain).

Kringos DE, WGW B, Hutchinson A, Saltman R. Building primary Care in a Changing Europe. Eur Observ Health Syst Health. Observatory Studies Series. 2015;38;172.

Sheiman I, Shevsky V. Concentration of health care providers: does it contribute to integration of service delivery? Risk Manag Healthc Policy. 2019;12:153-66. https://doi.org/10.2147/RMHP.S205905 .

Rugol LV, Son IM, Starodubov VI, Pogonin AV. Some results of healthcare reforming. Soc Aspects Popul. Health. 2021;67(1):4. https://doi.org/10.21045/2071-5021-2018-64-6-1 .

Article   Google Scholar  

Urbanski D, Reichert A, Amelung V. Discharge and Transition Management in Integrated Care. In: Amelung V, Stein V, Goodwin N, Balicer R, Nolte E, Suter E, editors. Handbook Integrated Care; 2017. p. 97–111.

Hernández-Quevedo C, Maresso A, Merkur S, Quentin W, Richardson E, Ginneken E. 20 years of health system reforms in Europe: what’s new? Eurohealth. 2018;24(2):23–8.

Spasova S, Baeten R, Vanhercke B. Challenges in long-term care in Europe. Eurohealth. 2018;24(4):7–12.

The presidium of the presidential Council for the Strategic Development and Priority Projects. The Federal Project ‘the senior generation’. Protocol No 16 dated December 24, 2018. The Ministry of Labour and Social Protection; 2018. https://mintrud.gov.ru/ministry/programms/demography/3 . Accessed 15 Nov 2020.

Government of the Russian Federation. Resolutions of the government of the Russian Federation ‘on the program of the state guarantees of free provision of medical care to Citizens’ 1998–2018. The Federal Compulsory Medical Insurance Fund. 2018. https://www.ffoms.gov.ru/system-oms/about-fund/questions-oms/programm-gos-guarantee/ . Accessed 11 Sept 2021.

The Ministry of Health of the Russian Federation. Reports on the implementation of the program of the state guarantees of free provision of medical care to Citizens, 1999–2019. Moscow: The Ministry of Health of the Russian Federation, 2000-2019.

Avxentyeva MV, Omelyanovskiy VV, Petrovskiy AV, et al. New approaches to the development of diagnostic related groups for Cancer pharmacotherapy in Russian Federation. Med Technol Assess Choice. 2018;2(32):8–22.

Kornai J. Economics of shortage. Amsterdam: Elsevier Science. 1980.

Kuhlmann E, Shishkin S, Richardson E, et al. Understanding the role of physicians within the managerial structure of Russian hospitals. Health Policy. 2019;123(8):773-81. https://doi.org/10.1016/j.healthpol.2019.05.020 .

Sazhina SV, Shevsky VI, Sheiman IM, Shishkin SV. Interaction of health care providers: physicians' survey outcomes. Soc Aspects Popul Health. 2021;67(1):4 http://vestnik.mdnet.ru/content/view/1230/30/lang,ru/ .

Download references

Acknowledgements

Authors’ information (optional).

Sergey Shishkin – DSc in Economics, Director, Centre for Health Policy, HSE University.

Igor Sheiman – PhD in Economics, Professor, Health Economics and Management Department, HSE University.

Vasily Vlassov – DSc in Internal Diseases, Professor, Health Economics and Management Department, HSE University.

Elena Potapchik – PhD in Economics, Leading Research Fellow, Centre for Health Policy, HSE University.

Svetlana Sazhina – MPA, Leading Analyst, Centre for Health Policy, HSE University.

The study was funded by the grant provided by the Ministry of Science and Higher Education of the Russian Federation (Grant Agreement No. 075–15–2020-928).

Author information

Authors and affiliations.

Centre for Health Policy, National Research University Higher School of Economics, 4 Slavyanskaya Ploshchad, Building 2, Moscow, Russia, 109074

Sergey Shishkin, Elena Potapchik & Svetlana Sazhina

Department of Health Care Administration and Economics, School of Politics and Governance, Faculty of Social Sciences, National Research University Higher School of Economics, 20 Myasnitskaya Ulitsa, Moscow, Russia, 101000

Igor Sheiman & Vasily Vlassov

You can also search for this author in PubMed   Google Scholar

Contributions

Sergey Shishkin conceptualized, designed the study and supervised the work. All authors collected, analyzed and interpreted the data. Elena Potapchik, Svetlana Sazhina made statistical analysis. Sergey Shishkin, Igor Sheiman and Vasily Vlassov wrote a first draft of the manuscript. All authors critically reviewed the draft. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sergey Shishkin .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Shishkin, S., Sheiman, I., Vlassov, V. et al. Structural changes in the Russian health care system: do they match European trends?. Health Econ Rev 12 , 29 (2022). https://doi.org/10.1186/s13561-022-00373-z

Download citation

Received : 19 January 2022

Accepted : 05 May 2022

Published : 26 May 2022

DOI : https://doi.org/10.1186/s13561-022-00373-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health care system
• Health service provision
• Primary care
• Inpatient care
• Structural changes
• Semashko model

Health Economics Review

ISSN: 2191-1991

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Health Care Access and Quality

Goal: increase access to comprehensive, high-quality health care services..

Many people in the United States don’t get the health care services they need. Healthy People 2030 focuses on improving health by helping people get timely, high-quality health care services.

About 1 in 10 people in the United States don’t have health insurance. 1 People without insurance are less likely to have a primary care provider, and they may not be able to afford the health care services and medications they need. Strategies to increase insurance coverage rates are critical for making sure more people get important health care services, like preventive care and treatment for chronic illnesses.

Sometimes people don’t get recommended health care services, like cancer screenings, because they don’t have a primary care provider. Other times, it’s because they live too far away from health care providers who offer them. Interventions to increase access to health care professionals and improve communication — in person or remotely — can help more people get the care they need.

Objective Status

Learn more about objective types

Related Objectives

The following is a sample of objectives related to this topic. Some objectives may include population data.

Health Care Access and Quality — General

• Adolescents

Drug and Alcohol Use

Family planning.

• Health Care

Health Communication

• Health Insurance
• Oral Conditions

Pregnancy and Childbirth

Sensory or communication disorders, sexually transmitted infections, other topics you may be interested in.

Berchick, E.R., Hood, E., & Barnett, J.C. (2018). Health Insurance Coverage in the United States: 2017. Retrieved from https://www.census.gov/content/dam/Census/library/publications/2018/demo/p60-264.pdf  [PDF - 1.4 MB]

The Office of Disease Prevention and Health Promotion (ODPHP) cannot attest to the accuracy of a non-federal website.

Linking to a non-federal website does not constitute an endorsement by ODPHP or any of its employees of the sponsors or the information and products presented on the website.

You will be subject to the destination website's privacy policy when you follow the link.

• Open access
• Published: 01 July 2022

Challenges to implementing artificial intelligence in healthcare: a qualitative interview study with healthcare leaders in Sweden

• Lena Petersson 1 ,
• Ingrid Larsson 1 ,
• Jens M. Nygren 1 ,
• Per Nilsen 1 , 2 ,
• Margit Neher 1 , 3 ,
• Julie E. Reed 1 ,
• Daniel Tyskbo 1 &
• Petra Svedberg 1  

BMC Health Services Research volume  22 , Article number:  850 ( 2022 ) Cite this article

28k Accesses

62 Citations

30 Altmetric

Metrics details

Artificial intelligence (AI) for healthcare presents potential solutions to some of the challenges faced by health systems around the world. However, it is well established in implementation and innovation research that novel technologies are often resisted by healthcare leaders, which contributes to their slow and variable uptake. Although research on various stakeholders’ perspectives on AI implementation has been undertaken, very few studies have investigated leaders’ perspectives on the issue of AI implementation in healthcare. It is essential to understand the perspectives of healthcare leaders, because they have a key role in the implementation process of new technologies in healthcare. The aim of this study was to explore challenges perceived by leaders in a regional Swedish healthcare setting concerning the implementation of AI in healthcare.

The study takes an explorative qualitative approach. Individual, semi-structured interviews were conducted from October 2020 to May 2021 with 26 healthcare leaders. The analysis was performed using qualitative content analysis, with an inductive approach.

The analysis yielded three categories, representing three types of challenge perceived to be linked with the implementation of AI in healthcare: 1) Conditions external to the healthcare system; 2) Capacity for strategic change management; 3) Transformation of healthcare professions and healthcare practice.

Conclusions

In conclusion, healthcare leaders highlighted several implementation challenges in relation to AI within and beyond the healthcare system in general and their organisations in particular. The challenges comprised conditions external to the healthcare system, internal capacity for strategic change management, along with transformation of healthcare professions and healthcare practice. The results point to the need to develop implementation strategies across healthcare organisations to address challenges to AI-specific capacity building. Laws and policies are needed to regulate the design and execution of effective AI implementation strategies. There is a need to invest time and resources in implementation processes, with collaboration across healthcare, county councils, and industry partnerships.

Peer Review reports

The use of artificial intelligence (AI) in healthcare can potentially enable solutions to some of the challenges faced by healthcare systems around the world [ 1 , 2 , 3 ]. AI generally refers to a computerized system (hardware or software) that is equipped with the capacity to perform tasks or reasoning processes that we usually associate with the intelligence level of a human being [ 4 ]. AI is thus not one single type of technology but rather many different types within various application areas, e.g., diagnosis and treatment, patient engagement and adherence, and administrative activities [ 5 , 6 ]. However, when implementing AI technology in practice, certain problems and challenges may require an optimization of the method in combination with the specific setting. We may therefore define AI as complex sociotechnical interventions as their success in a clinical healthcare setting depends on more than the technical performance [ 7 ]. Research suggests that AI technology may be able to improve the treatment of many health conditions, provide information to support decision-making, minimize medical errors and optimize care processes, make healthcare more accessible, provide better patient experiences and care outcomes as well as reduce the per capita costs of healthcare [ 8 , 9 , 10 ]. Even if the expectations for AI in healthcare are great [ 2 ], the potential of its use in healthcare is far from having been realized [ 5 , 11 , 12 ].

Most of the research on AI in healthcare focuses heavily on the development, validation, and evaluation of advanced analytical techniques, and the most significant clinical specialties for this are oncology, neurology, and cardiology [ 2 , 3 , 11 , 13 , 14 ]. There is, however, a current research gap between the development of robust algorithms and the implementation of AI systems in healthcare practice. The conclusion in newly published reviews addressing regulation, privacy and legal aspects [ 15 , 16 ], ethics [ 16 , 17 , 18 ], clinical and patient outcomes [ 19 , 20 , 21 ] and economic impact [ 22 ], is that further research is needed in a real-world clinical setting although the clinical implementation of AI technology is still at an early stage. There are no studies describing implementation frameworks or models that could inform us concerning the role of barriers and facilitators in the implementation process and relevant implementation strategies of AI technology [ 23 ]. This illustrates a significant knowledge gap on how to implement AI in healthcare practice and how to understand the variation of acceptance of this technology among healthcare leaders, healthcare professionals, and patients [ 14 ]. It is well established in implementation and innovation research that novel technologies, such as AI, are often resisted by healthcare leaders, which contributes to their slow and variable uptake [ 13 , 24 , 25 , 26 ]. New technologies often fail to be implemented and embedded in practice because healthcare leaders do not consider how they fit with or impact existing healthcare work practices and processes [ 27 ]. Although, understanding how AI technologies should be implemented in healthcare practice is unexplored.

Based on literature from other scientific fields, we know that the leaders’interest and commitment is widely recognized as an important factor for successful implementation of new innovations and interventions [ 28 , 29 ]. The implementation of AI in healthcare is thus supposed to require leaders who understand the state of various AI systems. The leaders have to drive and support the introduction of AI systems, the integration into existing or altered work routines and processes, and how AI systems can be deployed to improve efficiency, safety, and access to healthcare services [ 30 , 31 ]. There is convincing evidence from outside the healthcare field of the importance of leadership for organizational culture and performance [ 32 ], the implementation of planned organizational change [ 33 ], and the implementation and stimulation of organizational innovation [ 34 ]. The relevance of leadership to implementing new practices in healthcare is reflected in many of the theories, frameworks, and models used in implementation research that analyses barriers to and facilitators of its implementation [ 35 ]. For example, Promoting Action on Research Implementation in Health Services [ 36 ], Consolidated Framework for Implementation Research (CFIR) [ 37 ], Active Implementation Frameworks [ 38 ], and Tailored Implementation for Chronic Diseases [ 39 ] all refer to leadership as a determinant of successful implementation. Although these implementation models are available and frequently used in healthcare research, they are highly abstract and not tailored to the implementation of AI systems in healthcare practices. We thus do not know if these models are applicable to AI as a socio-technical system or if other determinants are important for the implementation process. Likewise, based on a new literature study, we found no AI-specific implementation theories, frameworks, or models that could provide guidance for how leaders could facilitate the implementation and realize the potential of AI in healthcare [ 23 ]. We thus need to understand what the unique challenges are when implementing AI in healthcare practices.

Research on various types of stakeholder perspectives on AI implementation in healthcare has been undertaken, including studies involving professionals [ 40 , 41 , 42 , 43 ], patients [ 44 ], and industry partners [ 42 ]. However, very few studies have investigated the perspectives of healthcare leaders. This is a major shortcoming, given that healthcare leaders are expected to have a key role in the implementation and use of AI for the development of healthcare. Petitgand et al.’s study [ 45 ] serves as a notable exception. They interviewed healthcare managers, providers, and organizational developers to identify barriers to integrating an AI decision-support system to enhance diagnostic procedures in emergency care. However, the study did not focus on the leaders’ perspectives, and the study was limited to one particular type of AI solution in one specific care department. Our present study extends beyond any specific technology and encompasses the whole socio-technical system around AI technology. The present study thus aimed to explore challenges perceived by leaders in a regional Swedish healthcare setting regarding implementation of AI systems in healthcare.

This study took an explorative qualitative approach to understanding healthcare leaders’ perceptions in contexts in which AI will be developed and implemented. The knowledge generated from this study will inform the development of strategies to support an AI implementation and help avoid potential barriers. The analysis was based on qualitative content analysis, with an inductive approach [ 46 ]. Qualitative content analysis is widely used in healthcare research [ 46 ] to find similarities and differences in the data, in order to understand human experiences [ 47 ]. To ensure trustworthiness, the study is reported in accordance with the Consolidated Criteria for Reporting Qualitative Research 32‐item checklist [ 48 ].

The study was conducted in a county council (also known as “region”) in the south of Sweden. The Swedish healthcare system is publicly financed based on local taxation; residents are insured by the state and there is a vision that healthcare should be equally accessible across the population. Healthcare responsibility is decentralized to 21 county councils, whose responsibilities include healthcare provision and promotion of good health for citizens.

The county council under investigation has since 2016 invested financial, personnel and service resources to enable agile analysis (based on machine learning models) of clinical and administrative data of patients in healthcare [ 49 , 50 ]. The ambition is to gain more value from the data, utilizing insights drawn from machine learning on healthcare data to make facts-based decisions on how healthcare is managed, organized, and structured in routines and processes. The focus is thus on overall issues around management, staffing, planning and standardization for optimization of resource use, workflows, patient trajectories and quality improvement at system level. This includes several layers within the socio-technical ecosystem around the technology, dealing with: a) generating, cleaning, and labeling data, b) developing models, verifying, assuring, and auditing AI tools and algorithms, c) incorporating AI outputs into clinical decisions and resource allocation, and d) the shaping of new organizational structures, roles, and practices. Given that AI thus extends beyond any specific technology and encompasses the whole socio-technical system around the technology, in the context of this article, it is hereafter referred to generically as ‘AI systems’. We deliberately sought to understand the broad perspectives on healthcare leaders in a region that has a high level of support for AI developments and our study thus focuses on the potential of a wide range of AI systems that could emerge from the regional investments, rather than a specific AI application or AI algorithms.

Participants

Given the focus on understanding healthcare leaders’ perceptions, we purposively recruited leaders who were in a position to potentially influence the implementation and use of AI systems in relation to the setting described above. To achieve potential variability, these leaders belonged to three groups: politicians at the highest county council level, managers at various levels, such as the hospital director, manager for primary care, manager for knowledge and evidence, head of research and development center, and quality developers and strategists with responsibilities for strategy-based work at county council level or development work in various divisions in the county council healthcare organization.

The ambition was to include leaders who had a range of experiences, interests and with different mandates and responsibilities in relation to funding, running, and sustaining the implementation of AI systems in practice. A sample of 28 healthcare leaders was invited through snowball recruitment; two declined and 26 agreed to participate (Table 1 ). This sample comprised five individuals originally identified on the basis of their knowledge and insights. They were interviewed and they then identified and suggested other leaders to interview.

Data collection

Individual semi-structured interviews were conducted between October 2020 and May 2021 via phone or video communication by one of the authors (LP or DT). We start from a broad perspective on AI focusing on healthcare leaders’ perceptions bottom-up and not on the views of AI experts or healthcare professionals who work with specific AI algortihms in clinical practice. The interviews were based on an interview guide, structured around: 1) the roles and previous experiences of the informants regarding the application of AI systems in practice, 2) the opportunities and problems that need to be considered to support implementation of AI systems, 3) beliefs and attitudes towards the possibilities of using AI systems to support healthcare improvements, and 4) the obstacles, opportunities and facilitating factors that need to be considered to enable AI systems to fit into existing processes, methods and systems. The interview guide was thus based on important factors previously identified in terms of implementing technology in healthcare [ 51 , 52 ]. Interviews lasted between 30 and 120 min, with a total length of 23 h and 49 min and were audio-recorded.

Data analysis

An inductive qualitative content analysis [ 46 ] was used to analyze the data. First, the interviews were transcribed verbatim and read several times by the first (LP) and second (IL) authors, to gain familiarity. Then, the first (LP) and second (IL) authors conducted the initial analyses of the interviews, by identifying and extracting meaning units and/or phrases with information relevant to the object of the study. The meaning units were then abstracted into codes, subcategories, and categories. The analytical process was discussed continuously between authors (LP, IL, JMN, PN, MN, PS). Finally, all authors, who are from different disciplines, reviewed and discussed the analysis to increase the trustworthiness and rigour of the analysis. To further strengthen the trustworthiness, the leaders’ quotations used in this paper were translated from Swedish to English by a native English-speaking professional proofreader and were edited only slightly to improve readability.

Three categories consisting of nine sub-categories emerged from the analysis of the interviews with the healthcare leaders (Fig.  1 ). Conditions external to the healthcare system concern various exogenous conditions and circumstances beyond the direct control of the healthcare system that the leaders believed could affect AI implementation. Capacity for strategic change management reflects endogenous influences and internal requirements related to the healthcare system that the leaders suggested could pose challenges to AI implementation. Transformation of healthcare professions and healthcare practice concerns challenges to AI implementation observed by the leaders, in terms of how AI might change professional roles and relations and its impact on existing work practices and routines.

Categories and subcategories

Conditions external to the healthcare system

Addressing liability issues and legal information sharing.

The healthcare leaders described the management of existing laws and policies for the implementation of AI systems in healthcare as a challenge and an issue that was essential to address. According to them, the existing laws and policies have not kept pace with technological developments and the organization of healthcare in today’s society and need to be revised to ensure liability.

The accountability held among individuals, organizations, and AI systems regarding decisions based on support from an AI algorithm was perceived as a risk and an element that needs to be addressed. However, accountability is not addressed in existing laws, which were perceived by the leaders to present problematic uncertainties in terms of responsibilities. They raised concerns about where responsibilities lie in relation to decisions made by AI algorithms, such as when an AI algorithm run in one part of the system identifies actions that should be taken in another part of the system. For example, if a patient is given AI-based advice from a county council-operated patient portal for triaging suggesting self-care, and the advice instead should have been to visit the emergency department, who has the responsibility, is it the AI system itself, the developers of the system or the county council. Additionally, concerns were raised about accountability, if it turns out that the advice was not accurate.

The issue of accountability is a very difficult one. If I agree with what doctor John (AI systems) recommended, where does the burden of proof lie? I may have looked at this advice and thought that it worked quite well. I chose to follow this advice, but can I blame Doctor John? The legislation is a risk that we have to deal with. Leader 7.

Concerns were raised as to how errors would be handled when AI systems contributed to decision making, highlighting the need for clear laws and policies. The leaders emphasized that, if healthcare professionals made erroneous decisions based on AI systems, they could be reported to the Patients Advisory Committee or have their medical license revoked. This impending threat could lead to a stressful situation for healthcare professionals. The leaders expressed major concerns about whether AI systems would be support systems for healthcare professionals’ decisions or systems that could take automated and independent decisions. They believed based on the latter interpretation that there would be a need for changes in the laws before they could be implemented in practice. Nevertheless, some leaders anticipated a development where some aspects of care could be provided without any human involvement.

If the legislation is changed so that the management information can be automated, that is to say that they start acting themselves, but they’re not allowed to do that yet. It could, however, be so that you open an app in a few years’ time, then you furnish the app with the information that it needs about your health status. Then the app can write a prescription for medication for you, because it has all the information that is needed. That is not allowed at present, because the judicial authority still need an individual to blame when something goes wrong. But even that aspect will be gradually developed. Leader 2.

According to the leaders, legislation and policies also constituted obstacles to the foundation in the implementation of AI systems in healthcare: collecting, using, merging, and analyzing patient information. The limited opportunities to legally access and share information about patients within and between organizations were described as a crucial obstacle in implementing and using AI systems. Another issue was the legal problems when a care provider wanted to merge information about patients from different providers, such as the county council and a municipality. For this to take place, it was perceived that a considerable change of the laws regulating the possibilities of sharing information across different care providers would be required. Additionally, there are challenges in the definition of personal data in laws regulating personal integrity and in the risk of individuals being identified when the data is used for computerized advanced analytics. The law states that it is not legal to share personal data, but the boundaries of what is constituted by personal data in today’s society are changing, due to the increasing amounts of data and opportunities for complex and intelligent analysis.

You are not allowed to share any personal information. No, we understand that but what is personal information and when is personal information no longer personal information? Because legally speaking it is definitely not just the case of removing the personal identity number and the name, as a computer can still identify who you are at an individual level. When can it not do that? Leader 2.

Thus, according to the healthcare leaders, laws and regulations presented challenges for an organization that want to implement AI systems in healthcare practice, as laws and regulations have different purposes and oppose each other, e.g., the Health and Medical Services Act, the Patient Act and the Secrecy Act. Leaders described how outdated laws and regulations are handled in healthcare practice, by stretching current regulations and attempts to contribute to changing laws . They aimed to not give up on visions and ideas, but to try to find gaps in existing laws and to use rather than break the laws. When possible, another way to approach this was to try to influence decision-makers on the national political level to change the laws. The leaders reported that civil servants and politicians in the county council do this lobbying work in different contexts, such as the parliament or the Swedish Association of Local Authorities and Regions (SALAR).

We discuss this regularly with our members of parliament with the aim of influencing the legislative work towards an enabling of the flow of information over boundaries. It’s all a bit old-fashioned. Leader 16.

Complying with standards and quality requirements

The healthcare leaders believed it could be challenging to follow standardized care processes when AI systems are implemented in healthcare. Standardized care processes are an essential feature that has contributed to development and improved quality in Swedish healthcare. However, some leaders expressed that the implementation of AI systems could be problematic because of uncertainties regarding when an AI algorithm is valid enough to be a part of a standardized care process. They were uncertain about which guarantees would be required for a product or service before it would be considered “good enough” and safe to use in routine care. An important legal aspect for AI implementation is the updated EU regulation for medical devices (MDR) that came into force in May 2021. According to one of the leaders, this regulation could be problematic for small innovative companies, as they are not used to these demands and will not always have the resources needed to live up to the requirements. Therefore, the leaders perceived that the county council should support AI companies to navigate these demands, if they are to succeed in bringing their products or services to implementation in standardized care processes.

We have to probably help the narrow, supersmart and valuable ideas to be realized, so that there won’t be a cemetery of ideas with things that could have been good for our patients, if only the companies had been given the conditions and support to live up to the demands that the healthcare services have and must have in terms of quality and security. Leader 2.

Integrating AI-relevant learning in higher education for healthcare staff

The healthcare leaders described that changes needed to be made in professional training, so that new healthcare professionals would be prepared to use digital technology in their practical work. Some leaders were worried that basic level education for healthcare professionals, such as physicians, nurses, and assistant nurses has too little focus on digital technology in general, and AI systems in particular. They stated that it is crucial that these educational programs are restructured and adapted to prepare students for the ongoing digitalization of the healthcare sector. Otherwise, recently graduated healthcare professionals will not be ready to take part in utilizing and implementing new AI systems in practice.

I am fundamentally quite concerned that our education, mainly when it comes to the healthcare services. Both for doctors and nurses and also assistant nurses for that matter. That it isn’t sufficiently proactive and prepare those who educate themselves for what will come in the future. // I can feel a certain concern for the fact that our educations do not actually sufficiently prepare our future co-workers for what everybody is talking now about that will take place in the healthcare services. Leader 15.

Capacity for strategic change management

Developing a systematic approach to ai implementation.

The healthcare leaders described that there is a need for a systematic approach and shared plans and strategies at the county council level, in order to meet the challenge of implementing AI systems in practice. They recognized that it will not be successful if the change is built on individual interests, instead of organizational perspectives. According to the leaders, the county council has focused on building the technical infrastructure that enables the use of AI algorithms. The county council have tried to establish a way of working with multi-professional teams around each application area for AI-based analysis. However, the leaders expressed that it is necessary to look beyond the technology development and plan for the implementation at a much earlier stage in the development process. They believed that their organization generally underestimated the challenges of implementation in practice. Therefore, the leaders believed that it was essential that the politicians and the highest leadership in the county council both support and prioritize the change process. This requires an infrastructure for strategic change management together with clear leadership that has the mandate and the power to prioritize and support both development of AI systems and implementation in practice. This is critical for strategic change to be successful.

If the County Council management does not believe in this, then nothing will come of it either, the County Council management have to indicate in some way that this is a prioritized issue. It is this we are going to work with, then it’s not sufficient for a single executive director who pursues this and who thinks it’s interesting. It has to start at the top and then filter right through, but then the politicians have to also believe in this and think that it’s important. Leader 4.

Additionally, the healthcare leaders experienced that there was increasing interest among unit managers within the organization in using data for AI-based analysis and that there might be a need to make more prioritizations of requests for data analysis in the future. The leaders expressed that it would not be enough to simply have a shared core facility supporting this. Instead, management at all levels should also be involved and active in prioritization, based on their needs. They also perceived that the implementation of AI systems will demand skilled and structured change management that can prioritize and that is open to new types of leadership and decision-making processes. Support for innovative work will be needed, but also caution so that change does not proceed too quickly and is sufficiently anchored among the staff. The implementation of AI systems in healthcare was anticipated to challenge old routines and replace them with new ones, and that, as a result, would meet resistance from the staff. Therefore, a prepared plan at the county council level was perceived to be required for the purpose of “anchoring” with managers at the unit level, so that the overall strategy would be aligned with the needs and views of those who would have to implement it and supported by the knowledge needed to lead the implementation work.

It’s in the process of establishing legitimacy that we have often erred, where we’ve made mistakes and mistakes and mistakes all the time, I’ve said. That we’re not at the right level to make the decisions and that we don’t follow up and see that they understand what it’s about and take it in. It’s from the lowest manager to the middle manager to executive directors to politicians, the decisions have to have been gained legitimacy otherwise we’ll not get the impetus. Leader 21.

The leaders believed that it was essential to consider how to evaluate different parts of the implementation process. They expressed that method development is required within the county council, because, at the moment, there is a lack of knowledge and guidelines on how to evidence-base the use of AI systems in practice. There will be a need for a support organization spanning different levels within the county council, to guide and supervise units in the systematic evaluation of AI implementations. There will also be a need for quantitative evaluation of the clinical and organizational effects and qualitative assessment that focuses on how healthcare professionals and patients experience the implementation. Additionally, validation and evaluation of AI algorithms will be needed, both before they can be used in routine care, and afterwards, to provide evidence of quality improvements and optimizations of resources.

I believe that one needs to get an approval in some way, perhaps not from the Swedish Medical Products Agency, but the AI Agency or something similar. I don’t know. The Swedish National Board of Health and Welfare or some agency needs to go in and check that it is a sufficiently good foundation that they have based this algorithm on. So that it can be approved for clinical use. Leader 10.

Furthermore, the leaders described a challenge around how the implementation of AI systems in practice could be sustainable and last over time. They expressed that the county council should develop strategies in the organization so that they are readied for sustainability and long-term implementation. At the same time, this is an area with fast development and high uncertainty about the future, and thus what AI systems and services will look like in five or ten years, and how healthcare professionals and patients will use them. This is a challenge and requires that both leaders and staff are prepared to adjust and change their ways of working during the implementation process, including continuous improvements and uptake, updating and evolution of technologies and work practices.

The rate of change where digitalization, technology, new technology and AI is concerned is so high and the rate of implementation is low, so this will entail that as soon as we are about to implement something then there is something else in the market that is better. So I think it’s important to dare to implement something that is a little further on in the future. Leader 13.

Ascertaining resources for AI implementation

The leaders emphasized the importance of training for implementation of AI systems in healthcare. The county council should provide customized training at the workplace and extra knowledge support for certain professions. This could result in difficult decisions regarding what and whom to prioritize. The leaders discussed whether there was a need to provide all staff with basic training on AI systems or if it would be enough to train some of them, such as quality developers, and provide targeted training for some healthcare professionals who are close to the implementation of the AI system at a care unit. Furthermore, the leaders described that the training had to be connected to implementing the AI system at a specific care unit, which could present a challenge for the planning and realization. They emphasized that it could be a waste of resources to educate the staff beforehand. They need to be educated in close connection to the implementation of a specific AI system in their workplace, which thus demands organizational resources and planning.

I think that we often make the mistake of educating first, and then you have to use it. But you have been educated, so now you should know this? Yes, but it is not until we use something that the questions arise. Leader 13.

There could also be a need for patient education and patient guidance, if they are to use AI systems for self-care or remote monitoring. Thus, it is vital to give all citizens the same opportunities to access and utilize new technical solutions in healthcare.

We treat all our patients equally now, everyone will receive the same invitation, and everyone will need to ring about their appointment, although 99% could really book and do this themselves. Then we should focus on that, and thus return the impetus and the power to the patient and the population for them to take care of this themselves to a greater extent. But then of course information is needed and that in turn needs intuitive systems. That is not something we are known for. Leader 14.

Many of the healthcare leaders found financial resources and time, especially the prioritization of time, to be critical to the implementation process of AI system. There is already time pressure in many care units, and it can be challenging to set aside time and other resources for the implementation.

Involving staff throughout the implementation process of AI systems

The healthcare leaders stated that anchoring and involving staff and citizens is crucial to the successfully implementation of AI systems. The management has to be responsible for the implementation process but also ensure that the staff are aware of and interested in the implementation, based on their needs. Involvement of the staff together with representatives from patient groups was considered key to successful implementation and to limit risks of perceiving the AI system as unnecessary and erroneously used. At the same time, the leaders described that it would be important for unit managers to “stand up” for the change that is required, if their staff questioned the implementation.

I think for example that if you’re going to make a successful implementation then you have to perhaps involve the co-workers. You can’t involve all of them, but a representative sample of co-workers and patients and the population who are part of it. // We mess it up time after time, and something comes that we have to implement with short notice. So we try to force it on the organization, so we forget that we need to get the support of the co-workers. Leader 4.

The propensity for change differs both among individuals and within the organization. According to the leaders, that could pose a challenge, since the support and needs differ between individuals. The motivational aspect could also vary between different actors, and some leaders claim that it is crucial to arouse curiosity among healthcare professionals. If the leaders are not motivated and do not believe that the change benefits them, implementation will not be successful. To increase healthcare professionals’ motivation and engagement, the value that will be created for the clinicians has to be made obvious, along with whether the AI system will support them in their daily work.

It has to be beneficial for the clinics otherwise it’s meaningless so to speak. A big risk with AI is that you work and work with data and then algorithms emerge that are sort of obvious. Everyone can do this. It’s why it’s important to have clinical staff in the small agile teams, that there really is a clinical benefit, this actually improves it. Leader 10.

Developing new strategies for internal and external collaboration

The healthcare leaders believed that there was a need for new forms of collaboration and communication within the county council, at both organizational and professional levels. Professionals need to interact with professions other than their own, thus enabling new teamwork and new knowledge. The challenge is for different groups to talk to each other, since they do not always have the same professional language. However, it was perceived that, when these kinds of team collaborations are successful, there will be benefits, such as automation of care processes that are currently handled by humans.

To be successful in getting a person with expert knowledge in computer science to talk to a person with expert knowledge in integrity legislation, to a one who has expert knowledge in the clinical care of a patient. Even if all of them go to work with exactly the same objective, that one person or a few people can live a bit longer or feel a bit better, then it’s difficult to talk with each other because they use essentially different languages. They don’t know much about what knowledge the other has, so just getting that altogether. Leader 2.

Leaders’ views the implementation of AI systems would require the involvement and collaboration of several departments in the county council across organizational boundaries, and with external actors. A perceived challenge was that half of the primary care units are owned by private care providers, where the county council has limited jurisdiction, which challenges the dissemination of common ways of working. Additionally, the organization in the county council and its boundaries might have to be reviewed to enable different professions to work together and interact on an everyday basis.

The complexity in terms of for example apps is very, very, very much greater, we see that now. Besides there being this app, so perhaps the procurement department must be involved, the systems administration must definitely be involved, the knowledge department must be involved and the digitalization department, there are so many and the finance department of course and the communication department, the system is thus so complex. Leader 9.

There was also consensus among the healthcare leaders that the county council should collaborate with companies in AI systems implementation and should not handle such processes on their own. An eco-system of actors working in AI systems implementation is required, who have shared goals for the joint work. The leaders expressed that companies must be supported and invited to collaborate within the county council’s organization at an early stage. In that way, pitfalls regarding legal or technical aspects can be discovered early in product development. Similar relations and dialogues are also needed with patients to succeed with implementation that is not primarily based on technical possibilities, but patients’ needs. Transparency is essential to patients’ awareness of AI systems’ functions and for the reliability in outcomes.

This is born out of a management philosophy, which is based on the principle of not being able to command everything oneself, one has to be humble, perceptive about not being able to do it. One needs to invite others to be there and help with the solution. Leader 16.

Transformation of healthcare professions and healthcare practices

Managing new roles in care processes.

The healthcare leaders described a need for new professions and professional roles in healthcare for AI systems implementation. All professional groups in today’s healthcare sector were expected to be affected by these changes, particularly the work unit managers responsible for daily work processes and the physicians accountable for the medical decisions. The leaders argued that the changes could challenge traditions, hierarchies, conventional professional roles and division of labour. There might be changes regarding the responsibilities for specific work tasks, changes in professional roles, a need for new professions that do not exist in today’s labour market and the AI systems might replace some work tasks and even professions. A change towards more combined positions at both the county council and a company or a university might also be a result of the development and implementation of AI systems. However, the leaders perceived that, for some healthcare professionals, these ideas are unthinkable, and it may take several years before these changes in roles and care processes become a reality in the healthcare sector.

I think I will be seeing other professions in the healthcare services who have perhaps not received a healthcare education. It will be a culture shock, I think. It also concerns that you may perhaps not need to be medically trained, for sitting there and checking those yellow flags or whatever they are, or it could perhaps be another type of professional group. I think that it would actually be good. We have to start economizing with the competencies we now have and it’s difficult enough to manage. Leader 15.

The acceptance of the AI systems may vary within and between professional groups, ages, and areas of specialized care. The leaders feared that the implementation of AI systems would change physicians’ knowledge base and that there would be a loss of knowledge that could be problematic in the long run. The leaders argued that younger, more recently graduated physicians would never be able to accumulate the experience-based knowledge to the extent that their older colleagues have done, as they will rely more on AI systems to support their decisions. Thus, on one hand, professional roles and self-images might be threatened when output from the AI systems is argued to be more valid than the recommendation by an experienced physician. However, on the other hand, physicians who do not “work with their hands” can utilize such output as decision support to complement their experience-based knowledge. Thus, it is important that healthcare professionals have trust in recommendations from the AI systems in clinical practice. If some healthcare professionals do not trust the AI systems and their output, there is a risk that they will not use them in clinical practice and continue to work in the way they are used to, resulting in two parallel systems. This might be problematic, both for the work environment and the healthcare professionals’ wellbeing. The leaders emphasized that this would represent a challenge for the implementation of AI systems in healthcare.

We can’t add anything more today without taking something else away, I’d say it was impossible. // The level of burden is so high today so it’s difficult to see, it’s not sufficient to say that this will be of use to us in two years’ time. Leader 20.

Implementing AI systems can change existing care processes and change the role of the patient. The leaders described that, in primary care, AI systems have the best potential to change existing work processes and make care more efficient, for example through an automatic AI-based triage for patients. The AI system could take the anamnesis, instead of the healthcare professionals, and do this when patients still are at home, so the healthcare professionals will not meet the patient unless the AI system has decided that it is necessary. The AI system can also autonomously discover something in a patient’s health status and suggest that the patient contact healthcare staff for follow-up. This use of AI systems could open up opportunities for more proactive and personalized care.

The leaders also described that the implementation of AI systems in practice could facilitate an altered patient role. The development that is taking place in the healthcare sector with, for instance, patient-reported data, enables and, in some cases, requires an active and committed patient that takes part in his or her care process. The leaders mentioned that there might be a need for patient support. Otherwise, there might be a risk that only patients with high digital literacy would be able to participate with valid data. The leaders described that AI systems could facilitate this development, by recommending self-care advice to patients or empowering them to make decisions. Still, there were concerns that not all patients would benefit from AI systems, due to variations in patients’ capabilities and literacy.

We also deal with people who are ill, we must also have respect for that. Everyone will not be able to use these tools. Leader 7.

Building trust for AI systems acceptance in clinical practice

A challenge and prerequisite for implementing AI systems in healthcare is that the technology meets expectations on quality to support the healthcare professionals in their practical work, such as having a solid evidence base, being thoroughly validated and meeting requirements for equality. It is important to have confidence in the validity of the data, the algorithms and their output. A key challenge pointed out was the need to have a sufficiently large population base, the “right” type of data and the right populations to build valid AI systems. For common conditions, where rich data exists to base AI algorithms, leaders believed the reliability would be high. For unusual conditions, there were concerns that there would be lower accuracy. Questions were also raised about how AI systems take aspects around equity and equality into account, such as gender and ethnicity. The leaders expressed concern that, due to these obstacles, in relation to certain unusual or complex conditions AI systems might not be suitable.

Then there is a challenge with the new technology, whether it’s Ok to apply it. Because it’s people who are affected, people’s health and lives that are affected by the new technology. How can we guarantee that it delivers what it says it will deliver? It must be safe and reviewed, validated and evidence-based in order for us to be able to use it. If a bug is built in then the consequences can be enormous. Leader 2.

Lack of confidence in the reliability of AI systems was also described and will place higher demands and requirements on their accuracy than on similar assessments made by humans. Thus, acceptance depends on confidence in AI systems as highly sensitive and that they can diagnose conditions at earlier stages than skilled healthcare professionals. The leaders perceived that the “black box” needs to be understood in order to be reliable, i.e. what the AI algorithms calculations are based on. Thus, reliance on the outputs from AI algorithms depends on reliance on the algorithm itself and the data used for its calculation.

There are a number of inherent problems with AI. It’s a little black box. AI looks at all the data. AI is not often easy to explain, “oh, you’ve got a risk, that it passed the cut-off value for that person or patient”, no because it weighs up perhaps a hundred different dimensions in a mathematical model. AI models are often called a black box and there have been many attempts at opening that box. The clinics are a bit skeptical then when they are not able to, they just get a risk score, I would say. Leader 10.

Big data sets are important for quality, but the leaders stated that too much information about a patient also could be problematic. There is a risk that information about a patient is available to healthcare professionals who should not have that information. The leaders believed that this could already be a problem today, but that it would be an increased risk in the future. This challenge needs to be handled as the amount of patient information increases, and as more healthcare professionals get access to such information when it’s being used in AI systems, regardless of the reason for the patient’s contact with the healthcare unit. Another challenge and prerequisite for implementing AI systems in healthcare is that the technology is user-friendly and create value for both healthcare professionals and patients. The leaders expected AI systems to be user-friendly, self-instructing, and easy to use, without requiring too much prior knowledge or training. In addition to being easy to use, the AI systems must also be time-saving and never time-consuming or dependent on the addition of yet more digital operative systems to work with. Using AI systems should, in some cases, be equated with having a second opinion from a colleague, when it comes to simplicity and time consumption.

An easy way to receive this support is needed. One needs to ask a number of questions in order to receive the correct information. But it mustn’t be too complicated, and it mustn’t take time, then nothing will come of it. Leader 4.

The leaders expected that AI systems would place the patients in focus and thereby contribute to more person-centred care. These expectations are based on a large amount of data on which AI algorithms are built, which leaders perceive will make it possible to individualize assessments and treatment options. AI systems would enable more person-centred and value-creating care for patients. AI systems could potentially contribute to making healthcare efficient without compromising quality. It was seen as an opportunity to meet future increasing needs for care among the citizens, combined with a reduced number of healthcare professionals. Smart and efficient AI systems used in investigations, assessments, and treatments can streamline care and allow more patients to receive care. Making healthcare efficient was also about the idea that AI systems should contribute to improved communication within and between caregivers for both public and private care. Using AI systems to follow up the given care and to evaluate the quality of care with other caregivers was highlighted, along with the risk that the increased efficiency provided by AI systems could result in a loss of essential values for healthcare and in impaired care.

I think that automatization via AI would be a safe way and it would be perfect for the primary care services. It would have entailed that we have more hands, that we can meet the patients who need to be met and that we can meet more often and for longer periods and perhaps do more house calls and just be there where we are needed a little more and help these a bit more easily. Leader 13.

The perspectives of the challenges described by leaders in the present study are an important contribution to improving knowledge regarding the determinants influencing the implementation of AI systems in healthcare. Our results showed that healthcare leaders perceived challenges to AI implementation concerning the handling of conditions external to the healthcare system, the building of internal capacity for strategic change management and the transformation of professional roles and practices. While implementation science has advanced the knowledge concerning determinants for successful implementation of digital technology in healthcare [ 53 ], our study is one of the few that have investigated leaders’ perceptions of the implementation of AI systems in healthcare. Our findings demonstrate that the leaders concerns do not lie so much with the specific technological nuances of AI, but with the more general factors relating to how such AI systems can be channeled into routine service organization, regulation and practice delivery. These findings demonstrate the breadth of concerns that leaders perceive are important for the successful application of AI systems and therefore suggest areas for further advancements in research and practice. However, the findings also demonstrate a potential risk that, even in a county council where there is a high level of investment and strategic support for AI systems, there is a lack of technical expertise and awareness of AI specific challenges that might be encountered. This could cause challenges to the collaboration between the developers of AI systems and healthcare leaders if there is a cognitive dissonance about the nature and scope of the problem they are seeking to address, and the practical and technical details of both AI systems and healthcare operational issues [ 7 ]. This suggests the need for people who are conversant in languages of both stakeholder groups maybe necessary to facilitate communication and collaboration across professional boundaries [ 54 ]. Importantly, these findings demonstrate that addressing the technological challenges of AI alone is unlikely to be sufficient to support their adoption into healthcare services, and AI developers are likely to need to collaborate with those with expertise in healthcare implementation and improvement scientists in order to address the wider systems issues that this study has identified.

The healthcare leaders perceived challenges resulting from external conditions and circumstances, such as ambiguities in existing laws and sharing data between organizations. The external conditions highlighted in our study resonate with the outer setting in the implementation framework CFIR [ 37 ], which is described in terms of governmental and other bodies that exercise control, with the help of policies and incentives that influence readiness to implement innovations in practice. These challenges described in our study resulted in uncertainties concerning responsibilities in relation to the development and implementation of AI systems and what one was allowed to do, giving rise to legal and ethical considerations. The external conditions and circumstances were recognized by the leaders as having considerable impact on the possibility of implementing AI systems in practice although they recognized that these were beyond their direct influence. This suggests that, when it comes to the implementation of AI systems, the influence of individual leaders is largely restricted and bounded. Healthcare leaders in our study perceived that policy and regulation cannot keep up with the national interest in implementing AI systems in healthcare. Here, concerted and unified national authority initiatives are required according to the leaders. Despite the fact that the introduction of AI systems in healthcare appears to be inevitable, the consideration of existing regulatory and ethical mechanisms appears to be slow [ 16 , 18 ]. Additionally, another challenge attributable to the setting was the lack of to increase the competence and expertise among professionals in AI systems, which could be a potential barrier to the implementation of AI in practice. The leaders reflected on the need for future higher education programs to provide healthcare professionals with better knowledge of AI systems and its use in practice. Although digital literacy is described as important for healthcare professionals [ 55 , 56 ], higher education faces many challenges in meeting emerging requirements and demands of society and healthcare.

The healthcare leaders addressed the fact that the healthcare system’s internal capacity for strategic change management is a hugh challenge, but at the same time of great importance for successful and sustainable implementation of AI systems in the county council. The leaders highlighted the need to create an infrastructure and joint venture, with common structures and processes for the promotion of the capability to work with implementation strategies of AI systems at a regional level. This was needed to obtain a lasting improvement throughout the organization and to meet organizational goals, objectives, and missions. Thus, this highlights that the implementation of change within an organization is a complex process that does not solely depend on individual healthcare professionals’ change responses [ 57 ]. We need to focus on factors such as organisational capacity, climate, culture and leadership, which are common factors within the “inner context” in CFIR [ 37 ]. The capacity to put the innovations into practice consists of activities related to maintaining a functioning organization and delivery system [ 58 ]. Implementation research has most often focused on implementation of various individual, evidence-based practices, typically (digitally) health interventions [ 59 ]. However, AI implementation represents a more substantial and more disruptive form of change than typically involved in implementing new practices in healthcare [ 60 ]. Although there are likely many similarities between AI systems and other new digital technologies implemented in healthcare, there may also be important differences. For example, our results and other AI research has acknowledged that the lack of transparency (i.e. the “black box” problem) might yield resistance to some AI systems [ 61 ]. This problem is probably less apparent when implementing various evidence-based practices based on empirical research conducted according to well-established principles to be trustworthy [ 62 ]. Ethical and trust issues were also highlighted in our study as playing a more prominent role in AI implementation, perhaps more prominently than in “traditional” implementation of evidence-based practices. There might thus be AI-specific characteristics that are not really part of existing frameworks and models currently used in implementation science.

Transformation of healthcare professions and healthcare practice

The healthcare leaders perceived that the use of AI in practice could transform professional roles and practices and this could be an implementation challenge. They reflected on how the implementation of AI systems would potentially impact provider-patient relationships and how the shifts in professional roles and responsibilities in the service system could potentially lead to changes in clinical processes of care. The leaders’ concerns related to the compatibility of new ways of working with existing practice, which is an important innovation characteristic highlighted in the Diffusion of Innovation theory [ 63 ]. According to the theory, compatibility with existing values and past experiences facilitates implementation. The leaders in our study also argued that it was important to see the value of AI systems for both professionals and service-users. Unless the benefits of using AI systems are observable healthcare professionals will be reluctant to drive the implementation forward. The importance of observability for adoption of innovations is also addressed in the Diffusion of Innovation theory [ 63 ], being the degree to which the results of an innovation are visible to the users. The leaders in our study conveyed the importance for healthcare professionals of having trust and confidence in the use of AI systems. They discussed uncertainties regarding accountability and liability in situations where AI systems impacts directly or indirectly on human healthcare, and how ambiguity and uncertainty about AI systems could lead to healthcare workers having a lack of trust in the technology. Trust in relation to AI systems is well reflected on as a challenge in research in healthcare [ 30 , 41 , 64 , 65 , 66 ]. The leaders also perceived that the expectations of patient-centeredness and usability (efficacy and usefulness) for service users could be a potential challenge in connection with AI implementation. Their concerns are echoed in a review by Buchanan et al. [ 67 ], in which it was observed that the use of AI systems could serve to weaken the person-centred relationships between healthcare professionals and patients.

In summary, the expectations for AI in healthcare are high in society and the technological impetus is strong. A lack of “translation” of the technology is in some ways part of the initial difficulties of implementing AI, because implementation strategies still need to be developed that might facilitate testing and clinical use of AI to demonstrate its value in regular healthcare practice. Our results relate well to the implementation science literature, identifying implementation challenges attributable to both external and internal conditions and circumstances [ 37 , 68 , 69 ] and the characteristics of the innovation [ 37 , 63 ]. However, the leaders in our study also pointed out the importance of establishing an infrastructure and common strategies for change management on the system level in healthcare. Thus, introducing AI systems and the required changes in healthcare practice should not only be dependent on early adopters at the particular units. This resonates with the Theory of Organizational Readiness for Change [ 70 ], which emphasizes the importance of an organization being both willing and able to implement an innovation [ 71 ]. The theory posits that, although organizational willingness is one of the factors that may facilitate the introduction of an innovation into practice, both the organization’s general capacities and its innovation-specific capacities for adoption and sustained use of an innovation are key to all phases in the implementation process [ 71 ].

Methodological considerations

In qualitative research, the concepts credibility, dependability, and transferability are used to describe different aspects of trustworthiness [ 72 ]. Credibility was strengthened by the purposeful sample of participants with various experiences and a crucial role in any implementation process. It is considered of great relevance to investigate the challenges that leaders in the county council expressed concerning the implementation of various AI systems in healthcare, albeit the preparation for implementing AI systems is a current issue in many Swedish county councils. Furthermore, the research team members’ familiarity with the methodology, together with their complementary knowledge and backgrounds enabled a more nuanced and profound, in-depth analysis of the empirical material and was another strength of the study.

Dependability was strengthened by using an interview guide to ensure that the same opening questions were put to all participants and that they were encouraged to talk openly. Because this study took place during the COVID-19 pandemic, the interviews were performed either at a distance, using the Microsoft Teams application, or face-to-face, the variation might be a limitation. However, according to Archibald et al. [ 73 ], distance interviewing with videoconferencing services, such as Microsoft Teams, could be beneficial and even preferred. Based on the knowledge gap regarding implementation of AI systems in healthcare, the authors chose to use an inductive qualitative approach to the exploration of healthcare leaders’ perceptions of implementation challenges. It might be that the implementation of AI systems largely aligns with the implementation of other digital technologies or techniques in healthcare. A strength of our study is that it focuses on perceptions on AI systems in general regardless of the type of AI algorithm or the context or area of application. However, one potential limitation of this approach is the possibility that more specific AI systems and or areas of applications may become associated with somewhat different challenges. Further studies specifying such boundaries will provide more specific answers but will probably also require the investigation be conducted in connection with the actual implementation of a specific AI systems and based on participants' experiences of having participated in the implementation process. With this in mind, we encourage future research to take this into account when deciding upon study designs.

Transferability was strengthened by a rich presentation of the results along with appropriate quotations. However, a limitation could be that all healthcare leaders work in the same county council, so transferability to other county councils must be considered with caution. In addition, an important contextual factor that might have an impact on whether, and how, the findings observed in this study will occur in other settings as well, concerns the nature of, and approach to, AI implementation. AI could be considered a rather broad concept, and while we adopted a broad and general approach to AI systems in order to understand healthcare leader’s perceptions, we would, perhaps, expect that more specific AI systems and or areas of applications become associated with different challenges. Taken together, these are aspects that may affect the possibilities for our results to be portable or transferred to other contexts. We thus suggest that the perceptions of healthcare leaders in other empirical contexts and the involvement of both more specific and broader AI systems are utilized in the study designs of future research.

In conclusion, the healthcare leaders highlighted several implementation challenges in relation to AI within the healthcare system and beyond the healthcare organization. The challenges comprised conditions external to the healthcare system, internal capacity for strategic change management, and transformation of healthcare professions and healthcare practice. Based on our findings, there is a need to see the implementation of AI system in healthcare as a changing learning process at all organizational levels, necessitating a healthcare system that applies more nuanced systems thinking. It is crucial to involve and collaborate with stakeholders and users inside the regional healthcare system itself and other actors outside the organization in order to succeed in developing and applying system thinking on implementation of AI. Given that the preparation for implementing AI systems is a current and shared issue in many (Swedish) county councils and other countries, and that our study is limited to one specific county council context, we encourage future studies in other contexts, in order to corroborate the findings.

Availability of data and materials

Empirical material generated and/or analyzed during the current study are not publicly available, but are available from the corresponding author on reasonable request.

Buch VH, Ahmed I, Maruthappu M. Artificial intelligence in medicine: current trends and future possibilities. Br J Gen Pract. 2018;68(668):143–4.

Article   PubMed   PubMed Central   Google Scholar  

Mehta N, Pandit A, Shukla S. Transforming healthcare with big data analytics and artificial intelligence: A systematic mapping study. J Biomed Inform. 2019;100: 103311.

Article   PubMed   Google Scholar  

Horgan D, Romao M, Morré SA, Kalra D. Artificial Intelligence: Power for Civilisation - and for Better Healthcare. Public Health Genomics. 2019;22(5–6):145–61.

Union E. A definition of AI: Main capabilities and scientific disciplines. https://ec.europa.eu/futurium/en/system/files/ged/ai_hleg_definition_of_ai_18_december_1.pdf ; 2018.

Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J. 2019;6(2):94–8.

Shaw J, Rudzicz F, Jamieson T, Goldfarb A. Artificial Intelligence and the Implementation Challenge. J Med Internet Res. 2019;21(7):e13659.

Elish MC. The Stakes of Uncertainty: Developing and Integrating Machine Learning in Clinical Care. Ethnographic Praxis in Industry Conference Proceedings. 2018;2018(1):364–80.

Article   Google Scholar  

Lee JC. The perils of artificial intelligence in healthcare: Disease diagnosis and treatment. J Comput Biol Bioinformatics Res. 2019;9(1):1–6.

Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56.

Article   CAS   PubMed   Google Scholar  

Technology EIoIa. Transforming healthcare with AI. https://eithealth.eu/wp-content/uploads/2020/03/EIT-Health-and-McKinsey_Transforming-Healthcare-with-AI.pdf ; 2020.

Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230–43.

De Nigris S, Craglia M, Nepelski D, Hradec J, Gomez-Gonzales E, Gomez Gutierrez E, et al. AI Watch : AI Uptake in Health and Healthcare. 2020.

He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. 2019;25(1):30–6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alhashmi S, Alshurideh M, Al Kurdi B, Salloum S. A Systematic Review of the Factors Affecting the Artificial Intelligence Implementation in the Health Care Sector. 2020. p. 37–49.

Google Scholar  

Asan O, Bayrak AE, Choudhury A. Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians. J Med Internet Res. 2020;22(6):e15154.

Gooding P, Kariotis T. Ethics and Law in Research on Algorithmic and Data-Driven Technology in Mental Health Care: Scoping Review. JMIR Ment Health. 2021;8(6):e24668.

Beil M, Proft I, van Heerden D, Sviri S, van Heerden PV. Ethical considerations about artificial intelligence for prognostication in intensive care. Intensive Care Med Exp. 2019;7(1):70.

Murphy K, Di Ruggiero E, Upshur R, Willison DJ, Malhotra N, Cai JC, et al. Artificial intelligence for good health: a scoping review of the ethics literature. BMC Med Ethics. 2021;22(1):14.

Choudhury A, Asan O. Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review. JMIR Med Inform. 2020;8(7):e18599.

Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JMC. Clinical Decision Support Systems for Triage in the Emergency Department using Intelligent Systems: a Review. Artif Intell Med. 2020;102:101762.

Yin J, Ngiam KY, Teo HH. Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review. J Med Internet Res. 2021;23(4):e25759.

Wolff J, Pauling J, Keck A, Baumbach J. The Economic Impact of Artificial Intelligence in Health Care: Systematic Review. J Med Internet Res. 2020;22(2):e16866.

Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review. J Med Internet Res. 2022;24(1):e32215.

Safi S, Thiessen T, Schmailzl KJ. Acceptance and Resistance of New Digital Technologies in Medicine: Qualitative Study. JMIR Res Protoc. 2018;7(12):e11072.

Whitelaw S, Mamas MA, Topol E, Van Spall HGC. Applications of digital technology in COVID-19 pandemic planning and response. Lancet Digit Health. 2020;2(8):e435–40.

Alami H, Lehoux P, Denis J-L, Motulsky A, Petitgand C, Savoldelli M, et al. Organizational readiness for artificial intelligence in health care: insights for decision-making and practice. J Health Organ Manag. 2021;35(1):106–14.

Reed JE, Howe C, Doyle C, Bell D. Simple rules for evidence translation in complex systems: A qualitative study. BMC Med. 2018;16(1):92.

Reichenpfader U, Carlfjord S, Nilsen P. Leadership in evidence-based practice: a systematic review. Leadersh Health Serv (Bradf Engl). 2015;28(4):298–316.

Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19(1):189.

Chen M, Decary M. Artificial intelligence in healthcare: An essential guide for health leaders. Healthc Manage Forum. 2020;33(1):10–8.

Loh E. Medicine and the rise of the robots: a qualitative review of recent advances of artificial intelligence in health. BMJ Leader. 2018;2(2):59.

Ogbonna E, Harris LC. Leadership style, organizational culture and performance: empirical evidence from UK companies. Int J Human Res Manage. 2000;11(4):766–88.

Battilana J, Gilmartin M, Sengul M, Pache A-C, Alexander JA. Leadership competencies for implementing planned organizational change. Leadersh Q. 2010;21(3):422–38.

Denti L, Hemlin S. Leadership and innovation in organizations: a systematic review of factors that mediate or moderate the relationship. Int J Innov Manag. 2012;16(03):1240007.

Nilsen P. Overview of theories, models and frameworks in implementation science. In: Nilsen P, Birken, S. A., editor. Handbook on Implementation Science. Cheltenham: Edward Elgar Publishing Limited; 2020. p. 8–31.  https://www.elgaronline.com/view/edcoll/9781788975988/9781788975988.xml .

Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7(3):149–58.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JCJIS. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. 2009;4.

Blase KA, Van Dyke M, Fixsen DL, Bailey FW. Key concepts, themes, and evidence for practitioners in educational psychology. In: Handbook of implementation science for psychology in education. New York, NY, US: Cambridge University Press; 2012. p. 13–34.

Chapter   Google Scholar  

Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8(1):35.

Wangmo T, Lipps M, Kressig RW, Ienca M. Ethical concerns with the use of intelligent assistive technology: findings from a qualitative study with professional stakeholders. BMC Med Ethics. 2019;20(1):98.

Shinners L, Aggar C, Grace S, Smith S. Exploring healthcare professionals’ understanding and experiences of artificial intelligence technology use in the delivery of healthcare: An integrative review. Health Informatics J. 2020;26(2):1225–36.

Laï MC, Brian M, Mamzer MF. Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France. J Transl Med. 2020;18(1):14.

Diprose WK, Buist N, Hua N, Thurier Q, Shand G, Robinson R. Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator. J Am Med Inform Assoc. 2020;27(4):592–600.

Nelson CA, Pérez-Chada LM, Creadore A, Li SJ, Lo K, Manjaly P, et al. Patient Perspectives on the Use of Artificial Intelligence for Skin Cancer Screening: A Qualitative Study. JAMA Dermatol. 2020;156(5):501–12.

Petitgand C, Motulsky A, Denis JL, Régis C. Investigating the Barriers to Physician Adoption of an Artificial Intelligence- Based Decision Support System in Emergency Care: An Interpretative Qualitative Study. Stud Health Technol Inform. 2020;270:1001–5.

PubMed   Google Scholar  

Graneheim UH, Lindgren BM, Lundman B. Methodological challenges in qualitative content analysis: A discussion paper. Nurse Educ Today. 2017;56:29–34.

Krippendorff K. Content analysis : an introduction to its methodology. Thousand Oaks, Calif: SAGE; 2013.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Ashfaq A, Lingman M, Sant’Anna A, Nowaczyk S. Readmission prediction using deep learning on electronic health records. J Biomed Inform. 2019;97:103256.

Ashfaq A, Lönn S, Nilsson H, Eriksson JA, Kwatra J, Yasin ZM, et al. Data Resource Profile: Regional healthcare information platform in Halland. Sweden Int J Epidemiol. 2020;49(3):738–9.

Wärnestål P, Nygren J. Building an experience framework for a digital peer support service for children surviving from cancer. New York, New York, USA: Proceedings of the 12th International Conference on Interaction Design and Children Association for Computing Machinery; 2013. p. 269–72.

Reed MS, Curzon R. Stakeholder mapping for the governance of biosecurity: a literature review. J Integr Environ Sci. 2015;12(1):15–38.

Vis C, Bührmann L, Riper H, Ossebaard HC. Health technology assessment frameworks for eHealth: A systematic review. Int J Technol Assess Health Care. 2020;36(3):204–16.

Rosen MA, DiazGranados D, Dietz AS, Benishek LE, Thompson D, Pronovost PJ, et al. Teamwork in healthcare: Key discoveries enabling safer, high-quality care. Am Psychol. 2018;73(4):433–50.

Booth RG, Strudwick G, McBride S, O’Connor S, Solano López AL. How the nursing profession should adapt for a digital future. BMJ. 2021;373:n1190.

Article   PubMed Central   Google Scholar  

Foadi N, Varghese J. Digital competence - A Key Competence for Todays and Future Physicians. J Eur CME. 2022;11(1):2015200.

Nilsen P, Schildmeijer K, Ericsson C, Seing I, Birken S. Implementation of change in health care in Sweden: a qualitative study of professionals’ change responses. Implement Sci. 2019;14(1):51.

Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41(3–4):171–81.

Nilsen P, Birken S. Prologue. In: Nilsen P, Birken, S. A., editor. Handbook on Implementation Science. Cheltenham: Edward Elgar Publishing Limited; 2020. p. 1–6.  https://www.elgaronline.com/view/edcoll/9781788975988/9781788975988.xml .

Scott WR. Institutional change and healthcare organizations : from professional dominance to managed care. Chicago: University of Chicago Press; 2000.

von Eschenbach WJ. Transparency and the Black Box Problem: Why We Do Not Trust AI. Philosophy & Technology. 2021;34:1607+.

Li S-A, Jeffs L, Barwick M, Stevens B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: a systematic integrative review. Syst Rev. 2018;7(1):72.

Rogers EM. Diffusion of innovations. New York: Free Press; 1995.

Matheny ME, Whicher D, Thadaney IS. Artificial Intelligence in Health Care: A Report From the National Academy of Medicine. JAMA. 2020;323(6):509–10.

Moorman LP. Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring. Appl Clin Inform. 2021;12(4):888–96.

Lee D, Yoon SN. Application of Artificial Intelligence-Based Technologies in the Healthcare Industry: Opportunities and Challenges. Int J Environ Res Public Health. 2021;18(1):271.

Buchanan C, Howitt ML, Wilson R, Booth RG, Risling T, Bamford M. Predicted Influences of Artificial Intelligence on the Domains of Nursing: Scoping Review. JMIR Nursing. 2020;3(1):e23939.

Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57.

Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health. 2019;7:3.

Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4(1):67.

Scaccia JP, Cook BS, Lamont A, Wandersman A, Castellow J, Katz J, et al. A practical implementation science heuristic for organizational readiness: R = MC(2). J Community Psychol. 2015;43(4):484–501.

Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12.

Archibald MM, Ambagtsheer RC, Casey MG, Lawless M. Using Zoom Videoconferencing for Qualitative Data Collection: Perceptions and Experiences of Researchers and Participants. Int J Qual Methods. 2019;18:1609406919874596.

World Medical Association Declaration of Helsinki. ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4.

Article   CAS   Google Scholar  

Council SR. Good Research Practice: Swedish Research Council; 2017 [Available from: https://www.vr.se/download/18.5639980c162791bbfe697882/1555334908942/Good-Research-Practice_VR_2017.pdf .

Download references

Acknowledgements

The authors would like to thank the participants who contributed to this study with their experiences.

All authors belong to the Healthcare Improvement Research Group at Halmstad University, https://hh.se/english/research/our-research/research-at-the-school-of-health-and-welfare/healthcare-improvement.html

Open access funding provided by Halmstad University. The funders for this study are the Swedish Government Innovation Agency Vinnova (grant 2019–04526) and the Knowledge Foundation (grant 20200208 01H). The funders were not involved in any aspect of study design, collection, analysis, interpretation of data, or in the writing or publication process.

Author information

Authors and affiliations.

School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden

Lena Petersson, Ingrid Larsson, Jens M. Nygren, Per Nilsen, Margit Neher, Julie E. Reed, Daniel Tyskbo & Petra Svedberg

Department of Health, Medicine and Caring Sciences, Division of Public Health, Faculty of Health Sciences, Linköping University, Linköping, Sweden

Department of Rehabilitation, School of Health Sciences, Jönköping University, Jönköping, Sweden

Margit Neher

You can also search for this author in PubMed   Google Scholar

Contributions

LP, JMN, JR, DT and PS together identified the research question and designed the study. Applications for funding and coproduction agreements were put in place by PS and JMN. Data collection (the interviews) was carried out by LP and DT. Data analysis was performed by LP, IL, JMN, PN, MN and PS and then discussed with all authors. The manuscript was drafted by LP, IL, JMN, PN, MN and PS. JR and DT provided critical revision of the paper in terms of important intellectual content. All authors have read and approved the final submitted version.

Corresponding author

Correspondence to Lena Petersson .

Ethics declarations

Ethics approval and consent to participate.

The study conforms to the principles outlined in the Declaration of Helsinki (74) and was approved by the Swedish Ethical Review Authority (no. 2020–06246). The study fulfilled the requirements of Swedish research: information, consent, confidentiality, and safety of the participants and is guided by the ethical principles of: autonomy, beneficence, non-maleficence, and justice (75). The participants were first informed about the study by e-post and, at the same time, were asked if they wanted to participate in the study. If they agreed to participate, they were verbally informed at the beginning of the interview about the purpose and the structure of the study and that they could withdraw their consent to participate at any time. Participation was voluntary and the respondents were informed about the ethical considerations of confidentiality. Informed consent was obtained from all participants prior to the interview.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no potential conflicts of interest with respect to the research, authorship, and publication of this article.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Petersson, L., Larsson, I., Nygren, J.M. et al. Challenges to implementing artificial intelligence in healthcare: a qualitative interview study with healthcare leaders in Sweden. BMC Health Serv Res 22 , 850 (2022). https://doi.org/10.1186/s12913-022-08215-8

Download citation

Received : 04 April 2022

Accepted : 20 June 2022

Published : 01 July 2022

DOI : https://doi.org/10.1186/s12913-022-08215-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Digital transformation
• Implementation
• Healthcare leaders
• Organizational change
• Qualitative methods
• Stakeholders

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Admissions Visit Opportunities

The Charger Blog

Graduate Students, Community Health System Collaborate on Research Published in Prestigious Public Health Journal

Candidates in the Master of Public Health and Master of Healthcare Administration programs worked with Yale New Haven Health to study social determinants of health, and their findings were published in Frontiers of Public Health.

June 13, 2024

By Alisha Thapa ’24 MPH, Dawa Lhomu Sherpa ’23 MPH, Keerthi Katukuri ’24 MHA, Hiba Mohammed Jaidi ’24 MHA, Kashyap Ramadyani ’24 MHA, and Pavani Rangachari, Ph.D., CPH

Representing the Department of Population Health and Leadership in the School of Health Sciences , we had the opportunity to collaborate with Yale New Haven Health (YNHH) to publish the results of a months-long systematic review project in Frontiers in Public Health.

Entitled “ Hospital and Health System Initiatives to Address Social Determinants of Health (SDOH) in the United States: A Scoping Review of the Peer-Reviewed Literature,” the project began with a 2023 University of New Haven Summer Research Grant awarded to Pavani Rangachari, Ph.D., CPH, professor of healthcare administration and public health.

As part of the project, we collaborated with Lewis Goodrum, FACHE, associate director of regional practice operations for Yale New Haven Health. We represented two major disciplines: public health (MPH) and healthcare administration (MHA) . Our approach enabled the integration of three high-impact practices: faculty-mentored research, interdisciplinary learning, and community engagement into one project.

Working with Dr. Pavani , we were able to leverage the experience of working on a faculty-mentored research project to fulfill our curricular internship requirement in summer 2023, and all five of us as graduate students remained committed to supporting the project for nearly six months after the completion of our summer internship period.

Over the nine-month project period, many research challenges were overcome and milestones achieved, including a comprehensive search of three academic databases; article selection based on predetermined eligibility criteria and critical appraisal criteria; data collection based on research questions; data analysis; and the write-up of initial manuscript.

‘This work adds significant value’

Social Determinants of Health (SDOH) refer to “the conditions in which people are born, grow, work, live, and age, and the wider set of forces and systems shaping the conditions of daily life.” Nonprofit hospitals and health systems have historically invested little in addressing SDOH, however recent policy changes and environmental influences have accelerated healthcare’s attention to SDOH. This project makes an original and timely contribution in identifying the key characteristics of existing hospital and health system initiatives to address SDOH in the U.S. to gain insight into progress and gaps, and to identify implications for practice, policy, and research.

“I’m so excited to see the vision come to life,” added Mr. Goodrum, Associate Director for Practice Operations at YNHH. "This work adds significant value to the field.”

Below the students and their adviser reflect on their experiences.

Alisha Thapa ’24 MPH

The moment we have been waiting for is finally here. Working on this project has been very rewarding both academically and professionally. Every phase of this research has been a journey of growth and learning. It's been an honor collaborating with the team, navigating challenges, and celebrating the milestones together! I'm incredibly proud of our achievement.

Dawa Lhomu MPH ’23

As I reflect on our research journey together, I feel humbled and grateful to have had the opportunity to work on such an important topic as SDOH. This research is a big milestone and a great reference point to plan and implement specialized initiatives in the future. Thank you so much to Dr. Rangachari for your guidance and understanding throughout this journey. I look forward to creating a bigger impact together in future as well.

Keerthi Katukuri ’24 MHA

I am deeply grateful to my adviser Dr. Pavani Rangachari for granting me this exceptional internship opportunity, and I extend my thanks to the University librarians and other members who have offered their unwavering support for our research project. This journey has been transformational, providing me with the knowledge, skills, and motivation to make a meaningful impact in the field of public health.

Hiba Jaidi MHA ’24

This research is very impactful. Through our collective efforts, we have enriched the understanding of how hospitals are actively engaging with social determinants of health, which is an increasingly vital aspect of healthcare delivery. By identifying and analyzing the various interventions implemented across the United States, our research serves as a valuable resource for policymakers and healthcare professionals seeking to address health disparities at their roots. I am immensely proud to be part of a project that holds such significance in the realm of healthcare. It has been an enriching journey, honing my research skills and understanding the importance of addressing social determinants of health to create a more equitable healthcare landscape.

Kashyap Ramadyani ’24 MHA

I'm thankful for this fantastic learning opportunity and for the opportunity to work with a group of people who are passionate about changing the face of healthcare. I have improved my research and analytical abilities via this experience, and I have gained a deeper appreciation for the critical role of social determinants of health in creating a better and more just society. This effort has the potential to spark big changes in healthcare systems across the nation, and it is more than just a theoretical exercise. Healthcare professionals can execute focused interventions and better allocate resources if they have an awareness of the nuances of socioeconomic factors and how they affect health outcomes. We hope that our efforts will serve as a first step in encouraging teamwork across healthcare organizations, researchers, and communities. I feel privileged to have contributed to this outstanding initiative under the direction of Dr. Rangachari, and I eagerly anticipate seeing the great changes it can potentially bring throughout the healthcare industry and beyond.

Pavani Rangachari, Ph.D., CPH

It is delightful to see our hard work over so many months come to fruition with an impactful publication in a prestigious public health journal I am especially grateful for three opportunities provided by the project: 1) The opportunity to address a gap in research in an area of critical importance at the intersection of healthcare administration and public health, 2) the opportunity to engage graduate students and contribute to their professional development, and 3) the opportunity to engage community partners in our research. In addition to having a community partner as co-author, this project has provided an opportunity to engage with health equity leaders in the community in both disseminating our findings and identifying future avenues for collaboration. I am grateful to the community leaders who have already taken the time to meet with me in this regard.

‘Hopeful that this work will provide a foundation’

Prior to having our work published in Frontiers in Public Health , we used a multi-pronged approach to local and regional dissemination of the results, beginning with the presentation of posters at the 2024 UNewHaven Graduate Showcase and the 2024 Connecticut Public Health NextGen Workforce Showcase in New Britain, CT, in April. At the latter event, as students, we had an opportunity present the results one-on-one to Dr. Manisha Juthani, CT Public Health Commissioner, who greatly appreciated our presentation and later left a note of appreciation for us on LinkedIn.

In addition to our local and regional presentations, as part of the SHS Research Seminar Series in Spring 2024, Dr. Rangachari presented the results of the study to the broader faculty body in the School of Health Sciences, while Alisha and Keerthi served as guest speakers to present the results to a class on Healthcare Leadership (PUBH 6640). This student-led guest lecture, in turn, provided an opportunity for current MPH/MHA students to serve as guest lecturers to their peers, while lecturing on a topic directly relevant to the course theme on healthcare leadership. Moreover, since this course required students to complete a scoping review for their term paper, the presentation was timely in imparting reflections and lessons learned from our real-world (internship) experience to peers.

In addition to having our work published in an international peer-reviewed journal, we will have an opportunity to present a poster this fall at the 151st Annual Meeting of the American Public Health Association (APHA) to be held in Minneapolis in October 2024.

In addition to the opportunity to disseminate the results at the local, national, and international levels, we are hopeful that this work will provide a foundation for original future research targeted toward the reduction of healthcare disparities and promotion of health equity.

Alisha Thapa ’24 MPH and Dawa Lhomu Sherpa ’23 MPH are graduates of the University’s Master of Public Health program. Keerthi Katukuri ’24 MHA, Hiba Mohammed Jaidi ’24 MHA, and Kashyap Ramadyani ’24 MHA are graduates of the University’s Master of Healthcare Administration Program. Pavani Rangachari, Ph.D., CPH, is a professor of healthcare administration and public health.

Recent News

University of New Haven Team Races to Top Finish at Dragon Boat Regatta

A team of students and faculty (and their families) from the biology and environmental science department fittingly named Charlie’s Angels finished among the top teams at the annual dragon boat race organized by the Canal Dock Boathouse in New Haven.

Working to Crack a Cold Case Dating Back More than 400 Years

Professor Declan Hill is developing a study abroad program to take a team of University of New Haven students to England to try to find the remains of Matoaka, known to the world as Pocahontas, hoping to return her remains to her tribal home in Virginia.

• MBA/MPA Program - University of New Haven
• Arts & Sciences: Art & Design | University of...
• Make Your Gift

An official website of the Department of Health & Human Services

• Search All AHRQ Sites
• Email Updates

Advancing today's Discoveries, Transforming Tomorrow's Care

• Contact DHR

Health IT for Improved Chronic Disease Management

Chronic diseases--such as heart disease, cancer, and diabetes--are placing a growing burden on the U.S. health care system.  In response, some health care organizations are instituting chronic disease management (CDM) programs to reduce the incidence of preventable hospitalizations and adverse events by more effectively and comprehensively managing the health of patients with chronic conditions. Many of these organizations are implementing health information technology (health IT) to facilitate their chronic disease management programs.

The Agency for Healthcare Research and Quality (AHRQ) has funded a broad portfolio of research projects to foster innovation in using health IT to improve care for patients with chronic diseases.  This brief highlights early observations from 13 of these projects that incorporate the use of health IT in their programs and focus on the following implementation considerations:

• Types of health IT applications used in CDM programs.
• Implementation of technology solutions.
• Use of multidisciplinary care teams and staff.
• Health IT adoption and change management.
• Usability and system design.
• Facilitating collaboration for patients and providers.
• Sustainability of health IT efforts for chronic disease management.

More lessons from the AHRQ telehealth portfolio ( PDF , 114 KB,  HTML ) .

Types of Health IT Applications Used in CDM Programs

Lesson 1:  Grantees are using a variety of health IT applications -- and combinations of applications -- to address different aspects of CDM.

AHRQ grantees have incorporated the following technologies in their chronic disease management programs:

• Clinical decision support (CDS) systems help providers to interpret clinical results, document patients' health status, and prescribe medications through the use of alerts, reminders, and customized data entry forms.
• Health information exchanges (HIE) allow organizations to share information across organizational boundaries. Such systems enable all participating providers in a community to access patient information, thereby helping them to provide better patient care.
• Disease registries capture and track key patient information to assist care team members in proactively managing patients.
• Patient-centered applications such as patient portals, personal health records (PHRs), and integrated voice response (IVR) systems are designed to educate patients about their disease, their medications, and how they can self-manage chronic conditions such as diabetes, hypertension, or heart disease.
• Electronic health records (EHRs) with integrated decision support and chronic care management tools help providers manage patient information and monitor health outcomes for patients who are undergoing treatment for chronic diseases.  EHRs integrated with laboratory and pharmacy information systems can supply import information to support EHR CDM functions.
• Telehealth applications that remotely connect providers and patients in co-management of chronic diseases. Remote monitoring devices and electronic health records are components that extend traditional Telehealth networks to provide enhanced CDM functions for patients and providers.

Implementation of Technology Solutions

Lesson 2: Significant investments of time and resources are required to configure both off-the-shelf vendor products and internally developed technologies to meet stakeholders' needs.

An initial question that every health organization must answer in implementing projects for emerging areas of health IT-enabled care is whether to buy a commercially available product or build a customized application.  Regardless of which approach is taken, a significant investment of time and resources is required to configure IT systems to perform the functions desired by stakeholders.  A few of the AHRQ-funded CDM projects opted to purchase software or equipment off-the-shelf from commercial vendors; others developed their solutions internally.  Many of the examples below illustrate issues grantees faced when working with off-the-shelf solutions to implement their chronic disease management programs.

• There are few commercial off-the-shelf (COTS) solutions that provide comprehensive functionality to support CDM programs.  Consequently, few AHRQ grantees implemented COTS systems: only four of the thirteen AHRQ-funded CDM projects purchased COTS solutions. 
• None of the COTS technologies purchased by grantees were designed specifically for chronic care. As a result, each of the four grantees that adopted COTS systems needed to modify their systems to support their CDM requirements. 
• Of all health IT applications used by the AHRQ-funded CDM projects, clinical decision support systems required the highest degree of customization. Grantees resourced customization efforts with both IT staff and clinicians, with one grantee dedicating 50 percent of a physician's time to the effort.  The resources and expertise needed for implementing CDS systems should be carefully considered so that clinician and staff time is effectively utilized.  
• Grantees experienced discrepancies between COTS system vendor promises and delivery in both system functionality and delivery schedules.  The lack of maturity in COTS solutions in this area led one grantee organization to adopt a system in the Beta stage of development.  This necessitated that project staff members spend significant amounts of time testing and revising the system before it could be implemented.  Grantees recommend that organizations considering COTS systems balance a vendor's claims with the experiences of the vendor's other customers, and that organizations should build penalty clauses into vendor contracts.
• Projects should research the availability and cost of vendor technical support. In the grantees' experience, vendors often have technical support available only during business hours and have delayed response times and higher costs for support provided outside of business hours. A more comprehensive support agreement may be needed to ensure after-hours access. One project's vendor did not return phone calls after 5:00 p.m. or on weekends, even though the project often experienced problems at these times. Depending upon the scope of the implementation, it may be important for a project to have its own trained support staff rather than relying solely on vendor resources. 
• For projects that have focused needs and access to technical resources, open source solutions may provide a cost-effective mechanism for implementing CDM solutions.  One grantee found that the costs and capabilities for the standard technology components for an HIE exceeded the project's scope and budget.  It made this discovery after soliciting bids for a system to share clinical data relevant to chronically ill patients among community providers. Instead of purchasing a commercial solution, the project used available internal development resources to build an appropriate HIE infrastructure using open source software and information from the published literature.  
• Grantees considered their access to technical resources in deciding whether to develop solutions internally or customize vendor solutions.  One grantee had initially planned to develop a custom solution but then determined that the amount of resources required to develop and maintain the product exceeded the cost of purchasing a vendor solution.  In addition, this project had limited access to technical staff in its geographic area.

Use of Multidisciplinary Care Teams and Staff

Lesson 3: Chronic disease management health IT applications may enable the re-distribution of patient management tasks to non-physician personnel.

Many health IT solutions for chronic disease management are intended primarily for physician use. However, these systems also can be designed to engage other key members of the health care team in decision-making, such as nurses and case managers. The AHRQ-funded CDM projects are deploying health IT applications to non-physician personnel to assist in the management of patients with chronic diseases. 

• Nurse Educator: One project sought to improve the project site's performance on CMS core measures for chronic heart failure (CHF) patients by providing IT-enabled patient education. The hospital created a new staff position, a full-time nurse educator, to help coordinate care and educate patients with CHF and other conditions about self-management. The hospital information system alerts the nurse educator when a chronically ill patient is admitted. The alert prompts her to attend bedside meetings with other members of the care team and to educate the patient directly about how to perform self-care after being discharged from the hospital. 
• Nurse Case Manager: Two projects use nurse case managers to triage clinical decision support alerts for patients with chronic conditions. Instead of sending alerts and reminders to physicians, the CDS system sends the messages to nurse case managers who help sort through issues that are not an immediate priority, such as non-emergency alerts. 
• Case Manager: Another project uses case managers, employed by the State's Medicaid office, to triage CDS alerts for some patients. Specifically, the system identifies patients who miss appointments or have not had a hemoglobin A1-C (HbA1C) test in over a year, (HbA1C is a recommended marker for the effective management of diabetes over time). The system can automatically generate letters to patients from clinics and the Medicaid system, and it can easily notify providers when their patients have been hospitalized for an issue related to their chronic illness.
• Non-Clinical Assistants: An integrated delivery network uses non-clinical assistants to review incoming secure messages from patients and to forward them to the appropriate clinical staff for response. This prevents overloading of physician inboxes with questions that other providers could answer. The same assistant can monitor when providers respond to ensure that patient questions are answered in a timely manner.

Health IT Adoption and Change Management

Lesson 4: Securing user buy-in and trust is critical to the success of health IT implementations.

A review of AHRQ-funded CDM projects yielded several preliminary findings about how to incorporate adoption of IT solutions into routine clinical practice.

• Short-term health IT solutions may be put in place to fill a need while long-range system design plans are under development. One project set out to convene community stakeholders to form a regional health information exchange. Initial conversations with several smaller physician practices revealed an immediate need for improved regional referral processes. The project team decided there that the existing infrastructure provided enough overlap to support a rudimentary (but useful) referral system and a CDM system.  These systems were implemented immediately and were enthusiastically adopted by clinicians. At the same time, a master patient index (MPI) and other components of an HIE to support the CDM needs were developed and tested. The project now has a working data exchange that enables regional providers to easily refer patients and receive feedback on referral encounters. Once final agreement is reached on other aspects of clinical data exchange, the project will expand to facilitate sharing of additional forms of clinical data. 
• Another project learned that, to be successful, it is important to engage clinicians who are directly involved in the delivery of patient care in the development of practical electronic templates.  Standardized templates that have been created in a research environment or larger integrated delivery network could run the risk of being inadequate to secure clinician buy-in and adoption in other settings. 
• More features and equipment do not always translate to better care.  This is particularly true for applications developed for patient use, such as integrated voice response (IVR) and patient portals, which can become so complex that they discourage user adoption.  AHRQ projects that utilized patient-centered applications discovered that it was important to keep the user interfaces and options as simple as possible.

Usability and System Design 

Lesson 5: For both patients and providers, usability and system design are key factors driving the adoption and use of health IT systems to improve CDM.

AHRQ-funded CDM projects faced a number of usability challenges. Several projects discovered and emphasized that testing is of critical importance. To ensure the system is designed to optimize usability, a project needs to "test, test, and then test some more." Pilot testing with a subset of users enabled several projects to discover problematic issues related to workflow and system functionality. Several projects used an iterative design process to help eliminate major workflow and system issues before rolling their projects out to large groups of clinical staff. 

• Two projects learned through initial testing that the first versions of their health IT systems did not integrate well into clinical workflow.  During testing of a template designed to capture pediatric obesity information, the small group of physicians involved reported that, although they loved the template, it was hard to find within the organization's electronic health record (EHR) system. Investigators worked closely with the EHR system staff to better integrate access to the template into existing patterns of EHR use. 
• Another project discovered problems with its clinical decision support algorithm during a pilot test and returned to development to solve the problem. Testing also revealed that many of the project's remote monitoring devices did not work properly. 
• Aligning health IT projects with stakeholders' priorities is also crucial to their success.  Pilot testing and post-implementation analysis can offer insights into usability and adoption from a small subset of individuals before undertaking a larger rollout.  For pilot-testing, it is important to select a pilot group of enthusiastic and IT-ready end users who are willing to work through the early phases of implementation and provide valuable feedback.  Upon working through the "kinks" of the initial implementation during a pilot, a project also must validate that the pilot group accurately represents the majority of end users on a project from technology-savvy individuals to those less familiar with computers. 
• Health IT solutions need to be tailored for the end user to improve usability and avoid "information overload."  One project faced a huge task of incorporating information on thousands of medications into its patient-focused application. Patients at the project's renal practice took on average 11 different medications. The project narrowed the list of medications to be included in the application to those most frequently prescribed. Although the volume of information was still large, it was more manageable and required less input from end users who were often very ill and/or possessed limited computer skills. 

Facilitating Collaboration for Patients and Providers  

Lesson 6: Health IT can enable opportunities for remote patient management, patient education, and provider information-sharing for patients with chronic conditions.

The AHRQ CDM projects have used health IT to help both providers and patients access up-to-date information concerning clinical practice, medications, and treatment options. Some examples of the ways that the projects are using IT to educate patients and providers are described below.

• One project's telehealth network helps primary care physicians receive up-to-date information about clinical practices for chronic conditions. Physicians also can interact with other primary care physicians and specialists at the closest academic medical center to discuss complex cases. The group environment of the telehealth network enables physicians to learn from one another. The telehealth network also educates nurses and office managers about processes for teaching patients about self-management of their chronic illnesses. Telehealth also can be used to provide education directly to patients if providers choose to integrate this technology into their clinical workflow. 
• Technology can alert medical staff when a patient needs educational interventions. This may assist organizational efficiency, while ensuring that patients get the information they need. One project employed a clinical decision support system to notify a nurse educator when a chronically ill patient was admitted to the hospital.  The educator then scheduled time to work with the patient's care team and to educate the patient directly about self-management. 
• Another project implemented an interactive voice response (IVR) system that patients can use in their homes. Through a telephone, the IVR provides health data to a central IT system and sends feedback to the patient based on decision support logic. The computer-generated feedback helps patients better understand what changes in their health status should prompt them to seek advice or treatment from their physician. 

Sustainability of Health IT Efforts for Chronic Disease Management

Lesson 7:  To obtain additional funding from health care executives, payors, or through grants, health IT projects will need to demonstrate return on investment or alignment with potential funders' strategic goals.

The AHRQ projects received limited-term funding, and thus needed to identify mechanisms for sustaining their health IT applications. Some projects are planning to complete implementation, transitioning into an operations and maintenance mode. However, many others intend to expand their scope. While a challenge, the projects report that sustainability may be achieved when organizations and communities make CDM a top priority for the future and are able to demonstrate improved clinical management of their patients. Below are some examples of how projects plan to continue activities after the end of their AHRQ awards.

• Many payors are interested in innovative approaches to chronic disease care because of its impact on health care costs. Aligning HEDIS (Healthcare Effectiveness Data and Information Set) and CMS measures with health IT projects allowed several grants to demonstrate that health IT systems can impact these measures to improve health care quality. One project worked with its State government to develop a pilot project testing the capacity of EHR and clinical decision support systems to report key measures for their Medicaid population.  This information is transmitted to care managers and health-risk management professionals, who can then respond appropriately. The pilot project demonstrated that the system was more efficient and timely in its reporting of CMS measures than the current State reporting process. The project team is now working with the State to implement their process statewide.
• Some grantees are working with payors in their area to investigate opportunities for Health IT-based reimbursement policies and pay-for-performance (P4P) initiatives.  One project worked with physicians, the provider organizations, and local payors to reach agreement on the reimbursement process for physicians' use of secure messaging.  Another project is working with payors who are interested in exploring P4P initiatives by seeking to demonstrate how its quality dashboard for chronic diseases can help providers and payors to measure the quality of care provided to these patients.
• Improving care for chronically ill patients provides benefits not only to patients but also the community.  Several AHRQ-funded projects have achieved sustainability forchronic care initiatives by securing support from community organizations.  An HIE project secured support from public health agencies, as better CDM provides value beyond a single organization.  The same project partnered with school nurses to support asthma treatment for children and received funding from the CDC to do additional research on this chronic condition.  Community relationships take time to build, and they require energy to sustain.  However, integration of one innovative project has led to possible expansion statewide to provide greater quality to a larger population.  Many investigators spoke about the need for continued support for innovative uses of health IT for chronic care, and they advocated that these interventions can target the populations that are the sickest and the neediest and that consume the most health care resources.

Measuring the Impact of Chronic Disease Management Health IT Solutions

The AHRQ -funded projects listed below are measuring the impact of health IT on health care quality, safety, and efficiency in managing patients with chronic disease.

• Trial of Decision Support to Improve Diabetes Outcomes  (Randall Cebul; Cleveland, OH)
• Improving Pediatric Safety and Quality with Health Care IT  (Timothy Ferris; Boston, MA)
• Santa Cruz County Diabetes Mellitus Registry  (Eleanor Littman; Santa Cruz, CA)
• Statewide Implementation of Electronic Health Records  (David Bates; Boston, MA)
• Evaluating Smart Forms and Quality Dashboards in an EHR  (Blackford Middleton; Boston, MA)
• The Chronic Care Technology Planning Project  (John M. Branscombe; Presque Isle, ME)
• New Mexico Health Information Collaborative  (Maggie Gunter; Albuquerque, NM)
• Showing Health Information Value in a Community Network  (David Lobach; Durham, NC)
• Home Heart Failure Care Comparing Patient-Driven Technology Models  (Lee Goldberg; Billings, MT)
• Patient-Provider Electronic Messenger in Chronic Illness  (James D. Ralston; Seattle, WA)
• Project ECHO Extension for Community Healthcare Outcomes  (Sanjeev Arora; Albuquerque, NM)
• Telewoundcare Network  (Charles A. Bryant; Oklahoma City, OK)
• Director's Corner
• Current Priorities
• Executive Summary
• Research Spotlight
• Research Themes and Findings
• Research Dissemination
• Research Overview
• 2020 Year in Review
• 2019 Year in Review
• Engaging and Empowering Patients
• Optimizing Care Delivery for Clinicians
• Supporting Health Systems in Advancing Care Delivery
• Our Experts
• DHR 20th Anniversary Blog Series
• DHR 20th Anniversary Timeline
• Milestones and Achievements
• Search AHRQ-Funded Projects
• AHRQ-Funded Projects Map
• AHRQ Digital Healthcare Research Publications Database
• A Practical Guide for Implementing the Digital Healthcare Equity Framework
• ePROs in Clinical Care
• Guide to Integrate Patient-Generated Digital Health Data into Electronic Health Records in Ambulatory Care Settings
• Health IT Survey Compendium
• Time and Motion Studies Database
• Health Information Security and Privacy Collaboration Toolkit
• Implementation in Independent Pharmacies
• Implementation in Physician Offices
• Children's Electronic Health Record (EHR) Format
• Project Resources Archives
• Archived Tools & Resources
• National Webinars
• Funding Opportunities
• Digital Healthcare Research Home
• 2018 Year in Review Home
• Research Summary
• Research Spotlights
• 2019 Year in Review Home
• Annual Report Home

Here’s how you know

• U.S. Department of Health and Human Services
• National Institutes of Health

Complementary, Alternative, or Integrative Health: What’s In a Name?

We’ve all seen the words “complementary,” “alternative,” and “integrative,” but what do they really mean?

This fact sheet looks into these terms to help you understand them better and gives you a brief picture of the mission and role of the National Center for Complementary and Integrative Health (NCCIH) in this area of research. The terms “complementary,” “alternative,” and “integrative” are continually evolving, along with the field, but the descriptions of these terms below are how we at the National Institutes of Health currently define them.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Complementary Versus Alternative

According to a 2012 national survey, many Americans—more than 30 percent of adults and about 12 percent of children—use health care approaches that are not typically part of conventional medical care or that may have origins outside of usual Western practice. When describing these approaches, people often use “alternative” and “complementary” interchangeably, but the two terms refer to different concepts:

• If a non-mainstream approach is used  together with  conventional medicine, it’s considered “complementary.”
• If a non-mainstream approach is used  in place of  conventional medicine, it’s considered “alternative.”

Most people who use non-mainstream approaches also use conventional health care.

In addition to the terms complementary and alternative, you may also hear the term “functional medicine.” This term sometimes refers to a concept similar to integrative health (described below), but it may also refer to an approach that more closely resembles  naturopathy  (a medical system that has evolved from a combination of traditional practices and health care approaches popular in Europe during the 19th century).

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Integrative Health

Integrative health brings conventional and complementary approaches together in a coordinated way. Integrative health also emphasizes multimodal interventions, which are two or more interventions such as conventional health care approaches (like medication, physical rehabilitation, psychotherapy), and complementary health approaches (like acupuncture, yoga, and probiotics) in various combinations, with an emphasis on treating the whole person rather than, for example, one organ system. Integrative health aims for well-coordinated care among different providers and institutions by bringing conventional and complementary approaches together to care for the whole person.

The use of integrative approaches to health and wellness has grown within care settings across the United States. Researchers are currently exploring the potential benefits of integrative health in a variety of situations, including pain management for military personnel and veterans, relief of symptoms in cancer patients and survivors, and programs to promote healthy behaviors.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} What is whole person health?

Whole person health refers to helping individuals, families, communities, and populations improve and restore their health in multiple interconnected domains—biological, behavioral, social, environmental—rather than just treating disease. Research on whole person health includes expanding the understanding of the connections between these various aspects of health, including connections between organs and body systems.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Integrative Approaches for Pain Management for Military Personnel and Veterans

• An NCCIH-funded study is developing an innovative, collaborative treatment model involving chiropractors, primary care providers, and mental health providers for veterans with spine pain and related mental health conditions.
• Other NCCIH-funded studies are testing the effects of adding mindfulness meditation, self-hypnosis, or other complementary approaches to pain management programs for veterans. The goal is to help patients feel and function better and reduce their need for pain medicines that can have serious side effects.
• For more information on pain management for military personnel and veterans, see NCCIH’s  Complementary Health Practices for U.S. Military, Veterans, and Families  webpage.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Integrative Approaches for Symptom Management in Cancer Patients and Survivors

• Massage therapy may lead to short-term improvements in pain and mood in patients with advanced cancer.
• Yoga may relieve the persistent fatigue that some women experience after breast cancer treatment, according to the results of a preliminary study.
• Tai chi or qigong have shown promise for managing symptoms such as fatigue, sleep difficulty, and depression in cancer survivors.
• For more information, see  NCCIH’s fact sheet on cancer .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Integrative Approaches and Health-Related Behaviors

• Preliminary research suggests that yoga and meditation-based therapies may help smokers quit.
• In a study funded by the National Cancer Institute, complementary health practitioners (chiropractors, acupuncturists, and massage therapists) were successfully trained to provide evidence-based smoking cessation interventions to their patients.
• An NCCIH-funded study is testing whether a mindfulness-based program that involves the whole family can improve weight loss and eating behavior in adolescents who are overweight.
• For more information, see the NCCIH  Quitting Smoking  and  Weight Control  webpages.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Complementary Health Approaches

Complementary approaches can be classified by their primary therapeutic input (how the therapy is taken in or delivered), which may be:

• Nutritional (e.g., special diets, dietary supplements, herbs, and probiotics)
• Psychological (e.g., mindfulness)
• Physical (e.g., massage, spinal manipulation)
• Combinations such as psychological and physical (e.g., yoga, tai chi, acupuncture, dance or art therapies) or psychological and nutritional (e.g., mindful eating)

Nutritional approaches include what NCCIH previously categorized as natural products, whereas psychological and/or physical approaches include what was referred to as mind and body practices.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Examples of complementary health approaches that fall within the categories: Psychological, Physical, and Nutritional

Click image to enlarge

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Nutritional Approaches

These approaches include a variety of products, such as  herbs   (also known as botanicals),  vitamins and minerals , and  probiotics . They are widely marketed, readily available to consumers, and often sold as  dietary supplements .

According to the 2012 National Health Interview Survey (NHIS), which included a comprehensive survey on the use of complementary health approaches by Americans, 17.7 percent of American adults had used a dietary supplement other than vitamins and minerals in the past year. These products were the most popular complementary health approach in the survey. (See chart.) The most commonly used nonvitamin, nonmineral dietary supplement was fish oil.

Researchers have done large, rigorous studies on a few dietary supplements, but the results often showed that the products didn’t work for the conditions studied. Research on others is in progress. While there are indications that some may be helpful, more needs to be learned about the effects of these products in the human body, and about their  safety  and potential  interactions with medicines  and other natural products.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Psychological and Physical Approaches

Complementary physical and/or psychological approaches include tai chi , yoga , acupuncture , massage therapy , spinal manipulation , art therapy, music therapy, dance, mindfulness-based stress reduction, and many others. These approaches are often administered or taught by a trained practitioner or teacher. The 2012 NHIS showed that yoga, chiropractic and osteopathic manipulation , and meditation are among the most popular complementary health approaches used by adults. According to the 2017 NHIS , the popularity of yoga has grown dramatically in recent years, from 9.5 percent of U.S. adults practicing yoga in 2012 to 14.3 percent in 2017. The 2017 NHIS also showed that the use of meditation increased more than threefold from 4.1 percent in 2012 to 14.2 percent in 2017.

Other psychological and physical approaches include relaxation techniques   (such as breathing exercises and guided imagery),  qigong ,  hypnotherapy , Feldenkrais method, Alexander technique, Pilates, Rolfing Structural Integration, and Trager psychophysical integration.

Research findings suggest that several psychological and physical approaches, alone or in combination, are helpful for a variety of conditions. A few examples include the following:

• Acupuncture  may help ease types of pain that are often chronic, such as low-back pain, neck pain, and osteoarthritis/knee pain. Acupuncture may also help reduce the frequency of tension headaches and prevent migraine headaches.
• Meditation  may help reduce blood pressure, symptoms of anxiety and depression, and symptoms of irritable bowel syndrome and flare-ups in people with ulcerative colitis. Meditation may also benefit people with insomnia.
• Tai chi  appears to help improve balance and stability, reduce back pain and pain from knee osteoarthritis, and improve quality of life in people with heart disease, cancer, and other chronic illnesses.
• Yoga  may benefit people’s general wellness by relieving stress, supporting good health habits, and improving mental/emotional health, sleep, and balance. Yoga may also help with low-back pain and neck pain, anxiety or depressive symptoms associated with difficult life situations, quitting smoking, and quality of life for people with chronic diseases.

The amount of research on psychological and physical approaches varies widely depending on the practice. For example, researchers have done many studies on acupuncture, yoga, spinal manipulation, and meditation, but there have been fewer studies on some other approaches.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Other Complementary Health Approaches

Some complementary approaches may not neatly fit into either of these groups—for example, the practices of traditional healers, Ayurvedic medicine , traditional Chinese medicine , homeopathy , naturopathy , and functional medicine.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} NCCIH’s Role

NCCIH is the Federal Government’s lead agency for scientific research on complementary and integrative health approaches.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} NCCIH’s Mission and Vision

The mission of NCCIH is to determine, through rigorous scientific investigation, the fundamental science, usefulness, and safety of complementary and integrative health approaches and their roles in improving health and health care.

NCCIH’s vision is that scientific evidence informs decision making by the public, by health care professionals, and by health policymakers regarding the integrated use of complementary health approaches in a whole person health framework.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} For More Information

Nccih strategic plan.

NCCIH’s current strategic plan, Strategic Plan FY 2021 – 2025: Mapping a Pathway to Research on Whole Person Health , presents a series of goals and objectives to guide us in determining priorities for future research on complementary health approaches. 

NCCIH Clearinghouse

The NCCIH Clearinghouse provides information on NCCIH and complementary and integrative health approaches, including publications and searches of Federal databases of scientific and medical literature. The Clearinghouse does not provide medical advice, treatment recommendations, or referrals to practitioners.

Toll-free in the U.S.: 1-888-644-6226

Telecommunications relay service (TRS): 7-1-1

Website: https://www.nccih.nih.gov

Email: [email protected] (link sends email)

This publication is not copyrighted and is in the public domain. Duplication is encouraged.

NCCIH has provided this material for your information. It is not intended to substitute for the medical expertise and advice of your health care provider(s). We encourage you to discuss any decisions about treatment or care with your health care provider. The mention of any product, service, or therapy is not an endorsement by NCCIH.

Related Topics

NCCIH Strategic Plan FY 2021–⁠2025 Mapping a Pathway to Research on Whole Person Health

Related Fact Sheets

Are You Considering a Complementary Health Approach?

Be an Informed Consumer

Digital transformation: Health systems’ investment priorities

Health systems around the world are facing a host of challenges, including rising costs, clinical-workforce shortages, aging populations requiring more care (for example, to treat chronic conditions), and increasing competition from nontraditional players. 1 Rupal Malani, “ 2024 health systems outlook: A host of challenges ahead ,” McKinsey, December 19, 2023. At the same time, consumers are expecting new capabilities (such as digital scheduling and telemedicine) and better experiences from health systems across their end-to-end care journeys. 2 Rupal Malani, “ 2024 health systems outlook: A host of challenges ahead ,” McKinsey, December 19, 2023. In response, health systems are increasing their focus on digital and AI transformation to meet consumer demands, address workforce challenges, reduce costs, and enhance the overall quality of care. 3 Shubham Singhal and Drew Ungerman, “ Healthcare’s next chapter: What’s ahead for the US healthcare industry, ” McKinsey, December 19, 2023. However, despite acknowledging the importance of these efforts to future sustainability, many health system executives say their organizations are still not investing enough.

AI, traditional machine learning, and deep learning are projected to result in net savings of up to $360 billion in healthcare spending.

AI, traditional machine learning, and deep learning are projected to result in net savings of $200 billion to $360 billion in healthcare spending. 4 David M. Cutler et al., The potential impact of artificial intelligence on healthcare spending , National Bureau of Economic Research working paper, number 30857, January 2023. But are health systems investing to capture these opportunities? We recently surveyed 200 global health system executives about their digital investment priorities and progress. 5 The survey was conducted from July 24 to August 7, 2023, with 200 respondents who currently work as executives at healthcare delivery organizations. Fifty percent of respondents hold technical executive roles (for example, IT, digital, or analytics), while respondents in nontechnical positions represent all other executive roles (such as strategy, operations, administration, finance, marketing, and medical). Approximately 60 percent are based in the United States; 40 percent are based in 26 other countries across Africa, Asia, Europe, and South America; 40 percent have academic affiliations; 20 percent are from public institutions; and one-third represent not-for-profits. Seventy-five percent of respondents reported their organizations place a high priority on digital and analytics transformation but lack sufficient resources or planning in this area.

Increasing prioritization

In line with other industries, the majority (nearly 90 percent) of health system executives surveyed, in both technical roles (such as chief information officer or chief technology officer) and nontechnical roles (for example, CEO or CFO), reported that a digital and AI transformation is a high or top priority for their organization. At the same time, 75 percent of respondents reported their organizations are not yet able to deliver on that priority because they have not sufficiently planned or allocated the necessary resources.

Health system digital investment priority areas and anticipated impact

For health system executives, current investment priorities do not always align with areas they believe could have the most impact. There is alignment in some areas, including virtual health and digital front doors, where about 70 percent of respondents expect the highest impact. 1 Virtual health and digital front door include consumer-facing platforms providing a range of access points for patients to manage their health (for example, scheduling care, navigating care, managing episodic care, paying bills, supporting education, and managing long-term care). In other areas, such as AI, 88 percent of respondents reported a high potential impact, 2 Top three selected. yet about 20 percent of respondents do not plan to invest in the next two years. The absence of investment in a robust, modern data and analytics platform could delay value creation in areas that depend on these capabilities—such as efforts to close gaps in care, improve timely access for referrals, and optimize operating room throughput.

Major headwinds and slow progress

Given the current macroeconomic climate and increasing cost pressures on health systems, most respondents identified budget constraints as a key obstacle to investing at scale across all digital and AI categories of interest (51 percent of respondents ranked this obstacle among the top three). For example, a health system that is building a digital front door may lack the resources to simultaneously invest in the latest generative AI (gen AI) capabilities.

Respondents called out challenges with legacy systems as the second-greatest concern (after budget constraints). Core tech modernization is key to delivering on the digital promise, 1 Eric Lamarre, Kate Smaje, and Rodney W. Zemmel, Rewired , Hoboken, NJ: John Wiley & Sons, 2023, chapter 8. but health systems have typically relied on a smaller set of monolithic systems that have become a challenge to untangle.

Additional highly ranked challenges include data quality (33 percent), tech talent and recruiting (30 percent), and readiness to adopt and scale new technology (34 percent).

Satisfaction with digital investment

Most executives of health systems that have invested in digital priorities (72 percent) reported satisfaction across all investment areas. Among the comparatively fewer respondents who reported investing in robotics and advanced analytics, satisfaction was even higher, at 82 percent and 81 percent, respectively. Given that investments result in a high level of satisfaction and that 75 percent of executives reported they are not yet able to deliver on their digital transformation ambitions (as noted above), health systems may be facing a failure to scale their digital programs.

What health systems can do and how they can learn from other industries

The goal of digital and AI transformation is to fundamentally rewire how an organization operates, building capabilities to drive tangible business value (such as patient acquisition and experience, clinical outcomes, operational efficiency, and workforce experience and retention) through continuous innovation. Delivering digital value for health systems requires investment and new ways of working.

Building partnerships. Scale is crucial to value creation. But the definition of at-scale systems has changed in the past few years; today, it takes more than $13 billion to be a top 20 system by revenue, and many have reached their current position through inorganic growth. 6 Shubham Singhal and Drew Ungerman, “ Healthcare’s next chapter: What’s ahead for the US healthcare industry, ” McKinsey, December 19, 2023. Partnerships (joint ventures and alliances) may offer a promising avenue to access new capabilities, increase speed to market, and achieve capital, scale, and operational efficiencies. 7 “ Overcoming the cost of healthcare transformation through partnerships ,” McKinsey, August 11, 2022.

Moving beyond off-the-shelf solutions. History shows that deploying technology—such as electronic health records (EHRs)—on top of broken processes and clinical workflows does not lead to value. Realizing value from healthcare technology will require a reimagination (and standardization) of clinical workflows and care models across organizations. For example, optimizing workflows to enable more appropriate delegation, with technical enablement, could yield a potential 15 to 30 percent net time savings over a 12-hour shift. This could help close the nursing workforce gap by up to 300,000 inpatient nurses. 8 Gretchen Berlin, Ani Bilazarian, Joyce Chang, and Stephanie Hammer, “ Reimagining the nursing workload: Finding time to close the workforce gap ,” McKinsey, May 26, 2023.

Using the cloud for modernization. Health systems are increasingly building cloud-based data environments with defined data products to increase data availability and quality. Health systems can also use cloud-hosted, end-user-focused platforms (such as patient or clinician apps) that integrate multiple other applications and experiences to simplify stakeholders’ interactions with the system.

Operating differently. Operating differently entails fundamental changes in structure (flatter, empowered, cross-functional teams), talent (new skill sets and fully dedicated teams), ways of working (outcome orientation, agile funding, and managing products, not projects), and technology (modular architecture, cloud-based data systems, and reduced reliance on the monolithic EHR). With these changes, some health systems have begun to see real value within six months. Building a digital culture helps the transformation succeed over time. 9 Matt Banholzer, Laura LaBerge, Andy West, and Evan Williams, “ How innovative companies leverage tech to outperform ,” McKinsey, December 14, 2023.

Cautiously embracing gen AI. Gen AI has the potential to affect everything from continuity of care and clinical operations to contracting and corporate functions. Health system executives and patients have concerns about the risks of AI, particularly in relation to patient care and privacy. Managing these risks entails placing business-minded legal and risk-management teams alongside AI and data science teams. 10 Juan Aristi Baquero, Roger Burkhardt, Arvind Govindarajan, and Thomas Wallace, “ Derisking AI by design: How to build risk management into AI development ,” McKinsey, August 13, 2020. Organizations could also implement a well-informed risk-prioritization strategy.

Digital and AI investments provide health systems with opportunities to address the many challenges they face. Successful health systems will invest in areas with the greatest potential impact while removing barriers—for example, by upgrading legacy infrastructure. Health systems that make successful investments in digital and analytics capabilities could see substantial benefits and position themselves to benefit from the $200 billion to $360 billion opportunity. 11 The potential impact of artificial intelligence, January 2023.

Jack Eastburn is a partner in McKinsey’s Southern California office; Jen Fowkes is a partner in the Washington, DC, office; and Karl Kellner is a senior partner in the New York office. Brad Swanson is a consultant in the Denver office.

The authors wish to thank David Bueno, Camilo Gutierrez, Dae-Hee Lee, Audrey Manicor, Lois Schonberger, and Tim Zoph for their contributions to this article.

Explore a career with us

Related articles.

McKinsey on Healthcare: Innovate to thrive

Leadership rundown: How US healthcare leaders are scaling innovation and transformation

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Glob Health
• v.8(1); 2018 Jun

Health systems around the world – a comparison of existing health system rankings

Stefanie schütte.

1 Centre Virchow-Villermé, Université Sorbonne Paris Cité, Charité-Universitätsmedizin Berlin, France-Germany

Paula N Marin Acevedo

Antoine flahault.

2 Institute of Global Health, Faculty of Medicine, University of Geneva, Geneva, Switzerland

Existing health systems all over the world are different due to the different combinations of components that can be considered for their establishment. The ranking of health systems has been a focal points for many years especially the issue of performance. In 2000 the World Health Organization (WHO) performed a ranking to compare the Performance of the health system of the member countries. Since then other health system rankings have been performed and it became an issue of public discussion. A point of contention regarding these rankings is the methodology employed by each of them, since no gold standard exists. Therefore, this review focuses on evaluating the methodologies of each existing health system performance ranking to assess their reproducibility and transparency.

A search was conducted to identify existing health system rankings, and a questionnaire was developed for the comparison of the methodologies based on the following indicators: (1) General information, (2) Statistical methods, (3) Data (4) Indicators. Overall nine rankings were identified whereas six of them focused rather on the measurement of population health without any financial component and were therefore excluded. Finally, three health system rankings were selected for this review: “Health Systems: Improving Performance” by the WHO, “Mirror, Mirror on the wall: How the Performance of the US Health Care System Compares Internationally” by the Commonwealth Fund and “the Most efficient Health Care” by Bloomberg.

After the completion of the comparison of the rankings by giving them scores according to the indicators, the ranking performed the WHO was considered the most complete regarding the ability of reproducibility and transparency of the methodology.

Conclusions

This review and comparison could help in establishing consensus in the field of health system research. This may also help giving recommendations for future health rankings and evaluating the current gap in the literature.

Identifying simple, practical and understandable ways to assess health system performance, with its complex interlinked dimensions, remains a challenging goal. Health systems are complex, may be seen as the sum of all the organizations, institutions and resources whose primary purpose is to improve health with limited resources [ 1 , 2 ].

All health systems are different due to the different combinations of components they can consider. Ranking health systems is important for informing policy-makers and for strengthening health systems as well as prompt attention to inequalities amongst different populations. It is also in the interest of the United Nations (UN) and the World Health Organization (WHO) for systems to be assessed and compared for policies to be developed and so that the Sustainable Development Goals signed by the 193 member countries can be achieved [ 3 ]. Efficiency of a health system is often considered as the degree of achievement of the goals of a health system given the resources utilized to achieve these goals [ 4 ].

More than a decade ago, the WHO was pioneer in conducting the first health system performance ranking of the 191 member nations of the WHO [ 5 ]. They focused on how nations could improve the efficiency of health system performance by development of evidence based on the outcomes of health systems and their determinants [ 6 ]. This served as the basis for many rankings focusing on the performance of health systems. The methods for this ranking were published in a series of discussion papers by the WHO [ 7 - 10 ].

However, the performance of rankings may be a very complex process. First, a set of appropriate and available indicators such as health-relevant measures to represent the inputs and outputs of the systems has to be identified. Second, different weights, usually based on surveys, on statistical methods or on a collective decision among experts [ 11 , 12 ] are assigned to each indicator. Finally, statistical analyses are conducted to obtain the scores of health systems.

There exists several rankings for health systems and the main difference amongst them is the methodology used to conduct the ranking. As far as we know neither methodological gold standard nor consensus for the methodology to be used to conduct a ranking for health systems does exist. Indeed, while rankings are a popular method for comparison, there is much confusion and debate over which indicators to use and how to present the information in ranked format. Moreover, transparency is essential to the success of any ranking system. The openness of the process in terms of how the indicators were chosen, the approach taken to present this information in ranked format, and access to the original data are a very crucial point.

Hence, we propose to review and assess existing health system rankings as this may also play a role in improving the transparency and the ability of reproducibility of available health rankings. The objective of this review is to evaluate the transparency of existing health system rankings by assessing the completeness and comprehensiveness of the ranking methodology from a systematic perspective. This may also help giving recommendations for future health rankings, evaluating any current gaps in the literature and to encourage future discussions in this area.

In order to identify existing health rankings a search was performed using different search engines: PubMed, Web of Science, Science Direct, Google scholar and Google. The keywords used to perform the search were the following: “health rankings”, “health system rankings”, “health system performance”, “health system efficiency”. Google and Google Scholar was used to be able to find rankings that were not published in scientific journals. No ranking was found through the other scientific databases (PubMed, Web of Science, Science Direct) in addition to those already found through Google and Google Scholar.

Inclusion was based on the objectives of the rankings. Health systems are not easy to compare, mostly because the health sector produces more than one outcome. The most obvious is the health status of the population, followed usually with some measures of financial protection for the population such as paying out of pocket for care in order to measure the performance of the health system. Based on this definition, only rankings that included a financial dimension to evaluate health system performances as an input-output structure were included. Excluded were rankings that did not focus on health systems or that included only measurements of population health without any financial input.

A total of nine health rankings were identified when the search was performed. After the evaluation of each ranking, three rankings were selected to be compared, based on the inclusion criteria shown in Figure 1 .

Flowchart indicating health ranking selection process.

Table 1 shows the list of the excluded rankings that had been identified through our key search terms but had mainly a focus on the measurement of population health rather than a focus on the health system performance and did not include any financial dimension.

List of excluded rankings based on the inclusion and exclusion criteria

To evaluate the transparency of the methodologies, four categories of evaluation were chosen. They were based on information that is necessary to be able to replicate a ranking. The four categories of evaluation are the following: (1) General information, (2) Statistical methods, (3) Data and (4) Indicators. For each category a number of criteria were established. Scores were determined for each criterion on a scale of 0 for No and 1 for Yes, with a higher score representing better attainment in each category. Scores attained by each category were then added and divided by the number of criteria taken into account as some criterion were not applicable (N/A) to get a comparable average score. The highest summation was considered the most complete and transparent methodology. Table 2 shows the list and description of all the categories and the criteria that compose them.

List and description of categories used for the comparison of the rankings

Table 3 shows the general characteristics of the three selected health system rankings.

General characteristics of the selected health system rankings

The WHO published the year 2000 the “World Health Report – Health Systems: Improving Performance” which had as objective the assessment of the performance of the 191 WHO member countries used overall five indicators [ 13 ]. The Commonwealth Fund (CWF) had its first publication of their ranking entitled “Mirror, Mirror on the Wall” in 2004 and has kept on publishing; their last publication was in 2014 [ 14 ]. Its aim is to compare the health system of 11 industrialized countries by using overall 80 indicators to make its comparison. The third selected ranking was published by Bloomberg with the title “Most Efficient Health Care 2014” [ 15 ]. It compared 51 countries and ranked them according to what they considered to be efficiency of health care system by taking into account only three indicators.

Table 4 shows the categories and criteria established for the comparison of the three selected rankings. In the General information category, we can see that the WHO scored 1 for every criterion, the CWF scored 1 in every criterion with the exception of the open accessibility of the surveys conducted towards the population. The criterion “Methods used to determine weights assigned to each indicator” received a “not applicable (N/A)” as all indicators were weighted equally. The Bloomberg ranking received the least points in this category. For published literature they received 0 and for reporting of weights a 1. Regarding the surveys, they received N/A because they did not conduct any surveys to perform their ranking.

Categories and criteria established for the comparison of the three selected health system rankings

WHO – World Health Organization, CWF – Commonwealth Fund

For the Statistical methods category Bloomberg scored 0 in all criteria as no information of the calculation of the ranking was mentioned or manually found. The CWF scored a 2 overall for “models mentioned” and “sensitivity to different statistical methods said to be calculated”, and it received a N/A for “sensitivity to weight change said to be calculated” because they weighted every indicator equally. The WHO received the scored 1 in every criterion except for the criterion “Statistical Software used mentioned.” The ranking from the WHO scored in each criterion a 1. The CWF scored 0 in “Data taken from the same year” and 1 in the others. Bloomberg scored 0 in “Data taken from the same year” and “Years of data used provided”, it scored 1 in the other two remaining criteria.

For the final category Indicators , the WHO scored 1 in each criterion. The CWF scored 0 in “Reason for the use of each indicator provided” and 1 in each of the others. The Bloomberg ranking scored 1 in “Ranking for each individual indicator provided” and 0 in the others.

Table 5 demonstrates the average score for each category and for the overall comparison. From the scores, we can see that the methodology and information provided by the WHO ranking scores highest in comparison with the CWF and Bloomberg. A higher score means criteria were better met (from 0 until 1 being best). Criteria that did not apply to the ranking were marked as N/A.

Overall average score of the comparison

Overall the WHO methodology scored the highest according to the criteria that were chosen, 3.9 out of 4. It also scored the highest in every category. The CWF was next with an overall mean of 2.6. Bloomberg had the lowest score with 1.2 with no points in the statistical methods category.

In this paper we have established a set of criteria used to compare the transparency of the published methodologies of health system performance rankings. After having set the inclusion and exclusion criteria, we looked at the methodology of the three selected health system rankings. The rankings were the following: “Health Systems: Improving Performance” by the WHO [ 13 ]”, “Mirror, Mirror on the wall: How the Performance of the US Health Care System Compares Internationally” by the CWF [ 14 ] and “the Most efficient Health Care” by Bloomberg [ 15 ]. The choice and number of indicators for each ranking were very different. The WHO used in total 5 indicators plus 2 other variables considered (GDP and education attainment), the CWF used 80 indicators and Bloomberg used 3 indicators.

Our objective was to assess the transparency of the different health system ranking reports regarding the methodology. Transparency is not only about access to the data, it is built on the free flow of enough provided information to understand the process behind a health system ranking. Therefore, it involves detailed general information including definitions, complete access to the methodology and any statistical techniques, the data itself, the ability to search, filter and manipulate the results, and also the explanations for the chosen indicators and the assigned weights.

Therefore, we divided our review criteria in four categories that are important to evaluate the transparency of these rankings. The four categories were (1) General information, (2) Statistical methods, (3) Data and (4) Indicators.

According to our proposed methodology for the comparison of the ranking methodologies we found that the most complete and transparent methodology was that of the WHO. It obtained the highest score in all of the categories. The WHO provided the most complete information compared to the other two rankings. The CWF did average overall, but it lacked mostly on the statistical methodology category. Bloomberg scored poorly in every category, but also, mostly in the statistical method category.

Indeed, the ranking of the WHO seems to be the most complex one. To calculate the efficiency index, the WHO used a fixed effect panel data model in which the health system is seen as a macro-level production unit and in this case, the overall efficiency combines both technical and allocative efficiency [ 5 ]. Three variables were considered: outcome indicator, health system inputs, and effect of controllable non health system determinants of health. The variable outcome indicator is represented by the outcome of the health system and was used by calculating a composite index of five indicators which according to the WHO are the 3 main goals of a health system [ 6 ]: (1) Health (level and distribution) (2) Responsiveness (level and distribution) and (3) Financial fairness. They used weights that were assigned to each indicator to calculate the composite index based on Internet surveys and expert opinions. For the variable Health System inputs that contribute to producing outcomes total health expenditure per capita (public and private) was used. The third variable effect of controllable non-health system determinants of health was measured by considering the educational attainment of the population, which is calculated by the average years in the population older than 15 years of age. The maximum efficiency index, also called frontier of maximum attainment, of the health system was calculated and the best performing country was used as the reference the other countries were compared to it. The frontier of minimum attainment was calculated by assuming absence health system and this is expressed in the calculation of the efficiency score by considering health inequality and responsiveness level as nonexistent. Uncertainty intervals were estimated and to obtain the confidence intervals Monte Carlo simulation techniques were used. In their conclusions the WHO states that health care system efficiency can be increased without increasing health expenditure and that determinants of relative efficiency are what they aim to focus on studying next [ 5 , 7 ].

The CWF ranking used survey data collected from patient and physician and data taken from the WHO and OECD. It assessed 5 dimensions: (1) Quality, (2) Access, (3) Efficiency, (4) Equity and (5) Healthy lives. Each dimension score was calculated by averaging the score of the different indicators used to evaluate health systems. The indicators used were taken from three surveys performed on patients and primary physicians and the Healthy lives dimension was calculated from data obtained from the WHO and OECD. As mentioned earlier, all indicators in this study were weighted equally [ 14 ].

The Bloomberg ranking considered three indicators: (1) life expectancy of the population in each country (2) percentage of GDP per capita cost of health care (3) absolute per capita cost of health care. Bloomberg gave each country an efficiency “score,” with a score of 100 representing a perfect system whereas life expectancy accounted for 60%, the second indicator for 30% and absolute per capita cost of health care accounted for 10%. However, the reasons for the choice of indicators and the dedicated weights are not explained making the transparency and reproducibility of this ranking weak in comparison to the previous two described rankings.

In addition, having a high public spending does not mean that countries will have better health [ 7 ]. The problem with these rankings is that health expenditure plays a role but it is not the main component of assessing the health outcomes of a population.

This review is not an in depth analysis of the methods used in the three rankings nor it does assess the methodological validity of the statistical methods including indicators and weights used. In addition, the conduct of health system rankings may be culturally restricted. It is to be noted that we may have missed rankings in other languages that are not included in our review as the initial search was done in English. Moreover, we used a very restrictive definition of health system performance, meaning that a ranking should include at least a financial dimension to be included in this review.

The dates of publications of each health ranking are fairly recent and became a source of great debate after the WHO published its first report [ 16 , 17 ]. Indeed, health system rankings may be seen as controversial as not everyone agrees that the performance of a health system can be quantified and compared in an international context. It may be unclear why a particular definition was chosen, how well it is founded, by whom it was decided and how open and reflective the decision process was. In particular, as no gold standard exists for health system rankings, there are several points to be taken into consideration: first, the choice of indicators rests with those doing the ranking. Consequently, the set of indicators used will vary according to the value system of the person or group doing the ranking. Second, the choice of weights is itself a value judgment and thus can vary depending on who is making the decision. Depending on the number of criteria and their weights, one dimension may dominate all the others, or several trivial dimensions may swamp more crucial ones.

However, such rankings may have considerable influence and may be used to capture public attention and to “sell” magazines or capture advertising revenues by attracting “views” on the internet. While the lack of appealing alternatives has legitimated the use of rankings in the eyes of many, there is still a lively debate over the issue of how to rank in the mainstreaming media as well as in academic circles [ 17 - 19 ].

There is no doubt that there is an increased interest in international health system rankings. In particular, in times of globalization such as the use of internet, but also travel and migration, have given the citizens and patients of many countries an image of life in other countries [ 20 ]. As stated by Papanicolas and Smith this exposure and trend towards a globalized world has put health systems around the world under pressure to deliver what is available elsewhere.

Despite of some research initiatives such as the “European Community Health Indicators” (ECHI project), the availability of data for such rankings remains a key challenge [ 21 ]. Rankings may be based on convenient data or, in the case of international rankings, on data that is available in a wide range of countries. There is a serious problem of available statistical information at international level that is objective, independent and comparable among countries at the same time. Definitional inconsistencies of measurements across countries may also exist [ 22 ].

Another challenge in some of these ranking publications is the fact that they do not go through a peer review board like scientific articles published in journals, and this impedes exchange among scientist and constructive criticism with regards to how data are used, indicators are chosen, weights are determined, and what kind of statistical methods are chosen to be used. It should be also noted, that the ranking of the WHO has been only performed once in 2000, also the Bloomberg ranking was not conducted on a regular basis. Perhaps this may underline the critical and sensitive issue of an international health system ranking whereas other internal rankings comparing the population health within one single country have been repeatedly done [ 23 , 24 ].

In the area of academic rankings that compare the quality and performance of universities, the UNESCO initiated an International Ranking Expert Group as there was a growing criticism of the existing approaches to and methodological problems. In 2006 they adopted a document containing principles of quality and good practice called the Berlin principles on ranking of higher education institutions [ 25 ]. We strongly recommend for future studies or expert groups to develop such principles and recommendations in the area of health system rankings.

CONCLUSIONS

To the best of our knowledge, this review is the first that assesses the transparency of existing health system performance ranking methodologies, which is important for the advancement of the health system research field.

Based on this review, an in-depth evaluation of the statistical methods used in each ranking would be insightful to know how accurate the applied statistical methods are in assessing performance of health systems. Also a report on the comparison of how weights are chosen would be valuable.

Acknowledgements

Ethics approval: Not needed for this review.

Funding: The Centre Virchow-Villermé for Public Health in Paris is funded by the French governmental programme “Investments for the future” (Investissements d'Avenir) and received a donation from Sanofi.

Authorship contributions: SS contributed to the study design, analysed the data, and wrote the first draft of this paper. PM assisted with the study design, advised on the method and revised the draft paper. AF initiated the project, designed the study and revised the draft paper.

Competing interests: The authors completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author), and declare no conflict of interest.