Article Text
PDF
Abstract
Background/aim In the past decades, there has been an increasing focus on defining, identifying and reducing unwarranted variation in clinical practice. There have been several attempts to monitor and reduce unwarranted variation, but the experience so far is that these initiatives have failed to reach their goals. In this article, we present the initial process of developing a safety, quality and utilisation rate dashboard (‘clinical dashboard’) based on a selection of data routinely reported to executive boards and top-level leaders in Norwegian specialist healthcare.
Methods We used a modified version of Wennberg’s categorisation of healthcare delivery to develop the dashboard, focusing on variation in (1) effective care and patient safety and (2) preference-sensitive and supply-sensitive care.
Results Effective care and patient safety are monitored with outcome measures such as 30-day mortality after hospital admission and 5-year cancer survival, whereas utilisation rates for procedures selected on cost and volume are used to follow variations in preference-sensitive and supply-sensitive care.
Conclusion We argue that selecting quality indicators of patient safety, quality and utilisation rates and presenting them in a dashboard may help executive hospital boards and top-level leaders to focus on unwarranted variation.
- management
- unwarranted variation
- variation
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Unwarranted variation utilisation of procedures and outcomes is a major challenge in modern healthcare systems.1 Reducing unwarranted variation in utilisation rates and outcomes are key elements in improvement work that benefit patients and reduce wasteful, unnecessary or even harmful care.1
The effects of various initiatives to monitor and reduce unwarranted variation in healthcare utilisation rates remain uncertain.2 3 An internal audit conducted in the South-Eastern Norway Regional Health Authority (HSO) revealed that clinical leaders were not sufficiently aware of HSO’s strategic expectations to focus on measures to reduce unwarranted variation in quality, patient safety and hospital utilisation rates. Moreover, many clinical leaders experienced lack of scrutiny of their efforts to reduce unwarranted variation. There was also a varying use of readily available data from national quality registers and national health atlases4 in clinical quality improvement work.
We argue that reducing unwarranted variation is a responsibility for executive boards and top-level leaders. However, board members and top-level leaders are exposed to an overload of information from various data sources. In addition to reports on numerous quality indicators, they oversee fiscal and activity reports, specific process indicators and clinical activities at an aggregated level. This abundance of data might contribute to a loss of the ability to focus on specific areas of unwarranted variation in quality, patient safety and utilisation rates. One way of addressing this problem is making a selection of the most relevant data and to present these in a dashboard. We had an interest in narrowing the scope of information presented to executive boards and top-level leaders using a dashboard as a tool.
In this article, we present the initial process of developing a safety, quality and utilisation rate dashboard (‘clinical dashboard’) based on a selection of data routinely reported to executive boards and top-level leaders in Norwegian specialist healthcare.
Overview of the Norwegian healthcare system
The specialist healthcare system in Norway is predominantly a publicly funded universal health coverage system designed to care for 5.4 million citizens (2022). Services are provided by four regional health authorities responsible for delivering specialist healthcare in their respective regions. HSO provides specialist healthcare for approximately 3.1 million inhabitants (57% of the population in Norway). HSO consists of seven health trusts and two private non-profit hospitals with catchment areas covering populations between 150 000 and 550 000. The health trusts and private hospitals are independent legal entities governed by independent boards with an overall responsibility for the clinical services they provide. The regional referral centre or dedicated hospitals offer highly specialised functions for all patients in the region.
Wennberg as a starting point
HSO developed a clinical dashboard to define, monitor, detect and reduce unwarranted variation in healthcare outcomes and utilisation rates in hospital-based healthcare services within the region. We used Wennberg’s categorisation of healthcare delivery5 to develop the dashboard, focusing on variation in (1) effective care and patient safety, (2) preference sensitive care and (3) supply sensitive care. The first category represents services whose effectiveness has been established in clinical trials and does not involve substantial trade-offs that depend on patient preferences. The second category represents treatment for conditions where two or more medically acceptable options exist, and the choice of treatment should depend on patient preferences. The third category describes a group of services that are directly related to the supply of physicians, healthcare facilities and medical equipment.
Since unwarranted variation for effective care generally is a matter of underuse, whereas overuse is more profound for preference-sensitive and supply-sensitive care, we argue that effective care should be monitored with result indicators whereas the latter should be followed with utilisation rates. Furthermore, the overlap between preference-sensitive and supply-sensitive care made us choose not to distinguish between the two categories when monitoring utilisation rates. We think these categories of services are easy to monitor with data available for most healthcare providers.
The clinical dashboard
The main purpose of the clinical dashboard was to provide the HSO board as well as the local executive hospital boards of the underlying local health trusts and top-level leaders at both levels a set of indicators monitoring unwarranted variation in quality and utilisation rates. The aim of the dashboard was to display relative underuse of effective care and overuse and misuse of preference-sensitive and supply-sensitive care.
We believe that significant underuse of effective care might be revealed by deviating result indicators whereas overuse and underuse of preference-sensitive and supply-sensitive care might be detected through utilisation rates. The dashboard is based on data from the following national registries: The Norwegian Patient Register (NPR),6 The Cancer Registry of Norway, Institute of Population-Based Cancer Research,7 and the Norwegian Health Atlas, Centre for Clinical Documentation and Evaluation.4
The selection of indicators was based on available data to avoid the need for additional reporting and data analyses. Moreover, we have only included data that are publicly available. We think that media exposure may contribute to creating a momentum for quality improvement—both for improving quality and patient safety and reducing overuse, and that such exposure can draw attention towards insufficient capacity for certain procedures.
The intention of the dashboard is to display quality of care, using data for 30-day mortality for patients admitted with acute myocardial infarction (AMI), stroke and hip fracture in addition to hospital standardised mortality rate (HSMR) and 5-year survival after cancer based on standard methods previously described by the data providers.8 9
Variation in utilisation rates is displayed by means of data from NPR where all clinical activity reimbursed by the public health system is registered. HSO developed a novel method based on the Nordic Medico-Statistical Committee Classification of Surgical Procedures and some radiological procedures,10 with all relevant procedures classified into a hierarchical system in four levels. The lowest level is the single procedures; the second level grouped specific procedures to categories (eg, hip replacement); the third grouped second level categories into procedure families (eg, joint replacement); while the fourth and final level grouped third level groups into clinical specialties (eg, orthopaedics).
In order to estimate aggregated cost for each procedure as classified in the four hierarchical levels, we used the median of diagnosis-related group weight across a timespan of 5 years. All procedures within an episode are sorted according to the median weight, and the procedures with highest weight were selected for further analysis. Thus, only one procedure per contact is analysed for the distribution of cost. Standardised utilisation rates for all performed procedures are calculated on the third (eg, joint prosthesis) and fourth level (eg, orthopaedics) by patient’s residence (municipality). We obtained central tendency and dispersion measures depending on distribution at the hospital catchment area (consisting of multiple municipalities) and at the regional level compared with the national level. Although this method allows investigation of both within-variation and between-variation across the hospital catchment areas, it needs to address extreme values on the municipal level to produce robust estimates. Thus, outliers (below 2.5 percentile and above 97.5 percentile) are removed and analysed separately.
The clinical dashboard displays indicators of quality and clinical activity with the aim to reduce unwarranted variation. The plan was to establish the clinical dashboard in four phases:
Interventional treatment including surgical and radiological procedures.
Non-procedural somatic treatment.
Mental healthcare including substance abuse.
Radiology and laboratory diagnostics.
In the following, we present the first phase of the clinical dashboard framework with outcome data in terms of mortality and utilisation rates of surgical and radiological procedures:
Effective care and quality in terms of 30-day mortality and long-term survival
Thirty-day mortality for patients admitted with AMI, cerebral stroke and hip fracture (both in-hospital and post discharge mortality are monitored) (figure 1).
Thirty-day HSMR (both in-house and postdischarge mortality are monitored) (figure 1).
A 5-year survival after initial treatment for patients with breast cancer, colorectal cancer, lung cancer, prostate cancer and pancreatic cancer (figure 2).
Utilisation rate of surgical procedures with documented impact on life expectancy.
Obesity surgery.
The results from the national quality indicator of the hospital standardised mortality rates. The analysis is performed on local hospitals and casemix adjusted. The results for each hospital are compared with a national average.
A 5-year survival after initial treatment for patients with breast cancer, colon cancer rectal cancer, lung cancer, prostate cancer and pancreatic cancer (2015–2019) (data obtained from The Cancer Registry of Norway, Institute of population based cancer research).
Variations in preference-sensitive care and supply-sensitive care
Utilisation rates are measured per capita in the catchment area categorised on total cost per clinical specialty.
Orthopaedic surgery.
General surgery.
Labour and childbirth, including caesarean sections.
Neurosurgery.
Interventional cardiology.
Plastic surgery.
Cardiothoracic surgery.
Vascular surgery.
Urology.
Gastroenterology inclusive upper and lower gastrointestinal tract endoscopy.
The total number of procedures selected and ranked based on total cost (figure 3).
Joint prosthesis (hip, knee).
Surgical fracture treatment.
Non-traumatic orthopaedic surgery lower extremity.
Labour and childbirth.
Percutaneous treatment of coronary artery and heart valve disease.
Neurosurgery (intracranial).
Non-traumatic orthopaedic surgery upper extremity.
Non-traumatic cervical and lumbar spine surgery.
Open heart surgery.
Upper gastrointestinal tract endoscopy including Endoscopic retrograde cholangiopancreatography (ERCP).
Standardised utilisation rates per 100 000 inhabitants for top 10 procedures with high total cost over hospital catchment areas within South-Eastern Norway Regional Health Authority compared with national rates.
Assessment of unwarranted variation
Significant variation in effective care and quality monitored through the described outcomes for quality and patient safety is generally considered as not acceptable. Variation in preference-sensitive and supply-sensitive care that cannot be explained by variation in the prevalence of illnesses or patient’s preferences is measured with utilisation rates and assessed according to following criteria to identify unwarranted variation:
Utilisation rates for procedures selected on cost and volume ranked on aggregated level are used to identify procedures with the most profound variation in utilisation rates between the hospitals in the region (figure 3).
Overall activity level in the region and variation in utilisation rates for procedures defined by others as low value (eg, coronary angiography and percutaneous coronary intervention (PCI) in stable ischaemic myocardial ischaemia).11
For selected procedures, we have used age recommendations to identify potential overuse (eg, upper gastrointestinal tract endoscopy in patients < 55 years, knee arthroscopy patients > 40 years).12
For diagnostic procedures, we have evaluated the proportion of examinations without significant clinical findings (eg, coronary angiograms).
In collaboration with the national quality registries, we plan to explore the value of using available data to assess the association between utilisation rates and average clinical effect of various procedures as described by Danielsen et al13 to estimate recommended utilisation rates.
Discussion
Several ‘top down’ initiatives focusing on reducing overuse and unwarranted variation (such as The Evidence-Based Interventions programme —reducing low value surgery) have so far not led to measurable significant changes in clinical practice.3 The documented effects of the clinician-driven ‘choosing wisely’ campaign have also been marginal.2 These findings may be due to the complexity of defining, measuring and taking actions to reduce unwarranted variation and not at least conflicts of interest between different stakeholders (patients, clinicians, private and public healthcare providers and policy-makers). Further, the production of clinical guidelines and use of quality registry data does not necessarily lead to changes in clinical practice.14
In this viewpoint, we have described a dashboard that narrows the scope of information presented to executive boards and top-level leaders. We chose the term ‘clinical dashboard’ for the ‘safety, quality and utilisation-rate dashboard’ we present to emphasise the need of access to a selection of relevant clinical indicators. Our experience is that executive hospital boards and top-level leaders in Norway are informed by an overweight of fiscal and activity reports. The indicators should be chosen and aligned with the strategic goals and objectives of the hospitals, which includes patient safety and quality including utilisation rates, a major determinant of use of human and monetary resources.
Reducing unwarranted variation is accordingly a key responsibility for both top-level leaders and hospital board members. They need access to relevant data and development of methods that help to differentiate between random fluctuations and unwarranted variations of clinical significance.15 The dashboard is based on outcome indicators to provide data relevant for their governance responsibilities. However, leaders on division and department level will need access to additional process-indicators to plan, execute and monitor effects of quality improvement initiatives addressing unwarranted variation displayed in the described dashboard. We believe that the described dashboard may be used to establish dashboards for division and department levels based on the described data sources.
We have previously published our experience with using indicators and utilisation rates displayed in the dashboard to identify and implement measures to reduce unwarranted variations in hospital mortality and utilisation rates.8 16 Based on data published in the initial edition of the dashboard, the board of HSO has launched regional initiatives to reduce the utilisation rates of upper gastrointestinal tract endoscopy, and coronary angiograms and PCI in patients with chronic myocardial ischaemia.
Norway is a country with a scattered population with areas with a low population density, and it is challenging to find the right balance between centralisation and accessibility with appropriate quality of care and patient safety. Accordingly, a few small hospital areas are still delivering services in a decentralised structure with 24/7 emergency care function for internal medicine, general and orthopaedic surgery. This might be challenging for the equality of the services, which is the responsibility of the board of regional health authority, and we suggest that the presented dashboard could be used to find an acceptable balance between these considerations. We, therefore, think the dashboard may be useful on a national scale to allow comparisons and benchmarking between specific hospitals, hospital trusts and regions. Since it uses internationally accepted quality metrics, it may be further developed and adjusted for use in other countries as well.
In conclusion, we believe that the abundance of available data might contribute to a loss of focus on unwarranted variation in quality and patient safety. We, therefore, think that presenting a selection of data reflecting clinical outcomes and utilisation rates in a dashboard might contribute to an increased focus and level of competence that members of hospital boards and top-level leaders may use to fulfil their governance duties. The dashboard we describe is designed to meet this need, and we think that sharing the model could stimulate a dialogue about ways of increasing the involvement of top-level leaders and board members in reducing unwarranted variation in quality, patient safety and utilisation rates.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Acknowledgments
Data from the Norwegian Patient Registry have been used in this publication.
Footnotes
Twitter @IngebrigtsenTor, @DrFrich
Contributors We here confirm that all authors have contributed to the development of the described dashboard and to the writing and revision of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Disclaimer The interpretation and reporting of these data are the sole responsibility of the authors, and no endorsement by the Norwegian Patient Registry is intended or should be inferred.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.