Background Indian healthcare is rapidly growing and needs efficiency more than ever, which can be achieved by leveraging healthcare analytics. National Digital Health Mission has set the stage for digital health and getting the right direction from the very beginning is important. The current study was, therefore, undertaken to find what it takes for an apex tertiary care teaching hospital to leverage healthcare analytics.
Aim To study the existing Hospital Information System (HIS) at AIIMS, New Delhi and assess the preparedness to leverage healthcare analytics.
Methodology A three-pronged approach was used. First, concurrent review and detailed mapping of all running applications was done based on nine parameters by a multidisciplinary team of experts. Second, capability of the current HIS to measure specific management related KPIs was evaluated. Third, user perspective was obtained from 750 participants from all cadres of healthcare workers, using a validated questionnaire based on Delone and McLean model.
Results Interoperability issues between applications running within the same institute, impaired informational continuity with limited device interface and automation were found on concurrent review. HIS was capturing data to measure only 9 out of 33 management KPIs. User perspective on information quality was very poor which was found to be due to poor system quality of HIS, though some functions were reportedly well supported by the HIS.
Conclusion It is important for hospitals to first evaluate and strengthen their data generation systems/HIS. The three-pronged approach used in this study provides a template for other hospitals.
- learning organisation
- operating system
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. All relevant data are part of the manuscript and in online supplemental material. They are being shared.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Information technology enabled systems have made it possible to collect huge quantum of data in digital form which constitutes the bedrock of the Big Data revolution. In essence, data analytics helps in connecting the dots, making sense of available data and in turn informed decision making all of which are precursors for overall better outcomes.
Digitisation in healthcare has enabled creation of Big Data over time with many benefits like better clinical outcomes, improved efficiency, better overall performance and lesser costs.1–6 Improving quality of care and bridging the quality divide7 8 have also been evidenced. Health insurance claims leveraging big data to detect fraud, abuse, waste and errors9 are also among use cases.
Secondary use of data has helped to cut costs and time of clinical trials by streamlining recruitment of subjects in Europe.10 Beyond hospitals, analytics is playing vital role at public health level through infodemiology which is at the intersection of healthcare informatics and epidemiology and infoveillance which employs infodemiology methods for surveillance purposes.11 The Institute of Medicine has estimated the missed prevention opportunities cost the US$55 Billion and Batarseh et al in their work have shown how big data and machine learning can be used for screening, early diagnosis and management of chronic diseases.12
The potential of big data, AI and healthcare analytics are increasingly being demonstrated during COVID-19 pandemic from different countries. Predictive models with data inputs from different devices backed by IoT (internet of things) have helped in improving patient outcomes during COVID-19.13 Better resource allocation and management,14 more efficient systems and processes,15 more accurate predictions of clinical progressions among others benefits have been very helpful during the black swan event of COVID-19 pandemic.
The concept of a data driven self-learning system that is capable of learning and continuous change, called the ‘Learning Health System’ (LHS) is what some would hope to be an eventual outcome of the data revolution and healthcare analytics. The crux of the LHS lies in the cyclical processes of learning with continuous improvement. LHSs can improve their own capacity to learn with increasing availability of data.16
Right from microlevel—Omics data to clinical data extending to community level epidemiological data, the Healthcare Data Spectrum (HDS) shows just how much diverse and vast healthcare data can be. HDS is a patient-centric data model/ framework by Feldman et al that catalogues and organises the range of healthcare data available for analysis.17
As evident from HDS, there are many stakeholders involved. Hospitals and healthcare facilities generate a huge spectrum of data and are an important piece in the healthcare ecosystem puzzle. With a population of 1.2 billion, economic growth and changing demography, healthcare market in India is rapidly growing with industry compound annual growth rate (CAGR) (2015–20) of 22.9%.18 In the light of rising costs, need for higher efficiency and unequivocal benefits of digitisation, the government of India has been pushing for digital health.
Initiatives like Digital India Mission, Ayushman Bharat scheme, electronic health record (EHR) guidelines of 2016, National Digital Health Blueprint and the most recent one being National Digital Health Mission (NDHM) clearly indicate government’s direction. Adoption of EHR so far has been low and a few corporate hospitals have their own EHR systems with little or no interoperability.19 Therefore, healthcare IT in India being Greenfield offers enormous opportunities to build systems ground-up. It is important to know where we stand and what it would take to leverage healthcare analytics to the maximum. This study is intended to find out what it takes at the level of a tertiary care teaching hospital, to get to the enigmatic ‘holy grail’ of healthcare analytics.
This study was carried out at an apex tertiary care teaching hospital in New Delhi, India. The institute has a main hospital and different superspecialty centres with a total 2428 beds, footfall of about 12 000 patients per day in outpatient departments and more than 700 admissions per day. There have been many efforts towards digitisation since 2012 and the system has been evolving with time.
These efforts have yielded large amounts of data and potential for even more, with an ever-increasing patient load. It was therefore found pertinent to study the system and explore ways to improve it. A three-pronged approach was adopted for holistic evaluation of the system. A blend of Systems’ review, users’ perspectives and ability of the Hospital Information System (HIS) to monitor and improve select performance indicators of the hospital was done.
To study the existing HIS at an apex tertiary care teaching institute in New Delhi and assess the preparedness to leverage healthcare analytics.
Concurrent review of the existing HIS at an apex tertiary care teaching institute in New Delhi.
To explore the opportunities and role of healthcare analytics in monitoring and improving select management performance indicators of the hospital.
Formative evaluation for users’ perspective through a questionnaire based on the updated DeLone and McLean model.
Main Hospital and Trauma Centre, an apex tertiary care teaching institute in New Delhi.
Cross-sectional descriptive study.
All the residents and faculty from the departments that use the HIS at Main Hospital. All the nursing staff up to the level of assistant nursing superintendent (ANS) and nursing informatics specialist (NIS) in main hospital. NISs are nurses trained to manage IT related work. This cadre was created at the beginning of computerisation for training and facilitation.
The sample size was310 residents, 310 nursing staff, 110 faculty and 20 NIS with 95% CI and 5% margin of error. It was based on the table from research advisors.20
September 2016 to October 2018.
Universal sample of all the applications running as part of the HIS in the main hospital and trauma centre were taken for concurrent review of the existing HIS as part of objective 1.
Stratified random sampling was done to administer the questionnaire for objective three among nurses, residents and faculty. Universal sample was taken for administering the questionnaire to the NIS as there were only 30 NIS in the main hospital at the time of administering the questionnaire.
Data collection method
The concurrent review of the existing HIS was done on select parameters as mentioned under objective 1. The questionnaire based on updated DeLone and McLean model was taken from a study with due permission from the author Clauss Bossen (associate professor, information studies, department of aesthetics and communication, Aarhus University, Denmark).21 After content and face validation, this self-administered questionnaire was given to the study sample (online supplemental annexure-A).
Data entry and statistical analysis
Data were entered in Epi-Info V.7.0 and then transferred into Microsoft Excel 2016. Data cleaning was done using Microsoft Excel 2016. Analysis was done using StataMP V.13 (64 Bit). Results of descriptive analysis have been presented as proportions with 95% CIs or as mean (SD), wherever applicable. One-way analysis of variance test was applied for differences in mean across professions while analysing the responses to different questions in the questionnaire. Post hoc analysis with Bonferroni test was done wherever required.
Patient and Public Involvement statement
Not applicable. Neither patients nor public were involved in this study.
Limitations and scope for future developments
The study was conducted in an apex tertiary care public sector hospital and the proposed three-pronged approach was used here for evaluation of HIS. The utility of this approach in different kinds of facilities needs to be evaluated in further studies. The focus of this study was primarily on the desirable features of the HIS and its associated factors, which constitute the stage of data generation/capture. Other aspects required for leveraging analytics like different data management systems, data mining tools, training and sensitisation of staff and decision makers has not been covered in this study.
Subsequent studies can focus on aspects of perceived benefits of healthcare analytics and willingness of healthcare workers to learn and adopt it in their daily functioning. There are a wide variety of hospitals with different sizes/bed strength and from different sectors, which cater to broad variety of patients. Based on their location, leadership and revenue management, their perception about healthcare analytics, expectations and requirements from HIS would be different. Better understanding of these aspects is important to fully leverage healthcare analytics and need to be explored in future studies.
All running applications at main hospital and trauma centre were studied in detail in terms of under mentioned parameters:
Type (web/cloud/desktop based).
Function (What does it do)?
Data pulled/ auto fill.
Format in which data is stored (Database details)
Data standards (If any?). Eg: International Classification of Diseases-10 (ICD-10), Logical Observation Identifiers Names and Codes (LOINC), Systematized Nomenclature of Medicine- Clinical Terms (SNOMED-CT), etc.
Is there inter-operability between applications? If yes, what data is automatically pulled?
Login credentials and portal (Separate/common with other applications?)
For the purpose of this study, selected KPIs (Key Performance Indicators) for managerial structures, processes and outcomes were taken from the National Accreditation Board for Hospitals (NABH) and Healthcare Providers, fourth edition (December 2015), section on Continuous Quality Improvement-4 (CQI-4).22 It was ascertained if the existing HIS captures the necessary data for measuring and monitoring these indices/ KPIs.
The DeLone and McLean paper postulated a comprehensive, multi-dimensional model of IS success based on the communications research of Shannon and Weaver, the information ‘influence’ theory of Mason and other empirical studies from 1981 to 87. This was first published in 1992 based on theoretical and empirical IS research.23 Since then a number of studies have empirically tested and validated relationships within the model and discussed its practical applications.21
The D&M model was updated in 2003 that included seven inter-related dimensions: information quality, system quality, service quality, use, intention to use, user satisfaction and net benefits.23 Since this is widely used, well validated and has seven dimensions that cover wide and inclusive range of aspects, it was chosen for a study by Claus Bossen for comprehensive evaluation of EHR at Randers Regional Hospital in Denmark.21 Therefore, in view of similar study settings (application in healthcare) and wide acceptance of the updated DeLone and McLean model, questionnaire based on it was used for the current study after validation.
Responses to self-administered questionnaire were obtained from 110 faculty and 310 residents (junior and senior residents) from the departments that routinely use HIS in their functioning and from 310 nursing staff up to the level of ANS and 20 NIS in Main Hospital. Follow-up was done for non-responders.
A total of 30 running applications at Main Hospital and 25 running applications at trauma centre were studied in detail in terms of 9 parameters mentioned in methodology. Information on all these parameters of each application studied has been complied in the tables (online supplemental annexure-B).
Observations and analysis
During the course of the study, all running applications, their functions and relations with respect to the overall framework was studied. The following key aspects were observed (figures 1 and 2).
All running applications can broadly be classified into four categories
NIC e-Hospital and its modules.
Computerised patient record system (CPRS) and EWD (Enterprise Web Design) Vista.
In-house applications (developed by computer facility).
Other third party applications (From different vendors, meant for specific purposes).
NIC functions on a PostgreS server and in-house applications function on MySQL servers, both of which are Relational Databases. CPRS runs on VistA server which is based on MUMPS (Massachusetts General Hospital Utility Multi-Programming System) database.
In the current scenario, each actor/user has to access multiple login portals, which makes the whole process tedious and time consuming. There is no sharing of data between applications meant for different processes. Therefore lack of structural interoperability is also impairing process interoperability. Contextual interoperability also is hampered, as standards like SNOMED-CT are not universally linked to all applications. Semantic interoperability in terms of parameters like diagnosis, physical findings etc, therefore, does not exist either (figure 3).
Impaired informational continuity of care
Computer systems are expected to ensure Informational Continuity of care and support quality care delivery.24–26 In the existing system, except basic patient data, no other parameters are being shared among different applications. This is evident in lack of interoperability between NIC e-Hospital, CPRS VistA, in-house applications and other third party applications. There is lack of portability even in the same application (CPRS VistA) between Main hospital and Trauma Centre. Only basic patient data are drawn from the master table of NIC e-Hospital in main hospital and Vista EWD in trauma centre by all other applications running in respective centres. This leads to duplication of work with repeated entry of data in different applications making them error prone and unstandard.
Limited device interface and automation
Most of vital devices like cardiac monitors, ventilators, pulse oximeters among others are not linked to the HIS and function as standalone devices with their proprietary data management systems and displays. Information from these devices are manually entered into applications meant for vitals monitoring by nursing staff. The vitals entry application is not interoperable with other applications with patients’ data like CPRS and e-Hospital.
Selected KPIs for the purpose of this study were taken from the NABH and Healthcare Providers, fourth edition (December 2015), section on CQI-4. CQI-4 has eight objective elements (listed below) with each objective element further having specific KPIs.
Objective elements include:
Procurement of medications essential to meet patient needs.
Utilisation of space, manpower and equipment.
Patient satisfaction and waiting time for services.
Adverse events and near misses.
Availability and content of medical records.
Priority managerial activities.
Out of a total of 33 KPIs, the HIS had requisite data to calculate only 9 KPIs. Five of the KPIs for which data are available are from Objective element C (utilisation of space, manpower and equipment) with little or no data to measure other KPIs and objective elements. Complete details of data availability for each KPI, also taken from concurrent review of HIS (objective 1) are compiled in the table (online supplemental annexure- C).
Out of a total of 660 respondents (n=660), there were 20 faculty, 310 residents, 310 nursing staff and 20 NIS. Out of 110 faculty only 20 reported to be using the HIS for their clinical work. The rest 90 reported they do not use the HIS and thus were not required to proceed with the questionnaire. Details of each category of respondents, analysis of their responses with statistical differences between responses of different groups/categories and post hoc analysis with Bonferroni test (where applicable) have been elaborated below and tabulated in separate tables.
Faculty and residents have been critical about the quality of information. Both the groups have expressed that it is not easy to establish an overview of the required information and thus information not always being sufficient (table 1). The nursing staff and NIS on the other hand have given a more favourable response in this regard. Post hoc Bonferroni test was applied and the difference between the nurses and residents’ responses was statistically significant (table 2). This could be due to the very nature of work/job responsibilities of the two groups, with the residents requiring more holistic information which are not available through a single application, as seen in results of objective 1.
Supplementary questions on how well the HIS supports specific work tasks were also asked. For forming an overview of individual patient and simultaneously forming an overview of more patients at the same time, nurses felt the HIS is good, contrary to the residents who said the HIS is poor to very poor in this aspect. About 60% of the residents said the HIS is poor to very poor with respect to medication prescribing and administration.
Finding documentation on nursing or treatment is not supported well by the HIS as expressed by 41% of residents. All groups have expressed concerns about information in the HIS not being updated always. Residents have expressed concerns that it is not always easy to document information in the right place and therefore documentation and record keeping is not supported well by the HIS.
It was found that the most common reasons for insufficient information in the HIS are information can be found elsewhere (on paper or in another system), too much unsorted information and the concrete menus and windows in the HIS are not well structured. These responses can easily be corroborated with the results of objective 1.
All groups have said it has been easy for them to learn how to use the HIS. They find the login time and shifting between screens as somewhat satisfactory. However, there are concerns from all four groups that the HIS is not stable and crashes often. The other remarkable finding that correlates well with findings in objective 1 is that there is no single login portal thus making working difficult. Due to presence of multiple login portals with their own login credentials, it becomes difficult for the user to remember multiple user ID and passwords, shift between screens and finds it difficult to get a complete picture (table 3).
Residents have been critical about not having a unified login portal, thus making their work more difficult. The nurses however were less critical about the same (difference between responses from residents and nurses being statistically significant in the Bonferroni test) (table 4). This could be due to the fact that many of the nurses’ functions are catered to by the e-Hospital application; on the contrary the residents have to access e-hospital, CPRS, PACS among others to establish an overview of a patient. The most common reasons for experiencing difficulty in using the HIS are there are too few computers, technical problems and no time for manually entering patient data into the HIS.
This has been a strong point for the institute in the whole process of implementation of HIS (table 5). All the four groups have agreed that they are satisfied with the support they received during the first 14 days and thereafter. They have expressed satisfaction regarding the overall support/help functions. Sometimes when they do not get help, it is because they cannot find the support staff.
All groups agree that appointment booking, receiving a patient, ordering tests and getting test results, transferring patients between units and discharging patients are supported very well by the current HIS (table 6).
But there are downsides as well. All groups clearly agree that implementation of the HIS has entailed new tasks for them, which means the overall workload has increased. This can also be corroborated with the results of objective 1 wherein there is redundancy with the users having to enter the same data in different applications, due to lack of interoperability. All groups have unequivocally and strongly agreed that the paper work is continuing/has increased despite implementation of HIS, which contributes to increased workload.
Future expectations of net benefits
All groups have clearly expressed that the HIS will bring benefits to patients, staff and hospital in the future. 58.5% feel that their profession has been positive and 40.6% feel that other professions have been positive towards the implementation of HIS (table 7).
However, there are qualms about the way in which HIS was introduced and implemented. Respondents have expressed concerns over inadequate allocation of staff for HIS implementation, not involving staff while designing functions and screen displays and suggestions for changes in the HIS have not been listened to/followed up. About one-third of the respondents (32.3%) do not agree that implementation of the HIS was well planned and well run.
Overall, the existing HIS has many limitations which can also be seen from results of objective 1 but there are certain definite strengths in terms of quality of support/services, positive attitude of staff and optimistic future expectations.
Healthcare organisations are gradually but surely catching up with digitisation. Right from patient demographics and clinical data to feedback systems27 and managerial data, HISs are building up the much talked about ‘Big data’ in healthcare. Healthcare analytics though has unequivocal benefits, has thus far remained largely an enigma. To realise maximum benefits from healthcare analytics, all the four stages viz. data generation, data extraction, analysis, visualisation and reporting28 need to be worked on. The stage of data generation is the very foundation as it provides data in digital format, which enables further analyses and visualisation. Therefore, a robust HIS is of paramount importance for any healthcare organisation to leverage healthcare analytics.
Indian healthcare is a US$372 bn industry with CAGR of 39%.18 More than three-fourth healthcare services are being provided by private sector and around 80% of them are less than 50 bedded, small mom-and-pop facilities often run by doctors themselves.29 Currently, there is very little digitisation in Indian healthcare but with Digital India Mission, NDHM among other initiatives, hospitals have a nudge to adopt some form of digital solution/HIS. Therefore, there is a strong need for a simple, technically less demanding yet robust approach or tool that can help these decision makers and leaders while implementing new HIS or improving existing ones.
The three-pronged approach used in this study can prove to be a handy option. Systems’ review along with users’ perspective gave valuable insights on mismatch between HIS and existing workflow. Workflows in healthcare settings are fluid and unpredictable.30 More often than not, the basis/rules used for development of computer systems are different from the actual processes of clinical work.31 The end result being, ‘The processes and systems are forced to fit into the software rather than other way round.’32 This brings about rigidity that eventually makes it difficult for doctors using these systems. The same is reflected in the current study in the form of poor data quality, continued documentation on paper, HIS entailing new tasks and increasing workload due to HIS, which indicate the dissonance of HIS with the workflow.
Also friendly user interface is very important to ensure more compliance and usability. In a quest towards collecting more structured data, there has always been a trade-off in conventional HIS/EHR systems between having checkboxes/radio-buttons and free text fields28 (figure 4). More the checkboxes, radio-buttons or drop-downs, more will be the structured data. But the fall-out is that in the process of forcefully categorising into one of the drop-downs or radio-buttons, the actual narrative of the patients’ story is lost.33 In the current study, systems’ review showed redundancy in data entry due to lack of interoperability between existing applications. This concurred with findings of unsorted information, poorly structured windows, and multiple login portals as part of users’ perspective. Issues with data quality were reflected in inability to monitor select management-related KPIs.
As a consequence, the overall satisfaction among the users is low with staff from across cadres expressing their discontent about certain aspects of the HIS. Many studies from the west have reported about burnout among clinicians,34 35 lack of human touch, more screen-time,36 37 lack of overall work satisfaction and fulfilment among clinicians33 38 due to compulsory implementation of EHR, which were also seen in the current study.
Specialised clinical workflow technologies backed by artificial intelligence (AI) can streamline the process of data capture and consequently address many of above mentioned concerns. Speech recognition,39 deep learning40 and natural language processing41 have been found to have promising outcomes in this regard. Implementing AI-based tools/technologies can improve system usability, thus strengthening the first step of ‘data generation’.
The three-pronged approach used in this study has helped in holistic assessment, identifying gaps and suggesting improvements in HIS. It can therefore be useful for Healthcare leaders in critical decision making pertaining to their choices of HIS or any other digital solution for that matter. Digitisation of Indian Healthcare is likely to catch pace and this approach can be handy for many organisations, especially the smaller ones who make up for a significant chunk of the industry.
A robust data generation mechanism sets the foundation for the subsequent stages of healthcare analytics viz. data extraction, analytics and visualisation. The prerequisites include interoperability, better data quality and data governance. There are many tools to help with analytics and visualisation,42 provided quality data is available in the first place. It is, therefore, important for any healthcare organisation to first assess their HIS/EHR, which is the very foundation and simultaneously work on implementing robust data governance mechanisms in order to realise the potential of ‘healthcare analytics’.
An important question that often confronts medical administrators and leaders is how to assess their existing system(s) and where to invest to make it better. The three-pronged approach of systems’ review, users’ perspectives and ability of the HIS to monitor and improve select performance indicators, provides a holistic view and helps identify gaps (online supplemental video). The three pronged approach used in this study also provides a template for other hospitals looking to evaluate where they stand and what it takes to get to the holy grail of healthcare analytics.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. All relevant data are part of the manuscript and in online supplemental material. They are being shared.
Patient consent for publication
Approval was obtained from the AIIMS Institute Ethics Committee Vide approval No:IECPG/452/27.07.2016.
Thanks to Dr. Nitin Agarwal, Department of Hospital Administration, B.P.Koirala Institute of Health Sciences, Ms. Nayana Kollalackal Narayanan, Nursing Officer, AIIMS, New Delhi and Mr. Linto C.P, Nursing Officer, AIIMS, New Delhi for their assistance in data collection. Thanks to Dr. Vikas H, Dr. Meghana Prabhu, Dr. Shashikiran, Dr. Manjunath Bale, Dr. Anant Gupta and Dr. Nishant Sharma for their assistance in data compilation, analysis and interpretation.
Contributors The study was conceptualised by SS, ARS and SDB. They were instrumental in designing the study methodology. NRG collected, compiled and analysed the data. Data interpretation was done by all authors. Manuscript draft was prepared by NRG which was reviewed and approved by other authors. NRG is the guarantor for the overall content of the article.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.