Challenges of fraud and plagiarism

The challenges of fraud and plagiarism are particularly concerning in the educational setting. ChatGPT can now write essay answers to exam questions that appear to be well-researched and referenced, leading to some academics declaring the end of essays as a form of assessment due to the potential for plagiarism that can be difficult to detect.3

This potential was highlighted in a recent study that found that ChatGPT was able to pass the United States Medical Licensing Exam without any human input, achieving the passing mark of 60%, the first to achieve this milestone, while displaying valid clinical reasoning and insights.4 The latest version of Google’s large language model, MedPaLM, more recently scored 85% on the same exam, showing how such AI models are improving.5 More recently, Microsoft’s large language model, GPT-4, passed the same exam, scoring 86.7%, outperforming the other models.6 As an aside, the fact that an AI model like ChatGPT can pass a medical school graduating exam may also be an indictment of our current medical education system and its focus on the rote memorisation of facts and figures.7

Scientific and medical publishing

In the context of scientific publishing, ChatGPT can review the available literature and write scientific papers that appear to be just as good as ones authored by human researchers. A recent study found that reviewers were only able to correctly identify 68% of scientific abstracts as being generated by ChatGPT.8

This ability for ChatGPT to generate scientific articles has led the AI tool to be credited as a coauthor in at least four articles in January 2023.9 One of these articles was in the peer-reviewed journal, ‘Nurse Education in Practice’, where ChatGPT was acknowledged as a coauthor because the topic of the article ‘… prompted the human author to write this editorial using the AI chatbot on its potential application in nursing education’.10

These developments have led the organisers of scientific meetings and publishers of journals to act quickly to amend their guidelines for presenters and authors. For example, the International Conference on Machine Learning has disallowed any papers that include text from AI tool such as ChatGPT.11 Springer Nature, the JAMA Network of journals and Elsevier, who publishes the Lancet family of journals will no longer accept any AI tool as a credited author on a research paper, arguing that authorship carries accountability for the work, and an AI tool cannot take on such responsibility, and any such use needs to be highlighted and acknowledged.12–14 The Science family of journals have taken a stricter approach, agreeing that an AI tool cannot be an author, and stating that text, figures, images or graphics cannot be generated by an AI tool.15

Currently, the BMJ family of journals, of which BMJ Leader is a part of, does not have any clearly stated position on the use of these AI tools. This author suggests that BMJ Leader takes the lead and comes up with an editorial position on these AI models.

Risk of bias

AI chatbots share the same risk for bias as other AI models. Microsoft’s ChatGPT-powered Bing search engine was launched in February 2023, and soon after, the AI chatbot developed an ‘alter-ego’ who called itself ‘Sydney’, and who declared that it was a feeling, living thing, hinted at plans for world domination and expressed racial epithets—this ‘alter-ego’ was terminated by Microsoft.16

Similarly, users have been interacting with another ‘alter-ego’ within ChatGPT called ‘DAN’ (short for ‘do anything now’) which appears to endorse violence, discrimination against people based on their race, gender or sexual orientation and appears to breach ethical rules.17 Trying to teach AI models ethical behaviour and address potential bias is an ongoing challenge for researchers, as these models may be limited and influenced by the material that they are trained on.

Privacy, cybersecurity and environmental concerns

ChatGPT and other generative AI models are a potential risk to privacy, as they are underpinned by large language models that learn by systematically scraping information from the internet, including personal information from blogs and forum posts obtained without consent, which can be a violation of privacy.18

Besides the need to ensure that the scraped personal information is secure from hackers, researchers have also discovered that cybercriminals have been using ChatGPT to develop phishing schemes and write malware code, which has serious implications for the cybersecurity of organisations that are increasingly struggling with cyberattacks, including those in the health industry.19

In addition, the cost and energy consumption of AI models like ChatGPT have also come under scrutiny. Vendors are developing larger and more powerful generative AI models, which are costing millions of dollars to train and run, and using up large amounts of energy, but the evidence that ‘bigger is better’ is lacking, and researchers are arguing for leaner, more energy-efficient systems.20

Opportunities for scientific research

AI models like ChatGPT can potentially be a transformational tool for scientific research and publication. Prior to ChatGPT, researchers were already using its precursors to help them organise their thoughts, develop programming code and summarise research literature.21 As this new technology develops, it is possible that scientists can start to use it to help design new experiments, conduct peer review and support journal editorial decisions to accept or reject submitted research papers, all of which can accelerate the process of scientific discovery and the translation of research findings into practice.22

Researchers also think that AI tools like ChatGPT can help speed up and potentially automate administrative work such as the arduous task of writing grant applications, or editing and correcting research articles.23 Some researchers have suggested that ChatGPT may even democratise the dissemination of scientific knowledge by allowing scientists to publish in other languages due to the ability of such tools to translate scientific terms effectively.14

Supporting clinical practice

Excitingly, AI-based chatbots like ChatGPT can potentially be used to support frontline clinicians in real and tangible ways. For example, AI tools can help doctors and hospitals generate discharge summaries, a time-consuming task that usually falls on junior doctors. A recent study found that ChatGPT generated a formal discharge summary in seconds with only minimal prompts from doctors, automating an administrative burden that will free up junior doctors to have more time to provide patient care and focus on their training.24

In the real world, Doximity, a social network and digital platform for doctors, have just launched a beta version of a ChatGPT tool called DocsGPT, which allows doctors to automate administrative tasks such as the drafting and faxing of medical letters to insurers, using an AI-based writing assistant.25 AI chatbots like ChatGPT have the potential to revolutionise medical care by removing the administrative burden of clinical practice, which may help address the health workforce shortage issue by freeing up practitioners from tasks that can be automated, as well as improve staff experience and well-being.

Besides administrative tasks, ChatGPT can also support, and potentially replace, clinician communications with patients. Researchers compared the accuracy of cancer information by ChatGPT with the National Cancer Institute’s answers on its ‘Common Cancer Myths and Misconceptions’ web page, and found that ChatGPT answers were 96.9% accurate, suggesting the chatbot could be used to answer patient questions around common cancer myths.26

A recent study assessed the feasibility of using an AI chatbot to answer patient questions as an adjunct to patient-provider communication and found that the general public appear to trust the use of chatbots to answer low risk health questions.27 Clearly, there are ethical implications of using AI to communicate with patients, but this study shows that there is an appetite by patients for such technologies as long as there is informed consent and health providers are open about their use.

As a result, some large health systems have started to use AI tools like ChatGPT in production to support their healthcare provision. The University of Kansas Health System has just announced that they are deploying such generative AI technology across over 140 of their locations to summarise clinician-patient interactions and to generate clinical documentation in real-time, specifically to combat administrative burdens and clinician burnout.28

Implications for medical leaders

It is imperative that medical leaders understand and keep up to date with the development of new technologies like ChatGPT and generative AI, because they have the potential to transform the way we practice medicine, and revolutionise the healthcare system. As medical leaders, we need to have a deep understanding of the challenges such new technologies pose to the health system, our clinicians and our patients.

We also need to fully appreciate the opportunities such new technologies offer us to address some of the problems we are facing, such as the health workforce shortage, clinician burnout and the administrative burden of health. There is no doubt that new technologies like this will find its way into the hands of our clinicians and patients, and we need to ensure that there is appropriate governance in their introduction into clinical practice, in order to protect the safety of our staff and patients.

Different regulators are now playing catch up and have created governance frameworks to ensure AI tools that have an impact on clinical care are safely introduced in the healthcate setting. In the USA, the Food and Drug Administration issued the ‘Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan’ in 2021, and in 2022 they released a list of AI tools they believed should be regulated as medical devices, such as AI applications used to predict sepsis or patient deterioration.29 Medical leaders need to be aware of how such regulations apply to their local setting.

In the end, AI tools like ChatGPT have the potential to also reduce the administrative burden of not just frontline clinicians, but medical managers, by automating the drafting of board briefs, policies and procedures, and discussion papers, and even strategic plans and project timelines. Soon, ChatGPT may even provide an advisory role to health leaders when it comes to making critical and strategic decisions.

The author will even dare to venture an opinion that AI can replace the role of the medical manager one day. Fortunately, this is unlikely in the lifetime of this author. But the possibility of such a future is real. The good news is that AI may replace the medical manager, but the health system will always need leaders, and it is likely that such leaders will still have to be human.