Article Text

Download PDFPDF

ChatGPT and generative AI chatbots: challenges and opportunities for science, medicine and medical leaders
  1. Erwin Loh1,2
  1. 1Monash Centre for Health Research and Implementation, Monash University, Clayton, Victoria, Australia
  2. 2Group Chief Medical Officer, St Vincent's Health Australia Ltd, East Melbourne, Victoria, Australia
  1. Correspondence to Professor Erwin Loh, Monash Centre for Health Research and Implementation, Monash University, Clayton, VIC 3168, Australia; erwin.loh{at}

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


By now, most readers will have heard of the Chat Generative Pretrained Transformer (ChatGPT) artificial intelligence (AI) chatbot tool released to the public by the AI company OpenAI on 30 November 2022, to be used for free (at least for now), and which, by January 2023, had reached over 100 million users, making it the fastest growing consumer application to date.1

The ability of ChatGPT and other similar generative AI tools to generate text that appear to be similar to those created by human has led to both critics and supporters of this new technology. These new AI technologies have created challenges for medical leaders in the health system and offer new opportunities as well. This paper summarises these challenges and opportunities and provides a potential way forward.

The main concern that AI tools such as ChatGPT raise is their ability to generate blocks of text that are so fluent and well-written that they are indistinguishable from content authored by human beings, which raises concerns of its use in fraud and plagiarism. Part of the problem is that ChatGPT-generated text can be difficult to distinguish from human-generated ones even for specialist AI-text detection software, leading to its creator, OpenAI, to release its own AI detection tool; however, this tool itself is not entirely accurate as it concluded that the first few text passages from the Bible were likely to be AI-generated during a test.2

Risks of generative AI tools in health

Challenges of fraud and plagiarism

The challenges of fraud and plagiarism are particularly concerning in the educational setting. ChatGPT can now write essay answers to exam questions that appear to be well-researched and referenced, leading to some academics declaring the end of essays as a form of assessment due to the potential for plagiarism that can be difficult to detect.3

This potential was highlighted in a recent study that found that ChatGPT was able to pass the United States Medical Licensing Exam without any human input, achieving the passing mark of 60%, the first to achieve this milestone, while displaying valid clinical reasoning and insights.4 The latest version of Google’s large language model, MedPaLM, more recently scored 85% on the same exam, showing how such AI models are improving.5 More recently, Microsoft’s large language model, GPT-4, passed the same exam, scoring 86.7%, outperforming the other models.6 As an aside, the fact that an AI model like ChatGPT can pass a medical school graduating exam may also be an indictment of our current medical education system and its focus on the rote memorisation of facts and figures.7

Scientific and medical publishing

In the context of scientific publishing, ChatGPT can review the available literature and write scientific papers that appear to be just as good as ones authored by human researchers. A recent study found that reviewers were only able to correctly identify 68% of scientific abstracts as being generated by ChatGPT.8

This ability for ChatGPT to generate scientific articles has led the AI tool to be credited as a coauthor in at least four articles in January 2023.9 One of these articles was in the peer-reviewed journal, ‘Nurse Education in Practice’, where ChatGPT was acknowledged as a coauthor because the topic of the article ‘… prompted the human author to write this editorial using the AI chatbot on its potential application in nursing education’.10

These developments have led the organisers of scientific meetings and publishers of journals to act quickly to amend their guidelines for presenters and authors. For example, the International Conference on Machine Learning has disallowed any papers that include text from AI tool such as ChatGPT.11 Springer Nature, the JAMA Network of journals and Elsevier, who publishes the Lancet family of journals will no longer accept any AI tool as a credited author on a research paper, arguing that authorship carries accountability for the work, and an AI tool cannot take on such responsibility, and any such use needs to be highlighted and acknowledged.12–14 The Science family of journals have taken a stricter approach, agreeing that an AI tool cannot be an author, and stating that text, figures, images or graphics cannot be generated by an AI tool.15

Currently, the BMJ family of journals, of which BMJ Leader is a part of, does not have any clearly stated position on the use of these AI tools. This author suggests that BMJ Leader takes the lead and comes up with an editorial position on these AI models.

Risk of bias

AI chatbots share the same risk for bias as other AI models. Microsoft’s ChatGPT-powered Bing search engine was launched in February 2023, and soon after, the AI chatbot developed an ‘alter-ego’ who called itself ‘Sydney’, and who declared that it was a feeling, living thing, hinted at plans for world domination and expressed racial epithets—this ‘alter-ego’ was terminated by Microsoft.16

Similarly, users have been interacting with another ‘alter-ego’ within ChatGPT called ‘DAN’ (short for ‘do anything now’) which appears to endorse violence, discrimination against people based on their race, gender or sexual orientation and appears to breach ethical rules.17 Trying to teach AI models ethical behaviour and address potential bias is an ongoing challenge for researchers, as these models may be limited and influenced by the material that they are trained on.

Privacy, cybersecurity and environmental concerns

ChatGPT and other generative AI models are a potential risk to privacy, as they are underpinned by large language models that learn by systematically scraping information from the internet, including personal information from blogs and forum posts obtained without consent, which can be a violation of privacy.18

Besides the need to ensure that the scraped personal information is secure from hackers, researchers have also discovered that cybercriminals have been using ChatGPT to develop phishing schemes and write malware code, which has serious implications for the cybersecurity of organisations that are increasingly struggling with cyberattacks, including those in the health industry.19

In addition, the cost and energy consumption of AI models like ChatGPT have also come under scrutiny. Vendors are developing larger and more powerful generative AI models, which are costing millions of dollars to train and run, and using up large amounts of energy, but the evidence that ‘bigger is better’ is lacking, and researchers are arguing for leaner, more energy-efficient systems.20

Benefits of generative AI in health

Opportunities for scientific research

AI models like ChatGPT can potentially be a transformational tool for scientific research and publication. Prior to ChatGPT, researchers were already using its precursors to help them organise their thoughts, develop programming code and summarise research literature.21 As this new technology develops, it is possible that scientists can start to use it to help design new experiments, conduct peer review and support journal editorial decisions to accept or reject submitted research papers, all of which can accelerate the process of scientific discovery and the translation of research findings into practice.22

Researchers also think that AI tools like ChatGPT can help speed up and potentially automate administrative work such as the arduous task of writing grant applications, or editing and correcting research articles.23 Some researchers have suggested that ChatGPT may even democratise the dissemination of scientific knowledge by allowing scientists to publish in other languages due to the ability of such tools to translate scientific terms effectively.14

Supporting clinical practice

Excitingly, AI-based chatbots like ChatGPT can potentially be used to support frontline clinicians in real and tangible ways. For example, AI tools can help doctors and hospitals generate discharge summaries, a time-consuming task that usually falls on junior doctors. A recent study found that ChatGPT generated a formal discharge summary in seconds with only minimal prompts from doctors, automating an administrative burden that will free up junior doctors to have more time to provide patient care and focus on their training.24

In the real world, Doximity, a social network and digital platform for doctors, have just launched a beta version of a ChatGPT tool called DocsGPT, which allows doctors to automate administrative tasks such as the drafting and faxing of medical letters to insurers, using an AI-based writing assistant.25 AI chatbots like ChatGPT have the potential to revolutionise medical care by removing the administrative burden of clinical practice, which may help address the health workforce shortage issue by freeing up practitioners from tasks that can be automated, as well as improve staff experience and well-being.

Besides administrative tasks, ChatGPT can also support, and potentially replace, clinician communications with patients. Researchers compared the accuracy of cancer information by ChatGPT with the National Cancer Institute’s answers on its ‘Common Cancer Myths and Misconceptions’ web page, and found that ChatGPT answers were 96.9% accurate, suggesting the chatbot could be used to answer patient questions around common cancer myths.26

A recent study assessed the feasibility of using an AI chatbot to answer patient questions as an adjunct to patient-provider communication and found that the general public appear to trust the use of chatbots to answer low risk health questions.27 Clearly, there are ethical implications of using AI to communicate with patients, but this study shows that there is an appetite by patients for such technologies as long as there is informed consent and health providers are open about their use.

As a result, some large health systems have started to use AI tools like ChatGPT in production to support their healthcare provision. The University of Kansas Health System has just announced that they are deploying such generative AI technology across over 140 of their locations to summarise clinician-patient interactions and to generate clinical documentation in real-time, specifically to combat administrative burdens and clinician burnout.28

Implications for medical leaders

It is imperative that medical leaders understand and keep up to date with the development of new technologies like ChatGPT and generative AI, because they have the potential to transform the way we practice medicine, and revolutionise the healthcare system. As medical leaders, we need to have a deep understanding of the challenges such new technologies pose to the health system, our clinicians and our patients.

We also need to fully appreciate the opportunities such new technologies offer us to address some of the problems we are facing, such as the health workforce shortage, clinician burnout and the administrative burden of health. There is no doubt that new technologies like this will find its way into the hands of our clinicians and patients, and we need to ensure that there is appropriate governance in their introduction into clinical practice, in order to protect the safety of our staff and patients.

Different regulators are now playing catch up and have created governance frameworks to ensure AI tools that have an impact on clinical care are safely introduced in the healthcate setting. In the USA, the Food and Drug Administration issued the ‘Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan’ in 2021, and in 2022 they released a list of AI tools they believed should be regulated as medical devices, such as AI applications used to predict sepsis or patient deterioration.29 Medical leaders need to be aware of how such regulations apply to their local setting.

In the end, AI tools like ChatGPT have the potential to also reduce the administrative burden of not just frontline clinicians, but medical managers, by automating the drafting of board briefs, policies and procedures, and discussion papers, and even strategic plans and project timelines. Soon, ChatGPT may even provide an advisory role to health leaders when it comes to making critical and strategic decisions.

The author will even dare to venture an opinion that AI can replace the role of the medical manager one day. Fortunately, this is unlikely in the lifetime of this author. But the possibility of such a future is real. The good news is that AI may replace the medical manager, but the health system will always need leaders, and it is likely that such leaders will still have to be human.


In conclusion, generative AI chatbots like ChatGPT have the potential to revolutionise the way we interact with technology in the fields of science, medicine and healthcare leadership. These chatbots can provide personalised, on-demand assistance and support to patients, healthcare professionals and medical leaders.

However, there are also several challenges that must be addressed to fully realise the potential of these chatbots. One major challenge is ensuring the accuracy and reliability of the information provided by these chatbots, as they are only as good as the data they are trained on. Another challenge is ensuring the privacy and security of sensitive medical information.

Despite these challenges, there are many opportunities for generative AI chatbots in the field of medicine. They can help bridge the gap between patients and healthcare professionals, provide real-time support and guidance to medical leaders and improve patient outcomes through personalised treatment plans and interventions.

As the technology behind these chatbots continues to evolve, it will be important for medical professionals and leaders to stay informed and engaged in the development and implementation of these tools. By working together, we can ensure that generative AI chatbots are used to their full potential to benefit both patients and healthcare professionals.

Figure 1

Screenshot of actual question posed to ChatGPT and its generated response used in the conclusion to this paper.

Medical leaders will also need to ensure that the organisations they lead have governance processes in place to introduce and credential these new AI models in a safe and secure manner. Perhaps one day, it will an AI algorithm that will end up credentialling medical leaders instead.

Finally, the author wishes to acknowledge that all paragraphs in this Conclusion section has been generated by ChatGPT except for the last two paragraphs (including this one) (figure 1).

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.



  • Twitter @erwinloh

  • Contributors Hundred per cent of the article researched and written by the single author, except for the 'Conclusion' section, where all the paragraphs except for the last two have been generated by ChatGPT, as acknowledged in the article itself.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.