Article Text
PDF
Abstract
Introduction Leadership and management training/development programmes have gained increasing institutional attention in healthcare organisations, and they have a wide variety of formats and approaches. However, limited evidence exists about effects of these programmes for the organisations that sponsor them. A minority of healthcare systems in the USA measure the impact of these programmes on organisational metrics such as staff turnover or cost savings. This systematic review sought to answer the question, ‘What evidence exists that leadership and/or management development and training programs yield benefits for health care organizations?’ These benefits could include return on investment, improved productivity/cohesion/teaming, or increased use of specific management skills (eg, strategic planning) that would directly benefit the organisation.
Methods We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to conduct a systematic review of the relevant literature. We conducted two searches in PubMed and one in ABI/Inform, a business literature database. All articles included for the study were further categorised according to their relevance for answering the research questions, using predefined criteria based on their methodology and reported findings.
Results Our search included 2462 studies, of which 55 met criteria for inclusion. We identified four potential organisation-level benefits to leadership and management training programmes: benefits to other staff (besides those who participate in the programmes), improved patient safety and satisfaction, tangible benefits from projects that were part of the programme and improved ability/confidence using leadership-related skills by programme participants. However, the research base on this topic is limited.
Conclusion Although this research identified potential benefits of leadership and management programmes at the organisation level, additional research is needed to make definitive conclusions about their impact.
- clinical leadership
- competencies
- improvement
- medical leadership
- management
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
The topic of leadership and management development in healthcare, both for clinical and non-clinical leaders, has gained increasing attention in recent decades as healthcare systems face new challenges and complexities.1 2 For example, between April 2014 and March 2015, over 35 000 people had enrolled in one of four programmes offered by the UK’s National Health Service Leadership Academy.3 A survey in 2010 of 104 healthcare systems in the USA found that over 50% had some kind of executive leadership development programme.4
Leadership programmes for healthcare professionals can focus on a wide range of topics, including foundational knowledge (eg, healthcare economics, regulatory and compliance issues), execution of projects, managing people and transforming organisations.2 They can also take on a wide variety of formats, lengths and other design criteria. This breadth of topics and formats is unsurprising given the multiple levels of leadership needed in healthcare (team, organisational and national) and the different types of competencies and behaviours required for leadership.5
Evidence suggests that factors specific to an individual organisation, such as the management team (as opposed to industry and corporate-parent effects), can account for up to one-third of the variation in its profitability.6 Further evidence also suggests that investment in human resource management more broadly can reduce turnover and improve profits.7
Therefore, as healthcare organisations invest in leadership and management development and training programmes, it is critical to understand the types of potential benefits from these programmes for the individual and for the sponsoring organisation.8 Healthcare organisations should have a clear understanding of the kinds of impact that they may expect to achieve from leadership and management development programmes in order to design them appropriately and assess their impact over time. Yet a minority of healthcare systems in the USA measure the impact of their programmes on organisational metrics such as turnover, cost savings or market share.4 Accordingly, this review aims to answer the research question: ‘What evidence exists that leadership and/or management development and training programs yield benefits for health care organizations?’
To date, at least one systematic review has addressed a similar topic. Frich et al 9 examined the impact of leadership development programmes for physicians on four outcomes: reactions to the programme, knowledge, behaviour/expertise and system results/performance. Frich et al found that ‘physician leadership development programs are associated with increased self-assessed knowledge and expertise; however, few studies have examined outcomes at a system level’ (p 656).
Our systematic review builds on this important work by Frich et al and differs from it in two critical ways. First, our research looks at leadership and management development programmes across all types of professionals working in healthcare, not just physicians. These could include nurse leaders, administrators, public health professionals and academics (ie, in academic medical centres). Second, our review aims to answer a more targeted question than Frich et al by specifically looking at the organisational benefits of these types of programmes. Given the investment required to undertake leadership development programmes and the importance of different types of professionals working in healthcare systems, this review can supplement the findings documented by Frich et al for healthcare administrators considering whether to undertake leadership or management development programmes.
Methods
We conducted a systematic review consistent with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.10
Search process and criteria
This search relied on an internal protocol developed by GS in partnership with two librarians at Harvard University (one affiliated with Harvard Medical School and one affiliated with Harvard Business School). The protocol was not registered externally. We conducted two separate searches in PubMed and one in ABI/Inform, a business literature database, on 5 January 2016 and a third search in PubMed in response to reviewer comments on 28 August 2017. Any references from these searches published before this date were eligible for inclusion. The earliest article that was included in this search was published in 1969.11 The search terms for the primary search in PubMed were as follows:
(‘Education, Medical, Continuing’[mesh] OR ‘Staff Development’[Mesh] OR ‘Education, Nursing, Continuing’[MeSH Terms] OR ‘Leadership/education’[Mesh] OR education[tiab] OR training[tiab]) AND (‘Leadership’[mesh] OR leadership[tiab]) AND (‘Physician Executives’[Mesh] OR ‘Nurse Administrators’[Mesh] OR ‘Health Facility Administrators’[Mesh] OR executive*[tiab] OR administrator*[tiab] OR manager*) AND (‘Program Evaluation’ OR ‘Organizational Objectives’[MeSH Terms] OR ‘Cost Benefit Analysis’[mesh] OR evaluat*[tiab] OR cost benefit*[tiab])
We searched reference lists from literature reviews and other highly relevant papers identified during the database searches, including reference lists from all publications by one author (A McAlearney4 12 13) whose work was highly pertinent for this review. We also reviewed any paper included in the Frich et al systematic review which had as its evaluation objective knowledge, behaviour/expertise (subjective or objective), or system results/performance (subjective or objective).9 (Papers included in Frich et al only evaluating participants’ reactions to a leadership programme or subjective knowledge were excluded.) We also conducted consultations with managers focused on leadership development at three academic medical centres in Boston (Beth Israel Deaconess Medical Center, Brigham and Women’s Hospital and Massachusetts General Hospital) and searched the websites of three top executive search firms (Korn Ferry, Boyden and Egon Zehnder), four management consulting firms (McKinsey, BCG, Bain and Deloitte) and the National Center for Healthcare Leadership for published literature on this topic.
Study selection and eligibility criteria
After conducting our search, all titles were reviewed for relevance. After excluding irrelevant titles, we read all abstracts and, when appropriate, full articles to determine both the relevance of the article for our research question and the availability of relevant data for inclusion. To be included in the review, the reference had to:
Analyse or document effects of a leadership or management development programme to an organisation, such as return on investment (ROI), improved productivity/cohesion/teaming, or increased use of specific management skills (eg, strategic planning) that would directly benefit the organisation.
Report on a programme which trained executive, practitioner or management-level staff.
Report on a programme which did not take place full-time at a college or university (ie, a degree programme); programmes which engaged professionals working full-time but which also culminated in them receiving a degree (eg, a leadership programme for residents that also resulted in the completion of an Masters of Public Health degree) were not excluded.
Focus on the healthcare/public health sectors.
Be original research with a full article in English available (as opposed to literature reviews, editorials, commentaries, interviews, research abstracts, and so on). In cases where only a research abstract was available but the corresponding full article was not available, the abstract was excluded from our review.
It is worth noting that leadership is critical for supporting quality improvement (QI) initiatives, and leadership and management development programmes often include a component specifically focused on QI.14 That being said, not all QI strategies specifically require leadership or management development/training. For example, some strategies, such as provider reminder systems, involve more task or process-oriented changes to the organisation, rather than leadership or management training.15 Therefore, to maintain an appropriate scope which addressed our research question, we did not include a broader review of all types of QI initiatives.
Data collection process
To extract the data for this review, we piloted an Excel-based data collection tool that was used to capture the results, including the author, title, year, journal, abstract, training programme type, relevant outcome metric, outcome result and quality of article data for answering our research question. We categorised articles based on their relevance for answering our research question based on a combination of methodological approach, methodological quality and findings, using the following five categories and corresponding definitions:
Articles with quantitative results
High relevance: Articles which tested for statistical significance against a control group to evaluate effects on or benefits to the organisation. Effects/benefits were quantified using some methodology other than self-report (eg, peer review, third-party observer, cost savings, and so on).
Medium relevance: Articles which used quantitative methods to identify effects on or benefits to the organisation but did not include measure of statistical significance against a control group. Benefits to the organisation quantified using some methodology other than self-report (eg, peer review, third-party observer, cost savings, and so on).
Low relevance: Articles which used quantitative methods based on self-report, with or without statistical significance against a control group, to determine effects on/benefits to the organisation, OR article focused only on reports of increased skills which could be construed as benefitting the organisation (eg, managers’ direct reports stating that their employees used conflict resolution skills more frequently) but which were not a direct measure of the organisation’s performance.
Null results: Articles whose methodology qualified as high or medium but found null results for benefits to the organisation.
Articles with qualitative results: Articles which explicitly sought to identify benefits beyond those that accrued to programme participants, and which included interviews with individuals other than participants in the development programme, were included in this review. In other words, articles using a qualitative methodology with self-report only were excluded. Descriptive studies of programmes without a summary of benefits for the organisation were (eg, case studies) excluded.
Preliminary results were first presented at a workshop with representatives from leading healthcare institutions including Harvard T H Chan School of Public Health, Harvard Medical School, three affiliated academic medical centres and KPMG. Presenting these preliminary results in this forum allowed us to validate our findings and identify research articles that we had missed during the initial review.
Risk of bias
As with any systematic review, this research has the potential for several forms of bias. First, there is a risk that organisations are more likely to publish results that reflect well on their organisations or programmes, and they are unlikely to publish information with negative results. Organisations are unlikely to publish proprietary financial data relating to their programmes. In addition, evaluations that rely on self-report, which are common for collecting participants’ feedback on training programmes, will have self-report bias. Accordingly, although studies with self-report are important for understanding the total evidence base on this topic, they are considered low relevance for making final conclusions about the impact of leadership and management training programmes, as described earlier.
Results
Study selection and study characteristics
Our search strategy identified 2462 articles for review. Of these, we excluded 2063 based on initial title/abstract review and included 399 for detailed review. Of these 399, we ultimately included 55 articles that met inclusion criteria. We excluded two articles that focused on leadership training but whose programmes also had a significant focus on clinical skills, which made disaggregating the impacts of the leadership training from the clinical training impossible.16 17 Of these 55 articles, 5 had high relevance, 12 had medium relevance, 30 had low relevance, 2 quantitative articles had null results and 6 articles used qualitative methods relevant for answering our research question. Of the articles with high relevance or null results, only two used a randomised controlled trial design.18 19 See figure 1 for the flow chart of study selection for inclusion in the systematic review, and table 1 for the full list of studies included in the systematic review.
Flow chart of studies included in this review.
Studies included in systematic review
Of the 21 articles included in Frich et al, 12 met inclusion criteria for our study. Of the nine included in Frich et al that we excluded, four were excluded because the outcomes measured were not applicable to our research question (eg, programme participants’ ratings of programme quality), and five were excluded because we were unable to find full copies of the articles after searching two academic libraries.20–22 The fact that only 12 of the 55 articles in our review were also included by Frich et al indicates that our search strategy achieved its goal of reviewing studies for a broader group of participants (eg, nurses and non-clinical managers).
Synthesis of results
The review found a wide range of types of programmes for leadership and management development and training, which included, among others: programmes specifically associated with QIs18 23; programmes focused on training women in leadership positions24 25; trainings for professionals working in specific clinical specialties26 27 or public health programmes28; and programmes focused on multidisciplinary teams and cross-team communication.29 These programme types were not necessarily mutually exclusive, and many programmes had multiple training components or multiple intended outcomes.
A limited number of high-relevance articles have attempted to determine the benefits of leadership and management training programmes for organisations. Low-relevance articles tended to use self-report by individuals who participated in training programmes on their changes in leadership-related skills or competencies. While these self-reported improvements in leadership behaviours could translate into other benefits for the organisation (such as increased profits or improved retention of staff), it would be premature to draw any conclusions of this sort from the available literature.
Of the four high-relevance studies, two found statistically significant evidence of improved workplace communication and teaming,18 29 one found higher retention rates for women who participated in a career development programme30 and one found improved performance of workshop participants on management skills that impacted their teams (eg, assigning roles).31 Across all studies included in our review, four key areas emerged as potential benefits that an organisation might receive from a leadership/management development or training programme. All findings reported below are from studies deemed to have high or medium-quality evidence unless otherwise specified. We also report here several miscellaneous positive effects and several null results.
Benefits to other staff in the organisation , such as increased staff support by leaders/managers or more effective teaming. This set of benefits had the greatest amount of evidence in the literature reviewed. Benefits to other staff took many different forms, which are not mutually exclusive and which may actually reinforce each other. For example, after the implementation of the programme in one of the high-relevance articles mentioned earlier, staff reported a statistically significant increase in support from their supervisors and in transformational (as opposed to transactional) leadership behaviours among programme participants, while also reporting a statistically significant decrease in ‘passive avoidant’ and ‘laissez-faire’ management styles.18 Similar benefits found in other studies included improved problem solving between groups and nursing leadership29; a reduction in the number of unprofessional behaviours that required intervention by the chair of a medical department or someone of similar stature32; increased practice of ‘enabling others to act’ by a leader33; increased valuing or practising of teamwork/collaboration34–37; increased respect or caring for staff36 37; increased availability and support to employees34 36; increased clarity of communication to staff38; increased demonstration of supportive leadership behaviours towards subordinates19; and increased general workplace culture and morale.39 The majority of these studies did not test for statistical significance, and several used qualitative, rather than quantitative, research methodologies.
Improvements in patient satisfaction or other patient-related metrics. Several studies found improved patient satisfaction scores on Press Ganey surveys, with improved metrics including ‘how well pain was controlled’,40 ‘nurses took time to listen’40 and general satisfaction with services.32 Studies also found that various programmes were associated with a reduction in patient complaints32; improved communication between staff and patients34 35 38; and improved continuity of care.38 None of these studies tested for statistical significance and some used qualitative research methodologies.
Achievement of a specific tangible benefit to the organisation associated with a project in the programme. Several development or training programmes included specific projects, and the success of these projects could directly benefit the sponsoring organisation. In one programme that focused on creating revenue-generating projects, 38% of all teams had generated revenue from their projects within 5 years after the training programme (for a total $4 million across all projects),41 and another programme focused on training middle managers helped identify £900 000 in cost savings (compared with £108 000 investment in the training programme). Several programmes focused on improving management indicators for various clinical topics (eg, reduction in time from abnormal mammogram findings to definitive diagnosis, reduction in patient registration time, improvement in patient experience, and so on) found that programme participants had completed their projects23 42 or achieved improvements on key indicators.43 A leadership development programme in Egypt taught local health managers and their teams leadership and management skills with the aim to ‘improve the quality and accessibility of health services’ through performance improvement projects. In the first year of the programme, 8 of the 10 projects in the programme had achieved 95% of their self-determined health performance targets.44
Increased confidence / use of leadership and management skills by programme participants. Although the demonstration of and/or confidence in leadership/management competencies are not necessarily tangible benefits to an organisation, an improvement on these dimensions after training programmes could lead directly or indirectly to more tangible benefits. For example, one study identified a statistically significant improvement in the ability of managers to use a specific management tool (strengths, weaknesses, opportunities and threats analysis) after training.45 Similarly, other studies found managers increasingly being proactive in leadership roles,34 acting as a facilitator after their training,34 37 ‘inspiring a shared vision’ among staff,46 47 becoming more comfortable with the demands expected of a leader27 and having more concrete knowledge of management practice26 48 49 or QI techniques. One high-relevance study found statistically significant improvement in an objective measure of team leadership skills by paediatric residents participating in a leadership workshop.31 Various other studies using self-report to measure confidence or use of specific leadership/management skills also found improvements after a training programme.50–62 Note that all articles relying only on self-report were considered ‘low relevance’ for the purposes of answering the research question and should not be considered strong evidence of a benefit to the organisation.
Miscellaneous effects / benefits. In addition to the four themes mentioned above, several articles identified miscellaneous effects or benefits. These effects were not normally documented across multiple studies. One study found that participants in leadership development programmes were more likely to hold a role as the chair of a national committee (eg, committees associated with professional associations), which could benefit the organisation by increasing its prestige.27 One study found that deans of academic medical centres who were more confident than female faculty who had participated in a leadership development programme were more likely than other female faculty to stay at their institution, although this finding was strictly a perception and not an actual observed effect.25 One study found a statistically significant increase in staff retention among participants in the training programme.30
Null results. Several studies found no statistically significant impact of leadership training on outcomes such as retention,18 63 job satisfaction,18 63 clinical outcomes18 or ‘worklife’ (ie, workload, control, fairness, and so on).64
Discussion
This research identified limited evidence of several benefits to organisations from leadership/management development and training programmes. However, there was not substantial, quantified evidence that such programmes have direct, tangible benefits for organisations in the form of ROI, reduced turnover or other potentially desired outcomes. This finding is consistent with the finding by Frich et al that ‘the majority of studies [on this topic] relied on learner satisfaction scores and self-assessed knowledge or behavioral change’ (p 656).9 Nonetheless, the absence of evidence does not necessarily equate to the absence of an effect. Rather, this review suggests that further research and more robust evaluation methodologies are needed, since a low percentage of training programmes use organisational metrics to measure their impact,4 and since our review found a limited research base on this topic.
This review included a diverse array of programmes designed for a variety of executives and managers in many different types of organisations, with potentially different goals in mind. Given the broad range of types of programmes, it would be difficult to develop a cohesive framework that would effectively categorise all the different types of programmes. The findings suggest that no single programme format will provide a ‘silver bullet’ to address the needs of all executive leaders or managers in all organisations and that different training goals require different designs. In the absence of very clear evidence for a single way to deliver training programmes, programmes should consider various factors to help promote clarity in the design and evaluation process. These factors include organisational context in which the leaders/managers operate, the goals of the programme given that organisational context, the knowledge, skills and abilities that will most benefit the participants and the organisation in achieving these goals, the pedagogical modes (eg, didactic training, experiential exercise, simulation) that will best facilitate retention and application of the learning and the evaluation mechanisms to be used in measuring programme outcomes against stated goals.
Limitations of this research and risk of bias
As already noted, the ability to draw conclusions about the impact of leadership and management training programmes at the organisational level is limited by the restricted evidence base on this topic. In addition to the fact that the research base and associated methods are limited in general (with only two randomised controlled trials), there are several risks for bias in the literature. First, institutions, particularly for-profit companies, are unlikely to publish ROI analyses or other financial data about their programmes. Among the information that is published, institutions are more likely to publish findings with positive results that reflect well on themselves. Indeed, while we found two articles with null results, we found that negative results were rarely if ever reported. Self-report introduces considerable bias into the evaluation of programmes’ effectiveness, which is why all articles relying solely on self-report were considered low relevance for our study.
Directions for future research
More research is needed on the impact of leadership and management training programmes at the organisation level. Although research on the direct financial implications of programmes at private institutions may be difficult to conduct or report publicly, research on other benefits to the organisation which could indirectly have positive financial impacts would also be relevant for the literature. Possible outcomes of interest include patient satisfaction, patient safety and other outcomes, staff retention and performance on quality criteria. In addition, research efforts should continue to analyse the various factors which promote or impede the success of leadership and management training programmes, and how these factors have differential impact on different types of outcomes. Academic medical centres could be particularly well positioned to conduct further research on this topic. Finally, despite the complex nature of evaluating programmatic effectiveness in large organisations, it is important for researchers to use rigorous methods in assessing leadership/management development programmes, given only two high-relevance articles included in this review used a randomised controlled trial design.
Conclusion
Although leadership and management training programmes are widespread, the research on their tangible impacts at the organisational level, such as a ROI, is very limited right now. Our research identified at least four potential effects/benefits of these types of programmes at the organisational level: benefits to other staff in the organisation, improvements in patient satisfaction or safety, achievement of specific project-related goals and increased confidence using leadership-related skills by programme participants. However, given the limited evidence base on this topic, more research is needed to identify the institution-level benefits of investing in these types of programmes.
Acknowledgments
The authors thank Jonty Roland at KPMG for his input in structuring the research question and overall approach. The authors thank Nancy Kane and Sara Singer at Harvard T H Chan School of Public Health for their extensive input into the research approach, multiple drafts of the manuscript and the list of articles to include in the review.
References
Footnotes
Contributors GS, LP and JM all conceived the research question and overall approach. GS designed the research protocol with two Harvard librarians. GS conducted a first review all articles in the database and wrote the first draft of the manuscript. GS, LP and JM all edited subsequent versions of the manuscript and recommended additional articles not originally included in the first database search.
Funding An original draft of this paper was commissioned by KPMG.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.