Article Text
PDF
Abstract
New findings The evidence presented demonstrates that this leadership programme improved financial performance through increased participant confidence, leading to improved individual and team performance. Observations confirm that margins were maximised by clinicians when provided with granular understanding of costs and training in leadership. Limitations of the study and methodology are discussed.
Background/aim The ‘Service Line Leadership’ programme was developed at Nottingham University Hospitals National Health Service Trust to support financial recovery, through leadership skills and improved understanding of cost. Over 3 years, the programme attracted 425 participants (69 consultants). This retrospective study quantifies the impact of the programme and how it influenced leaders.
Methods The five measures used were: (1) evaluation of the financial return on investment, (2) analysis of the mean differences between 360° psychometric assessment scores before and after the programme, (3) a survey utilising a focused interview and a follow-up questionnaire, (4) a post-programme questionnaire examining skills acquired and (5) a workshop evaluation using a net promoter score.
Results Savings of £3.3 million were identified through the delivery of 11 separate initiatives. Quality improvements were observed. Participants were found to have changed their leadership behaviours through improved communication of trust, leading to empowerment, confidence to challenge perceptions and being mindful of others. New techniques were acquired giving rise to increased confidence in personal leadership.
Conclusion The return on investment based on projects of leaders employing the techniques from the programme, demonstrated a positive impact on the business model of the hospital.
- behaviour
- value
- organisational effectiveness
- leadership assessment
- medical leadership
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
A strong correlation between the ‘best’ hospitals and clinical leadership has been established and is consistent with other industries where ‘domain experts’ or ‘expert leaders’ have been linked with better performance.1 The link between leadership and outcomes in healthcare has been clearly evidenced using metrics including patient satisfaction, patient mortality, staff turnover, quality of care and financial performance. There is also a compelling case for clinically led cooperation across boundaries within health economies to achieve best value.2 However, there has been limited formal opportunity to develop leadership skills for doctors in training with a ‘leadership gap’ being described. Evidence is emerging that the barriers to the development of leadership in healthcare are being overcome. The Centre for Creative Leadership has identified best practices,3 as clinical leadership has become an explicit subject of focus and training. This has resulted in an increasing body of evidence supporting the view that leaders are not just ‘born’ but can be ‘made’.4
Nottingham University Hospitals (NUH) National Health Service (NHS) Trust specified and commissioned a ‘hands on’ dynamic financial leadership programme, to develop business like behaviours in its clinical leaders. The resulting programme was adapted from a commercial model5 6 and branded ‘Service Line Leadership’ for use within the trust between May 2012 and June 2015.
Participation in the programme was voluntary, soon establishing a waiting list. It was run in multidisciplinary cohorts of 17–25 participants with the majority being consultants or ‘Agenda for change’ band 7 and above, see table 1.
Analysis of programme participants
The programme was designed to provide leaders with insight as to how they could influence the financial results, while ensuring the quality of care and the safety of their services. This was achieved over five face-to-face delivery days, ongoing telephone coaching and the use of a benchmarked 360-degree profile. Leaders were provided with worked examples to understand how to influence the trading position of services through their own, and their teams, actions and behaviours. Through interactive sessions, participants explored how the hospital had become more financially astute through the introduction of the financial management techniques of ‘service line reporting’ and ‘patient level costing and information systems’.7
It is estimated that only 10% of leadership development programmes evaluate the impact on the ‘behaviours’ of leaders.8 There is little evidence to describe what training works and why it is effective. A review of the return on investment (ROI) for leadership development initiatives in health examined more than 1000 articles and identified only 30 clear answers. These pointed to benefits such as ‘improved confidence to tackle bigger challenges, better functioning teams, improved patient satisfaction and increased likelihood of adopting transformational, rather than transactional behaviours’.9
Following the NUH programme, a study into the effectiveness of the intervention using Kirkpatrick’s model10 was undertaken combining qualitative and quantitative methods. The intention was to elicit the impact of the course on the Trust and participants by measuring the four Kirkpatrick levels:
Results
Behaviour change
Effective learning
Reaction to the programme.
Methods
Return on investment
Savings attributable to the course were used to establish a ROI using the formula: gain−cost/cost.
Over the 3 years of the programme participants were encouraged to prepare presentations for conferences. Savings achieved were identified through eleven discrete and measurable initiatives following the programme, see box 1.
Eleven service initiatives identified at Nottingham University Hospital
Plastic surgery: remapping of the elective pathway using patient level information. Theatre lists were converted from inpatient to day case, theatre time reduced by introducing a second surgeon on complex cases and job plans were rearranged to improve care for long stay patients.
Orthotics: using ‘win win’ agreements in previously difficult to manage relationships, the team focused on execution of a small number of projects with clearly defined targets including reduction of heel ulcers on orthopaedic wards and reducing footwear expenditure in department.
Waste management: established a new waste operational team to improve the management of waste arising from wards and departments resulting in improved satisfaction scores from the ward managers and reduced statutory compliance risks.
Medicine: established a framework for accountability to ensure that quality payments were earned. An engaging scorecard to measure the ‘Friends and Family’ test ensured all team members could see the progress towards the payment and understood their role in achieving the target.
Transitional care: correct recording of the transitional care provided to babies almost doubled as a result of all staff understanding the importance of completing paperwork correctly, ensuring payment of £410 per baby per day.
Paediatric diabetes: established a formal rolling education programme for hospital staff and school staff across Nottinghamshire to achieve the ‘Best practice tariff’ criteria. The team was strengthened, and the initiative led to better patient care, improved staff satisfaction and the criteria for the best practice tariff being achieved.
Theatres: led an initiative to ensure theatre lists started on time, across 50 theatres. A step change in productivity was achieved as improved session utilisation rates lead to more than 1000 additional cases without increasing theatre capacity.
Critical care: implemented an initiative to improve the arrangements for access to critical care beds for elective surgery. By building trust up to 2 hours was saved per day in determining availability of beds.
Pharmacy: established targets; to reduce medicines expenditure through effective procurement, ensuring prompt introduction of NICE appraised medicines and increased clinical trials unit income. The team focused on establishing ‘win/win agreements’ within the department, the execution of plans using scoreboards and net promoter scores to measure patient satisfaction.
Therapies: scorecards were developed and a cadence of accountability established to improve session utilisation and productivity (ie, 10% increase in patient contacts) and also leading to a saving of four inpatient beds.
Finance: a new financial framework was introduced using net promoter scores to determine confidence in the reports and establish monthly patient level reporting while reducing cost.
Margin improvement was measured using Nottingham University Hospital’s audited financial and management systems and records:
Service line reporting in the general ledger.
Patient level costing and information system.
Project management office records.
In each case, the first-year margin improvement was used, as NHS payment tariffs vary from year to year.
Attributing all of the margin improvement to the course would be inappropriate as participants often came to the course with an understanding of what they wished to achieve, but not how to achieve it. Savings attributable to participation on the course were therefore determined on using a standardised approach and classified as resulting from either:
Innovation effect: where the course had provided skills or methods to enable a participant or group of participants to identify a new way of solving the problem or facilitating change.
Catalytic effect: described as the number of months by which the scheme (and savings) had been brought forward. The catalytic effect was usually achieved through improvements in trust, execution of plans or clarity of purpose.
The cost of the course was determined by adding the direct training costs (ie, materials and trainers) to the costs of backfilling participants at waiting list or agency rates as appropriate.
360° leadership quotient (LQ) tool
A 360° LQ psychometric measure was supplied to all participants as part of the training programme. The main purpose of this was to ‘frame’ discussions during the programme, so learning was specific to the participant.
The LQ was completed by the participant, members of their team and their senior managers before the programme, providing a base line measure of the perceived leadership qualities of the participant. It was repeated later at the discretion of the participant to identify any changes in their scoring profile.
The instrument provided an assessment of the participant’s improved effectiveness as a leader using a summary score (the overall LQ) and scores for the four imperatives described in the programme: ‘Inspire Trust’, ‘Unleash Talent’, ‘Clarify Purpose’ and ‘Align Systems’.11 Additionally, a ‘Net Promoter Score’ (NPS) measure of peoples’ willingness to recommend the participant as a leader12 was included.
The anonymous paired scores for the LQ and NPS (prior to the programme and after the programme) were obtained from a central database maintained by the Franklin Covey organisation to which the programme participants submitted their completed questionnaires. The paired scores for LQ overall, the four imperatives and the NPS were processed and the means for each pair compared using the paired samples ‘t’-test.
Survey with focused interviews
This method allowed views and feelings to emerge using a topic guide.13 The interviews focused on the respondent’s subjective experience of the programme and its application. Commissioned after the programme, the interviews examined the effect on financial leadership behaviours.
A total of 30 participants were randomly selected from a list of all former participants of the programme. To ensure all professions were represented, the participant list was divided into the four categories of participants—medical, senior nurses, pharmacy and therapy services and management and accounting.
Other than an explanation of the topic guide, no specific questions were prepared or asked (except to probe, encourage a deeper discourse or clear up any ambiguity). Using the guide the interview focused on the subjective experiences of the interviewee and how they applied them in practice.
The topic guide was ‘How has the programme impacted on you, your leadership and the conduct of your work? and What has been the result and what did the experience mean to you?’
Interview recordings were transcribed, and the information analysed, interrogated and evaluated through the coding of pertinent extracts. Meaningful categories were identified which were further analysed to eliminate, combine or subdivide categories.
The analysis focused on what the respondents considered were the most influential aspects of the entire programme and how it impacted on their own work and leadership. In addition, the dialogue was interrogated to identify the required components of the Kirkpatrick’s Model for evaluating training specifically, learning, changes in behaviour and results.10
Survey to support interview analysis
The nine-item questionnaire consisting of positively worded statements drew on themes identified from the interviews. All participants of the programme still employed by NUH (N=Circa 400) were sent the questionnaire, which was scored using a five-point Likert type scale ranging from ‘Strongly Disagree’ (1) to ‘Strongly Agree’ (5). The main reason for the use of this survey was to support or challenge the results from the analysis of the interview dialogue using a wider group of participants than the interviewees.
Skills questionnaire
An eight-item questionnaire was prepared identifying individual skills within the four ‘imperatives of leadership’:
Inspire trust.
Clarify purpose.
Align systems.
Unleash talent.
Participants, who attended a 1-day follow-up session to the programme (N=142), were asked which skills from the programme they had used.
Workshop evaluation sheets
These provided immediate feedback following delivery of the programme measuring whether participants would recommend the programme to a friend or colleague.
The results
The impact on the trust’s financial performance and the ROI
The 11 schemes, involving 240 of the 425 participants, resulted in efficiencies of £9.4 million (m) (£22 067 per participant). £1.1 m cost savings were identified and £8.3 m income improvements were earned, see table 2.
Savings per identified scheme
Quality improvements were observed and described in the presentations made by participants, also see table 2. The majority of the schemes could be described as quality improvement initiatives which resulted in a positive financial result. The trust’s programme management office recorded the quality impact assessment when required (in the ‘Theatres’ scheme).
The total cost of the programme, including backfill of participants in the clinical setting, was £718 k (£1689 per participant). The savings attributed to the course due to ‘innovative’ or ‘catalytic’ effects were £3.3 m (£7833 per participant), see table 3.
Return on investment calculation
The ROI for the programme is calculated as 364% ((£3.3m−£0.72 m)/£0.72 m). Using the formula, working days/ROI, the payback period was 61 days.
In three initiatives, gross savings between £96 k and £204 k were achieved. When combined cost information at patient level, a further two initiatives provided returns of £305 k and £412 k. When the patient level information and clinical leadership were all present the programme generated six initiatives with larger margin improvements from £750 k to £2.4 m, see figure 1.
Factor analysis of gross returns achieved.
Assessment of leadership using LQ 360° tool
The paired results of the LQ 1 and LQ 2 and the NPS were compared using the paired samples t-test for 83 participants.
On average, respondents reported small, but statistically significant, greater scores on the post-programme LQ scores than those at the commencement of the programme. The results for each imperative and the NPS also improved, except for the ‘Inspire Trust’ imperative where the change was not significant, see table 4.
Learning questionnaire 360° tool overview of paired samples t-tests
Interviews
Thirty participants were interviewed for between 30 and 45 min. The interviews were transcribed and analysed. The following six categories of changes in behaviour were identified:
Learning—including learning about ‘leadership’ and ‘self’.
Acquiring new skills.
Empowering self and others including developing greater confidence.
Trust.
Mindful of others.
Challenging existing perceptions.
Box 2 describes a sample of the responses.
Analysis of six categories of behaviour changes identified from interview responses
1) Learning
‘I learnt about myself doing the LQ1 and 2’.
‘A lot of the stuff was not new, it was how it was presented…how to apply and use techniques to improve how I lead’.
‘How through my actions I can be (unwittingly) a barrier to developing potential in the team by trying to be in control’.
2) Acquiring new skills was an important part of the overall learning process.
‘The course has given me more skills and tools to work with’.
‘Some of the techniques were new, but on reflection it’s common sense…but they provided a good framework’.
‘The course gave me tools to manage the situation (a long term and difficult issue at work)’.
3) Empowering self and others including developing greater confidence
‘It’s (the course) helped me realise I am capable and able’.
‘It’s (the course) raised my level of confidence’.
‘I am now more likely to confidently step out of my comfort zone’.
‘I feel confident to leave them (team members and reports) to manage their own areas because I helped them to develop the skills and knowledge. I now know how to work effectively, so I feel confident’.
‘I now have the confidence to challenge regarding critical issues as I feel empowered through the things I learnt through the course’.
‘Using the ‘win–win’ helped me think about my team where I can take them and how they can develop ownership of their work’.
‘Now I don’t blame myself and others if things don’t go to plan; I ensure we all learn from it’.
‘I am much more likely to listen and try to understand other people’s agenda and help them align it with the bigger picture’.
‘I don’t … actually need to be there now, as long as I am kept informed and in the loop’.
‘Through applying the principles of the course I am much more aware of the potential of my team. I now do not feel threatened by people I think have greater knowledge or skills than myself. I don’t think I am a barrier to my team’s development as I think I might have been—I don’t have to be in total control now’.
4) Trust
‘I realised that trust in your team was an important aspect; this is in the heart of it and the key for me’.
‘We really did well on a particular project and that’s because I let it go and let it be theirs. They felt they were investing in it. They felt it was important; and not because I told them it was important’.
‘The programme helped me unpack what trust actually might mean and I have applied the course description to my own behaviours. I trusted a colleague with an important task in my absence she felt encouraged and empowered’.
5) Mindful of others
‘I am more likely to recognise traits in other people and use them to everyone’s advantage and my own. I am much more aware of how I could be perceived by others especially those I am leading’.
‘I am much more receptive to other people’s needs’.
‘I am now more willing; no that’s the wrong word, listen to my team. I probably am more empathetic’.
‘I am certainly more mindful of others and the impact and importance of giving feedback’.
6) Challenging existing perceptions
‘Aspects of the course made me think about leadership and how to get the best out of people’.
‘I’m much more assertive…I used to like being liked’.
‘If I have a service that is not effective I could invest my time mending the problem. Now, I build my relationship with the team members and support them and help them to solve the problem themselves’.
‘After listening to team members I realised there were hidden talents that I had not recognised and thought about using; its about talent spotting. The results have been phenomenal in what we have achieved’.
‘Using the ‘WIGs’ has made me really conscious about where I am focusing my time’.
‘I have a framework provided by the course of how I promote, manage, support my teams and understand more about myself as a leader—I have developed greater insight’.
‘By passing on my new skills and knowledge to the team I am more confident in really letting go without over managing them’.
The categories of behaviour change are related, with interdependency between them. Although not necessarily from the same interviewees, the statements in table 5 illustrate the inter-relatedness of the changes in behaviour when delivering the ‘imperatives’ of leadership identified in the programme.
Examples of the interrelated nature of the identified behaviour change categories
Learning was reported by all interviewees. This was not necessarily the rote learning of new facts or ideas about leadership, but rather the assimilation of new techniques, ideas and attitudes that enhanced team work through the application of the programme’s content. For example,
Within ‘Execution’ the identification of ‘wildly important goals’ were considered important. This does not mean before the programme participants did not have important goals; it was about how the limited number which were ‘wildly’ important were identified, communicated and the team involved in their achievement.
‘Win–win agreements’ are about dealing with other parties in a fair and equitable manner to ensure all involved benefit. They are contracts between two people.
Respondents made it clear that the programme helped them to develop a greater level of self-confidence and greater confidence in their teams. This was done through reflection and the questioning of motives, intentions and how they got things done.
The concept of ‘trust’ was important to respondents. This does not mean there was no trust before or that respondents thought that others were not to be trusted. Rather, it was clear the respondents understood that it is the communication of trust which is important. This approach to communicating trust led to less direct management and more to leadership to get things done. Respondents were much more likely to ‘let go’ and not manage directly.
Survey
Respondents indicated the strength of agreement or disagreement with ine statements, by use of a scale.
The returned questionnaires N=124 (a response rate of 31%) were processed and the descriptive and frequency statistics calculated, see table 6.
Frequency and descriptive statistics for questionnaire
Learning questionnaire
The ‘Skills used’ questionnaire was completed by 71 participants (50% of the sample). The main skills acquired by the participants were:
‘Reaction’—NPS
Workshop evaluation forms were completed by 297 participants. None of the evaluations scored 0–6 (ie, there were no ‘Detractors’). Twenty-four participants scored the experience 7 or 8 (ie, ‘Neutral’). Two hundred and seventy-three participants scored 9 or 10 (‘Promotors’). Using the NPS methodology of :
‘Promoters’−‘Detractors’/total the experience achieved a NPS of 92%.
Discussion and conclusion
The gross savings of £9.4 m per annum (1.1% of turnover of the trust) were identified over the 4 years of the programme.
In addition to acquiring new skills and enjoying the training, the results of the interviews and survey questionnaire illustrate a level of enthusiasm and a determined effort to employ the experiences of the programme. The questionnaire (although a self-reporting instrument and measure) also indicated an improvement in the respondent’s leadership.
These improvements are identified and are based on attitudes, the perception of self and others, evaluating through introspection and reflection, improvement in confidence and an understanding of the need to lead rather than just manage. This suggests why the programme led to the reported improvement in financial results.
The six categories of behaviour changes identified in the interviews clearly show changes in overt behaviour and, more importantly, cognitive changes in the individual. Respondents are not simply applying a recipe concocted by the programme and catalogued in the training, but taking the ideas, concepts and tools communicated in the course and applying them in a successful and rewarding manner, in their own way.
Although the results are encouraging, the study is not without its drawbacks and criticism:
Retrospective nature of study—when first commissioned, there was no intention to undertake a formal review of the programme beyond the workshop evaluation sheets. The value of the study became apparent as the programme matured. The opportunity to use the trust’s established management systems allowed the study to progress. As clear goals for the study were not expressed prior to undertaking the programme, conclusions regarding cause and effect are more difficult to apply.
Sample size—the study methodology included response rates which varied from 7% of population to 70%, see table 7. The sampling approach was determined by the retrospective nature of the study and the design would have been improved in a prospective study.
Response rates to study methods
Randomness of the interview group—initially, the group selected for interview was identified using a randomising method. On being invited to attend there were several refusals, cancellations and no shows amounting to 40% of the selected group. The interviewed group were enthusiastic, interested in the study, positive and clearly willing to be interviewed. They could be described as a self-selecting group as the invitation list continued to change until everybody invited, agreed to be involved. Where there was dropout, the most common reason given was pressure of work. It must be asked if the results would have been different (not so positive) had the original sample been used.
The questionnaire sent to all who had attended the programme had a low but acceptable return of 31%. Its use was primarily to confirm the qualitative results from the interviews as the questions were drawn from the categories identified. The percentage of scores indicating ‘agree’ or ‘strongly agree’ (4 and 5) at the lowest were 62.9% (question 3) to 91.1% (question 7). This indicates the dialogue and final analysis of the interviews is valid and offers a representation of the views of the wider population of participants.
The construction and administration of the LQ psychometric—the LQ results were less compelling. Although all the scores showed an increase in the mean difference, and all but one of the pairs (‘Inspire Trust’) were statistically significant, the actual improvement in LQ did not reflect the outcome of the ROI or interview analysis. Because the results of the survey suggested development and improvement in leadership, more significant findings were expected in the quantitative analysis of the LQ results. This suggests that there may be a problem with the validity, reliability or administration of the LQ instrument.
The ROI calculations—while appearing objective, subjective judgements were required when assessing the catalytic and innovative impacts of the training. Savings might also be understated with not all initiatives having been identified. One hundred and eighty-five participants did not report a financial impact, many of whom highlighted qualitative benefits arising from the course.
The financial assessment only measured the first year of financial savings—no attempt was made to quantify the ongoing benefit (eg, further improvements were achieved in plastic surgery as the initial £750 k improvement led to the elimination of a £3.5 m service deficit within 3 years).
Notwithstanding these criticisms, the study provides insights as to how leadership training can save money through increased participant confidence, leading to improved individual and team performance.
The observation that both ‘clinical leadership’ and the ‘granular understanding of costs’ were present when the savings achieved were greatest (ie, better than £0.5 m) supports the view that clinicians, when provided with the meaningful granular information and leadership training, can maximise financial performance.
Acknowledgments
The authors wish to acknowledge the support of Mr. R Egginton (Chief Finance Officer), the Finance department and Programme management office at Nottingham University Hospitals NHS Trust in providing information to determine the return on investment and quality impacts; and the contribution of Franklin Covey and Beffective Ltd in provision of the Leadership quotient analysis. Finally, without the participation of individual leaders on the programme, and their willingness to engage in the subsequent interviews and surveys this study would not have been possible, thank you.
Footnotes
Twitter @DuncanOrme
Contributors DO and CC planned the study. CC conducted the focused interviews with survey and the analysis of the LQ. DO conducted the analysis of ROI, skills questionnaire and workshop evaluation. DO submitted the study.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Author note DO is the Operational Director of Finance and Procurement at Nottingham University Hospital (NUH), one of the largest Trusts in the UK with over 14 500 staff and a turnover of £987 million. Duncan is a regular contributor to journals and conferences on costing and service line leadership in healthcare. He is also a member of the ‘Healthcare costing for value institute’ council. Previously Director of Finance at St Andrews Healthcare (a charity leading innovation in mental healthcare) and Director of Finance at Northampton General Hospital, DO is an experienced leader in finance and is also a certified facilitator in leadership. Duncan is currently co-authoring a guide to clinical business acumen. He keeps bees at his Northamptonshire home and races his 95 years old classic sailing boat in North Wales. CC After a varied and eventful career, he read psychology and education and worked primarily in mental health both as a practitioner and educator. Latterly, he advised and consulted on business and leadership programmes, supporting students through their health care business studies. He is now retired and living in rural Ireland where he also keeps bees.