Article Text
PDF
Abstract
Background The multifaceted nature of leadership as a construct has implications for measuring leadership as a competency in junior residents in healthcare settings. In Canada, the Royal College of Physicians and Surgeons of Canada’s CanMEDS physician competency framework includes the Leader role calling for resident physicians to demonstrate collaborative leadership and management within the healthcare system. The purpose of this study was to explore the construct of leadership in junior resident physicians using a new multisource feedback tool.
Methods To develop and test the Learning by Evaluation from All-Inclusive 360 Degree Engagement of Residents (LEADER) Questionnaire, we used both qualitative and quantitative research methods in a multiphase study. Multiple assessors including peer residents, attending physicians, nurses, patients/family members and allied healthcare providers as well as residents’ own self-assessments were gathered in healthcare settings across three residency programmes: internal medicine, general surgery and paediatrics. Data from the LEADER were analysed then triangulated using a convergent-parallel mixed-methods study design.
Results There were 230 assessments completed for 27 residents. Based on key concepts of the Leader role, two subscales emerged: (1) Personal leadership skills subscale (Cronbach’s alpha=0.81) and (2) Physicians as active participant-architects within the healthcare system (abbreviated to active participant-architects subscale, Cronbach’s alpha=0.78). There were seven main themes elicited from the qualitative data which were analogous to the five remaining intrinsic CanMEDS roles. The remaining two themes were related to (1) personal attributes unique to the junior resident and (2) skills related to management and administration.
Conclusions For healthcare organisations that aspire to be proactive rather than reactive, we make three recommendations to develop leadership competence in junior physicians: (1) teach and assess leadership early in training, (2) empower patients to lead and transform training and care by evaluating doctors, (3) activate frontline care providers to be leaders by embracing patient and team feedback.
- assessment
- patient involvement
- trainees
- competencies
- continuous improvement
Data availability statement
Data are not available due to ethical restrictions.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Background
The social contract describes the relationship between the medical profession and the society which it serves.1 2 This tacit commitment suggests ‘society grants physicians status, respect, autonomy in practice, the privilege of self-regulation, and financial rewards on the expectation that physicians would be competent, altruistic, moral, and would address the healthcare needs of individual patients and society.’1 2 For this reason, the Royal College of Physicians and Surgeons of Canada developed a CanMEDS competency framework, adding explicit structure and purpose to the physician’s social contract.3
Not unique to the Canadian healthcare training paradigm, physicians everywhere are expected to be leaders in how they meaningfully engage patients and caregivers; collaborate with interprofessional colleagues; and build coalitions to facilitate change for individual patients and to initiate systems change.4 Yet, despite this explicit manifestation of the social contract, physicians are rarely taught nor assessed for leadership skills at any stage of training.4–6 Perhaps, positing leadership assessment within the patient safety and quality improvement milieu will serve as an impetus to drive educators and health administrators to value leadership training and assessment.5 7
One way to help achieve a patient-centred healthcare system is to embed feedback mechanisms into training where patients and allied healthcare providers can share perspectives about patient care with resident physicians at a formative stage of their career.4 Gilbert describes the patient as a leader.8 One minor step towards appropriately elevating the patient to ‘patient leader’ requires equal authority in training assessment. Just as the junior resident physician should find it absurd not to receive performance feedback from their attending physician, they should find it equally absurd not to receive explicit feedback from their patient—the end user.
Another important step is what the resident physician does with patient feedback. It is not sufficient to be a passive recipient. The Institute of Medicine calls for academic health centres to ‘develop leaders at all levels who can manage the organisational and system changes necessary to improve health through innovation in health profession’s education, patient care and research’.9
Leadership competencies for medical education and healthcare providers have been shown to be comprised of five factors: (1) social responsibility, (2) innovation, (3) self-management, (4) task management and (5) justice orientation.10 Interestingly, some of these factors involve regulation of ‘self’, while others are focused on the external ‘system’ influences, suggesting leadership is a multi-dimensional construct. This has also been referred to as personal development and collaborative capacity.4 6
Moreover, from the quality improvement/patient safety (QI/PS) perspective, the construct of leadership lends itself to being a ‘champion’ for healthcare quality. Practitioners of QI/PS agree that such champions are central to implementing change; however, the ‘clinical champion role is a concept that has been widely promoted yet empirically underdeveloped in health services literature’.11 Leaders must help ‘define the future, align people with a vision and remove obstacles to allow people to see this vision’.9
The multifaceted nature of leadership as a construct has implications for measuring leadership as a competency in the healthcare setting. In Canada, the Royal College of Physicians and Surgeons of Canada’s (RCPSC) CanMEDS physician competency framework12 includes the Leader role (formerly referred to as Manager) calling for resident physicians to demonstrate collaborative leadership and management within the healthcare system, thereby contributing to a vision of high-quality healthcare. According to the RCPSC CanMEDS definition: As Leaders, physicians engage with others to contribute to a vision of a high-quality healthcare system and take responsibility for the delivery of excellent patient care through their activities as clinicians, administrators, scholars or teacher.13
Completion of specific competencies of intrinsic roles such as Leader from the CanMEDs roles represents ‘milestones’ that are fundamental to achieving Entrustable Professional Activities (EPA) in Competency-Based Medical Education (CBME).14 These milestones are expected to subsequently allow for clear targets of learning and assessment.15
Intrinsic CanMEDS roles are defined as the explicit labelling of traditional implicit identification of physician competence, both reiterated and emphasised through key stakeholder engagement. Intrinsic CanMEDS roles are considered the ‘armour that protects medical expertise, while claiming medical ownership of a broad arena of medical practice’.3 16–18 The multifaceted construct of Leader as a competency, coupled with the adoption of CBME and the importance of patient-centred care, calls for appropriate workplace-based assessment tools for intrinsic CanMEDS roles. More specifically, since leadership is associated with accolade, teamwork, management and administration, it is important to have empirically supported assessment tools available for resident physicians.9 10 12 19
One method of assessment called 360° feedback or multisource feedback (MSF), requires evaluations that are completed by multiple people in a person’s sphere of influence.20 Literature examining MSF assessment in Canadian residents has left the CanMEDS role of Leader underexplored. Around the world, efforts to develop an MSF tool to assess leadership have not specifically assessed the Leader role, have failed to seek feedback from patients or have done neither.20–24
Our research question was, ‘What does the construct of leadership look like in junior residents using an MSF tool in a healthcare setting?’ The purpose of this study was to describe the construct of leadership in junior resident physicians by implementing a new MSF survey tool called the Learning by Evaluation from All-inclusive 360 Degree Engagement of Residents (LEADER) tool. The LEADER tool was developed for assessing resident physicians with respect to the RCPSC Leader competencies. Our specific objectives were to (1) explain the construct of leadership with the LEADER tool; (2) establish a basis for preliminary validity evidence for the LEADER tool; and (3) ensure the feasibility and utility in implementing the LEADER tool for junior resident performance from three large programmes (internal medicine, general surgery and paediatrics).
Methods
Methodology
This study used multiple phases to develop and test the LEADER tool. On developing the LEADER tool and applying it in healthcare settings, a convergent-parallel mixed-methods study design was used.25 In developing and providing evidence for validity of the LEADER tool, we used the framework by Samuel Messick adopted by the American Educational Research Association, American Psychological Association and the National Council on Measurement in Education as a field standard.26 27 Figure 1 describes these phases (see online supplemental material for the methods of phases 1 and 2).
Supplemental material
Phases to develop and test the LEADER tool. LEADER, Learning by Evaluation from All-inclusive 360 Degree Engagement of Residents; MSF, multisource feedback.
The LEADER Questionnaire was developed as a tool for evaluation of the Leader CanMEDS role in resident physicians in phase 3 and is shown in figure 2.
The LEADER tool.
We created two online LEADER survey tools, one for residents (self-assessment) and one for the multiple assessor groups (eg, peer residents, nurses, patients/family members/caregivers, allied health care providers and attending physicians). The resident self-assessment LEADER tool only differed in that it was worded in the first person beginning with ‘I’ as opposed to ‘The resident’, with the same questions as the multiple sources tool. The use of the first person for the self-assessment was deemed important because there can be significant memory and response advantages for first person pronoun usage.28 29 For example, in referencing the self as an actor, the use of the pronoun ‘I’ can prompt narrative self-referencing which is essential for self-assessment.28 29 Furthermore, the use of ‘I’ has been shown as a vital strategy to present the persona of the leader and to demonstrate the qualities as a leader.28 29 Both surveys were designed by two authors (AK and AB) and reviewed for clarity by three programme directors (internal medicine, general surgery and paediatrics), a medical education researcher, as well as nurses, physicians and residents who were external to the study.
Both LEADER tools (self-assessment and multiple-sources assessment) had 10 closed-ended questions (quantitative) and 3 open-ended questions (qualitative). The 10 closed-ended questions were competency-based with questions relating to respect, honesty, verbal communication, written communication, listening, teamwork, management, awareness, anticipation and advocacy. Each closed-ended question had a short description about the knowledge, attitudes and skills the resident was to exhibit as part of the assessment. The assessor could answer according to a 5-point Likert scale, ‘Always’=5, ‘Often’=4, ‘Sometimes’=3, ‘Seldom’=2 or ‘Never’=1. In addition, each question always allowed the selection of ‘Unable to Answer’ (which corresponded to a missing value). In this way, assessors would not overassert or feel obligated to assess a resident on an item for which they perceived being unable to evaluate. The modified LEADER tool for residents’ self-assessment was the same format, but asked them to identify a positive experience in their work specific to the block rotation in which LEADER data were collected and areas they felt they could improve upon. What follows is the study of the LEADER tool in three residency training programmes using a convergent-triangulation mixed-methods study design.30
Participant recruitment & selection
Participants were recruited in 2015 and 2016. Four inpatient acute care hospital units in various medical facilities were selected for the study: one internal medicine unit, one general surgery unit, one neonatal unit and one paediatric unit. The units were selected based on the presence of continual internal medicine, general surgery and paediatric resident block rotations that were part of junior resident schedules. There were three main participant groups for this study: resident physicians, nursing staff, attending physicians, peer resident physicians (on and off service), other allied healthcare providers (including pharmacists) and patients/family (parents/caregivers).
At the beginning of each resident block rotation, all residents were informed of the study and asked to participate. All University of Calgary junior residents were eligible to participate if they were rotating through one of the four selected units and were registered in one of the three programmes. Participation was restricted to only junior (first and second year) residents for sampling convenience and as such purposive sampling was used. All University of Calgary junior internal medicine, general surgery and paediatric residents who were assigned to one of the four MSF study units were invited to participate for their block rotation for 4–8 weeks. Residents were informed that their participation in this study was not mandatory and their consent/dissent would not affect their residency grades. There were no additional exclusion criteria.
Nursing staff participants were either recommended as typically having knowledge and interaction with all residents on the respective units by the respective residency training programme directors, or randomly selected by the research assistant based on staff schedules that coincided with residents being present on the unit. Each nursing staff and allied health participant was approached and asked if they had interacted with the resident. This recruitment process was also used for staff physicians, peer residents (often off-service residents) and allied healthcare providers. Patient participants (or parents/caregivers if the patient was under the age of 18) were targeted for selection as having been seen or visited by the resident based on resident rotation schedules. Each patient evaluated only one resident physician. Included with each questionnaire was a photograph of the resident who was being assessed (excluding the self-assessment version). This ensured that the assessor was accurately recalling the specific resident for which they were assessing. Data were collected using an online Canadian survey programme.
The feedback was collected electronically via email or on an Apple iPad tablet. Each participant was asked to identify, who they were with respect to the assessor group and their gender. Each participant was assigned a unique identifier to ensure anonymity of feedback and reduction of researcher bias in data analysis.
Data analysis
Quantitative data
Data was analysed using SPSS V.24.31 Cronbach’s alpha reliability coefficient was computed for internal consistency for the overall LEADER tool. Kaiser-Meyer-Olkin Measure of Sampling Adequacy and Bartlett’s Test of Sphericity were computed to determine adequacy of sample for scale dimension reduction. Principal component analysis (PCA) with Varimax rotation was used to determine a simple representation of the items that maximise interindividual variance. Component loadings of 0.4 or above were retained for the analysis and subsequent analyses conducted on the subscales. In determining differences between subscale scores versus programme or assessor group, a one-way analysis of variance (ANOVA) was used with post-hoc testing using the least significant difference32 multiple comparisons adjustment.
Qualitative data
The open-ended questions in both the multiple assessor and self-assessment versions asked the assessor to identify a positive experience with the resident/during their block rotation and two areas of improvement for the resident. These data were analysed by two of the authors (AK and DS) using Braun and Clarke’s reflexive thematic analysis framework.33 We initially took an inductive thematic analysis approach, which began with the familiarisation of the data through reading and rereading. Coding then took place to identify important features of the data within and across the three residency programmes as well as within and across the assessor groups that were relevant to answering our research question. This involved coding the dataset and collating the codes and relevant data extracts for further analysis based on frequency and intensity (importance). Initial themes were then generated to identify significant ideas and where possible, comprehensive patterns of meaning (potential themes). Themes were then reviewed and refined by applying the multifaceted construct of leadership as described by the CanMEDS Leader role and existing literature.10 17 34 It is important to note however, no single definition of leadership is exclusively accurate. Themes were then based on developing a detailed analysis of each theme. The final phase involved cross referencing the analytical narrative with the data extracts and contextualising the analysis by discussing the results between the two coders. After the discussion, themes that were identified and were further coded deductively based on the intrinsic CanMEDS roles and key concepts of the Leader CanMEDS role found here: http://www.royalcollege.ca/rcsite/canmeds/framework/canmeds-role-leader-e.
Thematic analysis was conducted across programmes and participant groups. Inter-rater correlation was assessed using Cohen’s kappa.35
Results
Quantitative data
There were 230 assessments completed for 27 residents. Of these assessments, 12% (n=27) were resident self-assessments, 30% (n=70) were nurses, 11% (n=25) were pharmacists, 10% (n=22) were attending physicians, 13% (n=29) were peer residents (off-service residents from a different residency programme or senior residents in the same programme), 13% (n=30) were patients/family members and 12% (n=27) were other healthcare providers. Pharmacists were collapsed into the allied health category and peer residents were collapsed into a physician category with staff physicians to maximise sample sizes (n=30) so that the central limit theorem could be applied.
The frequencies of response by evaluator profession suggest that the study is feasible, with only a few categories that obtained less than a 75% response rate; these categories were for the written communication and anticipation of patient safety problems, from the patients and family member groups. A minimum of 4 and maximum of 17 assessments were completed per resident, with a median of 8 assessments completed per resident for each of the 27 residents, excluding self-assessments. Table 1 shows the demographic groupings of the assessments.
Demographic characteristics of assessor groups who completed assessments
Table 2 shows the results of the psychometric properties of the LEADER items and the subscales. The PCA with Varimax rotation showed a two-factor solution of Leader which accounted for 58% of the variance in scores on the LEADER tool. The two subscales were: (1) personal leadership skills (PLS subscale) and (2) physicians as active participant-architects within the healthcare system (abbreviated to active participant-architects or APA subscale) based on the RCPSC key concepts for the Leader role.
Principal component analysis (PCA) resulting in the personal leadership skills (PLS) and physician as active participant-architect in the healthcare system (APA) subscales*
When comparing the total mean score of the PLS subscale across the three programmes, there was no significant difference, as determined by the one-way ANOVA (F (2212)=1.186, p=0.307). Similarly, when comparing the total mean score of the APA subscale, there was no significant difference between programmes (F (2150)=0.626, p=0.536).
When comparing the subscale across assessor groups, there were significant differences across assessor groups for the PLS subscale, (F (4210)=10.697, p<0.0001) and the APA subscale (F (4148)=10.240, p<0.0001). Post-hoc testing showed significant differences (p<0.0001) across all assessor groups compared with resident self-assessments having the lowest scores (M=21.740, SD=1.55), followed by nurses (M=22.98, SD=2.20). There were no significant differences between the ratings of attending physicians/peer physicians, patients and other allied healthcare scores.
When comparing the APA subscale across assessor groups, there was no significant difference between nurses and other assessor groups, except nurses and the resident’s self-assessment, with nurses scoring higher than the residents’ own self-assessments (M=22.53, SD=1.89 vs M=19.92, SD=1.77, p<0.0001).
Qualitative data
The average percent agreement between coders on a sample of data was 93.2%, producing a Cohen’s kappa coefficient of 0.70, which was deemed satisfactory. There were seven main themes elicited from the qualitative data which were analogous to the remaining intrinsic CanMEDS roles of communicator, professional, scholar, collaborator and health advocate. The remaining two themes were related to (1) personal attributes unique to the junior resident and (2) skills related to management and administration. All seven themes were relevant to at least one key concept of the Leader role as determined by the RCPSC recognising leadership is a complex construct to define. Table 3 shows the themes elicited from the suggestions for improvement by assessor groups.
Themes and supporting quotes from suggested improvements for junior residents
The positive experiences open-ended question showed all residents self-assessed themselves as strongest in the following themes: (1) Communicator/negotiation, (2) Personal attributes/PLS, and (3) Collaborator/teamwork/systems thinking. Areas of suggested improvement focused on the following themes: (1) Scholar/knowledge base/career development and (2) Organisation/time management/administration/priority setting themes.
Across all evaluation sources, internal medicine residents were identified as strongest in the Personal attributes/PLS theme, with the suggested improvement in the Communicator/negotiation and Organisation/time management/administration/priority setting theme, particularly with respect to discharge planning for patients. Across all assessors, general surgery residents were identified as strong with respect to the Professional/physician roles and responsibilities in the healthcare system theme, with suggested improvements identified in the Communicator/negotiation and Organisation/time management/administration/priority setting themes. Paediatric residents were identified as strongest in the Personal attributes/PLS and Professional/physician roles and responsibilities in the healthcare system themes across all assessors, and suggested improvements in the Communicator/negotiation and Scholar/knowledge base/career development themes.
Discussion
To our knowledge, this study is the first to measure leadership as a competency in junior resident physicians across multiple programmes with responses from patients/family members, nurses, attending physicians, peer resident physicians, other allied healthcare providers and resident self-assessments. The main findings from this study were threefold. First, the LEADER tool was shown to have aspects of preliminary validity evidence and is feasible for use in three different residency training programmes. Second, the construct of the CanMEDS Leader role can be measured by MSF assessment of PLS and the APA subscales. Third, qualitative feedback identifies QI opportunities in the domains of ‘self’, ‘system’ or intersections of both. We also found our MSF tool to be feasible using a web and email-based tool administration, which was well-received by assessor groups.
Need for patient/family-centred and team-centred resident physician assessments
Our findings suggest educators should pay more attention to assessment by patients/families.8 In pursuit of quality improvement and a patient-centred healthcare system, patients ought to be elevated to the status of patient leaders whose assessments should shape the training of early-stage resident physicians.8
Contemporaneous feedback from multiple sources facilitates nuanced patient and team-driven communication about a resident’s strengths and weaknesses creating a culture of patient-centred healthcare. In turn, the resident receives feedback to inform future ‘self’ growth and ‘system’ transformation. This feedback aligns with the intrinsic CanMEDS roles, especially the Leader role. Few leadership development programmes focus on personal growth and self-awareness4; the LEADER tool attempts to leverage the humility of junior residency to drive personal and organisational realisation.
Our data corroborate the controversial claim that ‘no single professional group can assess clinical competence’.36 Different assessor groups observe different aspects of healthcare delivery through unique personal and professional lenses. This rich tapestry of feedback becomes valuable to a physician in training who benefits from a plurality of perspectives on different domains of care often invisible to the attending physician. Frich et al’s review of leadership development programmes noted current programmes focus more on skills of individual physicians rather than on increasing our capacity to collaborate across professional groups.4 The LEADER tool makes efforts to open the lines of communication between physicians and non-physicians during a trainee’s formative years.
In our study, residents did not choose their own assessors. This selection bias has been previously described in the literature as a problem with MSF.37 38 In our study, assessments occurred based on assigned block rotation schedules where trainees had no influence over their complement of assessors. Removing selection bias is important not only to protect the fidelity of feedback generated for the trainee but also by not cherry picking only ‘happy patients’ or ‘like-minded’ colleagues. All stakeholders deserve to be heard, perhaps especially so those that physicians would not naturally select to give feedback.
Interestingly, we found that residents self-assessed themselves lower than other assessor groups. This finding warrants further exploration, particularly if they believe they have a higher performance ceiling than colleagues and patients expect. Notably, communication was universally assessed by residents as a performance strength, however other assessors emphasised communication as an area for improvement. This discrepancy is a critical flag. Communication is not only important in the therapeutic relationship, but also as a conduit between the patient experience and health system transformation.
Contrary to the literature, we found physicians were not significantly more lenient than their non-physician colleagues when assessing resident physicians.23 24 Also contrary to the literature, we found nurses in our study were more likely to be ‘hawks’ than other non-physician assessor groups who were more likely to be ‘doves’.39 The notion of hawks versus doves warrants further exploration—especially when measuring leadership in residents. Given that nurses were all women in our sample, further examination is required to assess the interaction between gender and leadership assessment.
Our qualitative data highlighted personal attributes/PLS such as kindness, sense of humour and relatability, as a strength for cognitive specialty residents (internal medicine and paediatrics), but not surgical residents. Meanwhile surgical residents, but not cognitive specialty residents, were considered strong in Professional/physician roles and responsibilities in the healthcare system. These findings shed light on how patients and teams differentially experience specialty care provision.
These qualitative differences in resident attributes across specialties may be dismissed as peculiarities of specialty-specific subcultures in healthcare. For example, residents from medical specialties show higher levels of neuroticism while residents from surgical specialties show higher levels of extraversion and openness to experience.40 Additionally, medical students interested in procedural specialties tend to be more authoritative compared with those entering cognitive specialties.41
Such specialty-specific leadership subcultures in healthcare are significant given that they may be driven by selection and socialisation42 43 and have been found to impact patient safety.44 45 More research is needed to understand which leadership characteristics are specialty-specific performance-drivers. While specialties will always have different subcultures, some of those differences are ultimately important to outcomes whereas others are merely vestigial disciplinary legacies. Exposing any entrenched cultural silo to external feedback creates opportunity for introspection. For example, the LEADER tool might allow explicit consideration for which specialty-specific leadership characteristics could improve patient care and teamwork. By starting with junior doctors, we posit this tool may serve as a first step to examine prevailing cultures in healthcare systems.46 47
Utility and feasibility of the LEADER tool
Leadership is a complex construct that may be widely observed but poorly understood and measured.9 10 Regardless of their discipline and where they are on the continuum of training, physicians play a leading role in the healthcare system and key stakeholders such as other physicians, healthcare providers and patients can add value to assessing leadership in the healthcare setting. The LEADER tool can help with disentangling the multifaceted construct of leadership.
Though the qualitative data were intended to assess the Leader role, we found non-physicians also provided qualitative feedback about other intrinsic CanMEDS roles (eg, the Scholar role) which corroborated attending physician feedback. This reinforces the value of non-physician team experience, clinical skill and training in providing complementary feedback. Importantly, the LEADER tool could be used, without modifications, for all assessor groups including patients and allied healthcare providers alike. Contrary to the literature, we did not need to vary items to match the expected capability and ability of the different assessor groups.24 36
Preliminary validity evidence for the LEADER tool
While we could not establish validity evidence for every aspect of the LEADER tool, we were, by way of its development and application able to provide a basis for preliminary validity evidence. This serves as a jumping off point for future research and the application of the LEADER tool as a measure of leadership in junior residents. Furthermore, the LEADER tool can be used for peer benchmarking and hence for identifying outliers whose leadership competencies may be significantly less than expected.
We anticipate that with further application as residency programmes continue to move to a CBME paradigm in Canada and globally, the LEADER tool could be applied to identify individuals who appear refractory to leadership development; whose leadership competencies fall so far short of the mark that action can be taken to protect patient or team safety (physical or psychological); or conversely, to identify personal development needs that can then be associated with significant improvements in the scores over time.
Although the PLS and APA subscales only accounted for 58% of the variance, we believe that the remaining variance may be accounted for by the personal attributes of junior residents as well as their management and administration styles. Healthcare systems are dependent on the interaction between ‘self’ and ‘system’; end user and team feedback about patient care could sharpen the resident physician’s quality improvement mindset if they critically reflect and subsequently engage in quality improvement processes aimed at improving ‘self’ or ‘system’.
The LEADER tool effectively discriminated between assessor groups thereby demonstrating ‘relationships with other variables’ validity evidence.26 For example, nurses rated residents more severely than other assessor groups regarding PLS. We hypothesise that nurses see residents in less manicured, and more routine interactions with patients; consequently, nurses may have more data points, and higher fidelity data points arising from a more practical and therefore critical lens.
Future research
Future research should investigate why residents self-assessed themselves lower than other assessor groups and why communication was an area of importance for assessment and improvement. Further empirical work is warranted to assess effectiveness of communication education interventions on improved LEADER assessments. The assessment of leadership in medical education and how it relates to QI/PS in the healthcare system would also be advantageous. This would add to the validity evidence established for the LEADER tool.
Additionally, the LEADER tool does not intend to assess all aspects of the Leader construct. As a result, there remains a gap how to measure other concepts relating to the Leader role. These are: complexity of systems, consideration of justice, effective committee participation, health human resources, information technology for healthcare, leading change, management of personnel, organising, structuring, budgeting, and financing, physician remuneration, practice management and supervising others. These concepts associated with the Leader role may be more appropriate for assessment of more senior resident physicians or practising physicians.
Limitations
There were several limitations to this study. First, a generalisability theory study could not be used because we were not able to control for the number of assessor–occasion combinations, which impaired our confidence to extrapolate the results of this study.22 24 Second, we were unable to establish divergence and congruence with other measures to contribute additional validity evidence for the use of the LEADER tool. Third, we do not know if the feedback provided to the residents led to any changes in their practice (consequential validity)26 since we do not have data on impact and application of the tool. Fourth, we did not determine our assessor’s perceptions of the LEADER tool using Kirkpatrick’s hierarchy to determine the effectiveness of our tool. Fifth, while we assess presence of certain competencies, we cannot determine the degree of clinical competence, which requires a continuum of learning and the completion of EPAs that need to be assessed as part of CBME. Last, the use of purposive convenience sampling could have led to researcher bias, since subjective assumptions may have been made when choosing participants and those participants most interested were more likely to participate.
Conclusion
Leaders in medicine including patients have called for reform in healthcare in response to challenges in the system and the need for improvement in patient care.
For healthcare organisations that aspire to be proactive rather than reactive, we make three recommendations to develop leadership competence in junior physicians:
Teach and assess leadership early in training
There has been an increased drive to develop and deliver leadership education9 yet adequate assessment remains a challenge.4 Leadership should be taught and assessed as early in training as possible and continue through physicians’ career so they can take on important leadership roles within the healthcare system.
Empower patients to lead and transform training and care by evaluating doctors
Resident physicians ought to recognise, respect, and value differences in patient perspectives and experiences. By embracing pluralism in healthcare, resident physicians and hospitals have the potential to adapt as individuals and systems to provide inclusive and equitable care. However, this frameshift requires patients be included directly in resident physician assessment and early on in their training.
Activate frontline care providers to be health leaders by embracing patient and team feedback
Capturing patient, nurse and allied healthcare provider feedback can actively assist with system changes demonstrating the importance of ‘self’ and ‘system’, and the intersection of both as a critical aspect of leadership development. This is particularly important for residency education because becoming a physician requires leadership, management and administration but also the transfer of competencies, such as communication and collaboration, as active participant-architects who shape the healthcare system.
Data availability statement
Data are not available due to ethical restrictions.
Ethics statements
Patient consent for publication
Ethics approval
Study approval was obtained from the University of Calgary Conjoint Health Research Ethics Board (CHREB) (Ethics ID: REB13-1253).
Acknowledgments
We would like to thank Trenton Smith, Kathy Tobler, Jeff Schaefer, Jennifer Cowles and all participants.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @AleemBharwani, @academialiya
Contributors AB helped to design the study, recruit participants, coordinate data collection, interpret the results, and write and review the manuscript. DS helped to collect the data and analyse the results of the study, as well as write up findings. Part of this study was to satisfy DS’s honours thesis project for the Bachelor of Health Sciences (BHSc) program. EOP helped to coordinate data collection, interpret the results, and write and review the manuscript. CGB helped to coordinate data collection, interpret the results, and write and review the manuscript. LAM helped to coordinate data collection, interpret the results, and write and review the manuscript. AK designed the study, supervised DS and wrote the final manuscript.
Funding This work was supported by the University of Calgary Research Grants Committee (URGC) Seed Grant #10001359 and the Office of Surgical Research, Department of Surgery Grant #10008765.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.