Article Text

Download PDFPDF
Changing estimates of leadership ability before a programme: retrospective self-assessments and response-shift bias


Background Most evaluations of clinical leadership development programmes rely on self-assessments. Self-assessments are vulnerable to response-shift bias. Using retrospective then-tests may help to avoid this bias.

In this study, we investigate whether post-programme then-tests (retrospective self-assessments) are more sensitive to change in clinical leadership development programme participants than traditional pre-programme pre-tests when paired with post-test self-assessments.

Methods 17 healthcare professionals participated in an 8-month single-centre multidisciplinary leadership development programme. Participants completed prospective pre-test, retrospective then-test and traditional post-test self-assessments using the Primary Colours Questionnaire (PCQ) and Medical Leadership Competency Framework Self-Assessment Tool (MLCFQ). Pre–post pairs and then–post pairs were analysed for changes using Wilcoxon signed-rank tests and compared with a parallel multimethod evaluation organised by Kirkpatrick levels.

Results A greater number of significant changes were detected using then-test pairs than pre-test pairs for both the PCQ (11 of 12 vs 4 of 12 items) and MLCFQ (7 of 7 vs 3 of 7 domains). The multimethods data showed positive outcomes at all Kirkpatrick levels.

Conclusions In ideal circumstances, both pre-test and then-test evaluations should be conducted. We cautiously suggest that if only one post-programme evaluation can be conducted, then-tests may be appropriate means of detecting change.

  • clinical leadership
  • competencies
  • research
  • measurement
  • leadership assessment

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.