Background: Supervisors of some student selected components (SSCs) may appear to give higher grades than others. It is not known if feedback can influence the behaviour of supervisors in the grades they award. We have introduced feedback letters in our institution.
Aims: (1) To assess the feasibility of objectively identifying SSCs where grades awarded are consistently higher or lower than the average; (2) To assess the effect of feedback on the grades awarded by supervisors of SSCs.
Methods: The breakdown of SSC grades was examined over four consecutive years, before and after feedback letters were introduced in 2005. The grades awarded globally, and in five individual SSCs, were compared using the chi(2) goodness-of-fit test.
Results: (1) Individual SSCs were identified which awarded grades that were consistently different from the average. (2) Overall grades awarded in 2003/04 and 2004/05 (before feedback) were similar (chi2=0.37, df=2, p=0.83). Likewise, overall grades awarded in 2005/06 and 2006/07 (after feedback) were similar (chi2=1.72, df=2, p=0.42). Comparison of 2003/04 with 2005/06 (chi2=16.0, df=2, p<0.001), and 2006/07 (chi2=26.6, df=2, p<0.001), and of 2004/05 with 2005/06 (chi2=13.5, df=2, p=0.001), and 2006/07 (chi2=23.7, df=2, p<0.001), revealed highly significant differences. The grades awarded after feedback were higher than the grades awarded before feedback.
Conclusions: The chi2 goodness-of-fit test may be used to identify individual SSCs where the grades awarded are different from the average, although the interpretation of the results thus obtained is fraught with difficulty. Our data also suggest that it is possible to influence assessors in the grades they award.