The following is a Conversation Kick-starter guest blog post contributed by e-Council member, Dr. Ellie Hothersall pertaining to the article in the September 2015 issue of Medical Education :
A curious case of the phantom professor: mindless teaching evaluations by medical students
When I attend a teaching session of any kind, I try to be diligent with feedback. In fact, I try to give feedback whenever it’s asked for, although in these days of online shopping it’s all getting a bit much. However, there are times when I perhaps pay less attention than I should, particularly if the session was unremarkable, and maybe I too have been guilty of ticking 4/5 for everything and running out of the room. I know I worry that other people do that to me.
Now my anxiety is justified.
I do not know how Uijtdehaage and O’Neal came up with their ingenious study idea, but it certainly deserves some sort of award for cleverness. When asking students to evaluate a series of lectures, they included a fictional lecturer and lecture in the middle of the list. 66% of students still gave the lecturer a rating (bear in mind that “n/a” was an option), with this falling to 49% when a photograph of the fictional lecturer was provided.
Let’s just think about that: very nearly half of students rated a tutor’s session despite it not existing, even with the visual reminder of the tutor’s face. Some students even left free text comments (positive and negative).
Part of me finds this whole concept utterly impossible – how could so many students just not know who had taught them? Why would they make up feedback? And part of me knows how easy that would be to do: medical school teaching is a carousel of tutors and sessions; often you never see that face again, and some sessions are unmemorable.
Why does this matter? Well, for one thing we place a lot of reliance on student feedback. To quote the paper “A thoughtful and conscientious approach to [teaching evaluations] by medical students is critical, as their evaluations may impact both the careers of faculty members and the quality of instruction.” And yet the practice is evidently flawed. Similarly, it can be difficult to convince colleagues to change their practice if the feedback received is untrustworthy. In a letter on “feedback fatigue”, prompted by a similar type of event, Fysh tells us that in Wales fire alarms in public buildings have been replaced with spinklers as “no one pays attention to fire alarms any more” – when did a warning signal is dismissed as noise, it’s time to change.
In the United Kingdom students are required to participate in the evaluation of the teaching they recieve as part of Good Medical Practice. I had been thinking that it was unprofessional not to leave feedback after teaching. Now I’m wondering whether the reverse is worse?
So here are my questions for discussion:
Feedback collected immediately after a teaching session obviously does not suffer from the “phantom professor” problem. But is the feedback provided then more valuable? Less?
Does feedback or evaluation of teaching need reflection time built in? Would the “mindful evaluation” improve with discussion? Uijtdehaage and O’Neal propose “prospective evaluation” through designated students – would that help or just add to the burden for some students?
If the lack of impact of that evaluation, as aulluded to by Uijtdehaage and O’Neal, means students don’t take it seriously, how do we fix that?
If this is the death knell of the Likert scale, what should we use instead?
Fysh, T. Feedback Fatigue The Clinical Teacher Volume 8, Issue 4, pg 288
Dishman, L. Retailers: Your Surveys Are Making Customers Suffer http://www.forbes.com/sites/lydiadishman/2014/03/07/retailers-your-surveys-are-making-customers-suffer/ Accessed 16th July 2015
General Medical Council. Medical students: professional values and fitness to practise http://www.gmc-uk.org/education/undergraduate/professional_behaviour.asp Accessed 16th July 2015