Kudos: A new way for authors in Medical Education and The Clinical Teacher to increase readership and impact

The following is a guest blog post from Rosie Hutchinson, Senior Journals Publishing Manager at Wiley.

Kudos is a web-based service that helps authors explain, enrich and share their published work for greater readership and improved impact. It also provides direct access to a publication dashboard, enabling authors to measure the effects of their actions across a wide range of metrics.

Wiley’s recent partnership with Kudos now renders the service free for all authors publishing in #MedEdJ and #ClinTeach. Registering with Wiley Author Services and opting into the mailing list will streamline the process. Notification emails sent to those registered with Wiley Author Services contain a link for them to “claim” their article in Kudos. Authors who are either unregistered or published their article prior to 2014 can claim authorship in Kudos by searching for articles by DOI, article title or author name.

Once authors have claimed their articles, they are led through the following steps: 

  • Explain articles by adding lay summaries and highlighting what makes the work important. 
  • Enrich articles by adding links to related resources that put the research into context. 
  • Share via email and social media, while Kudos links across search engines and subject indexes

All of this produces many benefits for authors with a very limited time investment required:

  • Discoverability and Impact – Increases the likelihood of their articles being found, read, and cited.
  • Publication Metrics – Provides direct access to usage, citations, and altmetrics for their articles.
  • Networking – Encourages interactions that build relationships and visibility within their communities.

You can have a look at this introductory video and blog post for more info.

Conversation Kick-starter #MedEdJ: A curious case of the phantom professor

The following is a Conversation Kick-starter guest blog post contributed by e-Council member, Dr. Ellie Hothersall pertaining to the article in the September 2015 issue of Medical Education :

A curious case of the phantom professor: mindless teaching evaluations by medical students

When I attend a teaching session of any kind, I try to be diligent with feedback. In fact, I try to give feedback whenever it’s asked for, although in these days of online shopping it’s all getting a bit much. However, there are times when I perhaps pay less attention than I should, particularly if the session was unremarkable, and maybe I too have been guilty of ticking 4/5 for everything and running out of the room. I know I worry that other people do that to me.

Now my anxiety is justified.

I do not know how Uijtdehaage and O’Neal came up with their ingenious study idea, but it certainly deserves some sort of award for cleverness. When asking students to evaluate a series of lectures, they included a fictional lecturer and lecture in the middle of the list. 66% of students still gave the lecturer a rating (bear in mind that “n/a” was an option), with this falling to 49% when a photograph of the fictional lecturer was provided.

Let’s just think about that: very nearly half of students rated a tutor’s session despite it not existing, even with the visual reminder of the tutor’s face. Some students even left free text comments (positive and negative).

Part of me finds this whole concept utterly impossible – how could so many students just not know who had taught them? Why would they make up feedback? And part of me knows how easy that would be to do: medical school teaching is a carousel of tutors and sessions; often you never see that face again, and some sessions are unmemorable.

Why does this matter? Well, for one thing we place a lot of reliance on student feedback. To quote the paper “A thoughtful and conscientious approach to [teaching evaluations] by medical students is critical, as their evaluations may impact both the careers of faculty members and the quality of instruction.” And yet the practice is evidently flawed. Similarly, it can be difficult to convince colleagues to change their practice if the feedback received is untrustworthy. In a letter on “feedback fatigue”, prompted by a similar type of event, Fysh tells us that in Wales fire alarms in public buildings have been replaced with spinklers as “no one pays attention to fire alarms any more” – when did a warning signal is dismissed as noise, it’s time to change.

In the United Kingdom students are required to participate in the evaluation of the teaching they recieve as part of Good Medical Practice. I had been thinking that it was unprofessional not to leave feedback after teaching. Now I’m wondering whether the reverse is worse?

So here are my questions for discussion:

Feedback collected immediately after a teaching session obviously does not suffer from the “phantom professor” problem. But is the feedback provided then more valuable? Less?

Does feedback or evaluation of teaching need reflection time built in? Would the “mindful evaluation” improve with discussion? Uijtdehaage and O’Neal propose “prospective evaluation” through designated students – would that help or just add to the burden for some students?

If the lack of impact of that evaluation, as aulluded to by Uijtdehaage and O’Neal, means students don’t take it seriously, how do we fix that?

If this is the death knell of the Likert scale, what should we use instead?

 

References:

Fysh, T. Feedback Fatigue The Clinical Teacher Volume 8, Issue 4, pg 288

Dishman, L. Retailers: Your Surveys Are Making Customers Suffer http://www.forbes.com/sites/lydiadishman/2014/03/07/retailers-your-surveys-are-making-customers-suffer/ Accessed 16th July 2015

General Medical Council. Medical students: professional values and fitness to practise http://www.gmc-uk.org/education/undergraduate/professional_behaviour.asp Accessed 16th July 2015

Invitation to contribute to the 50th Anniversary special issue of Medical Education #MedEdJ

Deadline 30 September 2015

To celebrate the 50th anniversary of the publication of Medical Education we are pleased to announce a special issue aimed at demonstrating the liveliness of our field. Part of the inspiration for this special edition came from a desire to discount the claims made in a recent article that described academics as:

   ‘Serious people with no social lives’

This is certainly not true of medical educationalists, as there is a rich history of lightheartedness and good humour in our realm, as evidenced by a paper published in Medical Education in 1999 entitled, The Objective Santa Christmas Examination (OSCE). The paper was conceived during a convivial evening at a meeting of the Association for the Study of Medical Education (ASME).    That papers were being planned during such an evening might, come to think of it, be used to support the above quote.  Nonetheless, we are of the firm belief that many interesting ideas arise in those contexts, but never see the light of day.  Sometimes that is for the better as sober second thought settles in, but sometimes it is simply because the participants in the discussion do not pursue the idea due to lack of clear opportunities for dissemination.  Just as good comedians cast a light on society that has the potential to reveal new insight, we think there is much to be gained from taking a more whimsical look at health professional education and providing a voice to the playful demons within the academics amongst us.

To that end, we would like to invite you to submit the papers you’ve always wanted to write.   More specifically, we’re looking for papers on quirky topics of relevance to the lives and practices of health professional educators and trainees.  Novel empirical work is welcomed, but so are well‐ crafted essays that place a new and generative spin on well known topics/findings.  Priority will be given to papers that amuse and entertain. Articles should be submitted by clicking ‘write’ at www.mededuc.com, should be no more than 1,500 words, and must be received by the 30th of September 2015.  Queries can be sent to Trudie Roberts (Guest Editor) or Kevin Eva (Editor‐in‐chief) by emailing med@mededuc.com.

Reference:  1 Wood D, Roberts T, Bradley P, Lloyd D, O’Neil P. ‘Hello, my name is Gabriel, I am the house officer, may I examine you?’ or The Objective Santa Christmas Examination (OSCE). Medical Education 1999;33:915‐919.

The Purple Sticker Pilot Project

Hi all,

My name is Andrew Tabner and I’m this year’s e-Editor Intern for Medical Education and The Clinical Teacher. I’m delighted to be working with Dr Joshua Jacobs, the e-Editor, and the rest of the team at the journals; you’ll be hearing more from me over the coming months.

I’d like to take this opportunity to fill you in on an exciting new initiative we’re running – the “Purple Sticker Pilot Project.”

ASME (one of the journals’ “parent” organisations) recently held its annual scientific meeting in Edinburgh, UK; this focused on innovations and developments on the leading edge of medical education.

Session chairs at the conference were asked to make note of “hot topics” – these were papers or sessions that generated significant discussion or debate, ran out of time for questions, presented a particularly interesting new development etc. etc. They identified these sessions with a purple sticker in their programmes to highlight them to the organisers for future consideration.

Twenty-seven sessions earned a purple sticker and a few of these will be highlighted for continued discussion in this online forum. This discussion will hopefully include the speakers/authors of the session, and perhaps the session chairs, and will make reference to previously published supporting literature from the two journals. We’re currently working with the journals’ Editors in Chief to identify relevant articles and there will be future guest blog posts about the issues raised.

So watch this space and keep an eye on the journals’ Twitter hashtags (#MedEdJ and #ClinTeachJ) and accounts (@ClinicalTeacher and @asmeofficial) for further updates. We’d love you to join us to continue some of the fantastic discussions that took place at the ASME meeting, whether you were present for the initial meeting or not!

-A

@andrewtabner

Conversation Kickstarter #MedEdJ: Measuring Cognitive Load

This guest blog post was contributed by e-Council member Robert R. Cooney, MD, MSMedEd, RDMS, FAAEM, FACEP.

How do humans learn?  This question has prompted the exploration of multiple theories within the psychological and educational literature.  Cognitive load theory (CLT) is one such framework.  CLT posits that learning (and performance) becomes impaired when the learner exceeds his/her working memory capacity.  CLT also identifies three types of “load” that may arise during training: intrinsic (task complexity), extraneous (instructional design), and germane (cognitive resource use).

In this month’s issue of Medical Education, Haji et al set out to apply CLT to simulation, specifically trying to determine if the use of standardized scales (subjective rating of mental effort and simple reaction time) are sensitive measures of intrinsic cognitive load when novice learners are faced with a simple versus complex surgical knot tying exercise. The link to the article click: Measuring cognitive load: performance, mental effort and simulation task complexity.

Reading through their design, I am impressed by the complexity of measuring cognitive load.  The authors explain that, “the theory has recently come under intense scrutiny because many existing measures of CL cannot differentiate between the three types of load.”  From the standpoint of this study, the investigators attempted to hold extraneous and germane load constant, thus measuring only intrinsic load.

While this can be done from a research perspective, I don’t believe that it is as easily controllable in the “messiness” that is the medical classroom.  What this investigation does offer is an approach to considering how CL can be moderated in a simulated setting.

My questions for you:

Have you incorporated CL into instructional design?

If so, how have you modified instructional delivery to decrease extraneous load?

What about intrinsic load?

Please share with us your thoughts!

#ClinTeach Conversation Kickstarter | Standard setting in the age of CBME

This month’s Clinical Teacher’s Toolbox is by Dr. Richard Hays of Tasmania, Australia. Dr. Hays’ summary of standard setting procedures is most definitely a worthy addition to most graduate studies courses on assessment, and yet, as an educator whom is working on competency-based medical education (CBME) projects, this article made me think:

What and how are we setting standards in the age of CBME?

As the world moves towards outcome-based medical education, this is a critical question to consider. Traditional psychometric methods (including those mentioned in Dr. Hays’ paper: Ebel, Angoff, Hofstee) require population-level data to ‘set standards’. And in the K-12, university-level and possibly undergraduate medical education realms where we have traditionally used these techniques for setting standards, these have been helpful in determining passing grades.

That said, these methods primarily are based on some key assumptions – most specifically on the idea that a post-hoc analysis of data on a given population (or a aggregate analysis of historical data of similar populations) might be used to set a standard. And perhaps, if there is no clear outcome, this the best way to do this… But in the age of outcomes, where do these standard setting procedures now belong?

The achievement of the outcome IS the standard, no?

Please share with me your thoughts below! I have pondered this question quite extensively and I’m not sure what the answer is…

Conversation Kickstarter #MedEdJ: Can spaced testing help students manage their study time and improve their marks?

Contributed by e-Council member Karen Scott, PhD

Can spaced testing help students manage their study time and improve their marks?

Kerdijk and colleagues have compared the effects of cumulative and end-of-term assessment on medical students’ ability to manage their self-study time, as well as their assessment results.1 The authors found using spaced testing throughout the course helped students devote regular time to self-study, avoid procrastination and increase their overall study time. In contrast, students preparing for end-of-term assessment tended to cram in the final two weeks of the course.

When comparing student marks between the two assessment formats, the only difference was found in marks for content studied in the final two weeks of the course, with higher marks gained by students sitting the spaced tests. The authors speculated that these students had more cognitive resources available for learning in the final two weeks because they were not cramming for the final exam.

What do you think are the pros and cons of cumulative assessment compared with end-of-term assessment?

Do you think long-term retention of knowledge could be improved through cumulative rather than end-of-term assessment?

In what other ways could teachers help students manage their study time throughout a course?

 

  1. Kerdijk W, Cohen-Schotanus J, Mulder FBF, Muntinghe FLH and Tio RA. Cumulative versus end-of-course assessment: effects on self-study time and test performance. Medical Education 2015: 49: 709–716 doi:1111/medu.12756

Link to article click here

Link to related blog post click here

 

Presence with purpose: attitudes of people with developmental disability towards health care students

By Ginette Moores, Natalie Lidster, Kerry Boyd, Tom Archer, Nick Kates and Karl Stobbe

Click here to link to article.

Context

Early clinical encounters help medical and nursing students build professional competencies. However, there is a necessary emphasis on patient autonomy and appropriate consent. Although most individuals do not object to student involvement in clinical encounters, there are occasions when personal preference and health care education conflict. Many studies have evaluated patient attitudes towards students across a variety of specialties.

Objectives

The purpose of this study was to identify the attitudes, comfort level and preferences of individuals with developmental disability (DD) towards the presence and involvement of medical and nursing students during clinical encounters.

Methods

Adults with DD across the Hamilton–Niagara region were invited to participate. Focus groups were moderated by two students with a health care facilitator and physician-educator. Participants were provided with focus group questions in advance and encouraged to bring communication aids or care providers. Data were analysed for emerging themes by two independent reviewers, who then compared results.

Results

Twenty-two individuals participated. A wide range of opinions were expressed. Some participants were positively disposed towards students and perceived better care and improved communication with the health care team. Others were indifferent to students in a clinical setting. The final group was opposed to the presence of health care students, expressing confusion over their role and purpose, uneasiness with deviation from the norm, and concerns about confidentiality. Informative introductions with confidentiality statements and the presence of a supervising clinician were seen as helpful.

Conclusions

People with DD are affected by above-average health care needs. Their input into health care planning has been limited. Their opinions on health care learners varied considerably. Themes relating to attitudes, comfort and preferences about student involvement provide impetus for health care training practices that promote person-centred approaches and improvements to the quality of care received by people with DD.

DOI: 10.1111/medu.12751

Cumulative versus end-of-course assessment: effects on self-study time and test performance

By Wouter Kerdijk, Janke Cohen-Schotanus, B Florentine Mulder, Friso L H Muntinghe and René A Tio

Link to this article here.

Context

Students tend to postpone preparation for a test until the test is imminent, which raises various risks associated with ‘cramming’ behaviours, including that for suboptimal learning. Cumulative assessment utilises spaced testing to stimulate students to study more frequently and to prevent procrastination. This randomised controlled study investigated how cumulative assessment affects time spent on self-study and test performance compared with end-of-course assessment.

Methods

A total of 78 undergraduate medical students in a Year 2 pre-clinical course were randomly assigned to either of two conditions. Students in the cumulative assessment condition were assessed in weeks 4, 8 and 10. Students in the end-of-course assessment condition were assessed in week 10 only. Each week, students reported the number of hours they spent on self-study.

Results

Students in the cumulative assessment condition (n = 25) spent significantly more time on self-study than students in the end-of-course assessment condition (n = 37) in all weeks of the course except weeks 5, 9 and 10. Overall, the cumulative assessment group spent 69 hours more on self-study during the course than did the control group, although the control group spent 7 hours more in studying during the final week of the course than did the cumulative assessment group. Students in the cumulative assessment condition scored slightly higher on questions concerning the content of the last part of the course.

Conclusions

Cumulative assessment encourages students to distribute their learning activities over a course, which leaves them more opportunity to study the content of the last part of the course prior to the final examination. There was no evidence for a short-term effect of cumulative assessment on overall knowledge gain. We hypothesise that larger positive effects might be found if retention were to be measured in the long term.

DOI: 10.1111/medu.12756

How faculty members experience workplace-based assessment rater training: a qualitative study

By Jennifer R Kogan, Lisa N Conforti, Elizabeth Bernabeo, William Iobst and Eric Holmboe

Click here to link to article.

Context

Direct observation of clinical skills is a common approach in workplace-based assessment (WBA). Despite widespread use of the mini-clinical evaluation exercise (mini-CEX), faculty development efforts are typically required to improve assessment quality. Little consensus exists regarding the most effective training methods, and few studies explore faculty members’ reactions to rater training.

Objectives

This study was conducted to qualitatively explore the experiences of faculty staff with two rater training approaches – performance dimension training (PDT) and a modified approach to frame of reference training (FoRT) – to elucidate how such faculty development can be optimally designed.

Methods

In a qualitative study of a multifaceted intervention using complex intervention principles, 45 out-patient resident faculty preceptors from 26 US internal medicine residency programmes participated in a rater training faculty development programme. All participants were interviewed individually and in focus groups during and after the programme to elicit how the training influenced their approach to assessment. A constructivist grounded theory approach was used to analyse the data.

Results

Many participants perceived that rater training positively influenced their approach to direct observation and feedback, their ability to use entrustment as the standard for assessment, and their own clinical skills. However, barriers to implementation and change included: (i) a preference for holistic assessment over frameworks; (ii) challenges in defining competence; (iii) difficulty in changing one’s approach to assessment, and (iv) concerns about institutional culture and buy-in.

Conclusions

Rater training using PDT and a modified approach to FoRT can provide faculty staff with assessment skills that are congruent with principles of criterion-referenced assessment and entrustment, and foundational principles of competency-based education, while providing them with opportunities to reflect on their own clinical skills. However, multiple challenges to incorporating new forms of training exist. Ongoing efforts to improve WBA are needed to address institutional and cultural contexts, and systems of care delivery.

DOI: 10.1111/medu.12733