Conversations with Medical Education – revamp and expansion

Featured

The blog cum discussion board for the leading journal in research in medical education (Medical Education) has undergone a makeover and expansion. The blog will now encompass articles from The Clinical Teacher; the journal for practical application and best practices for teachers of and in clinical practice. Additionally, information from the parent organization of both journals (ASME) will be included.

The reason for this change is to provide added opportunity for the readerships of both journals to interact with each other and spur more ideas that will ultimately lead to better research and better practice in medical and clinical education. You will see a few important features on the blog, including:

  • New title: For those of you who pay attention to detail, you will notice a one-word change in the title of the blog. The web address (URL) remains the same at www.mededucconversations.com, but the title has changed from ‘Conversations with Medical Education’ to ‘Conversations in Medical Education’. The key change from ‘with’ to ‘in’ shows the blog is no longer simply conversations with one journal.
  • New look: The color scheme and layout has changed to reflect the broader coverage of the blog that includes information streaming in from the two journals and ASME.
  • New categories: Blog posts are now organized via a ‘table of contents’ of sorts, that allows you to easily find the type of information you’re looking for. You can still use the Search box in the upper right to search for specific keywords. Categories are listed below.
  1. Article Abstracts: This category links to all blog posts that simply contain the title and abstract of the article in the journal issues.
  2. Comments on articles: This category links to guest blog posts on specific articles in the journals. Guest blog posts are provided by invited authors and the e-Council for the journals on a monthly basis.
  3. Meetings: This category links to posts about virtual meetings, such as journal clubs held via Twitter, and to meetings facilitated by ASME.
  4. Virtual Issues: This category links to posts that draw together articles around a common theme from across multiple issues or volumes within a journal, or across the two journals.

As always, the benefit to you, the reader, is best when you join in active conversation with your colleagues via the ‘Comment’ bubble that is next to the title of every post. Comments are moderated to keep spam and advertisements out of the way, but this should not impede a free flow of ideas around different approaches, clarifying questions, constructive feedback, and sharing of experiences relating to the topics in the post. Go ahead; join the conversation.

Joshua Jacobs, MD

e-Editor

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

To read on journal website, click here.

Stephanie Montagne, Anja Rogausch, Armin Gemperli, Christoph Berendonk, Patrick Jucker-Kupper, and Christine Beyeler

Objectives

The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted.

Methods

Retrospective analyses of adapted mini-CEX forms (trainers’ and students’ assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables.

Results

A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer–student pairs most often identified LNs in the domains of ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation/efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were ‘physical examination’ (49%) and ‘history taking’ (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs.

Conclusions

Although trainer–student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students’ LNs raise questions about how the full potential of the mini-CEX as not only a ‘diagnostic’ but also an ‘educational’ tool can be exploited.

Article DOI: 10.1111/medu.12513

Reducing the number of options on multiple-choice questions: response time, psychometrics and standard setting

To read on journal website, click here.

Stephen Schneid, Chris Armour, Yoon Soo Park, Rachel Yudkowsky, and Georges Bordage.

Objectives

Despite significant evidence supporting the use of three-option multiple-choice questions (MCQs), these are rarely used in written examinations for health professions students. The purpose of this study was to examine the effects of reducing four- and five-option MCQs to three-option MCQs on response times, psychometric characteristics, and absolute standard setting judgements in a pharmacology examination administered to health professions students.

Methods

We administered two versions of a computerised examination containing 98 MCQs to 38 Year 2 medical students and 39 Year 3 pharmacy students. Four- and five-option MCQs were converted into three-option MCQs to create two versions of the examination. Differences in response time, item difficulty and discrimination, and reliability were evaluated. Medical and pharmacy faculty judges provided three-level Angoff (TLA) ratings for all MCQs for both versions of the examination to allow the assessment of differences in cut scores.

Results

Students answered three-option MCQs an average of 5 seconds faster than they answered four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no significant differences in item difficulty and discrimination, or test reliability. Overall, the cut scores generated for three-option MCQs using the TLA ratings were 8 percentage points higher (p = 0.04).

Conclusions

The use of three-option MCQs in a health professions examination resulted in a time saving equivalent to the completion of 16% more MCQs per 1-hour testing period, which may increase content validity and test score reliability, and minimise construct under-representation. The higher cut scores may result in higher failure rates if an absolute standard setting method, such as the TLA method, is used. The results from this study provide a cautious indication to health professions educators that using three-option MCQs does not threaten validity and may strengthen it by allowing additional MCQs to be tested in a fixed amount of testing time with no deleterious effect on the reliability of the test scores.

Article DOI: 10.1111/medu.12525

Validating relationships among attachment, emotional intelligence and clinical communication

To read on journal website, click here.

M Gemma Cherry, Ian Fletcher, and Helen O’Sullivan.

Context

In a previous study, we found that emotional intelligence (EI) mediates the negative influences of Year 1 medical students’ attachment styles on their provider–patient communication (PPC). However, in that study, students were examined on a relatively straightforward PPC skill set and were not assessed on their abilities to elicit relevant clinical information from standardised patients. The influence of these psychological variables in more demanding and realistic clinical scenarios warrants investigation.

Objectives

This study aimed to validate previous research findings by exploring the mediating effect of EI on the relationship between medical students’ attachment styles and their PPC across an ecologically valid PPC objective structured clinical examination (OSCE).

Methods

Year 2 medical students completed measures of attachment (the Experiences in Close Relationships–Short Form [ECR-SF], a 12-item measure which provides attachment avoidance and attachment anxiety dimensional scores) and EI (the Mayer–Salovey–Caruso Emotional Intelligence Test [MSCEIT], a 141-item measure on the perception, use, understanding and management of emotions), prior to their summative PPC OSCE. Provider–patient communication was assessed using OSCE scores. Structural equation modelling (SEM) was used to validate our earlier model of the relationships between attachment style, EI and PPC.

Results

A total of 296 of 382 (77.5%) students participated. Attachment avoidance was significantly negatively correlated with total EI scores (r = −0.23, p < 0.01); total EI was significantly positively correlated with OSCE scores (r = 0.32, p < 0.01). Parsimonious SEM confirmed that EI mediated the negative influence of attachment avoidance on OSCE scores. It significantly predicted 14% of the variance in OSCE scores, twice as much as the 7% observed in the previous study.

Conclusions

In more demanding and realistic clinical scenarios, EI makes a greater contribution towards effective PPC. Attachment is perceived to be stable from early adulthood, whereas EI can be developed using targeted educational interventions. The validation of this theoretical model of PPC in Year 2 medical students strengthens the potential educational implications of EI.

Article DOI: 10.1111/medu.12526

Professionalism: a framework to guide medical education

To read on journal website, click here.

Howard Brody and David Doukas.

Context

Despite considerable advances in the incorporation of professionalism into the formal curriculum, medical students and residents are too often presented with a mechanical, unreflective version of the topic that fails to convey deeper ethical and humanistic aspirations. Some misunderstandings of professionalism are exacerbated by commonly used assessment tools that focus only on superficially observable behaviour and not on moral values and attitudes.

Methods

Following a selective literature review, we engaged in philosophical ethical analysis to identify the key precepts associated with professionalism that could best guide the development of an appropriately reflective curriculum.

Results

The key precepts needed for a robust presentation of professionalism can be grouped under two headings: ‘Professionalism as a trust-generating promise’ (representing commitment to patients’ interests, more than a mere business, a social contract, a public and collective promise, and hard work), and ‘Professionalism as application of virtue to practice’ (based on virtue, deeper attitudes rather than mere behaviour, and requiring of practical wisdom).

Conclusions

These key precepts help students to avoid many common, unreflective misunderstandings of professionalism, and guide faculty staff and students jointly to address the deeper issues required for successful professional identity formation.

Article DOI: 10.1111/medu.12520

Medical student depression, anxiety and distress outside North America: a systematic review

To read on journal website, click here.

Valerie Hope and Max Henderson

Context

North American medical students are more depressed and anxious than their peers. In the UK, the regulator now has responsibility for medical students, which may potentially increase scrutiny of their health. This may either help or hinder medical students in accessing appropriate care. The prevalences of anxiety, depression and psychological distress in medical students outside North America are not clear. A better understanding of the prevalence of, risk factors for and results of psychological distress will guide the configuration of support services, increasingly available for doctors, for medical students too.

Objectives

The aim of this study was to examine the prevalences of depression, anxiety and psychological distress in students in medical schools in the UK, Europe and elsewhere in the English-speaking world outside North America.

Methods

A systematic review was conducted using search terms encompassing psychological distress amongst medical students. OvidSP was used to search the following databases: Ovid MEDLINE (R) from 1948 to October 2013; PsycINFO from 1806 to October 2013, and EMBASE from 1980 to October 2013. Results were restricted to medical schools in Europe and the English-speaking world outside North America, and were evaluated against a set of inclusion criteria including the use of validated assessment tools.

Results

The searches identified 29 eligible studies. Prevalences of 7.7–65.5% for anxiety, 6.0–66.5% for depression and 12.2–96.7% for psychological distress were recorded. The wide range of results reflects the variable quality of the studies. Almost all were cross-sectional and many did not mention ethical approval. Better-quality studies found lower prevalences. There was little information on the causes or consequences of depression or anxiety.

Conclusions

Prevalences of psychological distress amongst medical students outside North America are substantial. Future research should move on from simple cross-sectional studies to better-quality longitudinal work which can identify both predictors for and outcomes of poor mental health in medical students.

Article DOI: 10.1111/medu.12512

Exploring frontline faculty perspectives after a curriculum change

To read on journal website, click here.

Shannon L. Venance, Kori A LaDonna, and Christopher J. Watling

Context

Curriculum renewal is an essential and continual process for undergraduate medical education programmes. Although there is substantial literature on the critical role of leadership in successful curricular change, the voices of frontline faculty teachers implementing such change have not been explored. We aimed not only to examine and understand the perceptions of faculty members as they face curriculum change, but also to explore the influences on their engagement with change.

Methods

We used a constructivist grounded approach in this exploratory study. Sixteen faculty members teaching in the pre-clinical years were interviewed on their perspectives on a recent curricular change in the undergraduate medical programme at a single Canadian medical school. Constant comparative analysis was conducted to identify recurring themes.

Results

Faculty teachers’ engagement with curriculum change was influenced by three critical tensions during three phases of the change: (i) tension between individual and institutional values, which was prominent as change was being introduced; (ii) tension between drivers of change and restrainers of change, which was prominent as change was being enacted, and (iii) tension between perceived gains and perceived losses, which was prominent as teachers reflected on change once implemented.

Conclusions

We propose a model of faculty engagement with curricular change that elucidates the need to consider individual experiences and motivations within the broader context of the institutional culture of medical schools. Importantly, if individual and institutional values are misaligned, barriers to change outweigh facilitators, or perceived losses prevail; subsequently faculty teachers’ engagement may be threatened, exposing the medical education programme to risk.

Article DOI: 10.1111/medu.12529

Automated essay scoring and the future of educational assessment in medical education

To read on journal website, click here.

Mark J Gierl, Syed LatifiHollis LaiAndré-Philippe Boulais and André De Champlain

Context

Constructed-response tasks, which range from short-answer tests to essay questions, are included in assessments of medical knowledge because they allow educators to measure students’ ability to think, reason, solve complex problems, communicate and collaborate through their use of writing. However, constructed-response tasks are also costly to administer and challenging to score because they rely on human raters. One alternative to the manual scoring process is to integrate computer technology with writing assessment. The process of scoring written responses using computer programs is known as ‘automated essay scoring’ (AES).

Methods

An AES system uses a computer program that builds a scoring model by extracting linguistic features from a constructed-response prompt that has been pre-scored by human raters and then, using machine learning algorithms, maps the linguistic features to the human scores so that the computer can be used to classify (i.e. score or grade) the responses of a new group of students. The accuracy of the score classification can be evaluated using different measures of agreement.

Results

Automated essay scoring provides a method for scoring constructed-response tests that complements the current use of selected-response testing in medical education. The method can serve medical educators by providing the summative scores required for high-stakes testing. It can also serve medical students by providing them with detailed feedback as part of a formative assessment process.

Conclusions

Automated essay scoring systems yield scores that consistently agree with those of human raters at a level as high, if not higher, as the level of agreement among human raters themselves. The system offers medical educators many benefits for scoring constructed-response tasks, such as improving the consistency of scoring, reducing the time required for scoring and reporting, minimising the costs of scoring, and providing students with immediate feedback on constructed-response tasks.

Article DOI: 10.1111/medu.12517

Overview of the world’s medical schools: an update

To read on journal website, click here.

Robbert J Duvivier, John R Boulet, Amy Opalek, Marta van Zanten and John Norcini

Context

That few data are available on the characteristics of medical schools or on trends within medical education internationally constitutes a major challenge when developing strategies to address physician workforce shortages. Quality and up-to-date information is needed to improve health and education policy planning.

Methods

We used publicly available data from the International Medical Education Directory and Avicenna Directories, and an internal education programme database to gather data on medical education provision worldwide. We sent a semi-structured questionnaire to a selection of 346 medical schools, of which 218 (63%) in 81 different countries or territories replied. We contacted ministries of health, national agencies for accreditation or similar bodies to clarify inconsistencies among sources. We identified key informants to obtain country-level specific information. Descriptive statistics were used to analyse current medical school data by country.

Results

There are about 2600 medical schools worldwide. The countries with the largest numbers of schools are India (n = 304), Brazil (n = 182), the USA (n = 173), China (n = 147) and Pakistan (n = 86). One-third of all medical schools are located in five countries and nearly half are located in 10 countries. Of 207 independent states, 24 have no medical school and 50 have only one. Regionally, numbers of citizens per school differ: the Caribbean region has one school per 0.6 million population; the Americas and Oceania each have one school per 1.2 million population; Europe has one school per 1.8 million population; Asia has one school per 3.5 million population, and Africa has one school per 5.0 million population. In 2012, on average, there were 181 graduates per medical school.

Conclusions

The total number and distribution of medical schools around the world are not well matched with existing physician numbers and distribution. The collection and aggregation of medical school data are complex and would benefit from better information on local recognition processes. Longitudinal comparisons are difficult and subject to several sources of error. The consistency and quality of medical school data need to be improved to accurately document potential supply; one example of such an advancement is the World Directory of Medical Schools.

The good and bad of group conformity: a call for a new programme of research in medical education

To read on journal website, click here.

Tanya N Beran, Alyshah Kaba, Jeff Caird and Kevin McLaughlin

Context

Given that a significant portion of medical education occurs in various social settings (small groups, large classes, clinical environments), it is critical to examine how group members interact. One type of influence on these interactions is conformity, whereby an individual changes his or her own behaviour to match incorrect responses of others in a group. Conformity to peer pressure has been replicated in experimental research conducted in many countries over the last 60 years. There is newly emerging empirical evidence of this effect in medical education, suggesting that subtle motivations and pressures within a group may prevent students from challenging or questioning information that seems incorrect.

Objectives

This narrative review aims to present an overview of theory and findings in research into conformity in the fields of social psychology, business, sociology and aviation theory to demonstrate its direct relevance to medical education and the health professions.

Methods

We searched online databases (MEDLINE, PubMed, PsycINFO and ProQuest) from the University of Calgary catalogue. We also searched citations in articles reviewed and references provided by colleagues. We limited our narrative review to publications released between 1950 and 2012.

Results

Group conformity behaviour may be one of a number of communication challenges associated with interprofessional care, and may represent a factor contributing to the burden of adverse events. This paper calls for a new programme of research into conformity in medical education that provides systematic empirical evidence of its relevance and applications in education, health care and practice.

Conclusions

This review reveals decades of anecdotal and empirical evidence that conformity is a pervasive phenomenon across disciplines. Further research is needed to elucidate which situations pose the greatest risk for the occurrence of conformity, how to manage it in practice and its implications for patient safety.

‘You’re certainly relatively competent’: assessor bias due to recent experiences

Online abstract click here

Peter Yeates, Paul O’Neill, Karen Mann and Kevin W Eva

Context

A recent study has suggested that assessors judge performance comparatively rather than against fixed standards. Ratings assigned to borderline trainees were found to be biased by previously seen candidates’ performances. We extended that programme of investigation by examining these effects across a range of performance levels. Furthermore, we investigated whether confidence in the rating assigned predicts susceptibility to manipulation and whether prompting consideration of typical performance lessens the influence of recent experience.

Methods

Consultant doctors were randomised to groups within an internet experiment. The descending performance group judged videos of Foundation Year 1 (F1; postgraduate Year 1) doctors in descending order of proficiency; the ascending performance group judged the same videos in ascending order. For all videos, participants rated: (i) trainee competence; (ii) rater confidence and (iii) percentage better (the percentage of other F1 doctors who would perform better on the same task).

Results

Overall, the descending performance group assigned lower scores than the ascending performance group (2.97 [95% confidence interval 2.73–3.20] versus 3.50 [95% confidence interval 3.25–3.74]; F(1,47) = 9.80, p = 0.003, = 0.52). Pairwise comparisons showed differences were significant for good and borderline performances. The percentage better ratings showed a similar pattern (descending performance mean = 57.4 [95% confidence interval 52.5–62.3], ascending performance mean = 43.4 [95% confidence interval 38.4–48.5];F(1, 46) = 16.0, p < 0.001, = 0.67). Confidence ratings did not vary by level of performance and showed no relationship with the effect of group.

Discussion

Assessors’ judgements showed contrast effects at both good and borderline performance levels. Findings suggest that assessors use normative rather than criterion-referenced decision making while judging, and that the norms referenced are weakly represented in memory and easily influenced. Confidence ratings suggested a lack of insight into this phenomenon. Raters’ judgements could be importantly influenced in ways that are unfair to candidates.