Category Archives: Issue 6 June

Medical Education June 2012 Issue posts available for comment!

Posted on by
Reply

Hi Everyone,

No doubt you’ve received the eTOC (that’s ‘e-Table of Contents’) for the June 2012 issue of Medical Education. The posts are now up on the blog so feel free to join the conversation online with other readers and authors. Some ideas for comments include:

  • Provide your perspective on and experience with the topic written
  • Share how the article might change your practice or spur your research
  • See who else is engaged in this area of work; ask for collaborators!

If you’d like to jump to the journal website and read the full articles, you can do so by clicking here.

Welcome to the conversation!

Josh

How to construct and implement script concordance tests: insights from a systematic review

Posted on by
Reply

Online abstract click here

Valérie Dory, Robert Gagnon, Dominique Vanpee and Bernard Charlin

Medical Education 2012: 46: 552–563

Context  Programmes of assessment should measure the various components of clinical competence. Clinical reasoning has been traditionally assessed using written tests and performance-based tests. The script concordance test (SCT) was developed to assess clinical data interpretation skills. A recent review of the literature examined the validity argument concerning the SCT. Our aim was to provide potential users with evidence-based recommendations on how to construct and implement an SCT.

Methods  A systematic review of relevant databases (MEDLINE, ERIC [Education Resources Information Centre], PsycINFO, the Research and Development Resource Base [RDRB, University of Toronto]) and Google Scholar, medical education journals and conference proceedings was conducted for references in English or French. It was supplemented by ancestry searching and by additional references provided by experts.

Results  The search yielded 848 references, of which 80 were analysed. Studies suggest that tests with around 100 items (25–30 cases), of which 25% are discarded after item analysis, should provide reliable scores. Panels with 10–20 members are needed to reach adequate precision in terms of estimated reliability. Panellists’ responses can be analysed by checking for moderate variability among responses. Studies of alternative scoring methods are inconclusive, but the traditional scoring method is satisfactory. There is little evidence on how best to determine a pass/fail threshold for high-stakes examinations.

Conclusions  Our literature search was broad and included references from medical education journals not indexed in the usual databases, conference abstracts and dissertations. There is good evidence on how to construct and implement an SCT for formative purposes or medium-stakes course evaluations. Further avenues for research include examining the impact of various aspects of SCT construction and implementation on issues such as educational impact, correlations with other assessments, and validity of pass/fail decisions, particularly for high-stakes examinations.

Improving cultural competence education: the utility of an intersectional framework

Posted on by
Reply

Online abstract click here

Karen Powell Sears

Medical Education 2012: 46:545–551

Context  Most US medical schools have instituted cultural competence education in the undergraduate curriculum. This training is intended to improve the quality of care that doctors, the majority of whom are White, deliver to ethnic and racial minority patients. Research into the outcomes of cultural competence training programmes reveals that they have been largely ineffective in improving doctors’ skills. In varied curricular formats, programmes tend to teach group-specific cultural knowledge, despite the vast heterogeneity of racial and ethnic groups. This cultural essentialism diminishes training effectiveness.

Methods  This paper proposes key curriculum content changes and suggests the inclusion of an intersectional framework in the cultural competence curriculum. This framework maintains that racial and ethnic minority groups hold multiple social statuses, called social locations, which interact with one another to uniquely shape the health views, needs and experiences of the individuals within the groups. Social locations include those defined by race, ethnicity, gender, social class and sexuality, which are experienced multiplicatively, not additively, within a particular social context. Cultural competence education must go beyond simplified cultural understandings to explore these more complex meanings. Doctors’ ability to understand, communicate with and treat diverse groups can be vastly improved by applying an intersectional framework in academic research, self-awareness exercises and clinical training.

Results  Integrating an intersectional framework into cultural competency education can better prepare doctors for caring for racial and ethnic minority patients. This paper recommends curriculum elements for the classroom and clinical training that can improve doctor knowledge and skills for caring for diverse groups. Medical schools can use the proposed model to facilitate the development of new educational strategies and learning experiences. These improvements can lead to more equitable care and ultimately diminish disparities in health care. Although these recommendations are designed with US schools in mind, they may improve doctor understanding and care of marginal populations across the world.

Supporting students with disability and health issues: lowering the social barriers

Posted on by
Reply

Online abstract click here

Vivien Cook, Ann Griffin, Sheila Hayden, Joy Hinson and Peter Raven

Medical Education 2012: 46:564–574

Context  Legislation and General Medical Council guidance require medical schools to make reasonable adjustments to enable students with disabilities to progress in their studies without disadvantage. Current definitions of disability have moved from the traditional ‘medical’ model to a ‘social’ model that emphasises the role of institutions in removing unnecessary barriers to learning through flexible provision. In response, two medical schools introduced a Student Support Card to empower students with disabilities and health issues to request reasonable adjustments. A study was carried out to ascertain the effectiveness of the Student Support Card from the user’s perspective in both institutions.

Methods  A questionnaire survey of card-holders was conducted in 2009–2010. A total of 31 respondents took part, from among whom six volunteers were recruited to participate in semi-structured interviews. Thematic analysis was conducted and data independently analysed by two researchers.

Results  The results from the questionnaire indicated that the scheme was well received by students and brought clear benefits, and that the card’s value lay in ownership, as well as in use. Further themes emerged from the interviews which suggested that students were judicious in requesting adjustments, thus reflecting their concerns over the disclosure of disability, the threat that their behaviour might be misinterpreted and their acquiring of the competencies necessary to become a doctor.

Conclusions  The benefits of such a scheme are that it is flexible and addresses unique student needs. Further research needs to focus upon: how educators perceive the utility of the Student Support Card; whether it should be embedded in the wider medical school culture, and, subsequently, how a working consensus can be achieved with regard to reasonable adjustments within vocational education in which there are clear requirements for competence.

Ethnic disparities in undergraduate pre-clinical and clinical performance

Posted on by
Reply

Online abstract click here

Karen M Stegers-Jager, Ewout W Steyerberg, Janke Cohen-Schotanus and Axel P N Themmen

Medical Education 2012: 46:575–585

Context  Research from numerous medical schools has shown that students from ethnic minorities underperform compared with those from the ethnic majority. However, little is known about why this underperformance occurs and whether there are performance differences among ethnic minority groups.

Objectives  This study aimed to investigate underperformance across ethnic minority groups in undergraduate pre-clinical and clinical training.

Methods  A longitudinal prospective cohort study of progress on a 6-year undergraduate medical course was conducted in a Dutch medical school. Participants included 1661 Dutch and 696 non-Dutch students who entered the course over a consecutive 6-year period (2002–2007). Main outcome measures were performance in Year 1 and in the pre-clinical and clinical courses. Odds ratios (ORs) with 95% confidence intervals (CIs) were estimated by logistic regression analysis for ethnic subgroups (Surinamese/Antillean, Turkish/Moroccan/African, Asian, Western) compared with Dutch students, adjusted for age, gender, pre-university grade point average (pu-GPA), additional socio-demographic variables (first-generation immigrant, urban background, first-generation university student, first language, medical doctor as parent) and previous performance at medical school.

Results  Compared with Dutch students, Surinamese and Antillean students specifically underperformed in the Year 1 course (pass rate: 37% versus 64%; adjusted OR 0.40, 95% CI 0.27–0.60) and the pre-clinical course (pass rate: 19% versus 41%; adjusted OR 0.57, 95% CI 0.35–0.93). On the clinical course all non-Dutch subgroups were less likely than Dutch students to receive a grade of ≥ 8.0 (at least three of five grades: 54–77% versus 88%; adjusted ORs: 0.17–0.45).

Conclusions  Strong ethnic disparities exist in medical school performance even after adjusting for age, gender, pu-GPA and socio-demographic variables. More subjective grading cannot be ruled out as a cause of lower grades in clinical training, but other possible explanations should be studied further to mitigate the disparities.

Medical education accreditation in Mexico and the Philippines: impact on student outcomes

Posted on by
2

Online abstract click here

Marta van Zanten, Danette McKinley, Irene Durante Montiel and Concepcion V Pijano

Medical Education 2012: 46:586–592

Context  Accreditation of medical education programmes is becoming increasingly prevalent worldwide, but beyond the face validity of these quality assurance methods, data linking accreditation to improved student outcomes are limited. Mexico and the Philippines both have voluntary systems of medical education accreditation and large numbers of students who voluntarily take components of the United States Medical Licensing Examination (USMLE). We investigated the examination performance of Mexican and Philippine citizens who attended medical schools in their home countries by medical school accreditation status.

Methods  The sample included 5045 individuals (1238 from Mexico, 3807 from the Philippines) who took at least one of the three USMLE components required for Educational Commission for Foreign Medical Graduates (ECFMG) certification. We also separately studied 2702 individuals who took all three examinations (589 from Mexico, 2113 from the Philippines). The chi-squared statistic was used to determine whether the associations between outcomes (first attempt pass rate on USMLE components and rate of ECFMG certification) and medical school accreditation (yes/no) were statistically significant.

Results  For the sample of registrants who took at least one USMLE component, first attempt pass rates on all USMLE components were higher for individuals attending accredited schools, although there were differences in pass rates among the components and between the two countries. The distinction was greatest for USMLE Step 1, for which attending an accredited school was associated with increases in first attempt pass rates of 15.9% for Mexican citizens and 29.2% for Philippine citizens. In registrants from the Philippines who took all three examinations, attending an accredited medical school was also associated with increased success in obtaining ECFMG certification.

Conclusions  These findings support the value and usefulness of accreditation in Mexico and the Philippines by linking accreditation to improved student outcomes.

Understanding responses to feedback: the potential and limitations of regulatory focus theory

Posted on by
Reply

Online abstract click here

Christopher Watling, Erik Driessen, Cees P M van der Vleuten, Meredith Vanstone and Lorelei Lingard

Medical Education 2012:46:593–603

Objectives  Regulatory focus theory posits the existence of two systems of self-regulation underlying human motivation: promotion focus, which is concerned with aspirations and accomplishments, and prevention focus, which is concerned with obligations and responsibilities. It has been proposed that regulatory focus theory may help to explain learners’ variable responses to feedback, predicting that positive feedback is motivating under promotion focus, whereas negative feedback is motivating under prevention focus. We aimed to explore this link between regulatory focus theory and response to feedback using data collected in a naturalistic setting.

Methods  In a constructivist grounded theory study, we interviewed 22 early-career academic doctors about experiences they perceived as influential in their learning. Although feedback emerged as important, responses to feedback were highly variable. To better understand how feedback becomes (or fails to become) influential, we used the theoretical framework of regulatory focus to re-examine all descriptions of experiences of receiving and responding to feedback.

Results  Feedback could be influential or non-influential, regardless of its sign (positive or negative). In circumstances in which the individual’s regulatory focus was readily determined, such as in choosing a career (promotion) or preparing for a high-stakes examination (prevention), the apparent influence of feedback was consistent with the prediction of regulatory focus theory. However, we encountered many challenges in applying regulatory focus theory to real feedback scenarios, including the frequent presence of a mixed regulatory focus, the potential for regulatory focus to change over time, and the competing influences of other factors, such as the perceived credibility of the source or content of the feedback.

Conclusions  Regulatory focus theory offers a useful, if limited, construct for exploring learners’ responses to feedback in the clinical setting. The insights and predictions it offers must be considered in light of the motivational complexity of clinical learning tasks and of other factors influencing the impact of feedback.

The process of feedback in workplace-based assessment: organisation, delivery, continuity

Posted on by
Reply

Online abstract click here

Elisabeth A M Pelgrim, Anneke W M Kramer, Henk G A Mokkink and Cees P M van der Vleuten

Medical Education 2012: 46:604–612

Objectives  Feedback in workplace-based clinical settings often relies on expert trainers’ judgements of directly observed trainee performance. There is ample literature on effective feedback, but in practice trainees in workplace-based training are not regularly observed. We aimed to examine external conditions that impact feedback in observational workplace-based assessment (WBA).

Methods  Interviews were conducted and the resulting data analysed using a qualitative, phenomenological approach. Between October 2009 and January 2010, we interviewed 22 postgraduate general practice trainees at two institutions in the Netherlands. Three researchers analysed the transcripts of the interviews.

Results  A three-step scheme emerged from the data. Feedback as part of WBA is of greater benefit to trainees if: (i) observation and feedback are planned by the trainee and trainer; (ii) the content and delivery of the feedback are adequate, and (iii) the trainee uses the feedback to guide his or her learning by linking it to learning goals. Negative emotions reported by almost all trainees in relation to observation and feedback led to different responses. Some trainees avoided observation, whereas others overcame their apprehension and actively sought observation and feedback. Active trainers were able to help trainees overcome their fears. Four types of trainer–trainee pairs were distinguished according to their engagement in observation and feedback. External requirements set by training institutions may stimulate inactive trainers and trainees.

Conclusions  In line with the literature, our results emphasise the importance of the content of feedback and the way it is provided, as well as the importance of its incorporation in trainees’ learning. Moreover, we highlight the step before the actual feedback itself. The way arrangements for feedback are made appears to be important to feedback in formative WBA. Finally, we outline several factors that influence the success or failure of feedback but precede the process of observation and feedback.

Students’ workplace learning in two clerkship models: a multi-site observational study

Posted on by
Reply

Online abstract click here

Bridget C O’Brien, Ann N Poncelet, Lori Hansen, David A Hirsh, Barbara Ogur, Erik K Alexander, Edward Krupat and Karen E Hauer

Medical Education 2012: 46:613–624

Context  Longitudinal integrated clerkships (LICs) are established, rapidly growing models of education designed to improve the core clinical year of medical school using guiding principles about workplace learning and continuity. This study is the first to report data from direct observations of workplace learning experiences of students on LICs and traditional block clerkships (BCs), respectively.

Methods  This multi-institution study used an observational, work-sampling methodology to compare LIC and BC students early and late in the core clinical year. Trained research assistants documented students’ activities, participation (observing, with assistance, alone), and interactions every 10 minutes over 4-hour periods. Each student was observed one to three times early and/or late in the year. Data were aggregated at the student level and by in-patient or out-patient setting for BC students. One-way analysis of variance (anova) was used to compare two groups early in the year (LIC and BC students) and three groups late in the year (LIC, out-patient BC and in-patient BC students).

Results  Early-year observations included 26 students (16 LIC and 10 BC students); late-year observations included 44 students (28 LIC, eight out-patient BC and eight in-patient BC students). Out-patient activities and interactions of LIC and BC students were similar early in the year, but in the later period LIC students spent significantly more time performing direct patient care activities alone (25%) compared with out-patient (12%) and in-patient (7%) BC students. Students on LICs were significantly more likely to experience continuity with patients as 34% of their patients returned to them, whereas only 5% of patients did so for out-patient BC students late in the year.

Conclusions  By late year, LIC students engage in patient care more independently and have more opportunities to see clinic patients on multiple occasions than BC students. Consistent with the principles of workplace learning, these findings suggest that yearlong longitudinal integrated education models, that rely mostly on ambulatory settings, afford students greater opportunities to participate more fully in the provision of patient care.