Changes to the Step 1 Scoring System and Impact on Dermatology Residents: Parts 1 and 2
A study conducted by researchers in the department of dermatology at George Washington Medical School highlighted how residency program directors perceived the change of the Step 1 scoring system to pass/fail. In this video, Drs Adam Friedman, Ilana Rosman, and Kamaria Nelson discuss the impact of this change on future dermatology resident applicants.
In part 2 of this video series, Drs Friedman, Rosman, and Nelson discuss the impact of this change on future dermatology resident applicants and what is needed to assist future resident applicants.
Dr Rosman is residency program director, director of the Dermatopathology Center, and associate professor of dermatology and pathology at Washington School of Medicine in St. Louis, MO.
Dr Nelson is a research fellow and incoming resident with the department of dermatology at George Washington University School of Medicine in Washington, DC.
Dr Friedman is the residency program director, professor and interim chair of the department of dermatology, and director of translational research at George Washington School of Medicine in Washington, DC.
Melissa: Hi, everyone. With us is Dr Adam Friedman who is the interim chair of the Department of Dermatology and director of the residency program at George Washington University School of Medicine, Dr Ilana Rosman, who is the dermatology resident program director at Washington University, and Dr Kamaria Nelson, who is a research fellow and incoming resident with George Washington University’s dermatology program.
Dr Friedman and his colleagues at George Washington University School of Medicine recently published a study in The Journal of Drugs and Dermatology that assessed perceptions of the United States medical licensing exams change to the Step 1 scores among program directors to pass/fail.
Today we’re going to discuss this study and its implications. Thank you for joining us.
Dr Friedman: Thanks for having us.
Melissa: Dr Friedman, could you provide us a little background on your study and the results?
Dr Friedman: Sure. The concept of the Step 1 exam moving from a graded exam to pass/fail had been circulating the rumor mill. Actually, I distinctly remember from the 2019 Association of Professors of Dermatology meeting...I’m sure Ilana remembers, as well. They got into a pretty heated debate about it. Just to briefly summarize the two different sides, which both have their merits.
On the one hand, those who are pro, which I amidst, I think putting so much pressure on one day, one exam, and suggesting that dictates what type of dermatologist or dermatology resident, even, someone could ultimately become, I think is shortsighted.
I think it puts a lot of pressure on the student to do really well or, even more so, if someone doesn’t do well in Step 1, maybe push them away from pursing dermatology even though they would make a phenomenal dermatologist.
I think all three of us have heard that kind of lexicon that, “Oh, there’s no way you can become a dermatologist with that Step 1 exam.” I think highlighting only part of an overall puzzle and relying on that really undermines the quality someone could bring to the field.
On the flip side, also, I completely recognize, is that how does a—most dermatology programs are quite small—how does a small program, a small number of faculty, review 900 to 1000 applications in a meaningful way? The answer’s you can’t.
We really need standardized ways of approaching the applicant, something that we really can put people side by side, judge them based on some data. Certainly, an exam is one way to do it, though I’d argue there are limitations even with that given the resources that everyone has, the support that everyone has is going to vary, certainly, from school to school.
I think it’s also a numbers game. Using these kind of filters to say that we will only look at applications with a Step 1 exam score of X and that’s it certainly makes us seem elitist. I would argue dermatology is somewhat elitist, unfortunately.
It’s really more just a practical thing. How could we possibly look at all those applications in a meaningful way? I get both sides of it, even though I favor the pass/fail. However, there are things we have to consider by making it pass/fail and how we adapt to the new world that will emerge from that.
We had a good sense of what people were thinking, but we wanted to document that. That was the purpose for this survey, was to disseminate through to different dermatology program directors and associate program directors and see how they feel now that it’s in stone. Now that the exam is going to be pass/fail, what they thought. Were they happy with this decision and what they thought it meant for their ability to recruit high level applicants? Also, what did it mean for certain applicant populations? For underrepresented minorities in medicine.
Is this going to change the game for them, either good or bad? Also, does it put new pressures on other parts of the application overall, in terms of external rotations, research years, or even taking Step 2 earlier, which a lot of respondents thought that all it’s going to do is make Step 2 more important because that’s not pass/fail.
I think it was really important to put our finger on the pulse of dermatology at that moment in time and get a sense of where everyone was.
Melissa: Dr Rosman, what do you think about the findings?
Dr Rosman: I thought they were really interesting. I agree with everything that Adam has said. I have, for a while, been on the pro pass/fail side, although I recognize the challenges with Step 1 and with application review.
After the APD meetings, the Association of Professors of Dermatology, which is our academic dermatology community and our academic dermatology association...Our meeting last year, I gave a talk with a colleague on the application process. After that, we had this debate about a lot of the processes and a lot of the steps in the application process.
We formed this actual program director task force to look at the entire application process and see what we can do to make it better. I think in relation to Step 1 in particular, historically it has been used as a filter, as Adam said, to help us work through the immense number of applications that we get during this application season.
The problem is that there’s really not data to suggest that Step 1 scores are correlated with resident performance. There’s no data to suggest that they’re correlated with dermatology specialty board passage. There is data to suggest that there is racial bias…and also bias against women.
Women, underrepresented minorities, so black and Hispanic test takers all perform lower, up to 10 to 14 points, potentially, lower than white males on this test. Clearly, there are problems with this test. In using it to define what specialty someone goes into is incredibly problematic, I think. That’s what Adam is saying and alluding to.
I wasn’t entirely surprised. I was a little disappointed that a lot of program directors were against this change.
I do think that a lot of the people in our group, so the group that we interact with a lot in terms of our program director task force, they’re super engaged and they really are interested in reforming the process. I think a lot of them are actually for this change. That wasn’t necessarily reflected in the survey.
I was also disappointed with this idea that we should just replace it with Step 2. I think that’s a very easy and a path of least resistance but replacing one core metric with another core metric is not going to help the process and, if anything, is just going to lead us down the same path we’re in now.
I would like to see people open their mind a little bit more to other things and other ways we can reform the system, which I know we’ll get to and we’ll talk about.
I think the other thing that I noticed, also, which I think Adam alluded to, maybe, that one of the results is that people thought that it would mean that students would need to do more extracurricular activities to bolster their application because Step 1 scores are not there.
I think that’s also very dangerous. If you look at dermatology applicants, they have the highest mean number of publications, the highest mean number of volunteer experiences, research experiences. I think saying, “Oh, they need to do more extracurricular activities,” all that’s saying to applicants... “Hey, here’s a bunch of boxes that you check. Then you’ll match into dermatology.”
Really, we want students to be able to follow their passion. We want them to say, “You know what? I’m super interested in service. I’m going to do these volunteer experiences.” “I’m super interested in research. I’m going to do this.”
Each applicant doesn’t have to do everything. That’s not the point. None of us do everything. We have certain things that we’re particularly interested in. That’s what you want to develop with your students, as well, so they have a passion for their career and their field.
