Developing the CASSY Suicide Screening Tool
In Part 2, of this video, Cheryl King, PhD, professor in the Departments of Psychiatry and Psychology and Director of the Youth Depression and Suicide Prevention Program at the University of Michigan, Ann Arbor, discusses the 2 studies that determined the validity and accuracy of the CASSY suicide screening tool. Dr King addresses the personalization of this screening tool and how it assists emergency department physicians in creating a follow-up plan and correctly identifying underlaying mental health conditions, which can go undetected in the younger population.
In the upcoming Part 3, Dr King will address the acceptability to both parents and youth for this screening tool, next steps in implementation, as well as the stigma of suicidal ideation in youth. In Part 1, Dr King discussed the importance of developing the CASSY suicide screening tool for youth in emergency departments.
Read the transcript:
Dr. Cheryl King: How did we go about developing the CASSY? This was really a large-scale effort. I led this together with two colleagues in the field nationally. Jackie Grupp-Phelan, who is a pediatric emergency medicine physician -- she's currently at UCSF -- and David Brent, a child psychiatrist at the University of Pittsburgh.
We were a leadership team for this study that we did in collaboration with the Pediatric Emergency Care Applied Research Network. This is a federally-funded network of pediatric emergency departments that have infrastructure to be able to do research studies on a very large scale.
We conducted two studies. One, which was to develop the CASSY, develop the algorithms for it, which I'll say more about in a little bit. The second one was to validate the CASSY in an entirely new sample of youth who were coming in for emergency care at these emergency departments. A development study and a validation study.
We work with 13 emergency departments in study one to develop it, all from this pediatric network, and we work with 14 in study two in addition to an Indian Health Service hospital emergency services unit. A lot of sites were involved, and it involved a great deal of collaboration.
In study one, to develop it. What we did was have the youth coming into the emergency department, almost all of them, unless for some reason they were not capable if it had been a severe trauma.
If they were able to participate -- most of them were -- and if they had prior permission and they agreed to participate, we enrolled them. They completed a fairly long -- about 100 items -- youth suicide risk survey. In developing this survey, we wanted to include all the risk factors that were known at this point in time about risk for a suicide attempt.
We're including all of them in about a 100-item survey because then, that'll enable us to develop a screen to see which items and combinations items were more predictive of the youth making a suicide attempt within three months. We gathered data from thousands of youth, more than 6,000 youth.
Then, we followed up some of them at three months telephone interviews with the youth and the parent to learn who had made a suicide attempt during that relatively short period of time of three months. This enabled us to develop algorithms and develop the CASSY, which, as its name says, it's a Computerized Adaptive Screen for Suicidal Youth.
What is a computerized adaptive screen? This is a screen wherein the subsequent questions that a youth is asked depend on their responses to earlier questions. Not every youth receives the same number of questions when they complete a CASSY screen. Some may only have 5. Some may have 12 or 14.
What we're trying to do is have as much measurement precision as possible at the individual youth level. For some youth, we have to ask more to get a better estimate of their suicide risk. It's not only the number of questions, but different youth will get some different questions.
Every youth screen interludes some questions about suicidal thoughts, but they then vary. They might ask about "How much they feel they belong at their school?" "Are you a victim of bullying, peer victimization?" For different youth, the CASSY screen will have different items. It's adaptive. It's more personalized.
The reason we needed to enroll thousands in study one and have youth complete a longer suicide risk survey is so that we could develop these different algorithms. What questions to ask? What are the parameters that we're using and pulling from for these different youth? Then, we conducted study two. This is the validation study.
We went back to all of these emergency departments. We actually added another one. These emergency departments are geographically diverse in the United States. They included sites in Michigan where I'm from, Ohio, California, Arizona, Utah, Texas, Wisconsin. We're a good distribution nationally. We went back to them, and we had a new sample of youth.
This time, they're not completing the longer suicide risk survey. They're on a tablet just doing the CASSY. They're only getting the questions that they will get based on their responses to the first questions. Again, the number will vary and the content varies. They all are asked some questions about suicidal ideation.
In the validation study, then, we also did a three-month follow-up with telephone interviews separately with the parent and the youth to learn what had happened during those three months, and which of the youth had made a suicide attempt.
Our findings are very strong. The CASSY has quite good classification accuracy. The percent they were accurately classified as at-risk and not at-risk was 87 percent accuracy. In study one, if you went back and applied the algorithms to the data from the survey, it was 89 percent.
We did achieve a goal we had hoped to achieve, which is that we were able to have a sensitivity and a specificity both over 80 percent. Now, let me just take a minute in case those terms are unfamiliar to you.
A sensitivity is the extent to which we captured everyone who was at risk. "Did we screen as positive everyone who ended up making an attempt?" Specificity is, "Did we accurately predict who would make an attempt?" We don't have too many false positives where we predicted almost everyone would make an attempt. Then, obviously, we'd have really high sensitivity.
We captured all of them, but we're saying almost everyone's at risk. That's untenable for an emergency department because they don't have the resources to follow up in-depth with mental health evaluations of such a high proportion of the youth seeking emergency services.
Although we want high sensitivity, we don't want to miss too many youth at risk. We also need fairly high specificity, so that we're identifying youth who really are at risk and warrant further evaluation.
We were hoping that with a sensitivity over 80 percent, accurately identifying more than 80 percent at risk -- 80 or more -- we would have specificity of 80 or higher so that we didn't have too many false positives due to the resource burden, and the lack of feasibility and acceptability that would have on emergency departments.
We did achieve that, and we're very pleased with the accuracy of this CASSY screen and its adaptive nature. The fact that it's more personalized and adaptive and can account for the different patterns of risk in different youth is probably one of the reasons for that high level of accuracy.
That's important because we would like a widespread dissemination of suicide risk screening in emergency departments. It gives us the possibility of recognizing risk and potentially getting these youth life-saving treatments.
This widespread dissemination means we need a tool that is acceptable to EDs. They want to do the right thing, they are about saving lives, but they have resource limitations, space, and personnel. We have to develop a tool that meets a couple of criteria for them.
One is that it's accurate to take on screening all the youth who come into a pediatric emergency department for suicide risk. To do that, you want to have an accurate tool that's performing well, but second, it has to be highly feasible and acceptable in the emergency department setting.
The third is that the emergency department needs to have a capacity to follow up. If a youth does screen positive, they need to take action of some kind, take some next steps with clinical decision-making. Maybe it's a second brief evaluation, and for some of the youth, it may be an extended mental health evaluation.
An additional strength of the CASSY on top of its accuracy is because it's a continuous screen -- meaning, it's not just a positive screen, a negative screen -- the CASSY gives the provider in the emergency department a percentage risk, a number on a scale up to 100.
It makes it possible for the ED to set different thresholds depending on their resources and capacity for follow-up. Would they like to have slightly lower sensitivity and higher specificity? Then, for those youth they identify, they can be comfortable that they have followed up in the most beneficial way.
Giving emergency departments some choice in how they use a screen in addition to having an accurate screen, we believe, is extremely important to the possibility of truly widespread dissemination.
Cheryl King, PhD, is a professor in the Departments of Psychiatry and Psychology and Director of the Youth Depression and Suicide Prevention Program at the University of Michigan. Her research focuses on the development of evidence-based practices for suicide risk screening, assessment, and intervention. She has provided leadership for multiple NIMH-funded projects, including Emergency Department Screen for Teens at Risk for Suicide, which aims to develop a suicide risk screen that can be disseminated nationwide, and 24-Hour Risk for Suicide Attempts in a National Cohort of Adolescents. A clinical psychologist, educator, and research mentor, Dr. King has served as Director of Psychology Training and Chief Psychologist in the Department of Psychiatry and has twice received the Teacher of the Year Award in Child and Adolescent Psychiatry. She is the lead author of Teen Suicide Risk: A Practitioner Guide to Screening, Assessment, and Management. In addition, Dr. King has provided testimony in the US Senate on youth suicide prevention and is a Past President of the American Association of Suicidology, the Association of Psychologists in Academic Health Centers, and the Society for Clinical Child and Adolescent Psychology. She is a current member of the National Advisory Mental Health Council.
