How to Build a Value Assessment Model
How to Build a Value Assessment Model
Determining the vale of a drug may require less data than you realize
Value assessment models can be built using a number of approaches. Two tactics were described by Dan Danielson, MS, RPh, pharmacy manager, clinical services, Premera Blue Cross, during AMCP Nexus 2015. Before deciding which one to use, Mr Danielson emphasized the importance of knowing what you are aiming to accomplish.
Value assessment models help answer questions such as:
- Cost per event (eg, avoided emergency department visit, cost per cure)
- Contracting (ie, is it a good price?)
- Utilization management (ie, is this drug large enough to expand resources on managing?)
- Budget impact (ie, how large of an impact will this have on rates, customers, etc?).
“All these things come down to the question of ‘What’s value?’ ,” noted Mr Danielson. “Is the drug or technology improving health outcomes enough to warrant the price?”
DETERMINING VALUE: AN EXAMPLE
Using osteoporosis-related fracture prevention as an example, Mr Danielson presented the following information to the audience to determine which of 2 fictitious drugs (Drug A or Drug B) should be added to a formulary:
- He assumed, for illustrative purposes, that a 10- year baseline rate for hip fracture is 1/500 and vertebral fracture is 1/50.
- The average cost is assumed to be $25,000 for a hip fracture and $1100 for a vertebral fracture.
- Drug A decreases hip fractures by 75% and vertebral fractures by 20%, whereas Drug B decreases hip fractures by 65% and vertebral fractures by 50%.
- A one-month supply of Drug A costs $125, and a one-month supply of Drug B runs $100.
Is this enough information to decide which drug to add to a formulary? Though two-thirds of the audience said no, Mr Danielson revealed that, “You actually have all the information you need, it’s just not in the form you can use to answer the question.”
He suggested creating a 2 by 2 table (see Table 1 for an example) with the following information about each drug:
- Number of events
- Number of no events
- Total exposures/subjects.
Use the table as follows:
- To find the total number of exposures/subjects, add the number of events plus the number of no events
- To find the event rate, divide the number of events by the total number of exposures.
From there, you can calculate the absolute relative risk, relative risk reduction, and number needed to treat. See Table 2 for formulas that are used to determine event rates, absolute risk reductions, number needed to treat, and relative risk reduction.
After running the calculations, Mr Danielson was able to determine that Drug A needs to treat 667 patients to prevent 1 hip fracture and 250 patients to prevent 1 vertebral fracture. Drug B needs to treat 769 patients to prevent 1 hip fracture and 100 patients to prevent 1 vertebral fracture.
The 10-year cost of Drug A for fractures avoided is $10 million and $3.75 million for hip and vertebral fractures, respectively. For Drug B, 10-year cost for fractures avoided was $9.23 million and $1.2 Mr Danielson reviewed the core concept that effective size matters. He emphasized that while relative risk reductions are required for marketing approval, they are “awful, I think, for trying to form formulary decisions. It is not the right kind of data.” He called relative risk reductions “ratios of ratios.” On the other hand, “absolute risk reductions are a truer measure of clinical effect.”
Mr Danielson reviewed 2 distinctive decision models: decision analytic models and Markov models.
DECISION ANALYTIC MODELS
“Decision analytic models [take] a systematic approach to decision-making under uncertainty,” said Mr Danielson. Decision analytic models, which are best for answering short-term questions, use mathematical relationships to define a set of potential outcomes for a set of alternative therapies. He said each possible outcome has a cost and a probability. In order to build a decision analytic model, a problem must be defined and analyzed. Elements that are commonly assessed are costs, clinical practice, disease state, drugs used, outcomes, and populations.
He stressed that, while it may sound unnecessary, you should write down your question. The question is “very likely to evolve as you’re given information.” Writing the question down can help prevent losing sight of the goal for the decision analytic model.
There are 2 options when developing a decision analytic model: you could build your model from scratch or you can use reverse engineering.
When building from the ground-up, the first step is to gather information that is relevant to the answer you are seeking. Systematically review all comparators. Make sure that the populations are comparable across all the studies involved in the model. Because outcomes have to be standardized, you may need to adapt probabilities so they are based on the same timeframe.
Mr Danielson mentioned that studies could be included with slightly different timeframes, but be aware that an assumption was made. “[Building a model from scratch] takes a lot of time, and truthfully, this takes a team,” he said. “You probably want a PhD-level health economist to help you.”
Mr Danielson prefers what he calls reverse engineering. He suggested identifying a study that is already published. If it follows or closely follows US clinical practice, look for the specific way they built their model and mimic it.
It is important to validate your build. Do this by plugging the data from the study you mimicked into your model. If you do not get the same or a similar response to what the authors of the original study received, check the program you are using for glitches. Because Microsoft Excel is commonly used for building decision analytic models, also check for typos that could cause an incorrect calculation.
MARKOV MODELS
If you need to assess a chronic disease with defined states, use the Markov model. Patients tend to cycle from one stage of a condition to another over time. The transition between stages of a condition is governed by transition probabilities. “A cycle is the period over which you run the calculation.” Mr Danielson suggested thinking of the Markov model similar to running a decision analytic model over and over. He stressed, “Before you [start building a model], think, plan, consult.”
Mr Danielson reiterated when to choose which model:
- Decision analytic is best suited for short-term questions
- Markov model is best suited for chronic disease that has defined stages of progression.
It is essential that models include clinical data or information as close to clinical data as possible. “Your models need to be realistic,” stressed Mr Danielson.
