Application of PrecisionTree (Decision trees in Excel)
Pharmaceutical companies can use PrecisionTree (part of the DecisionTools Suite that creates multi-phase decision trees) to investigate the best clinical trial sequencing pathway for a novel drug in pursuit of multiple potential diseases (indications). They can then conduct sensitivity analysis to the dominant decision, determine the maximum cost of an initial proof-of-concept (POC) study, and then communicate findings to executives. This case study analyzed three different strategic alternatives (pathways) for indication sequencing for an anti-inflammatory drug in pursuit of 3 diseases (asthma; inflammatory bowel disease (IBD); lupus erythematosus (LE)):
- Sequencing POC trials for asthma first
- Sequencing POC trials for IBD first
- Sequencing POC trials for LE first
Summary of Decision Tree Analysis Example
Drug development is an expensive process that takes place over very long timeframes with high technical risk and commercial uncertainty. Advancing from the drug discovery phase through clinical trials to Food and Drug Administration (FDA) or European Union (EU) approval can take more than a decade and cost hundreds of millions of dollars.
Most modern compounds can be used as therapies for different diseases, and companies must decide on the order in which they will conduct clinical trials for the different diseases a drug may treat. This process, known as indication sequencing, can influence a company’s direction and future success.
Using PrecisionTree to create decisions trees that model the probability of phase-specific success a drug will encounter with treating different diseases and then conducting a sensitivity analysis to determine how far from those probabilities an option can deviate before altering the dominant decision can help companies break down the complexity and uncertainty of this decision at each stage of drug development and make informed choices.
Background
Dr. Richard Bayney is President and Founder of Project & Portfolio Value Creation (PPVC), a consulting boutique that provides services, training, and education in strategic planning, decision analysis, and portfolio management. He has more than 20 years of experience in Decision Analysis and Portfolio Management in the pharmaceutical industry with roles at Johnson & Johnson Pharmaceutical Research and Development, Bristol-Myers Squibb, Bayer, and Merck & Co.
He uses PrecisionTree and @RISK, Lumivero solutions contained within the Decision Tools Suite, to help pharmaceutical clients make informed decisions about indication sequencing and drug development.
Using PrecisionTree
When a pharmaceutical company is investigating anti-inflammatory compounds that can treat multiple indications PrecisionTree can help create multiple decision pathways that map out possible strategies for clinical trials, the likelihood of success of each strategy, and the estimated return on investment for each alternative. Three pathways are considered: testing the drug compound for effectiveness in treating asthma first via a Proof Of Concept (POC) trial, testing for IBD efficacy first with its own POC, or testing for LE efficacy first with its own POC.
Dr. Richard Bayney
President and Founder of Project & Portfolio Value Creation
Indication Sequencing with Decision Tree Analysis
The decision tree diagrams shown here indicate the following:
- The overall risk-adjusted value for each pathway. The number in red at the right-hand side of the analysis shows the risk-adjusted Net Present Value (eNPV) in millions of dollars. The result here indicates that testing the IBD proof of concept first is in the dominant decision with the highest eNPV ($552 million).
PrecisionTree model with positively correlated risks – well advised since the likelihood of the first indication success influences the likelihood of both second and third indication successes.
- A breakout of the decision tree in Excel for the asthma-first sequence. This model shows that there is only a 17% chance of approval for all three indications if this pathway is followed with a dominated eNPV of $348M.
PrecisionTree model asthma POC first – 17% probability of all three indications being successful. - A breakout of the lupus erythematosus-first sequence. This model shows that there is a 27% chance of successful development for all three indications if this pathway is followed with a dominated eNPV of $346M.
PrecisionTree model LE POC First – 27% probability of all three indications being successful. - A breakout of the Inflammatory Bowel Disease-first sequence. This model shows a 31% chance that a drug will reach approval for all three indications if this sequence is followed. As the option with the highest eNPV of $552M, this is known as the “dominant decision.”
PrecisionTree Model IBD POC First – 31% probability of all three indications being successful.
Sensitivity Analysis Using PrecisionTree
Once a dominant option has been identified, a sensitivity analysis can help interrogate the robustness of the analysis and determine any breakpoint—e.g. how low a phase-specific judgmental probability of success can go before the dominant decision changes.
In the PrecisionTree model for the IBD-first sequence above, the model shows a 70% chance that the drug under investigation will be successful in treating IBD during the POC phase. The illustration below shows how low that percentage would need to drop before one of the other pathways became a better option. The expected value of the IBD-first sequence only drops below the expected values of the other two decision pathways when the probability of success of the IBD POC phase is lower than 45%.
Strategy Region of Decision Tree “IND SEQ 1.0 (2) (4)’. Sensitivity analysis to IBD POC probability of success (POS).
This process can also be used to conduct any two-way sensitivity analysis, e.g. looking at the potential breakpoints for the combined probability of success of the IBD POC and the follow-up phase, Phase III, that involves testing the drug for statistical effectiveness against asthma as well. This generates a different strategy region visualization that allows for identification of breakpoints based on two variables.
Using Bayesian Revision to Determine How Much to Pay for a Proof-of-Concept Study
PrecisionTree makes it possible to test the conditional probability of success based on prior information in a process known as Bayesian Revision — a statistical calculation based on a theorem developed by the 18th-century mathematician William Bayes. In clinical practice, attending physicians routinely prescribe diagnostic assessments to revise their prior judgments that patients displaying symptomatic illness are afflicted by one condition or another. Based on the historical sensitivity and specificity of such diagnostic techniques, they will then revise their clinical judgments that may result in altered treatment.
In the case of pharmaceutical indication sequencing, Bayesian Revision can help determine how much to pay for the initial POC study for the IBD decision sequence. In this case, a prior judgment (prior probability) of Phase 3 success is estimated at 20% (bottom arm of decision tree) but is revised (posterior probability) to almost 37% if the IBD POC study is successful with its own judgmental probability of success of 38% (upper arm of decision tree).
Expected Value of Sample Information – determining how much to pay for a POC study that is informative but imperfect (ll).
The illustration above shows the expected values for a Phase III trial result conducted with a POC study (upper decision node) and the expected values for a Phase III trial result without a POC study (lower decision node). The expected value of sample information (ESVI) can be determined by subtracting the value of the lower node from the upper node. In this case, the result is $72 million.
Results
PrecisionTree identified the dominant indication sequencing strategy, provided a sensitivity analysis across multiple parameters, and even allowed for an estimate of the maximum amount the drug developer should be willing to pay for a POC study.
A Competitive Edge for Indication Sequencing
PrecisionTree has allowed Dr. Bayney to use decision trees in Excel to decompose the complexity of indication sequencing phase by phase. By conducting probabilistic analysis for different indication sequences, determining breakpoints through sensitivity analysis, and estimating the costs of conducting an initial POC study, PrecisionTree makes it possible to offer pharmaceutical executives the information they need to make the best possible decisions under conditions of high risk and uncertainty when scheduling clinical trials that will impact a company’s direction for years.
Make a Decision Tree for Yourself
To get started fast, download our free example model, Portfolio Evaluation of Multi-phase Project and a free 15-day trial of DecisionTools Suite. This interactive example model looks at cash flow of projects with multiple phases such as those typically found in the pharmacological industry.
Interested in learning more about how PrecisionTree, included in DecisionTools Suite, can help guide your pharmaceutical indication sequencing? Request a free demo today!
You can also join the Lumivero community to connect with current PrecisionTree users and discuss how they’ve used it to improve their decision making.