Welcome!

We hope you enjoyed reading the article ‘Quantitative Risk Analysis: Improving Decision Making by Quantifying Uncertainty’ in the April 2011 issue of the American Association of Pharmaceutical Scientists Magazine. We created this webpage to provide additional resources and information on quantitative risk analysis, including example models.

Download Paper

Additional resources

Quantitative risk analysis can be an arid and sometimes counter intuitive subject to learn. Fortunately, there are several electronic as well as printed resources that can help you learn more about quantitative risk analysis and its applications.  Below are a selected list of additional resources and literature that we have chosen based on our experience teaching quantitative risk analysis and risk modeling.

• Several of the authors of the current paper have also written a paper focusing on early drug development: Mudd P, Groenendaal H, Bush M, and Schmith V. (2010) Probabilistic Risk Analysis: Improving Early Drug Development Decision Making. Clinical Pharmacology & Therapeutics 88, 871-875.

• Thomas Shakespeare and his colleagues wrote an excellent paper on using probabilistic methods to interpret the results from clinical studies.

• And for those of you that like to read something on paper, here are some relevant books on QRA and simulation modeling:

o   Law A (2014) Simulation Modeling and Analysis, 3rd ed. McGraw-Hill.  Very thorough and mathematically oriented book. We recommend this book for people that would like to learn the nuts and bolts of quantitative risk analysis and simulation modeling, but it’s not the best book for a beginner.

o   Clemens R (2004) Making Hard Decisions.  South-Western College Pub. This is an interesting book that shows a variety of decision theory methodology from an applied point of view, while still presenting some of the theoretical background.

o   Lehman D, Groenendaal H, and Nolder G (2011) Practical Spreadsheet Risk Modeling for Management, 1st Ed. Chapman and Hall/CRC. A great general quantitative risk analysis book for beginner and intermediary practitioners. It includes many applied examples and accompanying models.

o   Gelman A, Carlin J, Stern H, Dunson H, Vehtari A, and Rubin D (2013) Bayesian Data Analysis, Third Edition, London: Chapman & Hall. Not a risk analysis book, but explains very clearly the Bayesian methods we often rely upon when performing quantitative risk analysis methods in pharma. Includes very good and entertaining examples.

• If you are looking for a more informal reading, please also check our white paper on the use of quantitative risk analysis in the pharmaceutical and medical device industries.

ModelAssist

ModelAssist is a FREE comprehensive quantitative risk analysis training and reference software. The ModelAssist software comes in two versions, one for @RISK users and one for Crystal Ball users (both software packages are Monte Carlo add-ins to Excel).

Three example models

We believe that one of the best ways to learn something is by seeing it in action. Therefore we have developed three relatively simple quantitative risk analysis models to illustrate some applications that may be useful to you. These examples are purposely simplified as they are meant as illustrations to demonstrate some of the basic concepts on risk analysis. Even though real-life situations are typically more complex, the same principles and ideas apply.

1. Comparing side effects
This model shows an example of how epidemiological data from literature can be taken into account and compared with clinical trial data. Instead of using a classical statistical test to determine if the rates of side effects are statistically different at some level of significance, we use here a Bayesian approach to estimate the confidence on the differences in rates, given the evidence provided by articles and clinical trial data. This concept is very similar to what is discussed by Thomas Shakespeake in his paper (see the resources section).

Use the following links to download the implementation of model #1 in @Risk®, ModelRisk®, and Crystal Ball®

2. Combining IC50 values in PK/PD model
The concept shown in this example is how data from difference sources and studies can be combined into one estimate (including all uncertainties within the different studies) and subsequently used in a model. In this example, IC-50 estimates from a rat study, a primate (“monkey”) study, and an in vitro human study are combined using different weightings to estimate (using a PK-PD model) the drug response a 6 hours post-dose.

Use the following links to download the implementation of model #2 in @Risk®, ModelRisk®, and Crystal Ball®

3. Cmax estimation at a certain dose
This model shows an example of how data from a very small clinical trial (n = 15) can be used to estimate variability within the overall patient population (using distribution fitting) as well as to estimate uncertainty around the model parameters given the small patient population.

Use the following links to download the implementation of model #3 in @Risk®, ModelRisk®, and Crystal Ball®