Monte Carlo Analysis - more information

Monte Carlo - more information

Risk in Project Planning

In this article we will discuss how to model risk in project planning. Two primary sources of risk are time and cost; we'll focus on time because it is slightly more complicated; a similar analysis can be applied to cost planning.

The risk in planning for project completion time is, naturally, that your projections are incorrect. Underestimating project time can result in additional costs and potential penalties. Overestimating project time can result in wasted resources.

With practice, and good analytical techniques, you can generally construct a reasonable projection for project completion time. But in the real world, it's not possible to anticipate every contingency, and even the best project planning will include some over- or underestimates.

To take this reality into account, it's common in project planning to produce multiple estimates for project completion time. Estimating best-case, worst-case, and expected estimates gives you a range of completion times, and tries to take into account the risk of unanticipated factors.

The Basic Project Plan

Consider a simple project plan. This plan includes a single project, with three tasks:

Task	Best-Case	Expected	Worst-Case
Task 1	10 Days	20 Days	30 Days
Task 2	10 Days	20 Days	30 Days
Task 3	5 Days	10 Days	20 Days

Let's further assume that these tasks must be completed in sequence, meaning each task is dependent on the task before it. How long will the project take to complete?

You can look at the projections for each task and add them; this will give you the best-case, expected, and worst-case scenario for the entire project.

Task	Best-Case	Expected	Worst-Case
Task 1	10 Days	20 Days	30 Days
Task 2	10 Days	20 Days	30 Days
Task 3	5 Days	10 Days	20 Days
Total	25 Days	50 Days	80 Days

This is useful, but still unrealistic; because it's unlikely that each task will take the best-case time, or the worst-case time. More likely, one task will take a little longer, and another will be completed ahead of schedule.

This is where Monte Carlo analysis can be helpful. Starting with the estimates for the project, we can run an analysis based on random estimates for each task. This will produce a model that takes into account variability, and also considers that each task is independent.

Adding Randomness

To do that, the first step is to add a random variable which models each task. There are two distributions commonly used to do this: the beta-PERT distribution (also called just PERT distribution), and the triangular distribution.

For this example, we will use the PERT distribution. This distribution is used for modelling expert data when, as here, we have estimates for the range of possible values. It takes a minimum, maximum, and most likely value, and returns a sample from that distribution. The distribution creates a smooth curve. For a discussion of the characteristics of the PERT distribution, see this article on our website.

Task	Best-Case	Expected	Worst-Case	Sample (PERT)
Task 1	10 Days	20 Days	30 Days	24 Days
Task 2	10 Days	20 Days	30 Days	16 Days
Task 3	5 Days	10 Days	20 Days	13 Days
Total	25 Days	50 Days	80 Days	53 Days

In this case, with random values selected for each task, you can see that the first task takes a little longer than expected. The last task takes longer as well. The second task is completed ahead of schedule.

Overall, this is the sort of thing we would expect to see in the real world; not the best case, not the worst case, but somewhere in between. A single random case is not useful, however, because it represents only one out of essentially limitless possibilities. This is where we can use Monte Carlo simulation to measure understand the risk.

Monte Carlo Analysis

In a Monte Carlo analysis, we run the same model — selecting a random value for each task — but we do it hundreds or thousands of times. Each time it runs, we record the values. When the simulation is complete, we can look at statistics from the simulation' to understand the risk in the model.

We'll start with the basic statistics: the minimum, maximum, and average.

Task	Best-Case	Expected	Worst-Case	Sample (PERT)	Minimum	Average	Maximum
Task 1	10 Days	20 Days	30 Days	...	11 Days	20 Days	29 Days
Task 2	10 Days	20 Days	30 Days	...	10 Days	20 Days	29 Days
Task 3	5 Days	10 Days	20 Days	...	5 Days	11 Days	19 Days
Total	25 Days	50 Days	80 Days	...	32 Days	51 Days	70 Days

Three notes about this table before we continue:

1. This data comes from a Monte Carlo analysis of 1,000 trials.
2. The last row contains statistics calculated from the total — these are not sums of the columns above them. 
3. The statistics are rounded to the nearest day. The actual average for the total, for example, was 50.83.

The first thing we can take away from this data is that the average completion time is close to our central estimate. That is what we would expect; it's the most likely case for each task. Next, we can see that the best and worst cases from the simulation — the minimum and maximum of the total — are not as extreme as the absolute best and worst cases, if we just summed up the estimates.

That is to be expected as well, because the tasks are independent. Even if one task takes the worst-case time, it's not necessarily likely that another task will meet the worst-case estimate. The Monte Carlo model helps capture this independent variability, and allows us to tighten up the estimates a bit. We can now say that the worst case scenario is 70 days, instead of 80.

Next, we can look at the range of data generated during the Monte Carlo simulation to understand more about the projections.

This chart plots the time, in days (at the bottom), against the probability (on the left) of completion within that time. It shows there is only a 5% chance that the project will be completed within 40 days; but there is a 94% chance the project will be completed within 60 days.

What does that mean? When we ran the Monte Carlo simulation, we used random values for each task (based on the estimates), and found the total time for the project. We did that 1,000 times, and each time different random values were selected, resulting in a new random total.

When the simulation was complete, we look at every trial and count the number of times that the total was 40 days or less. We find that in only 5% of the trials (50 out of 1,000) the total project time was 40 days or less. We can therefore say that, during the simulation, there was a 5% probability that the project was completed within 40 days.

Similarly, during the simulation there was a 94% chance that the project was completed within 60 days.

We can also reverse the question, and ask what the completion time was — based on probability from the simulation — at various probability levels. If we want to be 80% sure that the project will be complete, for example, we need to budget 57 days for the project; because in 80% of the cases the project took at least 57 days to complete.

Key Points

Based on this analysis, we can find a couple of key points from the model and the analysis:

• Estimated Range is 32-70 Days
  Even though our best- and worst-case estimates result in totals of 25-80 days, from the simulation we can project that the range will actually be narrower.
• Probability of Completion within 50 Days is 50%
  Our original estimate was 50 days. The probability of hitting this target, based on the model, is only 50%. That might seem obvious, but it's actually only because this is a very simple model. When you have a more complex model with dependencies between tasks and sub-projects, the probability of the "Most Likely" value can often be quite different.
• For 75% Confidence, Budget 55 Days; for 85% Confidence, Budget 58 Days
  We can say that in 75% of the simulation trials, the project was completed within 55 days. We can budget for various confidence levels by looking at probability from the simulation.

Caveats & Conclusion

The analysis above, and the data taken from the simulation, is entirely based on the project estimates we created in the first step. The validity and the usefulness of the analysis, therefore, is only as good as our ability to estimate.

If the estimates are too broad, or too narrow, or even if the "Most Likely" point is at the wrong place between the values, the analysis may misstate the overall risk.

It's also important not to create estimates that are too broad, and assume you can use the analysis to narrow the probabilities.

The estimates are the most important part of the model. That much, in project planning, is a combination of art and skill. Monte Carlo analysis is just a tool that compliments this. If you can generate viable project estimates, however, Monte Carlo analysis can be invaluable in helping identify and understand risks in your project planning models.

What is Monte Carlo Analysis?

What is Monte Carlo Analysis?

Monte Carlo analysis involves determining the impact of the identified risks by running simulations to identify the range of possible outcomes for a number of scenarios. A random sampling is performed by using uncertain risk variable inputs to generate the range of outcomes with a confidence measure for each outcome. This is typically done by establishing a mathematical model and then running simulations using this model to estimate the impact of project risks. This technique helps in forecasting the likely outcome of an event and thereby helps in making informed project decisions.

While managing a project, you would have faced numerous situations where you have a list of potential risks for the project, but you have no clue of their possible impact on the project. To solve this problem, you can consider the worst-case scenario by summing up the maximum expected values for all the variables. Similarly, you can calculate the best-case scenario. You can now use the Monte Carlo analysis and run simulations to generate the most likely outcome for the event. In most situations, you will come across a bell-shaped normal distribution pattern for the possible outcomes.

Let us try to understand this with the help of an example. Suppose you are managing a project involving creation of an eLearning module. The creation of the eLearning module comprises of three tasks: writing content, creating graphics, and integrating the multimedia elements. Based on prior experience or other expert knowledge, you determine the best case, most-likely, and worst-case estimates for each of these activities as given below:

Tasks	Best-case estimate	Most likely estimate	Worst-case estimate
Writing content	4 days	6 days	8 days
Creating graphics	5 days	7 days	9 days
Multimedia integration	2 days	4 days	6 days
Total duration	11 days	17 days	23 days

The Monte Carlo simulation randomly selects the input values for the different tasks to generate the possible outcomes. Let us assume that the simulation is run 500 times. From the above table, we can see that the project can be completed anywhere between 11 to 23 days. When the Monte Carlo simulation runs are performed, we can analyse the percentage of times each duration outcome between 11 and 23 is obtained. The following table depicts the outcome of a possible Monte Carlo simulation:

Total Project Duration	Number of times the simulation result was less than or equal to the Total Project Duration	Percentage of simulation runs where the result was less than or equal to the Total Project Duration
11	5	1%
12	20	4%
13	75	15%
14	90	18%
15	125	25%
16	140	28%
17	165	33%
18	275	55%
19	440	88%
20	475	95%
21	490	98%
22	495	99%
23	500	100%

This can be shown graphically in the following manner:

What the above table and chart suggest is, for example, that the likelihood of completing the project in 17 days or less is 33%. Similarly, the likelihood of completing the project in 19 days or less is 88%, etc. Note the importance of verifying the possibility of completing the project in 17 days, as this, according to the Most Likely estimates, was the time you would expect the project to take. Given the above analysis, it looks much more likely that the project will end up taking anywhere between 19 – 20 days.

Benefits of Using Monte Carlo Analysis

Whenever you face a complex estimation or forecasting situation that involves a high degree of complexity and uncertainty, it is best advised to use the Monte Carlo simulation to analyze the likelihood of meeting your objectives, given your project risk factors, as determined by your schedule risk profile. It is very effective as it is based on evaluation of data numerically and there is no guesswork involved. The key benefits of using the Monte Carlo analysis are listed below:

• It is an easy method for arriving at the likely outcome for an uncertain event and an associated confidence limit for the outcome. The only pre-requisites are that you should identify the range limits and the correlation with other variables.
• It is a useful technique for easing decision-making based on numerical data to back your decision.
• Monte Carlo simulations are typically useful while analyzing cost and schedule. With the help of the Monte Carlo analysis, you can add the cost and schedule risk event to your forecasting model with a greater level of confidence.
• You can also use the Monte Carlo analysis to find the likelihood of meeting your project milestones and intermediate goals.

Now that you are aware of the Monte Carlo analysis and its benefits, let us look at the steps that need to be performed while analysing data using the Monte Carlo simulation.

Monte Carlo Analysis: Steps

The series of steps followed in the Monte Carlo analysis are listed below:

1. Identify the key project risk variables.
2. Identify the range limits for these project variables.
3. Specify probability weights for this range of values.
4. Establish the relationships for the correlated variables.
5. Perform simulation runs based on the identified variables and the correlations.
6. Statistically analyze the results of the simulation run.

Each of the above listed steps of the Monte Carlo simulation is detailed below:

1. Identification of the key project risk variables: A risk variable is a parameter which is critical to the success of the project and a slight variation in its outcome might have a negative impact on the project. The project risk variables are typically isolated using the sensitivity and uncertainty analysis.
   Sensitivity analysis is used for determining the most critical variables in a project. To identify the most critical variables in the project, all the variables are subjected to a fixed deviation and the outcome is analysed. The variables that have the greatest impact on the outcome of the project are isolated as the key project risk variables. However, sensitivity analysis in itself might give some misleading results as it does not take into consideration the realistic nature of the projected change on a specific variable. Therefore it is important to perform uncertainty analysis in conjunction with the sensitivity analysis.
   Uncertainty analysis involves establishing the suitability of a result and it helps in verifying the fitness or validity of a particular variable. A project variable causing high impact on the overall project might be insignificant if the probability of its occurrence is extremely low. Therefore it is important to perform uncertainty analysis.
2. Identification of the range limits for the project variables: This process involves defining the maximum and minimum values for each identified project risk variable. If you have historical data available with you, this can be an easier task. You simply need to organize the available data in the form of a frequency distribution by grouping the number of occurrences at consecutive value intervals. In situations where you do not have exhaustive historical data, you need to rely on expert judgement to determine the most likely values.
3. Specification of probability weights for the established range of values: The next step involves allocating the probability of occurrence for the project risk variable. To do so, multi-value probability distributions are deployed. Some commonly used probability distributions for analyzing risks are normal distribution, uniform distribution, triangular distribution, and step distribution. The normal, uniform, and triangular distributions are even distributions and establish the probability symmetrically within the defined range with varying concentration towards the centre. Various types of commonly used probability distributions are depicted in the diagrams below:
   
   
4. Establishing the relationships for the correlated variables: The next step involves defining the correlation between the project risk variables. Correlation is the relationship between two or more variables wherein a change in one variable induces a simultaneous change in the other. In the Monte Carlo simulation, input values for the project risk variables are randomly selected to execute the simulation runs. Therefore, if certain risk variable inputs are generated that violate the correlation between the variables, the output is likely to be off the expected value. It is therefore very important to establish the correlation between variables and then accordingly apply constraints to the simulation runs to ensure that the random selection of the inputs does not violate the defined correlation. This is done by specifying a correlation coefficient that defines the relationship between two or more variables. When the simulation rounds are performed by the computer, the specification of a correlation coefficient ensures that the relationship specified is adhered to without any violations.
5. Performing Simulation Runs: The next step involves performing simulation runs. This is typically done using a simulation software and ideally 500 – 1000 simulation runs constitute a good sample size. While executing the simulation runs, random values of risk variables are selected with the specified probability distribution and correlations.
6. Statistical Analysis of the Simulation Results: Each simulation run represents the probability of occurrence of a risk event. A cumulative probability distribution of all the simulation runs is plotted and it can be used to interpret the probability for the result of the project being above or below a specific value. This cumulative probability distribution can be used to assess the overall project risk.

Probabilistic analysis of project

Quantitative Risk Analysis:

Probabilistic analysis of project 

Probabilistic analysis of the project. Forecasts of potential project schedule and cost results listing the possible completion dates or project duration and costs with their associated confidence levels.

How probable are we to achieve our Project Schedule? How probable are we to complete the project within budget? These are some of the probabilities that we aim at identifying and updating the Risk Register during this step

he PMBOK guide says that, the result of this kind of analysis is typically displayed as a Cumulative Distribution. These distributions are then used along with Stakeholder Risk Tolerances to allow for the quantification of Cost and Contingency Reserves. As with any value that is calculated by us, these reserves must be appropriate and realistic.

Quantitative Risk Analysis

Quantitative risk analysis – and how to perform it
Quantitative risk analysis assigns a projected value(usually this value is stated in terms of cost or time) to the risks that have already being ranked by the previous process ‘perform qualitative risk analysis’.
People often confuse these two processes which are normally performed at the same time. Perform quality of risk analysis however is determining the probability and impact of the risks to the project and going on to prioritize and rank them on the risk register. The outputs from this process will be used to plan risk responses and also to monitor and control risks.

There are five inputs to perform quantitative risk analysis:

The risk register. This contains a list of all of the identify risks so far on the project, and includes information on each such as their responses, their records is an categories.
The risk management plan. This document is in fact the risk management strategy because it defines the level of risk which is seen as tolerable, how such risks will be managed, who will be responsible for carrying out the risk activities, the time and cost aspects of each risk activity and how the communication of risk is to occur.
Schedule management plan. Because the schedule timings are presented in a quantifiable manner then risks concerned with timing and time scales can easily be quantified within this process.
Cost management plan. Similar to the above, costs are also quantifiable and can be used as an input for this process. Note that the scope management plan is not quantifiable and is therefore normally used within the qualitative risk analysis process.
Organizational process assets. These may consist of risk templates, policies procedures or guidelines, lessons learned from previous or similar projects, and any quantitative risk tools.