London Underground uses risk consultants Istria Ltd and its software tool IRIS, with @RISK embedded as the engine, to manage diverse risks from investment projects to passenger safety.

When it opened in 1863, London Underground was the world’s first underground railway. Today, this vital metro system, known locally as ‘The Tube’, provides over three million passenger journeys a day to and from 275 stations. The network stretches across 253 miles (408 km) and is operated, maintained and supported by 12 000 staff. London Underground is crucial to the economy of London and, in turn, of the United Kingdom as a whole. The risks inherent in the operation of such a complex and vital system are many.

IRIS and @RISK Central to London Underground Operation

London Underground uses Risk Consultants Istria Ltd and its software tool IRIS to manage its diverse risks. Everything from investment projects to passenger safety must be considered, and IRIS, with the embedded Palisade @RISK engine, are well-suited to the task.

Istria believes that, like quality, risk management should be designed-in to the planning and implementation of all plans and activities throughout an organisation, rather than being an add-on, peripheral ‘chore’ conducted at the behest of Head Office. Istria’s IRIS software (Istria Risk and Issue Support) provides a comprehensive risk solution based on Istria’s Enterprise Risk Management Method. @RISK is employed within IRIS to generate accurate management information using Monte Carlo simulation. This @RISK data analysis process is critical to effective decision-making.

Financial Benefits and Corporate Goals

London Underground’s Tylaar Haran (Group Facilities UIP Projects Manager) is quick to praise the value of using @RISK to augment IRIS. “London Underground has derived significant financial and performance benefits from using IRIS and its excellent @RISK-driven Monte Carlo functionality. The combination of IRIS and @RISK is a powerful tool to support decisions on the feasibility of potential projects and to drive-forward the identification and selection of options in complex, uncertain scenarios.”

Istria’s Risk Method links all risk-related activities directly to the attainment of corporate goals and objectives. As Chief Executive Colin Wheeler observes, “Increasingly, companies and government bodies are realising that risk management can provide tangible benefits in improving the ‘bottom-line’ or enhancing service delivery. All too often, however, the approaches used to introduce and operate risk management have no clear link to the overall strategic aims of the organisations. Istria’s method maintains a clear, auditable link to strategic and operational success criteria; this focuses the risk management effort on supporting the attainment of corporate goals and objectives.”

@RISK is used at AMSAA to help senior level officials avoid risks of schedule and cost overruns in equipping and sustaining weapons and materiel for U.S. soldiers in the field.

The US Army Materiel Systems Analysis Activity (AMSAA) conducts critical analyses to provide state-of-the-art analytical solutions to senior level Army and Department of Defense officials. Analyses by AMSAA support the equipping and sustaining of weapons and materiel for U.S. soldiers in the field, and inform plans for the future. @RISK is used at AMSAA to help senior level officials avoid the risks of schedule and cost overruns. New @RISK models integrate schedule and cost consequences, and employ Monte Carlo simulation to give decision-makers the best information possible.

Integrated Model Addresses Schedule and Cost Risks at AMSAA

One of the top priorities of the U.S. Army is to make better decisions in their acquisition programs. When considering any new materiel, decision-makers want to know the risk, cost, performance and operational effectiveness of each new acquisition. In this case, risk is the likelihood of not meeting the target time with schedule and cost consequences.

The US Army Materiel Systems Analysis Activity, known as AMSAA, conducts critical analyses to equip and sustain weapons and materiel for soldiers in the field and future forces. The Army is charged with determining the best possible choice among several acquisition options, taking care to examine alternatives in tradespace, sensitivity, and cost and schedule risk mitigation. Currently, with the help of @RISK, AMSAA conducts separate schedule and technical risk assessments, which are not connected to the cost risk assessment (approaches are in silo). However, AMSAA Mathematician and Statistician John Nierwinski decided to use @RISK to integrate schedule and cost consequences in order to provide a single risk rating that decision-makers can efficiently use to inform the overall decision. “This is cutting edge stuff,” says Nierwinski. Discussions are in process on ways to implement this in the future and gradually phasing out of the silo approach to risk evaluation. This modeling approach has not yet been applied to any acquisition studies; however, some of its submodels have been applied. Integrated risk ultimately translates to integrated schedule and cost that can be traded off with performance and operational effectiveness.

Monte Carlo Simulation Exposes the Risk of Project Overruns

The first step in Nierwinski’s analysis requires a schedule network model, with technology development, integration, manufacturing, and other events. Completion time for these events are assessed using data or subject matter experts (SME). If SME’s are used it is common to use a triangular time distribution with a minimum, maximum and most likely number of months to progress from Milestone (MS) B to MS C. @RISK uses this schedule network model as the main driver of the Monte Carlo process, which is an integrated risk model. For each iteration output of the @RISK model, a consequence determination is made using cost and schedule if the time exceeded the MS C target date. A cost model is used based on the technology cost information that is available. The main cost items are due to the fact that schedule delays can increase variable costs, such as labor expenses. Acquisition variable costs are typically estimated from database or expert information. Hence, if a program goes over the milestone date by a number of months, then the additional cost to the project will be the number of months times the variable cost per month.

With schedule and cost variables in place Nierwinski then creates integrated outputs from the model, which will then, with several thousand iterations, create scenarios that involve an overrun or underrun, depending on the iteration. For overrun situations there are two sets of consequences, cost and schedule. Each set of consequences range from 1-5 (best-case to worst-case scenarios). A given simulated run will yield a cost and a schedule consequence. The maximum of the two consequences is selected for each simulated run. This is called the max consequence. If the target time is not exceeded for a given simulated run, then the max consequence is set to none or 0.

Getting a Single Risk Rating

After running the integrated model, @RISK gives an output that tells the likelihood of not meeting the schedule, and the collection of maximum consequences from schedule overrun scenarios.

As the figure above indicates, ‘0’, is the chance that there will be no schedule overrun. Nierwinksi then translates this finding into a matrix to view the risk distribution in table form:

“This allows you to see your probabilities of being in each of these consequence ‘buckets’,” says Nierwinksi. Because this data is a probability distribution, it is easy to determine the probability of not meeting the schedule on time by subtracting the probability of meeting the milestone from 1, (1 – 0.18= 0.82) meaning there is an 82% likelihood of missing the schedule date.

Color-Coding Risk

Nierwinski then applies this distribution to a pre-established DOD color-coding system known as the DOD risk reporting matrix to determine the transformed risk distribution in the lower right hand corner of the figure below. The risk reporting matrix is typically used to report a risk level within a program. The level of risk is reported as low (green), moderate (yellow), or high (red) based on the mapping of the likelihood and consequence to a single square on the risk reporting matrix.

To compute probabilities in the transformed risk distribution, Nierwinski divides probabilities for consequence 1–5 by the probability of missing the schedule date, 0.82 (for example, consequence 1 = .25 / .82 = .30). The transformed consequence level probability is the conditional probability of the consequence level given a schedule overrun. The likelihood of 0.82 corresponds to a likelihood level of 5 in the risk reporting matrix (see figure above) therefore colors are green, yellow, red, red, and red for each consequence. Once Nierwinski formats the risk distribution, he determines an expected color by multiplying the probability of obtaining each color by 1, 2 or 3 (numerical rating: green = 1, yellow = 2, and red = 3). The expected color rating in the example above is:

1(.304) + 2(.12) + 3(.15 + .12 + .302) = 2.27

Next, he transforms the discrete scale of green/yellow/red colors to a continuous numerical scale. Since colors range from green (1) to red (3), the numerical difference is 2. Therefore, a range of 1 to [1 + 2/3 = 1.667] is green; a range of 1.667 to [1.667 + 2/3 = 2.333] is yellow; and 2.333 to 3 is red. Therefore, in our example, a 2.27 rating implies a yellow or medium risk rating for the alternative.

@RISK Brings Flexibility to Modeling Material Acquisitions at AMSAA

“Once we’ve established a risk rating for a certain materiel option, let’s say it’s a particular alternative for a kind of tank, we can do all sorts of studies (i.e. tradespace, what-if scenarios, risk mitigations)—we can examine what would happen to the risk if we were to swap out its engine for a cheaper one,” says Nierwinski. “This can change a lot of things. We can study how the risk rating changes: for example, it may go from high risk to low risk.”

Furthermore, we can extrapolate risk into integrated cost and schedule output, make joint probability statements, and conduct tradeoff decisions with performance and operational effectiveness. @RISK informs this acquisition decision making process.

Nierwinski says that @RISK was instrumental in modeling the inherent risk and uncertainty with materiel acquisitions that AMSAA faces. “@RISK enables us to build various kinds of risk models quickly, with lots of flexibilities.”

Different factory units need regular maintenance. During the maintenance work the production has to be shut down, which causes a reduction fall in the company’s income and profit.

The goal of applying risk management was to determine how a manufacturing company can ensure the highest level of income / profit by assessing and tackling different risks that occur during maintenance work.

The Szigma IntegRisk® method, developed by Hungarian Risk Management Company SzigmaSzervíz Ltd, uses Microsoft Project® 2013 and @RISK (risk analysis add-in for Microsoft Excel and Project) for the analysis (now known as ScheduleRiskAnalysis from a 2022 feature update included in DecisionTools Suite).

The Szigma IntegRisk® method is an integrated risk assessment, treatment and monitoring technique for supporting management decisions on strategic, tactic and operational level.

Dr. István Fekete, Managing Director for SzigmaSzervíz, believes that a combination of @RISK and Szigma IntegRisk® facilitates an integrated approach to quantifying schedule risks in case there are not historical data available.

Solution

The project managers had prepared a schedule for maintenance work with the help of Microsoft Project®. The length of the simplified project schedule’s critical path was calculated. In this example, the original duration before risk assessment is 26 days. (See Figure 1.) This same approach can be used with the new ScheduleRiskAnalysis which also features the ability to upload your Primavera P6 or Microsoft Project files to @RISK.

Figure 1: Simplified project schedule

The next step was to assign potential schedule risk sources/risk events for each project activity in a series of workshops. A risk database compiled on the basis of several years of experience was used (included in Szigma IntegRisk® method). An example can be seen in Table 1.

Scenario analysis was performed as the next step to describe a series of impacts and the probability of their occurrence for each identified risk event. (An impact refers to a risk event that can affect the original duration of a project activity). Scenario analysis is a useful tool to describe the perception of different experts and produces more reliable input data for Monte-Carlo simulation, if historical data are not available.

Selection of the critical risks was based on the previously defined threshold values. If the calculated mean value and/or standard deviation overrun the threshold, that risk is labelled as critical risk.

Following that, risk treatment actions were created for all critical risks. (See Table 1)

Table 1: Example of risk assessment

The next step was to run Monte Carlo simulation using @RISK. During the simulation, lognormal distributions have been included by using mean value and standard deviation from the scenario analysis (see Table 1). SzigmaSzerviz’s experience shows that lognormal distribution is the best way to describe the nature of the revised activity duration once identified risks have occurred. This identifies the risk events that might cause huge deviation, but have a very low probability of occurrence compared to original activities’ duration. As a result, the probability distribution of the modified length of critical path was calculated. (See Figure 2.)

Figure 2: Probability distribution of the length of critical path

After finishing Monte Carlo simulation, a Tornado-diagram was generated to show the activities most likely to be responsible for delays to the project. The identified risk events assigned to most of these activities might cause the biggest deviation compared to the original length of critical path so the risk treatment actions for these activities are first carried out. (See Figure 3)

Figure 3: Tornado diagram

As a result of Monte Carlo Simulation the difference between mean and original value of the length of critical path is roughly 4 days. (26 versus 30, 258 see Figure 2). If the duration of maintenance activities is longer than planned the restart of the production is delayed which causes significant income/profit deficit for the manufacturing company. For this reason it is vital to model the impact of the results of the risk assessment on production and income/profit plan of the company, as illustrated in Figure 4.

Figure 4: Essence of the developed model

The result demonstrates that, if the maintenance work is delayed by 4 days from the scheduled date due to inadequately managed risks, the profit before tax might decrease with HUF (24,981 – 23 606) = 1.375 Bn (€ 5M). However, undertaking risk management activity opens up the possibility of meeting the original deadline of maintenance works and avoiding the € 5M loss.

According to Mr. Fekete the key benefits of @RISK are as follows:

• A wide distribution gallery can be chosen for defining the probability distributions of input parameters
• Good communication between Microsoft Project® and @RISK
• Easy application
• Useful reporting opportunities for decision makers
• Combining Szigma Integrisk® and @RISK makes it possible to produce reliable output data if no historical data is available

About SzigmaSzervíz Ltd

SzigmaSzerviz Ltd., established in 2006, is the first Hungarian company engaged in integrated risk management. Szigma IntegRisk® risk management system and software is unique on the international market. This system gives an effective method for decision-makers, to support decisions ranging from strategic planning, to project management and annual planning, to internal auditing. SzigmaSzerviz Ltd. aims to foster risk awareness in organizations. Szigma IntegRisk® is not just software, but a complex business solution, which can provide a significant competitive edge to companies in the current economic environment.

New ScheduleRiskAnalysis with DecisionTools Suite

Solve schedule problems like one demonstrated in this SzigmaSzerviz Ltd. case study with the new ScheduleRiskAnalysis in DecisionTools Suite. Apply Monte Carlo simulation to Microsoft Project and Oracle Primavera P6 models, generate probabilistic Gantt charts, and easily calculate and report the Critical Index – all within Microsoft Excel! With probabilistic Gantt charts, you can see the likelihood of various durations and finish dates for tasks and entire projects. Plus, with critical indices, you can easily locate the tasks that matter most to your project’s critical path. Learn more about ScheduleRisksAnalysis.

Experience @RISK and DecisionTools Suite

To see how @RISK and DecisionTools Suite can help your business, request a free demo or download our free trial today.

Thales uses @RISK to quantify complex markets in of mission-critical electronic information systems for aerospace, defence, and security around the world.

Thales is a leading provider of mission-critical electronic information systems for aerospace, defence, and security markets around the world. With operations in 60 countries and 70,000 employees, it develops products for dual markets in recognition that civil and military systems benefit from many of the same technologies and innovations.

Risks and Challenges Addressed with @RISK

Thales operates in a highly competitive environment, with technologically advanced countries continuously providing tough opposition when it tenders for contracts. It is imperative therefore that Thales keeps up with the relentless pace of technology development—something made additionally challenging by the sophistication of the products in question. In addition, the critical and sensitive nature of its customers’ businesses demand that the equipment Thales develops and produces is rugged, robust, and failsafe.

Bringing products of this calibre to market is costly both in terms of time and resource. Consequently, for every opportunity to compete for new business (which in itself can be a long, and therefore expensive, process), Thales must be confident that it has a reasonable chance of success. It must also take into consideration that, while winning a contract is favourable, the long-term cost of then maintaining its market share can actually be prohibitive to pitching for the prospect in the first place. For example, once developed, sophisticated electronic systems for aircraft require continual upgrading—the cost of which may outweigh the original benefit (and profit projections) of the initial deal.

Thales uses @RISK software from Palisade to assist it in making these business-critical decisions. @RISK is an Excel add-in using Monte Carlo analysis to show all potential scenarios, as well as the likelihood that each will occur. @RISK enables Thales to calculate the competitiveness of complex markets, measure probabilities for project costs, quantify rate of return, and even account for the effects of cumulative business, thereby providing decision-makers with the most complete picture possible.

@RISK Quantifies Complex Markets

For example, Thales might need to decide whether to bid for developing a radar system for a particular type of combat aircraft. By developing a distribution model with @RISK, Thales can input unknown quantities such as production forecasts for that aircraft and account for them with distribution functions. Thales also inputs ‘known’ quantities, such as which countries and organisations are currently flying the same machine, and is then able to forecast the potential market for the product once it is completed. On the civil aviation front, the 25-year lifespan of aeroplanes offers the potential for several equipment upgrades during that time. For example, in-flight entertainment systems have changed dramatically over the past quarter of a century, with personalised systems that allow passengers to watch their individual choice of film now providing airlines with competitive advantage when selling tickets. Again, combining @RISK’s sophisticated prediction capability with human expertise and research enables Thales to ascertain whether the potential market is sufficient to warrant competing for a contract.

@RISK Weighs Up Project Cost vs. Probability

With the market potential ‘measured’, Thales needs to balance the overall cost for bringing the completed project to market against the probability of winning the business. Figures fed into the @RISK model include costs for staffing, new office openings, licensing, etc, as well as the adjustments needed for different countries and regions, such as providing instructions in different languages. Thales must also include a sum for ‘unknown’ or ‘unexpected’ costs.

To then calculate the probability of breaking even on a project, Thales must factor in the cost of preparing a tender with the probability of winning the business. Although bidding against three other operations could be perceived as offering a 25 percent chance of success, Thales believes it makes better business sense to err on the side of caution when making this estimation.

@RISK Measures Rate of Return

Adam Ogilvie-Smith, Senior Consultant at Thales Management Consultancy, explains: “In theory it would be tempting to try and win every piece of business that we are offered. But in the sophisticated and complex markets in which we operate, closer inspection of each case reveals this often is not possible, practical, or desirable. @RISK’s complex risk analysis capabilities provide us initially with a figure that is essentially an internal rate of return on the funds used to tender for and win a project—that is, will financing a business pitch and the subsequent product development bring an equivalent rate of return as investing the same amount of capital in a bank over the same period of time? In addition, the @RISK model helps us to determine the probability of securing the contract, and therefore whether it is strategically viable overall to pitch for the business.”

@RISK Accounts for Cumulative Business

Commercial agreements rarely occur in isolation. Thales’ @RISK decision models therefore ascend to another level by taking into account the cumulative effect of winning new business, for example including conditions such as winning project A being a prerequisite for pitching for project B. It also enables Thales to work with scenarios such as the initial probability of securing contracts in countries X and Y being factored as 40 percent and 30 percent respectively, but winning the contract in country X then making it easier to succeed in the next territory—country Y.

Taking this one step further, Thales can also calculate when it makes commercial sense to use this tactic strategically—for example, will winning a piece of business in one region be beneficial when tendering for another contract? @RISK helps Thales understand whether securing a contract with one region will be a route in to doing business in another.

@RISK Enables Business-Critical Decisions

Ogilvie-Smith concludes: “Our commercial success is determined by our ability to quantify both our internal operations, as well our position in the wider environment in which we operate—which is both dynamic and sensitive. It is essential therefore that we use a versatile and robust modelling tool that can handle the complexity of our queries. @RISK combines its powerful risk analysis capability with the added benefit of being easy and intuitive to use, and applicable for almost any task. As a result it is a key strategic tool for Thales, assisting us in our process of reaching informed business-critical decisions.”

The U.S. General Services Administration (GSA) provides contracts to U.S. Agencies for telecommunications and information technology. The newest set of services GSA facilitated are called Enterprise Infrastructure Solutions (EIS). GSA recently completed a project to acquire these services for Agencies’ use for the next 15 years, valuing at $50B. The GSA used @RISK throughout the acquisition project, which was performed on-time and on-budget. @RISK helped GSA executives understand the impacts of their decisions on schedule and cost within the framework of quantitative risk management.

Background

GSA provides contracts to U.S. Agencies for telecommunications and information technology services, with the newest set of contracts being Enterprise Infrastructure Solutions (EIS). EIS includes local, national and international voice and data services, optical services, wireless, satellite, security, cloud services, contact centers, building/campus networks, equipment and professional services.

The Hunt for Quality Contractors

In 2014 GSA began competitively acquiring EIS contracts, with the aim of providing government agencies EIS for up to 15 years for a total award value of $50B. A key goal was to award a robust set of contractors from which agencies could choose their services.

This EIS acquisition project was ambitious and complex, including hundreds of tasks performed over two years by teams of government staff and consultants. Team members included acquisition specialists, contracting officers, engineers, business operations professionals, and financial analysts. Moreover, the project depended on each bidding contractor understanding the requirements and developing responsive proposals.

@RISK Clarifies Consequences

To help manage the project, the GSA EIS Acquisition project manager and his team used @RISK. “The software helped GSA executives understand the quantitative risk of their decisions impacting both the cost and schedule of the project,” said the program manager.

Using Microsoft Project, the program manager and his team developed the schedule and resource loading in the traditional way and aligned with an Earned Value Management (EVM) approach to project management.

Once the schedule and resource loading was completed, the team imported the project into @RISK. The project management team then worked with the task teams to choose the probability distributions and parameters for each meaningful task. They next correlated the tasks, and then assigned probabilities to branches in the project. The team leadership then created a probabilistic calendar for resource availability.

Finally, the teams developed a thorough risk register of events that could impact schedule, and identified the tasks within the project that could be impacted by the risk events. GSA executives provided the required confidence level.

@RISK performed Monte Carlo simulations, usually with 5000 iterations. “The performance of @RISK was so fast and steady that we were able to perform the simulations in real time with GSA executives,” reported the program manager. “This allowed senior executives to see the Probability Distribution Function (PDF) of a milestone achievement based upon their decisions regarding scheduling tasks and resources, and escalating actions on solutions.”

By July of 2017, GSA had achieved this goal both on time and on budget, making 10 EIS awards.

“The initial @RISK simulations help set realistic expectations for milestone achievement,” explained the program manager, “and periodic simulations helped the team leadership and GSA executives understand where the risks were and provided an opportunity to test alternative mitigation strategies.”

Senior decision makers were also pleased with @RISK. “Project simulation enabled us to provide the agencies firm schedules,” reported one senior executive. “They could then stage their resources in a timely way to select their EIS vendor to obtain the earliest and greatest possible savings to the tax payer.”

Published: Dec. 17, 2021
Updated: Sept. 1, 2023

ScheduleRiskAnalysis

Manage Uncertainty in Project Schedules 

Start analyzing your schedule risk using Monte Carlo simulation with @RISK's ScheduleRiskAnalysis (SRA). SRA shows you virtually all possible outcomes in your schedule – and tells you how likely each is to occur. Plus, it works within the familiar Excel environment and lets you perform risk modeling on project files created in Primavera P6 and Microsoft Project. 

Request a Demo of DecisionTools Suite and access example models, case studies, webinars and more on the Lumivero resource page.