ENGCOMP is a Saskatchewan-based structural, mechanical and cost engineering consulting firm. Catering to Canada's commercial and heavy industrial market, it provides engineering services to the potash, uranium, oil and gas, pulp and paper, chemical processing, and food processing industries in Saskatchewan and Alberta.
With structural engineering as its core business, ENGCOMP also specialises in risk analysis, cost estimation, planning and computer task automation.
ENGCOMP was contracted to assist the Canadian Department of National Defence (DND) to define the budget for the fourth phase of construction of its ongoing Fleet Maintenance Facility Cape Breton (FMF CB) project located at the Canadian Forces Base Esquimalt, Victoria, BC. Using Palisade’s risk analysis software, @RISK, ENGCOMP conducted Monte Carlo simulations to quantify the uncertainty in defining the budget and schedule for this project.
The FMF CB project has been ongoing for more than 10 years. DND was looking to consolidate and upgrade the FMF, which includes smaller facilities, spread all over Base Esquimalt’s Dockyard area, into one large facility. To do this, DND needed to evaluate its remaining budget and existing schedule to complete the fourth phase of construction.
Whilst risk assessments had been conducted on the project in the past, a true risk analysis using Monte Carlo simulation had yet to be completed. Monte Carlo simulation, a quantitative statistical modelling tool, is important to this project as it can help reduce budget uncertainty and greatly increase the likelihood of achieving project success.
Jason Mewis, President, ENGCOMP, says, “Risk analysis is crucial to the cost and schedule management of any project, and must include a scientific approach to contingency and risk reserve estimation. As a concept, Monte Carlo simulation has been around for a long time, but is not widely used, and where it is used it is primarily applied to just capital cost estimation. A Monte Carlo-based simulation tool such as @RISK can help reduce uncertainty, greatly increasing the chances of project success.”
ENGCOMP developed a system that breaks down the Monte Carlo simulation into two – a simulation for Contingency Analysis and one for Project Risk Analysis.
Contingency is a very important aspect of budgeting and needs to be accounted for properly to ensure project success. It is the amount of money that needs to be added to a project budget to account for all the expected construction costs that haven’t been itemised at the time of budgeting.
As the first component of its Monte Carlo simulation-based risk analysis, ENGCOMP needed to determine the amount of contingency that would be required to be applied to the project budget, with a reasonable level of confidence, that the final approved budget for the FMF CB project would not be exceeded. Using @RISK, ENGCOMP aimed to quantify the potential variability of factors such as labour rates, material and equipment costs and productivity, to ascertain the amount of contingency that should be allocated to the overall project budget.
ENGCOMP assessed the contribution of the various work packages in the cost estimate to determine the total contingency required for the project. A work package represents a collection of work actions necessary to create a specific result. It is typically defined by statements of activity description, activity resources of skill and expertise, estimates of activity duration, activity schedule and activity risks.
ENGCOMP used @RISK to calculate the total project cost, both with and without estimated variability on the work packages. The difference between the two totals yielded the contingency for the project.
The results of the @RISK Monte Carlo simulation for the contingency analysis showed that the bulk of the budget uncertainty was due to market volatility and unknown site conditions. Existing FMF functions are located in facilities that were used for industrial functions that historically were relatively unregulated. Site investigations indicated potential contamination below these old structures. However, the requirement to maintain ongoing operations precluded the option of removing the buildings to conduct the comprehensive testing necessary to establish definitively the nature and extent of contamination and the related costs for removal and disposal. Therefore, the DND needed to assign significant contingency budget towards the demolition and decontamination activity.
The aim of the Project Risk Analysis was to take the Contingency Analysis to the next level, quantifying the effects of all reasonable risks and uncertainties on the project.
This component of the @RISK Monte Carlo simulation accounted for those items that are not required to construct the project, but should they occur, then DND would be expected to pay for them under the cost of the project. These factors included market conditions, environmental issues, internal operational issues and organisational changes that could collectively impact on the successful completion of the FMF CB project. For instance, the @RISK simulation aimed to quantify the uncertainty posed by unforeseen developments such as weather conditions preventing construction, labour strikes, delays in environmental approval, labour shortages, currency fluctuations, and safety, to name a few. Such occurrences can affect both the project’s cost and schedule.
@RISK enabled ENGCOMP to estimate the impact of the variability and uncertainties pertaining to risks, costs and scheduling. This assessment enabled them to estimate the project risk budget as well as the Risk Reserve and Schedule Contingency.
A key finding of the Project Risk Analysis was that, taking into account all the risk and uncertainties on the project, there is an 85 percent certainty that the FMF CB project will be completed by January 2014.
Canada’s DND operates infrastructure projects under highly regulated and controlled regime, as one might expect given the nature of the organisation. This means that securing project funding approval in a timely fashion represented a huge challenge for DND. For the FMF CB consolidation project, this challenge was further compounded as there were numerous other organisational and technical problems. Therefore, establishing and presenting the budget in a manner that would confidently demonstrate its successful completion was imperative.
Mewis explains, “We were able to help the DND define the budget as well as give them the tools to defend it. Based on our quantitative risk analysis, DND was able to clearly justify to the Federal Government’s Treasury Board why it should be allowed to get the capital appropriation for the project despite the level of uncertainty. This may not have been possible without the detailed and comprehensive analysis enabled by @RISK.”
The FMF CB project has been authorised and is in progress. In addition, due to the success of the risk analysis undertaken by ENGCOMP, DND is talking with the company about possibly preparing a policy on performing this level of detailed Monte Carlo simulation-based analysis for all future DND projects.
Distributions used Predominantly, ENGCOMP used the Trigen function, which enabled them to account for the inherent error relating to the subjective estimation of uncertainty. They used a structured calibration training process for all the analyses so that the ‘Subject Matter Experts’ (SME’s) could be as “good” as possible at estimating uncertainty. Also, as sometimes groups cannot be fully calibrated, the process highlighted to what degree the team was calibrated, giving ENGCOMP the ability to adjust the estimated values to reflect the appropriate level of confidence.
There were cases where ENGCOMP used a Pert distribution if there was a lot of skew towards a particular parameter in the estimated distribution. This helped smooth out the Triangular distribution and put less emphasis on the tails in the analysis.
Graph 1: ENGCOMP's Risk Induced Second S-Curve Diagram
This graph is used to clearly distinguish between the varying levels of cost uncertainty encountered on the project.
The blue line represents the base total cost estimate. The red S-curve is derived from the Contingency Analysis process and represents the variability in the cost estimate for the project. The difference between the blue line and the red S-curve is the ‘contingency’ cost that needs to be applied to the total project cost.
The green S-curve represents the results of the Project Risk Analysis (PRA), which adds the impact of schedule uncertainty and outside risk factors to the total project cost with contingency. At any given level of confidence, the difference between the two curves is the Risk Reserve budget.
Graph 2: Histogram for the Total Project Cost
This graph represents the population of data that was generated during the analysis and gives the general shape of the data distribution. It depicts the total project cost including the contingency budget.
Graph 3: The Tornado Graph
This graph represents the sensitivity analysis done within each simulation. It displays the the cost drivers that have the most impact on the bottom line and how they correlate to the total cost. The longer the bar, the larger the impact it has on the bottom line if it varies. This graph shows that the decontanmination activity induces the most uncertainty in the estimation of the total project cost.
As the world continues to become ever more digital, companies from all industries need to have a strong digital commerce strategy. However, launching a digital commerce platform can be a complex undertaking, especially for established companies which have built their businesses on more traditional, paper-based processes that are better suited for ‘brick and mortar’ transactions. This is where Netconomy has found its ‘sweet spot’: working with enterprise organizations to adapt and connect their business processes with state-of-the-art technologies to deliver scalable e-Commerce solutions. However, this is a fast-paced industry and the company is having to grow – and plan for that growth – rapidly. According to Andreas Schilk, CFO for Netconomy, “We are working in an extremely dynamic environment. Our customer base has grown significantly in a relatively short amount of time, expanding beyond Austria, Germany and Switzerland into Russia and Croatia. In addition, we are now offering even more services in our portfolio.”
For the first few years, the company utilized a classical budget process: taking a couple of months of the year to create a financial plan for the next year. However, this type of planning process began to be less effective for Netconomy. “The company is developing so quickly and there are so many variables with our business, including multiple currencies and fluctuating exchange rates. A plan we create today typically needs to change by tomorrow,” explained Schilk. The company needed a planning process that would combine risk management with its budgeting and forecasting.
Schilk has extensive experience of using Palisade’s @RISK in various business sectors. He knew Netconomy needed a way to plan not only for where it was today, but to create simulations of where it could go in the future, including potential associated risks. “This is exactly the type of problem that @RISK can solve,” added Schilk.
Every month, Netconomy creates a rolling, ‘top down’ 12-month plan for each of its three regionally-based divisions in Austria, Germany and Switzerland. Schilk then calculates distributions for each division, based on a variety of factors including product pricing, daily service rates and capacity utilization. As Netconomy is a service-based business, headcount forecast and personnel-related costs are also critical factors. He then uses @RISK, Palisade’s risk analysis add-in to Microsoft Excel, to simulate three possible scenarios for each division, as well as for the consolidated company to show realistic best case, worst case and median budget figures and the probability of their occurrence. The software runs Monte Carlo simulations to provide a range of probability distributions to represent uncertain variables, then computes hundreds or thousands of different scenarios. For the Netconomy models, they run 10,000 iterations each month, using 5-6 probability distributions, including PERT and triangle distributions.
Once Netconomy has created these scenarios, they compare current scenarios to forecasted scenarios from the previous months and determine if the results fit to the company’s targets. This enables the management team to easily see whether the targets are achievable, and make decisions quickly regarding any potential changes to the plans. “It’s an incredibly flexible approach,” added Schilk.
The graph below shows the development of the company’s rolling forecasts. Each scenario of a monthly forecast calculates the profit and loss, balance sheet and cash flow on a monthly basis for the entire business year.
Palisade’s @RISK software provides Netconomy with transparency and understanding about the risks involved in planning, and therefore facilitates decision-making without guesswork. By running the simulations on a monthly basis, the company is able to identify issues such as capacity shortage/idle capacity, efficiency and the development of the contribution margin before they have a significant impact on the business, potentially saving millions of Euros every year.
“With the help of @RISK, we don’t just wait to react to changes on our business. Instead, we are able to act early and quickly if we notice trends which could harm our business goals,” said Schilk. “In addition, we get full visibility regarding what we can afford, what we can risk and what we can invest. This gives the management team – and the board – much more confidence in our plans and our ability to proactively steer the business.”
Fitness First has over 1.2 million members, making it is the largest gym, health and fitness club group in the world. With 75 gyms in Australia and a high usage of active patrons wanting to keep cool as well as look better, they have a hefty electricity bill to manage.
Average business electricity costs in Australia rose (in real terms) by 60% in the 10 years between 2003 and 2013. This major cost item was recognised as a fiscal risk that needed to be addressed and managed.
Fitness First engaged Knowledge Global, an Australian company that has won The Oracle Eco Enterprise award on two separate occasions for their innovative approach to sustainability. Ross Sharman of Knowledge Global summarised, “The real trick to energy cost management is knowing your current costs and then managing well-orchestrated strategies to mitigate the cost risk for the greatest return. These strategies typically take the form of energy efficiency programs.”
Knowledge Global has been delivering verifiable and accurate financial return on investment on energy efficiency programs via their data analytics tool called NRG Insight. The purpose of this tool is to give an organisation’s executive management the ability to manage their energy costs in a coordinated and streamlined fashion. The product is fundamentally an independent billing engine that uses daily smart meter updates to provide automated bill validation, accruals, budgeting, forecasting, market analysis and procurement aids.
Budget model with example distribution
@RISK has allowed Knowledge Global to further enhance their offering by optimising expenditure of new capital in energy efficiency programs. Inputs into the model (such as energy cost, usage, and weather) can be modelled with appropriate distributions. The internal rate of return (i.e. NPV) on a portfolio of efficiency projects across a number of gyms is then modelled using RISKOptimizer, providing more confidence in future projects which are then validated by NRG Insight.
Energy Efficiency Program NPV Optimisation Simulation
“Fitness First and our other clients are provided with more confidence in the expected returns of energy efficiency strategies which makes capital sign off easier.” says Ross Sharman. He adds, “@RISK’s strong statistical capability has made NRG Insight much smarter.”
He goes on to explain how the in-depth analysis changed how Fitness First approached their energy needs.
“As a result of the success of the programs last year and with the continual analytics we provided, they have now allocated a larger energy efficiency budget for this financial year for a new program,” says Sharman. “We will continue to measure and analyse the delivery of these projects with ROI and also with effect to their overall energy budgets.”
Arc of Yates County, New York provides services for people in the community with developmental disabilities. The organization was facing funding shortfalls from state and local governments, and turned to Palisade’s @RISK to manage and mitigate these risks. Through their use of @RISK, Arc of Yates has been able to develop more effective strategic contingency plans and communicate their challenges to board members and other stakeholders.
Founded in 1975, Arc of Yates is a non-profit organization for people with developmental disabilities. It provides a wide range of community-based services, including service coordination, residential living, clinical services, employment opportunities, and industrial and educational development throughout Yates County in upstate New York.
With the recent economic downturn, Arc of Yates was unsure of how to account for potential shortfalls in their annual budget planning. As a non-profit organization, Arc of Yates primarily relies on funding from state and local governments. According to Stephen Johnson, CFO at Arc of Yates, “These funding streams are affected by state and local budget deficits, early retirements of knowledgeable government workers who are not being replaced, inconsistent and capricious rate-setting methodologies and heightened government audit protocols.” As a result, Arc of Yates could feel the strain on its own year-to-year budgets.
In addition, Arc of Yates also had to consider a number of additional variable and uncertain factors including Medicaid rate reductions, state contract reductions, county contribution, state audit, and inflation when planning their budget.
To quantify the magnitude and probability of risks in their funding streams that could impact their financial planning, Arc of Yates turned to Palisade’s @RISK. These risks are updated monthly or even weekly, and the group needed a dynamic tool that could keep up with the fast-paced environment. @RISK uses the Monte Carlo simulation technique: a powerful, yet simple tool for users of all levels to model uncertainty, utilizing ranges and probabilities. Using Palisade’s software, Arc of Yates was able to quantify the actual probability of funding shortfalls and mitigate these risks. These were crucial insights as Johnson and his team developed their three-year strategic financial plan.
@RISK works by providing a range of probability distributions to represent uncertain variables, and then computing hundreds or thousands of different scenarios. A probability distribution is simply a range of values with greater probability of certain values occurring than others. The normal distribution, or “bell curve,” is a common example. The values around the center – or mean – of the distribution are more likely to occur than the values at the ends or tails.
Due to the magnitude of uncertainties and no clear guidance from the State of New York, some of Arc of Yate’s uncertainties were initially modeled as uniform distributions, which show an equal probability of all values occurring. As more information became available, those distributions became more refined to beta general distributions. Due to the fact that they usually are given a range of funding cuts (e.g. 0 to 10%), normal distributions were not used because it’s impossible to determine the mean or standard deviation. Where there was uncertainty in funding methodology, Johnson and his team used a combination of a binomial and beta general distributions. Where historical data were available, the group used @RISK’s distribution fitting feature to determine the best distribution to use from the data. “We are not sophisticated statisticians,” noted Johnson, “and have found the above distributions to be most easily comprehended by our stakeholders.”
In addition to helping Arc of Yates manage uncertain funding, @RISK has been instrumental in communicating risks to the group’s board of directors. The graphs and output reports generated by @RISK are concise and easy to understand, enabling Johnson and his team to convey their challenges and strategies to senior leadership more effectively and in less time.
Figure 1 - Example of Arc of Yates results graph using @RISK. The red graph shows the range of projected surplus for 2011. There is a 90% chance of a surplus between $195,430 and $352,827. However, in 2012 and 2013, deficits are expected.
Johnson has used @RISK and other Palisade software since 2000. In addition to @RISK, he has used PrecisionTree for decision tree modeling and StatTools for statistical analysis. @RISK, PrecisionTree, and StatTools are available in an integrated bundle with the DecisionTools Suite. His experience stems from work done in R&D for the commercial printer RR Donnelly in 1999. There, he began to use a range of quantitative analytical techniques to make better decisions, including Monte Carlo simulation, decision trees, the Analytical Hierarchy Process, and others. He was particularly impacted by the work of the Strategic Decision Group (SDG) in his exploration of Monte Carlo simulation and decision trees, which led him to Palisade software solutions.
“The greatest benefit to Arc of Yates has been reducing a number of significant uncertainties down to a single output that provides our senior leadership and board of directors with a realistic expected outcome,” said Johnson. “Comparing the two outlying years in our three-year strategic financial plan has also provided advance warning that the next two years will be financially challenging. As a result, we have re-prioritized capital projects for 2011-2013 and will need to work more collaboratively with other similar agencies to reduce overhead and administrative expense.”
The utilization of @RISK empowered Arc of Yates with the direction it needed to foresee potential obstacles and short falls in its budgetary forecast. “By understanding risks, we are able to both quantify and focus other activities that will generate replacement revenue and contingency plans for cost reduction,” said Johnson.
To control costs, state governments are increasingly moving toward the privatization of social services. How much should a state pay its private vendors for providing those services? This is always a tricky question, and its answer hinges on ever-changing uncertainties. Dr. Anthony Broskowski’s consulting firm, Pareto Solutions, helps state governments plan the privatization process for mental health and child welfare services. Trained as a clinical psychologist and a statistician, Dr. Broskowski advises state governments on vendors’ administrative cost structures, the mix of appropriate services, and the long-range costs they can expect to pay for those services as well as traditional services. For more than a decade, he has relied on @RISK to make his budget projections.
“Contracting for social services is a risk-reward corridor that both government and vendors must travel through,” says Dr. Broskowski, and the doors to many uncertainties open onto this corridor. Take his recent work for the city of Washington, D.C. The district wanted to privatize the foster-care component of its child welfare program, under a system in which a vendor would receive a prepaid allotment for every child needing a foster care placement. Under this system, the vendor would assume the costs and the risks of cost fluctuations until the child can be reunited with his or her family or attains some other permanency status, such as adoption. To compensate for assuming these risks, the vendor would retain any portion of the government’s payment left over after expenses, within a range of plus or minus 85% of the expected total costs. Dr. Broskowski says that to figure out a fair rate to pay the service provider, “You can’t use a simple formula like number of children x number of days of care x price per unit.” There are too many other factors, such as:
In addition, Dr. Broskowski’s models must develop the formulas for distributing the risks more equally between the vendor and the government. These include repaying profits beyond a certain level and stop-loss provisions for the highest-cost cases, such as medically fragile children.
Clearly, the mix of corporate budgeting approaches and human social problems is not for the faint-hearted analyst. Dr. Broskowski’s models are big simulations spun out over long time spans. For a county-by-county projection of the costs for upgrading mental health services in the state of North Carolina, for instance, he estimates that his model incorporated about a thousand variables. But even so he shrugs this off as a problem of mere size. Having first used @RISK in the early ‘90s to apply managed care principles to carving-out mental health insurance benefits for top corporations like Federal Express and Chrysler, he has come to rely on its ease of use for even the largest models. In fact, although he now frequently trains his clients in the use of @RISK, he confesses that for many years he made little mention of his use of Palisade’s software. “I didn’t want those other guys—my competitors—to find out how easy it is to use.”
“With @RISK,” Dr. Broskowski says, “the real challenge isn’t the modeling. The real challenge is getting the folks in government to accept their role in risk sharing. Because the vendors are in business, risk-and-reward comes easier to them than to government officials letting out contracts. But here again, @RISK is so intuitive it makes it easy to show people how to balance the risks with the opportunities and come to some kind of equilibrium. It can help them actually see that this kind of contract is a simultaneous risk-opportunity equation.” Just the kind of equation to be solved by a company named Pareto Solutions.
