A group of CROs and senior risk managers unpick the for and against of quantitative risk tools versus a qualitative approach in a virtual meeting on influencing decision-making. Read on to find out their conclusion
Quantitative versus qualitative risk information: which one drives better decision-making? This was the ‘exam question’ that formed the core of debate during – and after – our recent New Thinking virtual meeting on influencing decision-making.
In fact, the big question was set by one of our Members, a senior ERM manager. And it elicited varying views from risk leaders and CROs who attended the meeting.
There were groups who argued strongly for a quantitative approach, others who were less convinced of its applicability to the c-suite and wider business, and some who advocated for both.
For or against? We think we have a conclusion.
But before we get to that, here’s how the discussion played out during the virtual meeting and after in our private messaging service for risk managers.
The exam question – explained
There was more to the exam question. In fact, it was more of a series of statements and questions – rather than on set question – which prompted a deeper debate.
For context, here is a summary of the initial comment.
Quantitative tools such as Monte Carlo simulation seem useful for areas such as project risks, financial and operational risks and in developing scenarios – all of which are very data driven. Here modelling can help risk managers get their heads around likelihood scenarios and impacts with the support of data analytics.
But how useful is it when analysing both non-tangible and strategic risks?
In addition, quantitative tools such as Monte Carlo simulation may not be understood by everyone outside of engineering/mathematics or, even by C-suite executives. Would a holistic and easy-to-understand presentation (without the complexities of statistical analysis) appeal more?
Plus, every model has its limitations in predictive analytics (especially those driven by historical data).
Shouldn’t we look at enterprise risks from both a qualitative and quantitative perspective? And asking difficult questions which leaders tend to brush under the carpet (questions that warrant a qualitative answer or output). Surely, this will help ensure that all aspects of risk and controls are considered from a strategic planning perspective?
The argument for quantitative tools
If you sit on the quantitative side of the fence, you may agree with this Member’s view:
“It is true that the more tangible the risk, the easier it will be to leverage data/facts and quantitative tools such as Monte Carlo simulation. However quantitative tools should always be used. ERM is not a standalone process but linked to strategic planning and/or budgeting and performance management in general. As such, you can always measure/monitor issues.
“It has been demonstrated by Nobel laureates (Kahneman & Tversky and Thaler) that people are terrible at ‘guessing/assessing’ – qualitative assessments are notoriously bad (exceptions do exist). Based on that, there is no such thing as ‘qualitative analysis’ and ‘qualitative data’ should be avoided.
“Management will not care much about having 4 ‘red’ risks but will care about a 60% likelihood of not meeting targets."
How about metrics and KRIs?
What about introducing data in the form of KRIs and applying performance management techniques to see if risk actions make a difference? The c-suite will want to get to the ‘so what’. They like numbers but only the ones they understand and recognise.
What if your c-suite doesn’t like maths?
If this is the case, do we shy away from quantitative risk analysis?
No is the advice from one Member. “This does not advocate that we as specialists refrain from leveraging hard numbers.” What we avoid is a lengthy explanation over how we arrive at the numbers.
What is important is the message: the extent to which a project or strategy will meet or exceed the target. What is the likelihood in percentage terms
You need both
But does it need to be ‘either/or?’
There is a strong and compelling argument for utilising both: that quantitative data can be complimented by qualitative input.
Here’s how one Member put it:
“Most risk managers, whether they like it or not, have to appeal to the traits that most senior executives regard and take comfort in –personal judgement based on a mix of factors.
“These are of course useful data points coming from quantitative modelling tools such as Monte Carlo simulation, as well as experience-driven, qualitative assessment.
“So, I would argue you need both. Why? Well with good data sets you can get over a 95% confidence level using tools such as Monte Carlo simulation. I absolutely see value in using quant tools and have previously done exactly this for Californian earthquake loss expectancies.
In situations like this, it is a powerful tool that can distill large amounts of data and deliver outcomes for consideration. However, in certain other situations you just don’t have comprehensive, reliable data sets available to model with the required degree of confidence.
“With human nature being what it is, qualitative assessment, whether we are fans of it or not, will always fill in that gap.”
So, the conclusion?
As one member succinctly put it: “Risk managers: use the full suite of tools available to you.”
We will be running many more virtual meetings on this very topic. If you are a risk manager and want to hear wide and varying views of risk managers from around the world – and grow your network at the same time – get in touch with me: kin.ly@riskleadershipnetwork.com
Are you an in-house risk manager who could benefit from collaborating with a global network of senior risk professionals? Talk to us about becoming a Member today.
