Ian Harragan
Director at i-confidential
You may have heard the story about the dart-throwing chimpanzees and how they are better at predicting future events than human experts. In the world of cyber and information security, it is ever more important for management teams to have access to the right information to understand where they are and where they are heading.
In his book, Super Forecasting, Philip Tetlock explores the accuracy of a range of forecasts across a wide number of fields and how they yield mixed results. One of the themes that comes through is that to make better forecasts you have to be able to measure accurately.
This sounds obvious, but a key finding in the book is that most predictions have one thing in common – after the event, no one thinks to formally measure how accurate they were. This lack of measurement means you have no sense of how accurate any particular source usually is. Without that baseline, how do you know who to listen to the next time you need to make a decision?
Another key theme is that predictions work best in, “learning friendly environments”, where the outcomes are easy to quantify and those involved are able to receive prompt and clear feedback, which is essential for improvement.
Maybe it’s not surprising then that when we look at large, complex organisations and their approach to measurement, we often find they are struggling to define and collect the ‘right’ information. This typically leads to incomplete sets of measures being used, which results in people forming a one-dimensional view of the issues they are facing. Coupled with inconsistent measurement collection approaches and unreliable output the decision-making becomes questionable. This position only gets worse if the culture of the organisation is to ‘hide the truth’.
How does this relate to the world of security? In business terms security is still a relatively ‘young’ discipline (compared to say, lending, which has been around for thousands of years). The upside is that we can build on good practice from other areas and ensure we get on with making up ground quickly. With the current array of threats being posed by the ‘bad guys’ we can’t be complacent.
Coming back to the themes highlighted in Super Forecasting, we can see that many organisations are not giving themselves the best chance to manage effectively today, let alone forecast what might be coming down the track.
At i-confidential, we always start with the old mantra that, ‘You can’t manage what you can’t measure’. To help our clients overcome these challenges we have developed a Metrics Capability Model to assess their current strengths and weaknesses, and outline a prioritised roadmap for improvement.
Feel free to reach out and discuss how we can help you beat ‘the chimp’.
Comments