A few years ago, if a CEO had posed the question of “Are we secure?” to the security team or CISO, many would have responded with statements revolving around new technology they’ve deployed or point to trailing indicators of success, anecdotally proving their worth by stating they haven’t been breached.
As the importance of security to protect businesses has increased (as well as the demand for the CISO to prove to the CEO that they are focusing on the right things to keep the company off the list of big breaches), it is now broadly recognized that there is a need to measure the effectiveness of the controls deployed within the infrastructure with metrics; proving value by fact not technical jargon, hope and a handshake.
When it comes to what should be measured to be able to demonstrate the effectiveness of controls, it is always wise to turn to best practice and existing research rather than reinvent the wheel.
The SANS (please don’t make me call it Council on CyberSecurity) “Top Twenty Critical Security Controls”, have some excellent examples of metrics that can be repurposed to suit each organisation’s requirements.
Without these best practices, security teams can use incorrect or false flag metrics which drive the wrong behaviours.
When measuring detected vulnerabilities (CSC#4) for example, we often see a total count as a measure rather than focusing on what is important.
If you do frequent vulnerability scans, something I strongly suggest you do, you’ll probably find thousands of vulnerable applications. Being able to trend how many easily exploitable, critical vulnerabilities are discovered is far more of an actionable metric that simply stating the total number of vulnerabilities irrelevant of ease of exploit or severity.
It is also important to measure time as a vector when looking at vulnerabilities, for example how quickly are known, easily exploitable vulnerabilities patched.
Just being able to trend these two metrics against business units or geographies would quickly give an insight into which parts of the organisation are taking security seriously, addressing issues quickly and efficiently verses those who are ignoring the problems faced, hoping they fade away.
With the approach of measuring the effectiveness of security though comes the need to effectively report on the metrics to communicate value upwards to management and expectations down.
In the recent “State of Metric Based Security” Tenable Network Security commissioned, 63% of the 400 respondents stated the frequency of reporting to be every quarter or more and the average time taken for all respondents to create these reports as two days.
It’s not uncommon to hear the CISO talk about spending hours manipulating data within Excel in preparation for a meeting with upper management, plugging away trying to abstract the relevant information from the “bits and bytes” that have no place in a boardroom.
At a recent roundtable I even heard one CISO remark that they’d employed a creative agency to add much needed style to the reports to make them more appealing to management.
The issue is further exacerbated by the perceived value of the data.
In the aforementioned survey 74% of respondents stated that the information provided is valuable or highly valuable to the CEO and their board of directors. Critical business decisions are being made on the data provided but the complexity of collection and collation of the data for the reports means many are having to make the tough decision between “What I want to show” versus “What I can show easily”.
When deciding on implementing any security control within an organisation, it is crucial that part of the deployment plan involves how to easily measure and report on the effectiveness of the control and what information, if any, should be communicated up to the CEO and board as evidence that the investment made is successful in mitigating the business risk it was purchased for.
If these steps are missed, the CISO will continue to spend time focused on providing the CEO with data that is difficult to take action on instead of designing solutions to further reduce the risk of cyber threat to the business.
What do you think? Is this an issue in your organisation? Leave a comment below detailing your experiences.
As a CISO I regularly update the c-suite on security. I find they do not want vulnerability metrics, lists of missing patches, or the number malware incidents, etc. They want to know what our exposure to risk is and what is being done to reduce it. They want to know the high level status (red, amber, green) of the security programmes and services (e.g. patching, endpoint security, threat detection, staff awareness, etc). They sometimes want detail on specific areas of interest, particularly areas reported in the media, but certainly not the technical details. My advice is to use high level dashboards, with pretty colours (I can understand getting a graphic designer in to blow some creativity into them).
This is my experience as well, using trends of relevant measurements is far more useful than bits and bytes of technical data. The most powerful tools I use to get appropriate metrics across to management are through an iPad app called Roambi or http://infogr.am/. Both very slick, powerful if used right and each metric demonstrated had relevance.
I'd be very interested in hearing the types of measures you share, if you are a Twitter user please DM me @gmillard or email me using the same @tenable.com.
Thanks for the insight.
Simply put, stop spending dollars to protect dimes.
Risk Management 101
Perhaps a better strategy? Estimate annual losses to defined assets from likely (frequent) threats to determine risks. THEN determine what controls are necessary for mitigation…THAT is "best practices" from a business perspective. www.cxoware.com