Start by using your information security objectives to determine your measurement information needs

  
  Interested in Cyber Essentials? Visit our advert landing page (under Marketplace) and learn how to migrate to partial or full conformity with ISO/IEC 27001, and gain greater control and defensive strength against cybersecurity threats
 

Monitoring, measurement, analysis, and evaluation

Introduction

Clause 9.1 of all management system standards is entitled “Monitoring, measurement, analysis, and evaluation”. This article explores the purpose of this clause and how it can be fulfilled in the context of ISO/IEC 27001.

What do Annex SL and ISO/IEC 27001 say?

Annex SLAppendix 2 of the ISO/IEC Directives (Part 1 Consolidated ISO Supplement), Annex SL specifies the high level structure and identical core text for all ISO management system standards (MSS)., Clause 9.1 begins with a requirement to determine what needs to be monitored and measured and ends with a requirement to evaluate the XXX performance and effectiveness of the XXX MS. ISO/IEC 27001 reproduces this requirement with the following changes:

  • XXX, the discipline place-holder, has been replaced with “information security
  • The last paragraph (to evaluate … performance and effectiveness), has been moved to the beginning of ISO/IEC 27001, Clause 9.1.
  • A note has been added: “The methods selected should produce comparable and reproducible results to be considered valid.
  • A new list item “d) who shall monitor and measure.“ has been added
  • A new list item “f) who shall analyse and evaluate these results.” has been added
  • The words “monitoring and measurement”’ have been added to what is now the final paragraph of ISO/IEC 27001, Clause 9.1.

The reason for moving the “evaluate … performance and effectiveness” text is because SC27ISO/IEC JTC 1 SC 27 WG1, the standards committee responsible for ISO/IEC 27001 and related standards considers it to be the raison d’être of Clause 9.1 and that the readers’ understanding is best served by placing it at the beginning of the clause, rather than at the end.

The note is problematic as there are many circumstances when measurement results are not repeatable (consider a roulette wheel, for example). However, ISO/IEC 27004 provides a detailed explanation.

What is ISO/IEC 27004?

ISO/IEC 27004 – Monitoring, measurement, analysis and evaluation – is a guidance standard in support of ISO/IEC 27001. It takes its name from Clause 9.1 of ISO/IEC 27001 and provides guidance to organisations on how to fulfil the requirements of that clause.

What are its key recommendations?

In keeping with the revised order of Annex SL text in ISO/IEC 27001, Clause 9.1, the key recommendation in ISO/IEC 27004 is to first determine the organisation’s information needs necessary to determine its information security performance and the effectiveness of its ISMS. Knowing what one wants to achieve in making measurements, and the setting of worthwhile goals are key to obtaining the greatest value from an ISMS. Clause 9 corresponds to the CHECK in the Deming PLAN-DO-CHECK-ACT model and allows organisations to take stock of their information security performance and ISMS effectiveness by direct measurements, thereby facilitating informed decisions to be made regarding corrective actions and improvements.

ISO defines monitoring as “determining the status of a system, process or an activity”, whereas measurement is a “process to determine a value”. Thus, monitoring is a series of measurements.  A measure is a variable to which a value is assigned as the result of measurement. Given these definitions, and having determined one’s information needs, the next step is to determine the corresponding measure (or measures), how the measure should be evaluated, calculated, or scored, and the desired result of the measurement or target.

To assist this process, ISO/IEC 27004 advocates the use of a form called a measurement construct and provides 35 completed examples. Organizations are advised to complete one such form for each information need.

Information descriptor Specification
Measure ID Specific identifier
Information need The ultimate purpose of the measure
Measure Measurement specification, e.g., …“percentage”, “number”, “frequency” and “average”.
Formula/scoring Formula for evaluating, calculating, or scoring the measurement result
Target Desired result of the measurement
Implementation evidence Evidence used to confirm the reliability of the measurement
Frequency How often measurements are to be made
Responsible parties Who is responsible for gathering and processing the measure
Data source The data sources to be used, e.g. databases, tracking tools, …
Reporting format How the results should be presented, e.g., as text, numerically, graphically (pie chart, line chart, bar graph etc.), as part of a ‘dashboard’ …

[Measurement construct based on the example given in ISO/IEC 27004:2016 “Information security management — Monitoring, measurement, analysis and evaluation”]

ISO/IEC 27004 also recommends the use of measurement programmes. For example, some organisations are known to have measurement programmes with Key Performance Indicators (KPIs), each with an associated measurement construct, which are derived from their business objectives and are updated each year. Note, however, that having a measurement programme is not an ISO/IEC 27001 requirement.

Where is a good place to start?

A good place to start is with your information security objectives. Although there is no direct linkage between Clauses 6.2 and 9.1, Clause 6.2 says that your information security objectives should be measurable if practicable. There seems little point in setting an objective if there is no way of determining if it is being met.

Some people argue that some information security objectives are long term goals, such as the preservation of confidentiality. However, such can be transformed into short term measurable objectives such as: “over the next reporting period, the number of incidents where information is undesirably disclosed should be zero”.

How should my measurement programmes develop?

ISO/IEC 27004 describes two types of measure — performance and effectiveness — terms which should not be confused with the same terms as used in the goal of evaluating “information security performance and ISMS effectiveness”.
Performance measures are measures based on intentions, whereas effectiveness measures consider how well something meets its objectives, for example:

  • performance: “is our training program meeting its planned targets of so many people being trained per week?”
  • effectiveness: “do our training sessions meet their training objectives?” and (to test the training objectives themselves) “are our training sessions having the desired effect on reducing the number of internal security incidents?”

Effectiveness measures are better at meeting the goal of evaluating information security performance and ISMS effectiveness. However, in practice they are more difficult to design. In this example, one would need to somehow measure the trainees’ understanding before and after the training, and the influence of training and other measurable quantities, such as the number of incidents caused by user error.

For this reason, ISO/IEC 27004 advocates starting with performance measures, and augmenting these with effectiveness measures as the ISMS matures.

Is there a scientific basis for this?

Yes — it is the science of metrology.

Metrology is the science of measurements. It refers to information need and shows how so-called base measures can be combined to create derived measures from which the information need can be satisfied.

As an example of base measures and derived measures consider the measurement of vehicle speed. The base measures are distance travelled and time, the distance travelled traditionally being measured by wheel rotation and latterly by GPS, and time by a clock. Speed is a derived measure and is calculated, i.e., derived, by dividing the distance travelled by time.

Conclusions

ISO/IEC 27001, Clause 9.1 is an important requirement as its diligent fulfilment facilitates informed decisions to be made about corrective actions and improvements to an ISMS.