Quality of security metrics and measurements

Quantification of information security can be used to obtain evidence to support decision-making about the security performance of software systems. Knowledge about the relational importance of the main quality criteria of security metrics can help build security metrology models based on practical needs. This paper presents the results of a quantitative security metrics expert survey of 141 respondents, and an associated interview study, regarding the prioritization of 19 quality criteria of security metrics identified in the literature. The interviews were used to validate the survey results and to obtain further information on the findings. The results identified three foundational quality criteria of security metrics: correctness, measurability, and meaningfulness. These criteria form the basis for credibility and sufficiency for security metrics and associated measurements. Moreover, usability was seen as an important criterion. The paper analyzes the foundational and related quality criteria and proposes a model of them.