Taxonomy of quality metrics for assessing assurance of security correctness

Assurance is commonly considered as “something said or done to inspire confidence” (Webster dictionary). However, the level of confidence inspired from a statement or an action depends on the quality of its source. Similarly, the assurance that the deployed security mechanisms exhibit an appropriate posture depends on the quality of the verification process adopted. This paper presents a novel taxonomy of quality metrics pertinent for gaining assurance in a security verification process. Inspired by the systems security engineering capability maturity model and the common criteria, we introduce five ordinal quality levels for a verification process aimed at probing the correctness of runtime security mechanisms. In addition, we analyse the mapping between the quality levels and different capability levels of the following verification metrics families: coverage, rigour, depth and independence of verification. The quality taxonomy is part of a framework for the Security Assurance of operational systems. These metrics can also be used for gaining assurance in other areas such as legal and safety compliance. Furthermore, the resulting metrics taxonomy could, by identifying appropriate quality security requirements, assist manufacturers of information technology (IT) in developing their products or systems. Additionally, the taxonomy could also empower consumers in IT security product selection to efficaciously and effectively match their organisational needs, while IT security evaluators can use it as a reference point when forming judgments about the quality of a security product. We demonstrate the applicability of the proposed taxonomy through access control examples.