Applicability of a Cryptographic Metric Taxonomy in Cryptosystem Procurement Process and in Evaluation of Open Standards

Measuring the security of cryptographic systems in a simple and effective way is a difficult problem. There are several metrics that need to be taken into account. Earlier studies have produced one taxonomy of these different metrics, but the applicability of the taxonomy and the different metrics have not been tested. In this paper, we present a revised taxonomy of metrics for cryptographic systems and show results of applying it in two different scenarios: a procurement process for cryptosystems and in evaluation of open standards, namely the TLS 1.2 and TLS 1.3 standards. Applicability and meaningfulness of a taxonomy depends on its ability to differentiate cryptosystems and thus enable comparisons. Our results show that the revised taxonomy can help in differentiating systems and standards, especially when examining implementation related metrics. Future work should streamline the overly complex evaluation process.