D.5.4 Evaluation Protocol and KPIs

The Cyber-MAR project aims to develop a framework to support maritime stakeholders in their security management programs and provider concrete measures, training, and tools, e.g. integrated cyber ranges.
The general challenge is that cyber readiness and awareness among businesses and individuals remain low, and there is a major shortage of cybersecurity skills in the workforce. The growing demand for the cybersecurity skills and cybersecurity professionals is boosting the development and implementation of cyber ranges and development of training. Cyber ranges are simulative operational representations of real systems, networks, IT and OT hardware and applications. They offer a plausible environment for enhancing cybersecurity skills of personnel or cybersecurity professionals, evaluate and test cybersecurity management or to develop cybersecurity solutions.
The purpose of the document is to define the Key Performance Indicators (KPIs) and Critical Success Factors (CSF) for measuring the efficiency of developed cyber-MAR solution, as it will be demonstrated during the scenarios performed in the pilots. Key Performance Indicators are composed of critical subsets of performance parameters representing the most critical capabilities and characteristics of the Cyber-MAR system.
The document first explains the general background of performance evaluation and background of key performance indicators and critical success factor implementation and their current state in cyber security management and cyber ranges’ development. It also discusses shortly the main resilience areas or preparedness and availability. Then the Cyber-MAR KPI and evaluation framework is presented. In the Cyber-Mar project, evaluation is based on the developed pilot use case scenarios. The project carries out two pilots at defined ports and one pilot for a vessel navigation, as defined in WP5. As annexes of the report the evaluation protocols for the pilot events are presented.
The results, methodologies and templates of this deliverable are used in T5.4 Pilot
Deployments & Simulations Activities.