The objective of this work was to quantify comparisons between several computational results and measurements performed during a pool fire scenario in a well-confined compartment. This collaborative work was initiated under the framework of the OECD fire research program and involves the most frequently used fire models in the fire community, including field and zone models. The experimental scenario was conducted at the French Institut de Radioprotection et de Sûreté Nucléaire (IRSN) and deals with a full-scale liquid pool fire in a confined and mechanically ventilated compartment representative for nuclear plants. The practical use of different metric operators and their ability to report the capabilities of fire models are presented. The quantitative comparisons between measurements and numerical results obtained from “open” calculations concern six important quantities from a safety viewpoint: gas temperature, oxygen concentration, wall temperature, total heat flux, compartment pressure and ventilation flow rate during the whole fire duration. The results indicate that it is important to use more than one metric for the validation process in order to get information on the uncertainties associated with different aspects of fire safety.