Framework for tool evaluation for a maintenance environment

Tool evaluation forms an essential part of tool development. Unfortunately, there are only a few solutions for this activity. As part of the AMES project, tool evaluation was implemented as part of a more comprehensive framework, used with the development and application of several AMES tools for application maintenance.
These included an application understanding tool‐set, a disabbreviation tool, a reverse‐engineering tool and an impact analysis tool‐set. In addition, the requirement for applicability both to on‐board space software and to tool software has directed our approach in setting requirements for tools evaluation.

The central element of the evaluation process was an AMES‐developed framework including evaluation criteria and the use of a goal/question/metrics (GQM) based approach. The criteria have produced detailed information about each tool.
The final result can be regarded as a combination of the score for each criterion and explicit metrics data. In addition, descriptive information about applications was used in tool evaluation.