Deliverable D9.1: Trial Evaluation Strategy

Abstract

In deliverable D1.4 Project Quality Plan and Procedural Handbook, the waterfall software life cycle model with feedback loops was presented as the guiding approach for the software development activities in the SPIKE project. As a part of the software development approach, it is required that a system evaluation plan must be developed at the end of the design phase. The evaluation tasks specified in the plan will be applied during system integration test phase.

This document lays out the common trial evaluation strategy. This strategy defines the scope and the responsibilities for the upcoming evaluation plan definition and execution tasks. Additionally, it must be ensured that all functional and non-functional requirements have been integrated to the trials according to their priority. The evaluation plan consists of a qualitative approach that is based on qualitative criteria measured at use case level and  a quantitative methodology that is first applied at a newly introduced task scenario level and which is then aggregated into expected and measured results at application level. The methodology allows the collection of measurable feedback, resulting in guidance for further user-oriented research and software development within the SPIKE project. 

This deliverable is the first document in a series of documents that will describe all evaluation activities to be performed on system integration level for software developed in the SPIKE project.

This report defines the methodology that will be applied to evaluate SPIKE, defines the scope of the trials, and determines the related responsibilities. A requirement mapping will be used to informally validate that all user requirements presented in D.2.2 are covered by the different application cases and are thus addressed by the trials. The evaluation approach is based on the following principles:

• Offer a qualitative level of measurement that is applied at use case level in order to evaluate the impressions of trial users according to a number of qualitative criteria.<//font>
• Offer a quantitative level of measurement based on assigning weights (specified as percentage values) at a newly introduced task scenario level. The assigned weights are first aggregated to task level, then to use case level, and finally to application level in order to have a measure of overall fulfilment in place.<//font>
• Support the planning and execution phases of the evaluation process by specifying expected (i.e. planned) values and measured values (actual results measured during evaluation activities). SPIKE developers will use these indicators to determine whether there are considerable conflicts between the expected and actual measured values in order to prioritize future implementation resources.<//font>

The complete evaluation cases will be described in the upcoming deliverable D9.2 using the methodology described in this report.

You can download a short version of this document here:

D9.1 Trial Evaluation Strategy

In order to obtain the full content of this document, please use the contact form below: