SE Validation Limited

Test and Evaluation (T&E)

Introduction

Test and Evaluation is the generic composite term applied to the planning and execution of data gathering and information synthesis activities and their subsequent assessment and presentation to support decisions. However, that is about the extent of general agreement because organisations involved in different countries have different views on the boundaries of systems to be evaluated and tested and on the type of decisions that these activities aim to inform. There are also differences of opinion on whether T&E is simply a 'control' activity that aims to prevent unsatisfactory systems being accepted into Service, or is a through life 'assurance' activity aimed at making sure that the system built is safe, survivable, suitable, cost-effective and delivered on-time. It is due to this uncertainty in the meaning of T&E that the terms verification and validation feature so much in this profile. Organisations with strong T&E policies have usually learnt the hard way about the negative impact of products that are released without being adequately tested and evaluated.

ITEA Planning

SE Validation has in-depth experience of T&E, with staff having been involved in developing the philosophy behind and the guidance for the UK military Integrated Test, Evaluation and Acceptance (ITEA) planning process.

Evaluation

There seems to be a general misunderstanding about why evaluation is required. This is allied to a belief that it is possible to establish that a solution is valid by test alone. In very simple cases it may be possible to demonstrate verification by test, but not validation. Thus evaluation includes deciding what tests need to be done to achieve adequate confidence at an acceptable price. It is the art of balancing the cost of knowing against the risk of assuming. At SE Validation we understand how to strike this balance and the subtleties of the trade-offs involved.

Test

Test is frequently the expensive and dangerous part. Invariably it is done under challenging time pressures, frequently because it stands in the way of recouping money or benefits from the investments put into development. With experience of complex and safety critical testing, SE Validation is well equipped to advise on many specialist test matters.

Example Papers

Integrated Test and Evaluation in support of Defence Capability, R.Ae.S Conference, Boscombe Down, March 1999

The Value of Test & Evaluation, International Test & Evaluation Association workshop, Annapolis, USA, July 2002

T&E in Smart Acquisition, DSEI, London, September 2003