Explorer

• Introduction to Requirements and Acceptance
• Organisations, Roles and Responsibilities
  • Organisations and Groups
  • Sponsor and User Responsibilities
  • Programme or Project Responsibilities
  • Stakeholder Responsibilities
• Principles
  • Requirements Management Principles
  • The Requirement and Acceptance Management Plan (RAMP)
    • RAMP: Requirements and Acceptance Strategies
    • RAMP: Management Aspects
  • User Requirements Principles
    • Key User Requirements (KUR) Principles
    • User Requirements Document (URD) Principles
  • System Requirements Principles
    • System Requirements Document (SRD) Principles
  • Effectiveness Envelope
  • Test, Evaluation and Acceptance Principles
    • Test and Evaluation Principles
    • Acceptance Principles
    • Integrated Test, Evaluation and Acceptance (ITEA) Principles
    • Verification and Validation
• Processes
  • Requirements Management Tasks
  • User Requirements Definition
    • Defining User Requirements: the Six Tasks
    • User Requirements Capture: Areas to Consider
    • User Requirements: Review Checks
  • System Requirements Definition
    • Defining System Requirements: the Six Tasks
    • System Requirements: Define Constraints
    • System Requirements: Review Checks
    • Do's and Don'ts when Defining System Requirements
  • Requirements and Acceptance Evolution
    • Stage 1: Project Initiation to Mid Concept
    • Stage 2: Mid to End of Concept
    • Stage 3: Initial gate to End of Concept
    • Stage 4: Start to Mid Assessment
    • Stage 5: Mid Assessment to Start of Demonstration
    • Stage 6: Start to End of Demonstration
    • Stage 7: End of Demonstration
    • Stage 8: Start of in-service to FOC
    • Stage 9: In Service (FOC)
  • Integrated Test, Evaluation and Acceptance (ITEA) Process
    • ITEA Process: the Six Tasks
• Products
  • Requirements and Acceptance Management Plan (RAMP)
  • User Requirements Document (URD)
    • URD Part 1: General Description
    • URD Part 2: Key User Requirements
    • URD Part 3: Individual Capability Requirements and Constraints
    • URD Parts 4 and 5: Context Documents and Glossary
    • User Requirements: Recommended Core Attributes
  • System Requirements Document (SRD)
    • SRD Part 1: General Description
    • SRD Part 2: Key System Requirements
    • SRD Part 3: Individual System Requirements and Constraints
    • SRD Parts 4 and 5: Context Documents and Glossary
    • System Requirements: Recommended Core Attributes
  • Integrated Test, Evaluation and Acceptance Plan (ITEAP)
    • ITEAP Schedule
    • Verification and Validation Requirements Matrix (VVRM)
• Resources
  • Acronyms
  • Glossary

Content

Integrated Test, Evaluation and Acceptance (ITEA) Process: the Six Tasks

This guidance is for those acting in the ITEA Manager role and describes how to define the ITEA products. The six tasks detailed here, form the ITEA Process .

The focus of the process is the ITEAP, associated ITEA Schedule and the Validation, Verification and Requirements Matrix (VVRM).

Further information on ITEA principles is also available.

The six tasks are:

• Task 1 – Acceptance strategy. Plan the strategic approach for ITEA.
• Task 2 – Define V&V. The Validation and Verification (V&V) criteria.
• Task 3 – Detail ITEAP. Consolidate the sets of tests to form an ITEAP.
• Task 4 – Execute ITEAP. Book trials facilities, carry out tests collate result set.
• Task 5 – Evidence management. Analyse the acceptance evidence and make recommendations.
• Task 6 – Declaration. Acceptance Authority (AA) declarations.

The process is iterative and the ITEA products should be reviewed throughout the project lifecycle, so that changes are recorded and the impact of trade off decisions are assessed.

Task 1 - Plan the approach

Plan the approach this is set out in the acceptance strategy documented in the Requirements and Acceptance Management Plan (RAMP).

Task 2 - Define the Validation and Verification criteria

As the user requirements document (URD) and system requirements document (SRD) are developed, the methods for validation and verification criteria should be recorded. This will be used as the basis for ITEA planning. Early identification of trials facilities should be planned with the DE&S TEST IPT.

Task 3 - Detail the ITEAP

Step 1 – Identify . Identify sources of need for Test and Evaluation building on the URD and SRD validation and verification methods. The ITEA process addresses:

• Validation of the capability, that may be incremental and covered by a Capability Implementation Plan (CIP).
• Verification of the capability that may be incremental.
• Design certification.
• Safety and environmental needs.
• Test and reference facilities.
• Technology readiness and research.
• Contract acceptance.

Early planning is required for:

• supporting roles
• operational trials
• calibration
• simulation and model validation.

Step 2 – Scope . Scope the outline test and evaluation requirement for individual sources of need.

As the URD and SRD start to evolve and solution options reduce, review the validation requirements to ensure their continuing appropriateness.

Identify the need for military exercises and range utilisation where long term planning is required.

Baseline the validation requirements with the rest of the URD at Main Gate. If the SRD is baselined at Main Gate and a negotiated test method differs from the SRD criteria then a formal approval will be required by submission of a Review Note.

The scope will include:

• Individual test cases that satisfy each requirement.
• Which tests are mandatory or desirable.
• The cost of Test and Evaluation (T&E) and whether it may be acceptable to test less than 100% of the requirement.
• A prioritised set of test and evaluation activities.
• Responsibility for evaluation of individual tests.
• Responsibility for management of the test and evaluation activities.
• Test, evaluation and acceptance dependencies.
• Estimated cost and elapsed time for each test.
• The impact on asset and contract delivery schedules.

Step 3 – Achievability - Compare estimated cost (including the production of test articles and development of facilities) to budget.

Compare schedule to critical project milestones and demand for resources (test facilities, government furnished assets (GFX) and test articles) to availability. Resolve problems, and inform risk management as appropriate.

Step 4 – Risk Assessment - Within the context of project risk management, perform a risk assessment on the draft ITEAP – areas to consider are:

• Non-availability of test articles and facility on the day of test.
• Failure of (integration) tests.
• Consequent collapse of project management plan.

Develop risk mitigation actions:

• Quantify the uncertainty of cost and schedule estimates.
• Consider the implications of schedule uncertainty on resource availability.
• Question if there is sufficient use of incremental verification to ensure no catastrophic surprises at completion of manufacture.

Step 5 – Consolidate . Consolidate the sets of tests and their implications as scoped in Step 2, to form a draft ITEAP , ITEA schedule and VVRM.

• Consolidate by removing duplication of test sets.
• Maintain links between test(s) and requirement(s).
• Refine the dependencies and sequencing to resolve bottlenecks and update the project master schedule.
• Identify when the Integrated Project Team (IPT) or Capability Working Group (CWG) will be required to evaluate evidence, and draft a review or acceptance meeting schedule.
• Develop the plan to include all sources of test and contributing organisations.
• Explicitly define accountabilities and liabilities.

  Check -

  • Does the Plan protect intellectual property rights (IPR) in test results?
  • Will there be any problems with accounting, liability and insurance where assets are 'loaned' across organisation boundaries?
  • Can each facility operate at the required security level?

Step 6 – Document . Document and cross reference to reflect the (draft) ITEAP in the:

• Stakeholder Responsibility Matrix (SRM)
• GFX Plan
• Asset and Contract Delivery Schedules.

Step 7 - Develop Implementation Processes - establish formal collaborative processes or procedures in order to agree administration of T&E activity across organisational boundaries.

These might embrace areas such as:

• The verification of individual DLoD.
• The collation and management of V&V evidence.
• The evaluation of evidence, and the conduct of acceptance decision-making.
• The management of disputes, provisos, and remedial action.
• The management of test articles.
• The management of test facilities and availability problems.
• The management of GFX and availability problems.
• Installation and test.
• Integration and interoperability testing.
• The Test Evaluation and Acceptance of Urgent Operational Requirements (UORs).

Ensure that there is an adequate process in place to progress delivery of the non-equipment DLoD of Military Capability (MC) into validation testing.

Step 8 - Optimise the ITEAP and obtain stakeholder endorsement - Reflect the conclusions of the achievability and risk assessment in the ITEAP, and obtain stakeholder agreement to it including regulatory stakeholders such as:

• Quality Assurance
• Air and Sea worthiness
• Safety
• Security.

Baseline and publish the ITEAP and Schedule maintaining it under configuration control.

Ensure that the SRM, Asset and Contract Delivery Schedules and other dependent plans remain aligned with the ITEAP and ITEA Schedule.

Task 4 – Execute the ITEAP

Book trials facilities, carry out tests and collate result set as acceptance evidence in the VVRM. Manage the DLoDs progress and record within the ITEAP.

Task 5 – Evidence management

Analyse the acceptance evidence and make recommendations.

Step 1 Within the ITEAP update the progress of the overall development including the other DLoDs. This should highlight any issues such as long term needs for test facilities and dependencies on other projects and programmes.

Step 2 The acceptance evidence is built up in the VVRM to collate the test results.

Step 3 Assessment - Analyse and assess the acceptance evidence contained in the reports from the VVRM.

Step 4 Record the outcome of the tests (pass/fail) for compliance or non-compliance.

Step 5 Following assessment of the results agreement is required between the stakeholders (Sponsor and user). These decisions may be influenced by political and operational pressures that will need to be identified in risk mitigation strategies.

Where agreement is not achieved the alternative options include:

• re-design
• quality investigation
• modification
• trade off
• re-trial or re-plan.

Step 6 Following the assessment of the tests a recommendation is made on acceptance to the acceptance authority.

Task 6 – Declaration

A recommendation to the acceptance authority is required for the following milestones:

• Contract Acceptance
• Systems Acceptance
• In Service Date (ISD)
• Initial Operating capability (IOC)
• Full Operating Capability (FOC).

Test and evaluation recommendation can also aid the customer in the down selection to the preferred solution and mitigation of risk at the assessment stage.

Change History

Change History