Testing
Overview
The General Mission Analysis Tool (GMAT) is a desktop space mission analysis tool tailored to support missions involving groups of spacecraft interacting throughout a modeled time period. The complexity of this problem makes GMAT an intricate software system and this complexity necessitates a rigorous testing environment to ensure that the system meets its objectives. GMAT is designed using an object-oriented architecture and coded using extensive object-oriented structures written in C++. The object based approach employed in GMAT's design and implementation makes the system robust and relatively easy to use for experienced analysts. The extent of the object model implemented to make GMAT a complete and robust system dictates a comprehensive testing philosophy. This document describes all levels of testing for GMAT.
Document Scope
This document serves as the test plan for all testing performed on the GMAT application. This includes developer tests such as unit and integration tests, as well as tests performed later in the development cycle such as system, stress, acceptance and regression tests. Preparation for system testing consists of three major stages:
- The Tests Environment and Tools chapter describes the systems and environments required for testing and procedures for configuring those systems for testing.
- The Test Process chapter sets the scope of system testing, the overall strategy to be adopted, the activities to be completed, the general resources required and the methods and processes to be used to test the release. It also details the activities, dependencies and effort required to conduct the System Test.
- Test Planning details the activities, dependencies and effort required to conduct the System Test.
- Test Cases documents the tests to be applied, the data to be processed, the automated testing coverage and the expected results.
This document covers the first two of these items, and established the framework used for the GMAT test case development. The test cases themselves exist as separate components, and are managed outside of and concurrently with this test plan.
Overview of GMAT Development and Testing Process
GMAT development is conducted as a cooperative effort between an analysis team, typically composed of flight dynamics specialists, and a development team consisting of talented software developers. New requirements for the system are defined and written by the analysis team. Mathematical and design specifications are derived from these requirements and compiled into a format that can be used to code the new functionality. Requirements, Specifications, and Designs are reviewed by the development team prior to implementation.
During the development process, new features of a component under development may be detected that need further specification. When that happens, the new features are discussed and collected together. This may result in an immediate update to the design documents, or it may result in collection of the new feature implementation for inclusion in a final update performed when the component is ready for integration. In either case, the design documentation is updated to reflect the implemented functionality prior to formal acceptance of the related components.
During development, the software undergoes internal testing in the development team at both a unit and an integration level. Unit testing is intended to exercise all of the executable paths through the code, validating that the internal working of the code behaves correctly. Integration testing takes unit tested components and builds those components, either one at a time or collectively, into the system. The development team may interact with the analysis team during integration testing to confirm that the observed behavior of the new code conforms to the.
When the GMAT development team completes integration of new functionality into the system, that new functionality is ready for system test. GMAT system testing follows a more formal test procedure than unit or integration testing. New components are exercised both from the GMAT scripting language and from the GMAT Graphical User Interface (GUI). The test cases exercised are documented using the procedures described later in this document. Test cases are managed using a traceability matrix that lists all of the elements of GMAT visible at the user level, and matches those elements to test cases that are executed in system testing. This master traceability matrix is used to generate a spreadsheet of test cases each time GMAT enters a system test cycle. All tests are tracked using this spreadsheet; formal system test is complete when every test case has been exercised and the results of the tests have been tabulated and accepted after review.
NASA IV&V
The activities to be provided to the GMAT project by the National Aeronautics and Space Administration (NASA) IV&V Facility in Fairmont, WV have not been specified. In the event of such participation by the IV&V facility, the GMAT test team will update this section of the test plan.
Test Objectives
The plan developed in this document is intended to demonstrate that:
- The functionality delivered in GMAT is as specified by the Mathematical and Design specifications.
- The software is stable and of high quality.
- The software models spacecraft missions faithfully.
- The software interfaces correctly with other systems, specifically MATLAB.
- The software user interfaces are stable, complete, and understandable by novice and experienced users.
These objectives are addressed through the development of a suite of test cases exercised on builds of the GMAT system. Each major release of GMAT is tested using this suite, and the results of the tests are collected and reviewed by all interested parties prior to release.
Applicable Documents
- GMAT System Test Plan
- GMAT Acceptance Test Plan
- GMAT Requirements Specification
- Minimum Requirements for Test Plan
- User's Guide for Resource Tree Test Complete Template
- GUI Test Matrix
- "How to Write Script Tests" Wiki page
- GUI Widget Test Procedures
- GMAT Compiler Config (Windows)