It should be noted that engineering organizations at different companies use terminology differently when describing test cases and scenarios. They use terms like “test case”, “test steps”, “check points”, “test suite”, and other similar terms. These may not correspond directly with the MxSuite definitions.
In MxSuite, a “TestCase” is characterized by the specification of preconditions, causal action(s), and effect(s). A TestCase identifies specific fundamental behavior that occurs when a set of inputs is provided to an entity under test while it is in a certain state. The behavior is characterized by signal inputs and signal outputs over time. A TestCase may or may not be associated with a pass-fail determination. That is, a TestCase may be used to actually test that output behaviors occur within time and value tolerances, or it may be used to set up preconditions (calibrations, states, or modes of the unit under test) for another TestCase without making any checks of output behavior itself. Any metric for number of TestCases passed or failed may not be meaningful by itself; it is the number of requirements tested that is usually more relevant. The user should take this into account when associating any meaning for numbers of TestCases passed or failed.
We could use the term test case as a quite generic term for a way of demonstrating that a system works as specified in its requirements specification. In this case, there are no assertions about the syntax for a TestCase except that we change inputs to the system and we expect a particular response from the system. TestCases are intended to be powerful and re-usable building blocks for a comprehensive suite of tests. Consider the statement:
If the Power Mode of the System is Powered Off, and I depress the Power Switch, the Power Mode will change to Powered On.
Most importantly note that this statement could be a Requirement for a system or it could be a TestCase. Let's break the statement down into its component parts. We have:
•Signals - Power Mode and Power Switch are the two Signals in use. It would be useful if we could reference a more concise definition of these Signals (in a Signal Dictionary say). We see the need to define and measure the Power Mode of the system. It appears that it has at least two states: Powered Off and Powered On.
•Precondition - The system is Powered Off - We specify a State of the system as a Precondition to continuing.
•Causal Action - Depress the power switch - We change an input to the system. In this case, the change is simple, however, we could have a pattern or sequence of changes to the Signal, and there may be several Signals involved in this Causal Action. The Causal Action results in some outcome - the Effect.
•Effect - The system powers on. There may be many subsequent and secondary effects. Here we identify the outcome or effect that is directly related to the Causal Action.
If we add to the statement information about the allowed latency between the Causal Action and the Effect, information about Tolerances if there are any, and we also restrict ourselves to referencing Signals that are formally defined in a Signal Dictionary, we use the term TestCase to describe such a statement. Note that the TestCase the statement strongly resembles (and indeed could be) a low-level functional requirement or at least a Use Case.
A TestCase serves many purposes. It can
•Be statement of a Systems Requirement or of a Use Case.
•Be executed as a TestCase in the context of the MxSuite.
•Advance the SUT from one operational state to another. That is, you can establish Preconditions by executing a TestCase.
•Be presented graphically to improve communication, since it has a somewhat formal syntax.
You can construct more complex tests by stringing together a sequence of simpler TestCases (in a Scenario). To take full advantage of the power of the MxSuite, you need to construct TestCases that fully describe the behavior of the system under test.
In MxVDev, when a TestCase is executed, it performs three basic functions:
•Transmit stimulus Signals
•Receive response Signals
•Perform pass/fail analysis on the response Signals
The MxSuite provides two ways to define TestCases: Graphical and Reactive. It is possible to mix reactive and nonreactive Signals in the same TestCase.
In a graphical TestCase, the commanded (stimulus) and expected (response) patterns are defined before the test is run and displayed graphically as shown here:
•This is the fastest and simplest method.
•No coding is needed.
•Patterns are predefined and therefore predictable.
•There is no interaction between Signals.
In a reactive TestCase, the patterns are defined by C# code. See Reactive TestCases.
•Signal patterns can be based on other Signals and response patterns.
•Supports conditional behavior. For example: If Signal A > Signal B assert failure and stop TestCase.
•Can interact with Reactive Scenarios.
•Requires some knowledge of C# code.
•The test may be more complex.