![]() |
![]() |
![]() |
![]() |
![]() |
Design Testing
One of the major benefits with a design notation like SDL that has a well-defined and complete semantics, is the possibility to test the application already in the design activity. This is feasible since the completeness of an SDL design makes it possible to simulate the design taking distribution and concurrence into account.
It is important to emphasis that the output of the object design activity is not only an SDL design but it is a tested SDL design that has been shown to fulfil its requirements. This implies that the design testing is an important task in the object design activity.
Testing Strategy
A traditional development/test strategy can be described by a "V" as in Figure 714. The design is performed top down, starting with a system design where the major components and their interfaces are defined, followed by a module design and an implementation phase where the application is implemented. The implementation of each module is then tested separately in a module test and finally the entire system is tested. The system test can also sometimes be divided into two parts, one focusing on the integration of the different modules and the other focusing on testing the functionality of the complete system.
This model works in practise fairly good but it has one problem: the complete functionality is not tested until the system tests are performed in the end of the development/testing process. To some extent this problem can be overcome by an iterative process that includes more than one "V" in a development project. Using techniques like SDL this can be even more improved by introducing one more line of testing in the model as in Figure 715.
The extra line of testing represents the design level testing that is the focus of this section. There are two practical aspects that differentiates the design testing from the traditional target testing:
- It is performed using a simulation of the design instead of on the real application in its target environment:
- It is performed as part of the design activity instead of as a separate testing activity performed after the implementation.
The second aspect is very important. Problems that otherwise would not show up until system test time is now found and solved during design at a much lower cost.
In the figure it may look like the testing effort has doubled in the improved "V" strategy since there are now two lines of testing instead of one. This is fortunately not the case. What has happened is that the effort of testing has switched from the target testing to the design testing, making the total amount of testing more or less the same. There are several reasons for this:
- The cost of testing is smaller for design testing than for target testing: the design testing on SDL level is very easy to perform. Since a large portion of the testing is performed at the SDL level this reduces the total testing cost.
- The target testing can focus on the integration and targeting aspects since the logic of the design has already been tested in the design tests. In practice the target module testing can even be skipped if automatic code generation is used when producing the implementation.
Test Case Sources
There are several sources from where the test cases may come:
The most important input, both to module and system test, is the design use case model from the system design. This model should capture most of the requirements on the system and using the implinks it is also possible to trace the dependencies from the original requirements all the way to the design use cases. Furthermore, the design use case should be in a format that is possible to test more or less automatically against the SDL system, like MSC or TTCN.
However, during the object design there are usually more test cases developed that tests other aspects of the design, and these form also an important part of the module testing tests.
The third source of tests is external sources. For example, in the telecommunication area it is common to have standardized tests suited for certain types of applications or interfaces. In other cases the customer may have specified acceptance tests that the system must comply to before it is approved. These type of tests should of course also be part of the design tests, in particular for system testing the external sources are important.
It is convenient, but not necessary, if the same notation is used for the design use cases, the tests developed during object design and the external test suites.
Tools for Testing
To perform the testing there is a need for tools, and fortunately there are several tools available that make a number of testing strategies possible:
- Manual or batch simulation of an SDL system using a standard SDL simulator
- MSC verification and automatic testing using a state space exploration tool
- Integrated-simulation of SDL and TTCN, using standard SDL and TTCN simulators
Using a standard SDL simulation tool it is of course possible to manually simulate the test cases and check that the system performs as expected, but it requires quite a lot of manual work. A better way is to produce test scripts that contain the simulator commands that are needed and then execute them automatically in a batch mode and log the results on a log file. To check the outcome of the test either the log files are manually inspected or checked by a post processing tool that e.g. compares the new log files with old, manually inspected log files.
Another approach to testing is to use a state space exploration tool that can automatically check if an MSC is consistent with the SDL system. The benefit with this method is that MSCs can be directly input to the tool and checked and furthermore there is no need for a manual inspection of the results: a verdict can be automatically generated by the tool. The drawback is that some features like the combination of user-written and automatically generated code can not always be handled by state space exploration tools.
A third approach that is useful if the tests are defined using TTCN is to use an integrated-simulation of an SDL system and a TTCN test suite as the means to perform design testing. Essentially this is similar to using an SDL simulator alone, but instead of specifying the input as simulator commands a TTCN simulator specifies the input to the SDL simulator and checks the outcome of the test.
Test Practices
A common situation in particular when performing module tests is that there is a need to test one part of an SDL system in isolation from the rest of the system. In an SDL system the part may be for example a block, a process or a data type definition. The simplest way to accomplish this in SDL is to use a package as a container of e.g. the block (which in this case will have be a block type) or data type and then use a special test system to specify the test environment.
As an example consider the DoorCtrl part of the access control system. Assume that this part is designed as a block type "DoorCtrlT" in a package DoorCtrlPack. To perform a module test on this block the simplest way is to create a special test system DoorCtrlTest that instantiates the DoorCtrlT and connects all its gates to the environment.
http://www.ibm.com/rational |
![]() |
![]() |
![]() |
![]() |