|
Test automation tools are increasingly being brought to the market to automate Test tasks. A number of automation tools
exist, but it's unlikely that a single tool is capable of automating all test tasks. Most tools focus on a specific
task or group of tasks, whereas some only address one aspect of a task.
When evaluating different tools for test automation, it's important to be aware of the type of tool you are evaluating,
the limitations of the tool, and what tasks the tool addresses and automates. Test tools are often evaluated and
acquired based on these categories:
Test tools may be categorized by the functions they perform. Typical function designations for tools include:
-
Data acquisition tools that acquire data to be used in the test tasks. The data may be acquired through
conversion, extraction, transformation, or capture of existing data, or through generating use cases or
supplemental specifications.
-
Static measurement tools that analyze information contained in the design models, source code, or other
fixed sources. The analysis yields information on the logic flow, data flow, or quality metrics such as complexity,
maintainability, or lines of code.
-
Dynamic measurement tools that perform an analysis during the execution of the code. The measurements
include the run-time operation of the code such as memory, error detection, and performance.
-
Simulators or drivers that perform tasks, which for reasons of timing, expense, or safety are not
available for testing purposes.
-
Test management tools that assist in planning, designing, implementing, executing, evaluating, and managing
the test tasks or work products.
Test tools are often characterized as either white-box or black-box based upon the manner in which tools are used, or
the technology and knowledge needed to use the tools.
-
White-box tools rely upon knowledge of the code, design models, or other source material to implement and
execute the tests.
-
Black-box tools rely only upon the use cases or functional description of the target-of-test.
Whereas white-box tools have knowledge of how the target-of-test processes the request, black-box tools rely upon the
input and output conditions to evaluate the test.
In addition to the broad classifications of tools previously presented, tools may also be classified by specialization.
-
Record and Playback tools combine data acquisition with dynamic measurement. Test data is acquired during
the recording of events (known as test implementation). Later, during test execution, the data is used to playback
the test script, which is used to evaluate the execution of the target-of-test.
-
Quality metrics tools are static measurement tools that perform a static analysis of the design models or
source code to establish a set of parameters that describe the target-of-test's quality. The parameters may
indicate reliability, complexity, maintainability, or other measures of quality.
-
Coverage monitoring tools indicate the completeness of testing by identifying how much of the target-of-test
was covered, in some dimension, during testing. Typical classes of coverage include use cases (requirements-based),
logic branch or node (code-based), data state, and function points.
-
Test case generators automate the generation of test data. Test case generators use either a formal
specification of the target-of-test's data inputs, or the design models and source code to produce test data that
tests the nominal inputs, error inputs, and limit and boundary cases.
-
Comparator tools compare test results with reference results and identify differences. Comparators differ in
their specificity to particular data formats. For example, comparators may be pixel-based to compare bitmap images
or object-based to compare object properties or data.
-
Data extractors provide inputs for test cases from existing sources, including databases, data streams in a
communication system, reports, or design models and source code.
|
