Test Prioritization Tool
The test prioritization tool, also known as the tselect tool, enables the profile-guided optimizations on all supported Intel® architectures, on Linux*, Windows*, and
operating systems, to select and prioritize tests for an application based on prior execution profiles.
The tool offers a potential of significant time saving in testing and developing large-scale applications where testing is the major bottleneck.
Development often requires changing applications modules. As applications change, developers can have a difficult time retaining the quality of their functional and performance tests so they are current and on-target. The test prioritization tool lets software developers select and prioritize application tests as application profiles change.
The information about the tool is separated into the following sections:
- Features and benefits
- Requirements and syntax
- Usage model
- Tool options
- Running the tool
Features and Benefits
The test prioritization tool provides an effective testing hierarchy based on the code coverage for an application. The following list summarizes the advantages of using the tool:
- Minimizing the number of tests that are required to achieve a given overall coverage for any subset of the application. The tool defines the smallest subset of the application tests that achieve exactly the same code coverage as the entire set of tests.
- Reducing the turn-around time of testing. Instead of spending a long time on finding a possibly large number of failures, the tool enables the users to quickly find a small number of tests that expose the defects associated with the regressions caused by a change set.
- Selecting and prioritizing the tests to achieve certain level of code coverage in a minimal time based on the data of the tests' execution time.
See Understanding Profile-guided Optimization and Profile an Application topics for general information on creating the files needed to run this tool.
Test Prioritization Tool
The test prioritization tool needs the following items to work:
- The .spi file generated by Intel® compilers when compiling the application for the instrumented binaries with the-prof-gen=srcpos(Linux* and) ormacOS*/Qprof-gen:srcpos(Windows*) option.
- The .dpi files generated by the profmerge tool as a result of merging the dynamic profile information .dyn files of each of the application tests. Run the profmerge tool on all .dyn files that are generated for each individual test and name the resulting .dpi in a fashion that uniquely identifies the test.
- User-generated file containing the list of tests to be prioritized. For successful instrumented code run, you should:
- Name each test .dpi file so the file names uniquely identify each test.
- Create a .dpi list file, which is a text file that contains the names of all .dpi test files.Each line of the .dpi list file should include one, and only one .dpi file name. The name can optionally be followed by the duration of the execution time for a corresponding test in the dd:hh:mm:ss format.For example:Test1.dpi 00:00:60:35states that Test1 lasted 0 days, 0 hours, 60 minutes and 35 seconds.The execution time is optional. However, if it is not provided, then the tool will not prioritize the test for minimizing execution time. It will prioritize to minimize the number of tests only.
The profmerge tool merges all .dyn files that exist in the given directory. Make sure unrelated .dyn files, which may remain from unrelated runs, are not present. Otherwise, the profile information will be skewed with invalid profile data, which can result in misleading coverage information and adverse performance of the optimized code. The tool uses the following general syntax:
-dpi_listis a required tool option that sets the path to the list file containing the list of the all .dpi files. All other tool commands are optional.
Windows* only: Unlike the compiler options, which are preceded by forward slash ("/"), the tool options are preceded by a hyphen ("-").
The following figure illustrates a typical test prioritization tool usage model.
Test Prioritization Tool Options
The tool uses the options that are listed in the following table:
Prints tool option descriptions.
Required. Specifies the name of the file that contains the names of the dynamic profile information (
.dpi) files. Each line of the file must contain only one .dpi file name, which can be followed by its execution time (optional). The name must uniquely identify the test.
Specifies the file name of the static profile information file (.SPI). Default is
Specifies the file name of the output report file.
Specifies the file name that contains the list of files of interest.
Instructs the tool to terminate when the cumulative block coverage reaches a preset percentage, as specified by
value, of pre-computed total coverage.
valuemust be greater than 0.0 (for example, 99.00) but not greater than 100.
valuecan be set to 100.
Instructs the tool to ignore the pre-compute total coverage process.
Instructs the tool to minimize testing execution time. The execution time of each test must be provided on the same line of
dpi_listfile, after the test name in dd:hh:mm:ss format.
Specifies a different top-level project directory than was used during compiler instrumentation run with the
prof-src-rootcompiler option to support relative paths to source files in place of absolute paths.
Instructs the tool to generate more logging information about program progress.
Running the Tool
The following steps demonstrate one simple example for running the tool on IA-32 architectures.
- Specify the directory by entering a command similar to the following:Exampleset PROF_DIR=c:\myApp\prof-dir
- Compile the program and generate instrumented binary by issuing commands similar to the following:Operating SystemCommandLinux andmacOS*