The squishrunner tool is used to execute and record test cases and test suites, and to set and retrieve the settings it needs at runtime. This section desribes its different modes of use:
Note: In almost every case, to make use of the
squishserver must already be running.
Every mode of use can accept zero or more server options. If the
--port options are not specified, Squish uses sensible defaults for them.
|By default, |
|By default, |
|Outputs additional debugging information to the |
The following command executes all the test cases in the specified test suite and outputs all the additional debugging information that is available:
squishrunner --debugLog alpw --testsuite /home/reggie/squish_addressbook_py
alpw is not a single command, but rather a list of one-letter commands:
w. Any of them can be specified, for example,
w. The meaning of the letters are:
a– log the application's start. This is the same as checking the Squish IDE's Application Start checkbox in the Preferences dialog's Squish pane's Logging child pane.
l– log dllpreload.exe (this only applies on Windows; if used on other platforms it is safely ignored).
p– log the preload phase. This is the same as checking the Squish IDE's Hooking checkbox in the Preferences dialog's Squish pane's Logging pane.
w– log the wrapping phase. This is the same as checking the Squish IDE's Wrapper Library checkbox in the Preferences dialog's Squish pane's Logging pane.
When the squishrunner is executed with the
--testsuite option, it reads in the
suite.conf file in the test suite's root directory, and its behavior will be affected by the specified options in those files. Many of these settings can be viewed and edited using the Squish IDE Test Suite Settings view.
|The AUT to start for each test case, and optionally any command line arguments that should be passed to the AUT when it is started. This can be overridden on squishrunner's command line using the |
|Sets the Java |
|The AUT's working directory when being tested. If this option isn't present or if its value is empty, the AUT's working directory is set to the path of the AUT's executable. If the value is not empty it is taken to be the path to use (e.g., |
|A file that contains one or more lines of |
|Whether Squish should hook into sub-processes launched by the AUT. The |
|Whether Squish should automatically start the AUT. This was common for Squish 3 scripts; but for Squish 4 this setting is normally |
|The scripting language used by the test scripts. Currently this must be |
|If present, the naming |
|By default, Squish reads the test suite's object map from the |
This configurating setting only applies to Text-Based Object Map.
|A space-separated list of one or more test case names. If the entire test suite is run, e.g., by using the |
In addition, this option is used by the Squish IDE to determine the order in which the test cases are shown in the Test Cases view. To have only particular test cases run, use the
|The wrappers which must be loaded for the tests to run successfully. Currently supported are |
The values used in the
suite.conf file can also refer to environment variables. The syntax for this is
$(ENVVAR). When such an entry is read, it is effectively replaced by the value of the environment variable of the same name. So if there was an environment variable called
/var/spool/mail/reggie, if we used
$(MAIL) in the
suite.conf file, it would be replaced with
Use squishrunner with the
--testsuite option to execute all of a test suite's test cases, or one or more specified test cases.
squishrunner [server-options] [settings-option]
--testsuite test-suite-dir [other-options]
All items in square brackets are optional. The
server-options are described in Server Options. The
other-options are discussed in the following sections. When used in this mode, the squishrunner's behavior is affected by the test suite's
suite.conf. For more information, see The suite.conf File.
There is one deprecated
This option makes it possible to specify which settings group to use. Settings groups are deprecated and have no support in the Squish IDE. This option exists purely for backward compatibility.
There are several
The square brackets are not part of the syntax—they indicate optional items. Here, every option is optional, and some parts of some options are optional. The
* indicates an option that can occur zero or more times, and
| indicates alternatives.
See Playback option --testcase or --skip-testcase for more details on playback related options.
squishrunner can automatically start and stop a local squishserver process as needed, during the playback of a test suite. Using this option means no
--port options are needed, and no separate squishserver process needs to be started.
--resultdir option is used in combination with file-based report generators (stdout, xmljunit, xml2), and determines the location to save the test report and failed screenshot files, if any.
Note: This option conflicts with the use of a directory-based report generator (json, html, xml3) and specifying it will yield an error. In cases where a file-based and a directory-based report generator are combined (like junit and xml3) this option needs to be removed and the directory-based report generator will set it implicitly to the specified directory from its configuration.
Squish supports several report generators simultaneously, with one limitation, there can be at most one directory based generator.
squishrunner --help for a list of supported generators.
Some generators output to the console unless a destination is specified.
Note: Generally, it is recommended to use the latest version available for any report generator.
Generates a static HTML page showing the results which are being stored in json data. This report generator is deprecated, please consider using the testcentercmd (see Processing Test Results) utility or the Squish Test Center report generator instead.
squishrunner --testsuite test-suite-dir --reportgen html,report-dir
Generates a Squish-specific custom JSON result format, intended to be used internally by the HTML report generator. This report generator is deprecated, please consider using the testcentercmd (see Processing Test Results) utility or the Squish Test Center report generator instead.
json1.2(Squish 6.3 and newer)
squishrunner --testsuite test-suite-dir --reportgen json1.2,report-dir
json1.1(Squish 6.1 and newer)
squishrunner --testsuite test-suite-dir --reportgen json1.1,report-dir
json(Squish 6.0 and newer)
squishrunner --testsuite test-suite-dir --reportgen json,report-dir
junit generator outputs reports in the same format as JUnit tests do, as single file in XML format.
squishrunner --testsuite test-suite-dir --reportgen junit,reportfile.xml
xmljunit generator outputs reports in the same format as JUnit tests do, as single file in XML format. However, since it pre-dates Squish' BDD support it does not preserve the test case and BDD test structure in its generated test report as well as the
junit report generator.
squishrunner --testsuite test-suite-dir --reportgen xmljunit,reportfile.xml
Generates no report, this is useful to not have the default stdout output.
squishrunner --testsuite test-suite-dir --reportgen null
An ASCII plain text table, the default used by squishrunner when no other generator is specified.
squishrunner --testsuite test-suite-dir
squishrunner --testsuite test-suite-dir --reportgen stdout
The stdout report generator can accept an optional destination filename:
squishrunner --testsuite test-suite-dir --reportgen stdout,reportfile.txt
For uploading to Squish Test Center.
squishrunner --testsuite test-suite-dir --reportgen testcenter,url/project/project-name?token=token
token is an upload token, created from Squish Test Center user settings. See Squish Test Center documentation for further details.
url can also include a query that specifies which labels or batches should be applied to the result. See Squish Test Center documentation for an example.
squishrunner --testsuite test-suite-dir --reportgen xls,reportfile.xml
The XML report generator comes in a variety of versions, each newer one adding more detail. It is mainly used by internal tools and Squish Test Center, the generated report is not meant for manual browsing.
xml3.5(Squish 6.8 and newer)
squishrunner --testsuite test-suite-dir --reportgen xml3.5,report-dir
Adds support for Video Capture. Also increases the precision of time attributes (now contains milliseconds).
xml3.4(Squish 6.5 and newer)
squishrunner --testsuite test-suite-dir --reportgen xml3.4,report-dir
Allows seeing result attachments, screenshots, as well as script backtraces.
xml3.3(Squish 6.4 and newer)
squishrunner --testsuite test-suite-dir --reportgen xml3.3,report-dir
Images can be seen in Squish Test Center.
xml3.2(Squish 6.3 and newer)
squishrunner --testsuite test-suite-dir --reportgen xml3.2,report-dir
Uses a new <retry> tag, and reports retries for test cases and scenarios.
Given a test suite called suite_myapp, a test case called tst_case1, a
my_app_results, and assuming that an image verification point fails, squishrunner will write:
Note: Legacy XML formats, such as
xml are kept for backward compatibility. Those older than
xml3 take a filename instead of a directory.
By default, all the test suite's test cases are executed, but if we want to specify exactly which test case or test cases are executed we can do so by supplying 1 or more of the
--testcase test-case-dir option for each test case we want executed. Alternatively to the
--testcase it's possible to skip test cases by using the
--skip-testcase test-case-dir option.
squishrunner --testsuite /home/reggie/suite_addressbook --testcase tst_add_address --testcase tst_edit_address
This example runs two specific test cases in the
suite_addressbook suite. If we wanted to run all of the suite's test cases we would simply omit the two
--testcase options since without them squishrunner defaults to running all of the suite's test cases. If we wanted to run all test cases but
tst_add_address, we would do something like this:
squishrunner --testsuite /home/reggie/suite_addressbook --skip-testcase tst_add_address
squishrunner will automatically start capture a video of the desktop of any application started via ApplicationContext startApplication(autName) or attached via ApplicationContext attachToApplication(autName). The videos are stored as part of the report, as if the test.startVideoCapture(message) function had been used and they are terminated once the application terminates or is disconnected from. The test report will have corresponding start and end entries when the capture starts and stops using a message that includes the application context's name.
By default, the AUT specified in the test suite's
suite.conf file will be used for the execution of the test cases, along with any command line arguments that are specified there. However, we can override this by specifying the name of the AUT using the
--aut aut option, followed by zero or more command line arguments. The specified AUT must be registered as a Mapped AUT already and the
Automatically start the AUT option in the test suite settings must be checked/enabled. (See also, The suite.conf File.)
By default, squishrunner works in non-interactive mode which allows it to run without access to any kind of graphical display. For using any of the
testInteraction functions (See testInteraction Functions) however squishrunner needs to be run in interactive mode. Pass the
--interactive option to enable interactive mode which will allow squishrunner to display dialogs and message boxes.
--testsuite option executes all test suite's test cases one by one. If the
--abortOnFail option is specified, squishrunner will terminate the suite execution as soon as a failed test case is detected.
When recording a web test, the browser to be used can be specified via the
--webbrowser browser option. The browser can be any of:
ie (Microsoft Internet Explorer),
edge (Microsoft Edge),
google-chrome (Google Chrome), or
chromium-based (Chromium-based Applications).
--webbrowserargs option can be passed to invoke the chosen web browser with your choice of command line arguments.
--device option is for the Squish Android edition only. It is required when more than one Android devices connected and/or emulators running, to specify on which the application should be tested.
Most modern tests use the Object waitForObject(objectOrName) function, but for various reasons some tests may have calls to the snooze(seconds) function. To influence the delay triggered by calls to the snooze(seconds) function, it is possible to use the
--snoozeFactor factor option. The factor must be a number—if it is less than 1 (i.e., a decimal fraction, like 0.5), it will cause shorter delays, and if it is greater than 1 it will cause longer delays. A factor of 0 will produce the fastest possible execution.
By default squishrunner returns non-zero value in case of fatal errors and 0 otherwise, regardless of the tests results. If
--exitCodeOnFail option is set and defines the specific exit code (between 0 and 255), squishrunner will return the custom exit code on fatal errors or if any of the test cases has failed, and 0 otherwise.
--timeout option defines the amount of time (in seconds), after which a test case will be terminated regardless of its state. The option applies in
--testsuite mode only and accepts positive integer values. After a test case is timed out, it terminates and the execution proceeds with the next text case, if any.
--retry option defines the number of times Squish should try to execute a failed test case again (at maximum). This retrying of the current test case will end as soon as it passes. The option only applies to the
--testsuite mode and accepts only positive integer values. For BDD tests the
--retry option applies to every scenario of a feature.
--tags option can be used to execute just those test scripts or BDD scenarios which match a given tag filter. The leading '@' sign from the feature file tags can be omitted. Some examples for the
--tags option follows:
|--tags foo||Execute all scenarios or test scripts with the tag |
|--tags ~foo||Execute all scenarios or test scripts not tagged |
|--tags foo,bar||Execute all scenarios or test scripts tagged |
|--tags foo --tags bar||Execute all scenarios or test scripts tagged both |
|--tags foo,bar --tags yoyo||Execute all scenarios or test scripts with the tag |
For an example of how to @tag a BDD scenario, see this page.
--random option is for executing testcases in random order. The used sequence number is printed in the report as log message. If it is necessary to reproduce a specific order the sequence number can be given as parameter to the
--random option. This can be used to reproduce a failure that occurs during random test execution. A sequence number of 0 indicates that a new sequence number is generated. It gives the same behavior as when the value for the option is omitted.
Use squishrunner with the
--record options to record a new test case in the test suite.
--record tst_test-case-dir [other-options]
All items in square brackets are optional. The server-options are described in Server Options. The other-options are discussed shortly. When used in this mode the squishrunner's behavior is affected by the test suite's
suite.conf; see The suite.conf File. The tst_test-case-dir is the test case's name (and the directory where it will be stored)—it must start with
squishrunner --testsuite /home/reggie/suite_addressbook --record tst_search_for_address --useWaitFor
This example will cause squishrunner to execute the AUT specified in the test suite's
suite.conf file. All your interactions with the AUT will be recorded in the test suite's
tst_search_for_address subdirectory in the
test.py file (or in
test.js, etc., depending on the language specified in the
To save the recorded script file and exit squishrunner, press Ctrl+c.
There are several other-options:
The square brackets are not part of the syntax—they indicate optional items. Here, every option is optional. The
* indicates an option that can occur zero or more times.
By default, squishrunner uses event compression during recording—this means that mouse moves and drags result in high-level API calls rather than lots of intermediate mouse hover and move events. If the
--disableEventCompression is used, this compression is switched off. This option is only supported for Qt Widgets.
--webbrowserargs have the same behavior in recording mode as in playback mode, please refer to Playback option --webbrowser.
By default, the AUT specified in the test suite's
suite.conf file will be used for the recording of the test case, along with any command line arguments that are specified there. However, we can override this by specifying the name (with full path) of the AUT using the
--aut aut option, followed by zero or more command line arguments—if any arguments are specified they will be passed to the AUT when it is started up. (See also, The suite.conf File.)
Use squishrunner with the
--testcase option to execute or record a specific test case. Using the
--testsuite option is easier and more convenient than using this advanced option; see squishrunner --testsuite: Running tests in batch mode and Recording a Test Case.
--testcase tst_test-case-dir [other-options]
All items in square brackets are optional. The server-options are described in Server Options. The other-options are discussed shortly.
Keep in mind that the squishrunner needs to know various items of information to work correctly in
--testcase mode. One way to ensure it has the information it needs is to specify each item individually using the
--wrapper option, the
--objectmap option, and so on.
squishrunner --testcase tst_update_address --aut addressbook
This example starts the addressbook application and executes the specified test case. Since no language has been explicitly specified Squish will assume Tcl. Similarly, if the
--record option is used, Squish will write Tcl. This can be changed of course, as we will see next.
The square brackets are not part of the syntax—they indicate optional items. Here, every option is potentially optional. The
* indicates an option that can occur zero or more times, and
| indicates alternatives.
squishrunner will assume that the scripting language (to execute, or to record) is Tcl unless the
--lang language option is used. If the script does not have a call to ApplicationContext startApplication(autName), then the
--aut aut option must be used to specify the AUT.
|Records the test case. Without this option, the test case will be executed.|
|Includes a call to the ApplicationContext startApplication(autName) function into the recording. This is useful for Test Suites created by the Squish IDE, which do not automatically start the AUT when executing a testcase.|
|Zero or more test data files.|
|By default, squishrunner uses event compression during recording. This means that common event sequences result in high-level API calls rather than lots of low-level events. This option switches off the compression, which is not recommended.|
|Most modern tests use the Object waitForObject(objectOrName) function, but for various reasons some tests may have calls to the snooze(seconds) function. Use this option to influence the delay triggered by calls to the snooze(seconds) function.|
The factor must be a number. If it is less than 1 (i.e., a decimal fraction, like 0.5), it will cause shorter delays. If it is greater than 1, it will cause longer delays. A factor of 0 will produce the fastest possible execution.
|Squish can output a report detailing what happened during test case execution. Several different report generators are supported. Use this option to specify the one to use, along with the file into which the report should be written. For example, |
|One or more wrappers needed by the test case.|
|The test case's entire environment. The filename specifies the full path to a file that has key=value entries, one per line, and that will be set as environment variables.|
|Overrides one or more environment variables.|
|By default, squishrunner will use the current working directory when executing or recording the test case. Use @app to use the AUT's directory as the working directory instead. Use @server to use squishserver's directory or path to specify an explicit absolute path.|
|When using Text-Based Object Map, the object map must be loaded by the test case itself using the objectMap.load(filename). Alternatively, you can use this option to specify the full path of the object map to read (if executing or recording a test) or create or append to (if recording a test).|
|The browser to use when executing a web test: |
|Invokes the chosen web browser with command line arguments.|
|By default squishrunner works in non-interactive mode which allows it to run without access to any kind of graphical display. To use any of the testInteraction functions, squishrunner needs to be run in interactive mode. Use this option to enable interactive mode, which will allow squishrunner to display dialogs and message boxes.|
|If the test case doesn't use the ApplicationContext startApplication(autName) function, the AUT (with full path) must be specified using this option, followed by zero or more command line arguments. If any arguments are specified, they will be passed to the AUT when it is started up.|
Use squishrunner with the
--info option to query for various items of information.
squishrunner --info topic
This will print a list of the registered AUTs and their paths on the console:
squishrunner --info applications
The following table describes the information that is output for each of the available topic values. As is usual for squishrunner, a squishserver should be running for squishrunner to work.
|A list of all attached Android emulator or device instances.|
|A list of all installed instrumentation packages on all attached Android emulator or device instances.|
|A list of all the applications which are located in the squishserver's application paths and that can be tested by Squish with the current settings.|
|A list of all the registered attachable applications.|
|A list of paths in which squishserver looks for applications when starting an AUT. This value can be changed by using the |
|How many milliseconds Squish should wait after an application has exited to end squishrunner. In some setups, a second process is started from the first one and the second Squish hookup happens after the first one has exited. This value can be changed by using the |
|How many seconds squishrunner will wait before timing out with a test failure if the AUT doesn't respond after being started. This value can be changed by using the |
|on, if the mouse cursor should be animated (visually moved between positions) during script playback. Otherwise, off. This value can be changed by using the |
|The name of the web browser that is used for web testing.|
|A list of available iOS simulator devices. Requires squishserver to run on a macOS machine with Xcode installed.|
|Installation prefix of the Java installation squishserver was configured to use. This value can be changed by using the |
|The number of seconds that Squish will wait before timing out with a test failure during (mostly network based) communication between the squishserver and the squishrunner, and also between other Squish components.|
|The settings key for this Squish installation, e.g., ver1.|
|A list of all the system's web browsers that were detected.|
|A list of the installed wrappers.|
Use squishrunner with the
--config option to change various squishserver settings. In addition to the
--config, you must also specify the
--port of the squishserver. Using squishrunner to configure squishserver is useful when the squishserver is not running on the local machine. Under the covers, it is squishserver that performs the actual configuration changes for you.
squishrunner --config action
Only a single configuration action can be specified each time.
squishrunner --config setAUTTimeout 60
This will set the AUT timeout to 60 seconds (the default is 20 seconds).
|Adds an application mapped to the specified path. If different versions of the same application have the same executable name and appear in different paths that have been registered using the addAppPath option, this option specifies the one that Squish should use, thereby avoiding any ambiguity.|
|Removes an application.|
|Creates a new wrapper with the given wrapper name and base directory.|
|A browser and the path to its executable. The browser should be one of: |
|The filename of a Tcl script that should be executed when the specified wrapper is used. The script should either be a filename with an absolute path or a filename with a path relative to the wrapper's base directory. Whenever a test case needs to use a wrapper, Squish first executes all the scripts registered with this action for the wrapper (if any), before the test case is executed. Although |
|How long Squish should wait before timing out with a test failure if the AUT doesn't respond after being started. This action's current setting can be output using the |
|How long Squish should wait before timing out with a test failure during (mostly network based) communication between the squishserver and the squishrunner, and also between other Squish components.|
|How long Squish should wait after an application has exited to end squishrunner. In some setups, a second process is started from the first one and the second Squish hookup happens after the first one has exited.|
|Whether the mouse cursor should be animated (visually moved between positions) during script playback (|
|For web applications, specifies the default web browser: |
|The AUT that should be attached to when the test case is run.|
|Unregisters an AUT that was previously registered using the |
|Verifies that the extension for the browser-id is installed and enabled and the extension version allows automation with this Squish version. This option currently supports using |
|Installs the browser extension needed to automate the browser identified by browser-id. The installation procedure will start the browser and point it to the extensio. The browser will then request your permission to install and activate the extension. Eventually, it will require you to restart the browser. Once all of this has completed, make sure to completely quit the browser by using the Quit menu entry.|
This option currently supports using
|A list of currently set GlobalScript locations.|
|GlobalScript locations, which will be included when interpreter searches for |
© 2023 The Qt Company Ltd.
Documentation contributions included herein are the copyrights of
their respective owners.
The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation.
Qt and respective logos are trademarks of The Qt Company Ltd. in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.