Running Autotests

Qt Creator integrates the Qt Test framework and Google C++ Testing Framework for unit testing applications and libraries. You can use Qt Creator to build and run Qt tests, Qt Quick tests (QML-based Qt tests), and Google tests for your projects.

To enable the experimental Auto Test plugin, select Help > About Plugins > Utilities > Auto Test. Restart Qt Creator to be able to use the plugin.

Creating Tests

You can use a wizard to create projects that contain Qt or Google tests.

Creating Qt Tests

To create a Qt test:

  1. Select File > New File or Project > Other Project > Auto Test > Choose to create a project with boilerplate code for a Qt test.
  2. In the Project and Test Information dialog, specify settings for the project and test:
    1. In the Test framework field, select Qt Test.
    2. Select the GUI Application check box to create a Qt application.
    3. In the Test case name field, enter a name for the test case.
    4. Select the Requires QApplication check box to add the include statement for QApplication to the main.cpp file of the project.
    5. Select the Generate initialization and cleanup code checkbox to add functions to your test that are executed by the testing framework to initialize and clean up the test.
    6. In the Build auto tests field, select Always to always build the test when building the project or Debug Only to only build it during debug builds.

Qt Creator creates the test in the tests\auto directory in the project directory. Edit the .cpp file to add private slots for each test function in your test. For more information about creating Qt tests, see Creating a Test.

Creating Google Tests

To create a Google test:

  1. Select File > New File or Project > Other Project > Auto Test > Choose to create a project with boilerplate code for a Google test.
  2. In the Project and Test Information dialog, specify settings for the project and test:
    1. In the Test framework field, select Google Test.
    2. In the Test case name field, enter a name for the test case.
    3. In the Test set name field, enter a name for the test set.
    4. Select the Enable C++ 11 check box to support C++ 11 features in the test.
    5. In the Build auto tests field, select Always to always build the test when building the project or Debug Only to only build it during debug builds.
    6. In the Google test repository field, select a directory that contains a clone of the googletest repository.

      To use an installed Google C++ Testing framework instead, see Setting Up the Google C++ Testing Framework.

Qt Creator creates the test in the tests\auto directory in the project directory.

Setting Up the Google C++ Testing Framework

To build and run Google tests, you must have the Google C++ Testing framework installed and configured on the development host. You can either clone it from Git Hub or install it from an installation package.

To configure a project to use a cloned Google testing framework, edit the INCLUDEPATH variable in the project file (.pro) to include the source and include folders of Google Test's googletest and googlemock. Usually, you need to add the following subfolders:

  • googletest
  • googlemock
  • googletest/include
  • googlemock/include

You also need to add the necessary files to the SOURCES variable. For example:

  • googletest/src/gtest-all.cc
  • googlemock/src/gmock-all.cc

To configure a project to use an installed Google testing framework package, add the following include paths to the .pro file:

  • <googletest_install_path>/include/gtest
  • <googletest_install_path>/include/gmock

Then add linker options to be able to find the libraries and to link against them. For example, for qmake based projects, you typically need to add the following values to the .pro file:

  • LIBS += -lgtest -L<path_to_gtest_lib>
  • LIBS += -lgmock -L<path_to_gmock_lib>

To specify settings for running Google tests, select Tools > Options > Test Settings > Google Test.

To run disabled tests, select the Run disabled tests check box.

To run several iterations of the tests, select the Repeat tests check box and enter the number of times the tests should be run in the Iterations field. To make sure that the tests are independent and repeatable, you can run them in a different order each time by selecting the Shuffle tests check box.

To turn failures into debugger breakpoints, select the Break on failure while debugging check box. To turn assertion failures into C++ exceptions, select the Throw on failure check box.

For more information about creating Google tests, see the Google Test Primer.

Building and Running Tests

To build and run tests:

  1. Open a project that contains tests.
  2. In the Test Results output pane, select (Run All Tests) to run all test or (Run Selected Tests) to run the selected tests.

    You can select the tests to run in the Tests view in the Projects pane. For more information, see Viewing Tests.

    Note: By default, Qt Creator builds a project before deploying and running it.

If a test takes more than a minute to execute, the default timeout might stop the test execution. To increase the timeout, select Tools > Options > Test Settings > General.

Qt Creator scans the project for tests when you open the project and updates the test list when you edit tests. To only update the test list when the Tests view or the Test Results output pane is open, deselect the Always parse current project for tests check box.

The code inside a benchmark test is measured, and possibly also repeated several times in order to get an accurate measurement. This depends on the measurement back-end that you can select in the Benchmark Metrics group in Tools > Options > Test Settings > Qt Test: walltime, CPU tick counter, event counter, Valgrind Callgrind, and Linux Perf. For more information, see Creating a Benchmark.

To allow the debugger to interrupt Qt tests on assertions, select the Disable crash handler while debugging check box.

Viewing Test Output

The test results are displayed in the Test Results output pane.

ResultDescription
BENCHBenchmark test.
BFAILBlacklisted test case failed. Since Qt 5.4, you can provide a BLACKLIST file for tests. It is mainly used internally by the Qt CI system.
BPASSBlacklisted test case passed.
DEBUGDebug message.
XFAILTest case is expected to fail, so it is marked by using the QEXPECT_FAIL macro. If the test case passes instead, an unexpected pass (XPASS) is written to the test log.
FAILTest case failed. Double-click the line for more information.
INTERNALInternal message.
PASSTest case passed.
SKIPTest case was skipped.
XPASSTest case passed even though it was expected to fail.
WARNWarning message.

To view only messages of a particular type, select (Filter Test Results), and then select the types of messages to show.

To hide internal messages and run configuration warnings, select Tools > Options > Test Settings > General.

© 2017 The Qt Company Ltd. Documentation contributions included herein are the copyrights of their respective owners. The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation. Qt and respective logos are trademarks of The Qt Company Ltd in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.