Neptune 3 UI - Squish Automated UI Testing
Squish is a commercial product from Froglogic which is used for automated GUI testing.
The goals of these tests and their setup are so that we can:
- Provide a basic set of scripts for Neptune 3 UI, QmlLive, and Qt Application Manager for smoke testing on a CI.
- Document any specifics in Qt Automotive Suite, especially the multi-processing aspect, which users should pay attention to for automated UI testing with Squish.
- Provide reference scripts as a part of Neptune 3 UI.
- Enable developers working with UI, in the Qt Automotive Suite, to write their own tests, if needed.
To use these tests, you need:
- Qt Automotive Suite
- Squish installed on the test machine
Note: Squish and the accompanying license must be obtained from Froglogic.
Set Up Your Test Environment
Squish can be run:
- within an IDE - the Squish IDE is available for most developer platforms.
- as a service - the service program,
squishserver, is either distributed with the IDE or can be adapted from an embedded source package. This source package runs on different platforms and is downloadable from Froglogic.
The target referenced embedded system is a Yocto Linux x64 build which, on the Qt Automotive Suite, can be installed as Intel NUC.
Tests are provided in Neptune 3 UI's
squishtest sub folder, and can be imported as a suite directly into the Squish IDE.
Application(s) Under Test (AUTs)
Whether you use the Squish IDE or squishserver, you must define all of the AUTs, that also includes all sub-processes that are called from within Neptune 3 UI apps. For multi-process, each of the Neptune 3 UI apps run as
To distinguish and attach to these processes you need to use the Squish hook plugin that extends Squish's attachable AUT feature. The
server.ini file must be adapted to specify where Neptune 3 UI and the other AUTs are located.
Because in multi-process, all Neptune 3 UI apps must be connected via Squish's attachable AUT feature, these apps must be defined as attachable AUTs with a corresponding port number. Additionally, the port in the
server.ini has to match the port defined in its
info.yaml manifest file as
Note: The Neptune 3 UI apps and their manifest files are located in
To run these tests directly from the IDE, it is possible and recommended to add or modify AUTs and attachable AUTs with port numbers, in the IDE itself.
For more details, see Squish Initialization Files.
[General] AUT/appman = "/home/user/qt/bin" AUT/appman-launcher-qml = "/home/user/qt/bin" AUT/neptune3-ui = "/home/user/install/neptune3" attachableAUT/neptune3-ui = "localhost:3334" attachableAUT/appman = "localhost:7777" attachableAUT/app_appstore = "localhost:7720" attachableAUT/app_calendar = "localhost:7721" attachableAUT/app_climate = "localhost:7722" attachableAUT/app_cluster = "localhost:7723" attachableAUT/app_downloads = "localhost:7724" attachableAUT/app_hud = "localhost:7725" attachableAUT/app_map = "localhost:7726" attachableAUT/app_music = "localhost:7727" attachableAUT/app_phone = "localhost:7728" attachableAUT/app_settings = "localhost:7729" attachableAUT/app_sheets = "localhost:7730" attachableAUT/app_tuner = "localhost:7731" attachableAUT/app_vehicle = "localhost:7732"
Exclude Disruptive Sub-Processes
Typically, Squish tries to connect to all sub-processes that are created from an initial AUT. However, this behavior can lead to issues where Squish stops, does not create these processes, or causes these processes to not behave as expected.
Remember, that these processes can also be external programs or services, which in our case are:
- D-Bus process
Consequently, for Neptune 3 UI you need to force Squish to ignore these processes to run the tests. Otherwise these tests might not succeed from the beginning - for example because of a stopped D-Bus session.
To ignore these processes you need to add or edit the
ignoredauts.txt file in your Squish installation directory. For macOS and other Unix-like Operating Systems, the file is in
For Linux, make the following change:
# ignore the following processes remotesettings-server dbus-daemon
For other Operating Systems, you might need to modify the D-Bus process name.
Note: It might be a good idea to ignore all sub-processes created that are not relevant to test in the test suite.
Squish needs additional Qt libraries to run Neptune 3 UI tests. One solution is to copy the following libraries from your installation folder to Squish's installation library folder
Note: Different Qt versions: If the Qt version for your Squish release differs from the Qt version used in Neptune 3 UI, when the tests start, you may notice a warning in the logs from
Loaded Qt version X.xx differs from Squish/Qt expected version X.yy, automating this AUT may not work."
Currently, with Neptune 3 UI's 5.12 and Squish's 5.11 there haven't been issues and tests can run using the additional libraries; but this warning mustn't be ignored.
Run Squish from the Command Line or on a Target
Once you have set up your test environment, you have the option to run tests via the Squish IDE, the command line, or as a service: such as a
systemd service (recommended), for your target Yocto environment.
Remember to use a D-Bus session beforehand, as this is how Neptune 3 UI communicates with its sub-processes.
Normally, for simple cases you can use a server,
squishserver, to run the tests and a corresponding client,
squishrunner, that can request the server to run these tests and return their results.
The AUT is on the server side, the test cases are transmitted from
squishserver; they need not be stored on the test machine. If you use the Squish IDE, when performing tests, both
squishrunner are run seamlessly on the same machine.
When you use the the command line, start the server first before the clients.
Ensure that all IP addresses for
squishrunner are able to connect to
squishserver, this is stored in
$SQUISH_DIR/etc/squishserverrc. Make sure that the IP ranges match, localhost is always valid.
The server starts with the following:
$SQUISH_DIR/bin/squishserver --port 3333
squishserver has a default port: 4322
Change the directory to the
squishtest path in your Neptune 3 UI installation, and run a single test case with the following command:
$SQUISH_DIR/bin/squishrunner --host SQUISH_SERVER_IP --port 3333 --testsuite suite_neptune3 --testcase tst_popup_climate
The test cases are run on a client, and the results are returned.
Squish on Target Environment (Yocto)
You can add a test environment on target hardware with Neptune 3 UI installed. As mentioned before, for a Yocto environment, you need to create a custom
squishserver using the Yocto SDK. A squish-embedded source package can be downloaded from the Froglogic website. Additionally, you need to have a binary desktop Squish installation to use the
Create a Custom (Yocto) Squishserver
The Yocto environment is an x64 Linux but unlike a standard Ubuntu x64 version, libraries in Yocto are stored in
Download and install the Yocto SDK separately, or use it from your Qt Automotive Suite installation (Qt Automotive Suite, Intel NUC).
Next, download the squish-embedded source package
squish-x.x.x-qt-embedded-src.tar.gz from the Froglogic website.
To enable cross compiling, run the following command:
Then, change the directory to where the squish-embedded source package is located. Typically, the binary desktop edition is installed in
mkdir -p yocto/build && cd yocto/build
Run the next two commands consecutively:
../../configure --with-qmake=<SDK-install-location>/sysroots/x86_64-pokysdk-linux/usr/bin/qmake --enable-qmake-config --with-squishidl=$SQUISH_DIR/bin/squishidl --disable-all --enable-qt --enable-server
Next, create a destination directory:
Build and install Yocto into it. Then, create an archive for transferability and storability. Make sure to use the same version as Squish.
./build install DESTDIR=../install && tar cfz ../squish-x.x.x-qt-yocto.tgz ../install
Install and Set Up Custom Yocto Squishserver on Target Hardware
The affected directory structure on the Neptune 3 target is:
./home/root ./home/.squish ./opt ./opt/squish
Perform the following steps on the target:
- Unpack the
/opt/squish-for-qt-x.x.xand rename the directory according to the version number, if needed.
- For convenience, create a symlink from the directory you just created to
/opt/squish. Consider this symlink as your
$SQUISH_DIR/etc/squishserverrcaccording to your network arrangement.
$SQUISH_DIR/etc/ignoredauts.txtto exclude disruptive sub-processes.
- Place your license key in the
- Transfer the
$USER\.squishconfiguration files from your working Squish installation directory on your local machine. This directory is created when you first start the Squish IDE.
- Then, adapt the paths to the Neptune 3 UI installation:
/opt/. The files involved are:
.squish/ver1/sub-folder. For more information, see Squish initialization files.
Note: Tests cannot be performed while Neptune 3 UI is running. You need to make sure that the
neptune3-ui process is terminated, prior to running the tests.
© 2019 Luxoft Sweden AB.
Documentation contributions included herein are the copyrights of
their respective owners.
The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation.
Qt and respective logos are trademarks of The Qt Company Ltd. in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.