Neptune 3 UI - Squish Automated UI testing

Squish is a commercial product from Froglogic which is used for automated GUI testing.

The goals of tests and their setup are:

  • Provide a basic set of scripts for Neptune 3 UI, Qt QmlLive, and Qt Application Manager for smoke testing on a CI.
  • Document any specifics in Qt Automotive Suite, especially the multi-processing aspect of it, a user should pay attention to when it comes to automated UI testing with Squish.
  • Provide reference scripts as a part of Neptune 3 UI.
  • Enable developers working with UI, in the Qt Automotive Suite, to write their own tests if needed.

The basic prerequisite to use the tests provided are:

Note: Squish and the accompanying license must be obtained from Froglogic.

Set Up Your Test Environment

Squish can be either run within an IDE or as a service. The IDE is available on most developers' platforms, the service program called squishserver is either distributed with the IDE or it can be adapted from an embedded source package, which can run on different platforms and is also provided as download from Froglogic.

The target referenced embedded system is a Yocto Linux x64 build which can be installed within the Qt Automotive Suite as "Intel NUC".

Tests are provided in squishtest sub folder in Neptune 3 UI and can be imported as suite directly into Squish IDE.

Application(s) Under Test (AUTs)

Whether you use the Squish IDE or squishserver, you must define all of the AUTs, which also includes all sub processes that are called from within the Neptune3 UI application. For multi-process each of the Neptune3 apps (see Neptune 3 UI - Application Development) run as a appman-launcher-qml process (Qt Application Manager). To distinguish and attach to these processes you need to use the Squish hook plugin that extends Squish's attachable AUT feature. The server.ini file has to be adapted to point to where the Neptune3 UI and other AUTs are located. Because in multi-process all Neptune3 apps must be connected via the attachable AUT feature of Squish, they must be defined as attachable AUTs with a corresponding port number.

For each Neptune3 app (as attachable AUT) the port in the server.ini has to match the port defined in its info.yaml manifest file as squishPort property (the Neptune3 apps and their manifest files are located in neptune3/apps).

To run the tests directly from the IDE it is possible and recommended to change and add AUTs and attachable AUTs with port number from inside the IDE.

For more details, see Squish Initialization Files.

[General]
AUT/NeptuneControlApp = "/home/user/install/neptune3"
AUT/RemoteSettingsServer = "/home/user/install/neptune3"
AUT/appman = "/home/user/qt/bin"
AUT/appman-launcher-qml = "/home/user/qt/bin"
AUT/neptune3-ui = "/home/user/install/neptune3"
ApplicationPaths = "/home/user/qt/bin,"
attachableAUT/RemoteSettingsServer = "localhost:5985"
attachableAUT/neptune3-ui = "localhost:3334"
attachableAUT/appman = "localhost:7777"
attachableAUT/app_appstore = "localhost:7720"
attachableAUT/app_calendar = "localhost:7721"
attachableAUT/app_climate = "localhost:7722"
attachableAUT/app_cluster = "localhost:7723"
attachableAUT/app_downloads = "localhost:7724"
attachableAUT/app_hud = "localhost:7725"
attachableAUT/app_map = "localhost:7726"
attachableAUT/app_music = "localhost:7727"
attachableAUT/app_phone = "localhost:7728"
attachableAUT/app_settings = "localhost:7729"
attachableAUT/app_sheets = "localhost:7730"
attachableAUT/app_tuner = "localhost:7731"
attachableAUT/app_vehicle = "localhost:7732"

Note: Different Qt versions

If the Qt version for Squish release differs from the Qt version used in Neptune3 UI (currently 5.12), when the tests start, you may notice a warning in the logs from squishserver indicating that:
"Loaded Qt version 5.xx differs from Squish/Qt expected version 5.yy, automating this AUT may not work."
In this case, you need to copy the following Qt libraries from your installation folder to Squish's installation library folder: libQt5Widgets.so, libQt5QuickWidgets.so

D-BUS

Neptune 3 UI needs a D-Bus session, on Linux this can be achieved by running

export $(dbus-launch)

before starting the IDE. In case of a target installation this should already be included during the boot process.

Squish from Command Line or on Target

After setting up the test environment there is the option to run tests via IDE, command line, or as a service such as a systemd service (recommended) for the target Yocto environment. Remember to use a D-Bus session beforehand, this is how Neptune 3 UI communicates with its sub processes.

For simple cases, normally, you can use a server, squishserver, to run the tests and a client, squishrunner, that can request the server to run these test cases and return their results.

The AUT is on the server side, the test cases are transmitted from squishrunner to squishserver, they need not be stored on the testing machine. In the IDE case when performing tests, both (squishserver and squishrunner) are executed seamlessly on the same machine.

When you use the the command line, start the server first before the clients.

Ensure that all IP addresses for squishrunner are able to connect to squishserver. On the squishserver, this is stored in $SQUISH_DIR/etc/squishserverrc. Make sure that the IP ranges match, localhost is always valid.

The server starts with (squishserver also has a default port)

$SQUISH_DIR/bin/squishserver --port 3333

Change directory to the squishtest path in your Neptune 3 UI installation, and run a single test case by

$SQUISH_DIR/bin/squishrunner --host SQUISH_SERVER_IP --port 3333 --testsuite suite_neptune3 --testcase tst_popup_climate

The test cases / suites on a client are executed and the results returned.

Squish on Target (Yocto) Environment

You can add a test environment on target hardware with Neptune 3 UI installed. As mentioned before, for a Yocto environment a custom squishserver needs to be created using the Yocto SDK, and a squish-embedded source package can also be downloaded from Froglogic website. A binary desktop Squish installation is also needed to use the squishidl command yet.

Creating a Custom (Yocto) Squishserver

The Yocto environment is a x64 Linux but unlike a standard ubuntu x64 version, libraries in Yocto are stored in /lib64 versus /lib .

Download and install Yocto SDK separately or use it from your Qt Automotive Suite installation (Qt Automotive Suite, Intel NUC). Then download squish-embedded source package squish-x.x.x-qt-embedded-src.tar.gz from the Froglogic website.

To enable cross compiling do

source <SDK-install-location>/environment-setup-corei7-64-poky-linux

and then change directory to where the squish-embedded source package is (SQUISH_DIR is where the installation of the binary desktop edition is stored):

mkdir -p yocto/build && cd yocto/build

then run

../../configure --with-qmake=<SDK-install-location>/sysroots/x86_64-pokysdk-linux/usr/bin/qmake --enable-qmake-config --with-squishidl=$SQUISH_DIR/bin/squishidl --disable-all --enable-qt --enable-server

followed by

./build

create a destination dir

mkdir ../install

build install into it, and create an archive to be better transferable and storable (use same version as squish)

./build install DESTDIR=../install && tar cfz ../squish-x.x.x-qt-yocto.tgz ../install

Install and Set Up Custom Yocto Squishserver on Target Hardware

The affected directory structure on the Neptune 3 target is

./home/root
./home/.squish
./opt
./opt/squish
  • squish.x.x.x-qt.yocto.tgz should be unpacked into /opt/squish-for-qt-x.x.x Adapt version, this is $SQUISH_DIR .
  • edit $SQUISH_DIR/etc/squishserverrc to match your network arrangements.
  • put your license key into /home/root
  • transfer the $USER\.squish configuration files from your working, such as created from the IDE, and adapt the paths to Neptune 3 UI installation (/opt/ ). Files covered are: server.ini and paths.ini in sub folder .squish/ver1/ , also look again at Squish initialization files.

Note: Tests cannot be performed while Neptune 3 UI is running. You need to make sure that the neptune3-ui process is terminated, prior to running the tests.

Also note that before you start squishserver you need to get a D-Bus session by running either

export $(dbus-launch)

on the console, or for an automatic start-up to use a D-Bus enabled systemd squishserver service.

© 2018 Pelagicore AG. Documentation contributions included herein are the copyrights of their respective owners. The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation. Qt and respective logos are trademarks of The Qt Company Ltd. in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.