Building and deploying the example

See specific steps relating to building and deploying Qt for Android Automotive examples.

Overview

The Surface Streaming example shows how to use a QAndroidRaaSApplication to render Qt Quick3D content offscreen within an Android Service. The service renders two separate outputs:

• One Qt Quick3D view rendered to an Android Surface provided by an activity (using an ObservableSurfaceView).
• Another view rendered to an ANativeWindow obtained from a QAndroidSurfaceStreamer, which encodes the content into an H.264 stream and serves it via an RTSP/RTP connection.

This example demonstrates how to combine Qt's offscreen rendering capabilities with Android's native surface management and media encoding APIs.

How It Works

The architecture consists of:

• Activity and Surface: The Android activity manages an ObservableSurfaceView, a small helper that simplifies surface lifecycle handling. Its Android Surface is passed to the service through an AIDL interface, allowing Qt to render into it.
• Qt Service: The service runs a QAndroidRaaSApplication with a QAndroidSurfaceRenderEngine. It manages multiple offscreen render targets, one bound to the activity’s surface, another to an ANativeWindow created by the QAndroidSurfaceStreamer. A QtRaaSApplication Java instance in the RenderingService will start and communicate with the Qt Quick application.
• Streaming Output: The streamer uses Android MediaCodec to set up an encoder. QAndroidSurfaceStreamer encode frames rendered on its surface into H.264 format, then transmits them over an RTSP/RTP connection.
• QML Scene: Both render targets share the same QML engine but can display independent View3D components or cameras, representing different viewpoints of the same 3D content.

C++ Entry Point

The main.cpp file creates a QAndroidRaaSApplication and initializes both the QAndroidSurfaceRenderEngine and the QAndroidSurfaceStreamer. The streamer's surface is used to encode the second Qt Quick3D view for network streaming.

    QAndroidRaaSApplication app(argc, argv);

    QAndroidSurfaceRenderEngine *renderEngine = new QAndroidSurfaceRenderEngine(&app);

Create and start the streamer.

    QAndroidMediaFormat mediaFormat;
    mediaFormat.setCodecType(QAndroidMediaFormat::CodecType::H264);
    mediaFormat.setSize(QSize(1080, 720));
    mediaFormat.setBitrate(8000000);
    mediaFormat.setFramerate(60.0f);
    mediaFormat.setIFrameInterval(1);
    mediaFormat.setOperatingRate(60.0f);
    mediaFormat.setAvcProfile(AVCProfileHigh);
    mediaFormat.setAvcLevel(AVC_LEVEL_4_2);

    QAndroidSurfaceStreamer *streamer = new QAndroidSurfaceStreamer(mediaFormat, &app);

    QObject::connect(
        streamer, &QAndroidSurfaceStreamer::error,
        [](QAndroidSurfaceStreamer::Error error, const QString &details) {
          qWarning() << "Stream error:"
                     << QAndroidSurfaceStreamer::errorString(error) << details;
        });

    QHostAddress host = QHostAddress::Any; // Listen on all interfaces
    quint16 port = 8554;                   // RTSP port

    if (!streamer->start(host, port)) {
        qFatal("Failed to start surface streamer");
        return -1;
    }

Pass the ANativeWindow instance to QAndroidSurfaceRenderEngine to start rendering the Quick item on it.

    QObject::connect(
        renderEngine, &QAndroidSurfaceRenderEngine::objectCreated, renderEngine,
        [streamer, renderEngine]() {
          ANativeWindow *surface = streamer->surface();
          if (surface)
            renderEngine->setNativeWindowForItem(surface, "orbitingCameraView");
          else
            qWarning(
                "The surface is invalid. Failed to start surface streamer!");
        });

    renderEngine->load(QLatin1String("qrc:/qt/qml/streaming_module/Main.qml"));

QML Scene

The QML scene is defined in Main.qml and uses an Item as its root object. Only direct children of this root Item can be referenced by their IDs from the Android side. Each surface registered through the AIDL interface is associated with one of these direct child items.

Android Integration

The Android service is implemented in Java and defined in the manifest as a service. It instantiates a QtRaaSApplication that loads the Qt runtime and starts the Qt Quick application. It implements the AIDL-defined IRenderingService interface to bridge the bound client(s) to the rendering engine.

interface IRenderingService {

    void setSurface(in Surface surface, String itemId);
    void unsetSurface(String itemId);
    void motionEvent(in MotionEvent event, String itemId);
}

The activity creates its own ObservableSurfaceView instances and binds to the service. When the surface becomes available, the activity passes it together with a QML item ID to the service. When the surface becomes invalid or is destroyed, it notifies the service to unset the surface for that particular item, so that the QtRaaSApplication stops rendering that item.