QML Video Shader Effects Example

Applying shader effects on video and camera viewfinder content

Applying shader effects on video and camera viewfinder content.

Running the Example

To run the example from Qt Creator , open the Welcome mode and select the example from Examples . For more information, visit Building and Running an Example.

Overview

QML Video Shader Effects demonstrates how a ShaderEffect can be used to apply postprocessing effects, expressed in GLSL, to QML VideoOutput type.

It also shows how native code can be combined with QML to implement more advanced functionality - in this case, C++ code is used to calculate the QML frame rate. This value is rendered in QML in a semi-transparent item overlaid on the video content.

The following screenshots show shader effects being applied. In each case, the effect is implemented using a fragment shader.

Here we see an edge detection algorithm being applied to a video clip (Sintel from blender.org).

This image shows a page curl effect, applied to the same video clip.

Here we see a ‘glow’ effect (edge detection plus colour quantization) being applied to the camera viewfinder.

This image shows a ‘wobble’ effect applied to the viewfinder.

The application includes many more effects than the ones shown here - look for Effect*.qml files in the list of files below to see the full range.

Application structure

Shader effects can be applied to video or viewfinder content using ShaderEffect , as shown in the following example, which applies a wiggly effect to the content:

import QtQuick 2.0
import QtMultimedia 5.0

Rectangle {
    width: 300
    height: 300
    color: "black"

    MediaPlayer {
        id: mediaPlayer
        source: "test.mp4"
        playing: true
    }

    VideoOutput {
        id: video
        anchors.fill: parent
        source: mediaPlayer
    }

    ShaderEffect {
        property variant source: ShaderEffectSource { sourceItem: video; hideSource: true }
        property real wiggleAmount: 0.005
        anchors.fill: video

        fragmentShader: "
            varying highp vec2 qt_TexCoord0;
            uniform sampler2D source;
            uniform highp float wiggleAmount;
            void main(void)
            {
                highp vec2 wiggledTexCoord = qt_TexCoord0;
                wiggledTexCoord.s += sin(4.0 * 3.141592653589 * wiggledTexCoord.t) * wiggleAmount;
                gl_FragColor = texture2D(source, wiggledTexCoord.st);
            }
        "
    }
}

In this application, the usage of the ShaderEffect and VideoOutput types is a bit more complicated, for the following reasons:

  • Each effect can be applied to either a VideoOutput or an Image item, so the type of the source item must be abstracted away from the effect implementation

  • For some effects (such as the edge detection and glow examples shown in the screenshots above), the transformation is applied only to pixels to the left of a dividing line - this allows the effect to be easily compared with the untransformed image on the right

  • Most effects have one or more parameters which can be modified by the user - these are controlled by sliders in the UI which are connected to uniform values passed into the GLSL code

The abstraction of source item type is achieved by the Content , which uses a Loader to create either a MediaPlayer , Camera or Image :

...
...

Each effect is implemented as a QML item which is based on the Effect , which in turn is based on the ShaderEffect :

The interface of Effect allows for derived effects to specify the number of parameters which they support (and therefore the number of sliders which should be displayed), and whether a vertical dividing line should be drawn between transformed and untransformed image regions. As an example, here is the implementation of the pixelation effect. As you can see, the pixelation effect supports one parameter (which controls the pixelation granularity), and states that the divider should be displayed.

The main.qml file shows a FileOpen , which allows the user to select the input source and an EffectSelectionPanel item, which lists each of the available shader effects. As described above, a Content item is used to load the appropriate input and effect type. A Divider item draws the vertical dividing line, which can be dragged left / right by the user. Finally, a ParameterPanel item renders the sliders corresponding to each effect parameter.

Here is the effect selection menu:

Calculating and displaying QML painting rate

The QML painting rate is calculated by the FrequencyMonitor class, which turns a stream of events (received via the notify() slot), into an instantaneous and an averaged frequency:

...

The FrequencyMonitor class is exposed to QML like this

and its data is displayed by defining a QML item called FrequencyItem, like this:

...

The result looks like this:

All that remains is to connect the afterRendering() signal of the QQuickView object to a JavaScript function, which will eventually call frequencyItem.notify():

...
...

filereader.cpp Example File

filereader.h Example File

qmlvideofx.svg Example File

trace.h Example File

Button.qml Example File

Content.qml Example File

ContentCamera.qml Example File

ContentImage.qml Example File

ContentVideo.qml Example File

Curtain.qml Example File

Divider.qml Example File

Effect.qml Example File

EffectBillboard.qml Example File

EffectBlackAndWhite.qml Example File

EffectEmboss.qml Example File

EffectGaussianBlur.qml Example File

EffectGlow.qml Example File

EffectIsolate.qml Example File

EffectMagnify.qml Example File

EffectPageCurl.qml Example File

EffectPassThrough.qml Example File

EffectPixelate.qml Example File

EffectPosterize.qml Example File

EffectRipple.qml Example File

EffectSelectionList.qml Example File

EffectSepia.qml Example File

EffectSharpen.qml Example File

EffectShockwave.qml Example File

EffectSobelEdgeDetection1.qml Example File

EffectTiltShift.qml Example File

EffectToon.qml Example File

EffectVignette.qml Example File

EffectWarhol.qml Example File

EffectWobble.qml Example File

FileBrowser.qml Example File

FileOpen.qml Example File

HintedMouseArea.qml Example File

Main.qml Example File

ParameterPanel.qml Example File

Slider.qml Example File

qmlapplicationviewer.cpp Example File

qmlapplicationviewer.h Example File

main.cpp Example File

qmlvideofx.pro Example File

qmlvideofx.qrc Example File

qmlvideofx.png Image File

Slider_bar.png Image File

Slider_handle.png Image File

Triangle_Top.png Image File

Triangle_bottom.png Image File

icon_BackArrow.png Image File

icon_Folder.png Image File

icon_Menu.png Image File

qt-logo.png Image File