TaskTree Class

Detailed Description

Use the Tasking namespace to build extensible, declarative task tree structures that contain possibly asynchronous tasks, such as Process, FileTransfer, or ConcurrentCall<ReturnType>. TaskTree structures enable you to create a sophisticated mixture of a parallel or sequential flow of tasks in the form of a tree and to run it any time later.

Root Element and Tasks

The TaskTree has a mandatory Group root element, which may contain any number of tasks of various types, such as ProcessTask, FileTransferTask, or ConcurrentCallTask<ReturnType>:

using namespace Tasking;

const Group root {
    ProcessTask(...),
    ConcurrentCallTask<int>(...),
    FileTransferTask(...)
};

TaskTree *taskTree = new TaskTree(root);
connect(taskTree, &TaskTree::done, ...);          // a successfully finished handler
connect(taskTree, &TaskTree::errorOccurred, ...); // an erroneously finished handler
taskTree->start();

The task tree above has a top level element of the Group type that contains tasks of the ProcessTask, FileTransferTask, and ConcurrentCallTask<int> type. After taskTree->start() is called, the tasks are run in a chain, starting with ProcessTask. When the ProcessTask finishes successfully, the ConcurrentCallTask<int> task is started. Finally, when the asynchronous task finishes successfully, the FileTransferTask task is started.

When the last running task finishes with success, the task tree is considered to have run successfully and the TaskTree::done() signal is emitted. When a task finishes with an error, the execution of the task tree is stopped and the remaining tasks are skipped. The task tree finishes with an error and sends the TaskTree::errorOccurred() signal.

Groups

The parent of the Group sees it as a single task. Like other tasks, the group can be started and it can finish with success or an error. The Group elements can be nested to create a tree structure:

const Group root {
    Group {
        parallel,
        ProcessTask(...),
        ConcurrentCallTask<int>(...)
    },
    FileTransferTask(...)
};

The example above differs from the first example in that the root element has a subgroup that contains the ProcessTask and ConcurrentCallTask<int>. The subgroup is a sibling element of the FileTransferTask in the root. The subgroup contains an additional parallel element that instructs its Group to execute its tasks in parallel.

So, when the tree above is started, the ProcessTask and ConcurrentCallTask<int> start immediately and run in parallel. Since the root group doesn't contain a parallel element, its direct child tasks are run in sequence. Thus, the FileTransferTask starts when the whole subgroup finishes. The group is considered as finished when all its tasks have finished. The order in which the tasks finish is not relevant.

So, depending on which task lasts longer (ProcessTask or ConcurrentCallTask<int>), the following scenarios can take place:

The differences between the scenarios are marked with bold. Three dots mean that an unspecified amount of time passes between previous and next events (a task or tasks continue to run). No dots between events means that they occur synchronously.

The presented scenarios assume that all tasks run successfully. If a task fails during execution, the task tree finishes with an error. In particular, when ProcessTask finishes with an error while ConcurrentCallTask<int> is still being executed, the ConcurrentCallTask<int> is automatically stopped, the subgroup finishes with an error, the FileTransferTask is skipped, and the tree finishes with an error.

Task Types

Each task type is associated with its corresponding task class that executes the task. For example, a ProcessTask inside a task tree is associated with the Process class that executes the process. The associated objects are automatically created, started, and destructed exclusively by the task tree at the appropriate time.

If a root group consists of five sequential ProcessTask tasks, and the task tree executes the group, it creates an instance of Process for the first ProcessTask and starts it. If the Process instance finishes successfully, the task tree destructs it and creates a new Process instance for the second ProcessTask, and so on. If the first task finishes with an error, the task tree stops creating Process instances, and the root group finishes with an error.

The following table shows examples of task types and their corresponding task classes:

Task Handlers

Use Task handlers to set up a task for execution and to enable reading the output data from the task when it finishes with success or an error.

Task's Start Handler

When a corresponding task class object is created and before it's started, the task tree invokes an optionally user-provided setup handler. The setup handler should always take a reference to the associated task class object:

const auto onSetup = [](Process &process) {
    process.setCommand({"sleep", {"3"}});
};
const Group root {
    ProcessTask(onSetup)
};

You can modify the passed Process in the setup handler, so that the task tree can start the process according to your configuration. You should not call process.start(); in the setup handler, as the task tree calls it when needed. The setup handler is optional. When used, it must be the first argument of the task's constructor.

Optionally, the setup handler may return a SetupResult. The returned SetupResult influences the further start behavior of a given task. The possible values are:

This is useful for running a task only when a condition is met and the data needed to evaluate this condition is not known until previously started tasks finish. In this way, the setup handler dynamically decides whether to start the corresponding task normally or skip it and report success or an error. For more information about inter-task data exchange, see Storage.

Task's Done and Error Handlers

When a running task finishes, the task tree invokes an optionally provided done or error handler. Both handlers should always take a const reference to the associated task class object:

const auto onSetup = [](Process &process) {
    process.setCommand({"sleep", {"3"}});
};
const auto onDone = [](const Process &process) {
    qDebug() << "Success" << process.cleanedStdOut();
};
const auto onError = [](const Process &process) {
    qDebug() << "Failure" << process.cleanedStdErr();
};
const Group root {
    ProcessTask(onSetup, onDone, onError)
};

The done and error handlers may collect output data from Process, and store it for further processing or perform additional actions. The done handler is optional. When used, it must be the second argument of the task's constructor. The error handler is also optional. When used, it must always be the third argument. You can omit the handlers or substitute the ones that you do not need with curly braces ({}).

Group Handlers

Similarly to task handlers, group handlers enable you to set up a group to execute and to apply more actions when the whole group finishes with success or an error.

Group's Start Handler

The task tree invokes the group start handler before it starts the child tasks. The group handler doesn't take any arguments:

const auto onSetup = [] {
    qDebug() << "Entering the group";
};
const Group root {
    onGroupSetup(onSetup),
    ProcessTask(...)
};

The group setup handler is optional. To define a group setup handler, add an onGroupSetup() element to a group. The argument of onGroupSetup() is a user handler. If you add more than one onGroupSetup() element to a group, an assert is triggered at runtime that includes an error message.

Like the task's start handler, the group start handler may return SetupResult. The returned SetupResult value affects the start behavior of the whole group. If you do not specify a group start handler or its return type is void, the default group's action is SetupResult::Continue, so that all tasks are started normally. Otherwise, when the start handler returns SetupResult::StopWithDone or SetupResult::StopWithError, the tasks are not started (they are skipped) and the group itself reports success or failure, depending on the returned value, respectively.

const Group root {
    onGroupSetup([] { qDebug() << "Root setup"; }),
    Group {
        onGroupSetup([] { qDebug() << "Group 1 setup"; return SetupResult::Continue; }),
        ProcessTask(...) // Process 1
    },
    Group {
        onGroupSetup([] { qDebug() << "Group 2 setup"; return SetupResult::StopWithDone; }),
        ProcessTask(...) // Process 2
    },
    Group {
        onGroupSetup([] { qDebug() << "Group 3 setup"; return SetupResult::StopWithError; }),
        ProcessTask(...) // Process 3
    },
    ProcessTask(...) // Process 4
};

In the above example, all subgroups of a root group define their setup handlers. The following scenario assumes that all started processes finish with success:

Groups's Done and Error Handlers

A Group's done or error handler is executed after the successful or failed execution of its tasks, respectively. The final value reported by the group depends on its Workflow Policy. The handlers can apply other necessary actions. The done and error handlers are defined inside the onGroupDone() and onGroupError() elements of a group, respectively. They do not take arguments:

const Group root {
    onGroupSetup([] { qDebug() << "Root setup"; }),
    ProcessTask(...),
    onGroupDone([] { qDebug() << "Root finished with success"; }),
    onGroupError([] { qDebug() << "Root finished with error"; })
};

The group done and error handlers are optional. If you add more than one onGroupDone() or onGroupError() each to a group, an assert is triggered at runtime that includes an error message.

Other Group Elements

A group can contain other elements that describe the processing flow, such as the execution mode or workflow policy. It can also contain storage elements that are responsible for collecting and sharing custom common data gathered during group execution.

Execution Mode

The execution mode element in a Group specifies how the direct child tasks of the Group are started. The most common execution modes are sequential and parallel. It's also possible to specify the limit of tasks running in parallel by using the parallelLimit() function.

In all execution modes, a group starts tasks in the oder in which they appear.

If a child of a group is also a group, the child group runs its tasks according to its own execution mode.

Workflow Policy

The workflow policy element in a Group specifies how the group should behave when any of its direct child's tasks finish. For a detailed description of possible policies, refer to WorkflowPolicy.

If a child of a group is also a group, the child group runs its tasks according to its own workflow policy.

Storage

Use the Storage element to exchange information between tasks. Especially, in the sequential execution mode, when a task needs data from another, already finished task, before it can start. For example, a task tree that copies data by reading it from a source and writing it to a destination might look as follows:

static QByteArray load(const QString &fileName) { ... }
static void save(const QString &fileName, const QByteArray &array) { ... }

static Group copyRecipe(const QString &source, const QString &destination)
{
    struct CopyStorage { // [1] custom inter-task struct
        QByteArray content; // [2] custom inter-task data
    };

    // [3] instance of custom inter-task struct manageable by task tree
    const TreeStorage<CopyStorage> storage;

    const auto onLoaderSetup = [source](ConcurrentCall<QByteArray> &async) {
        async.setConcurrentCallData(&load, source);
    };
    // [4] runtime: task tree activates the instance from [7] before invoking handler
    const auto onLoaderDone = [storage](const ConcurrentCall<QByteArray> &async) {
        storage->content = async.result(); // [5] loader stores the result in storage
    };

    // [4] runtime: task tree activates the instance from [7] before invoking handler
    const auto onSaverSetup = [storage, destination](ConcurrentCall<void> &async) {
        const QByteArray content = storage->content; // [6] saver takes data from storage
        async.setConcurrentCallData(&save, destination, content);
    };
    const auto onSaverDone = [](const ConcurrentCall<void> &async) {
        qDebug() << "Save done successfully";
    };

    const Group root {
        // [7] runtime: task tree creates an instance of CopyStorage when root is entered
        Storage(storage),
        ConcurrentCallTask<QByteArray>(onLoaderSetup, onLoaderDone),
        ConcurrentCallTask<void>(onSaverSetup, onSaverDone)
    };
    return root;
}

const QString source = ...;
const QString destination = ...;
TaskTree taskTree(copyRecipe(source, destination));
connect(&taskTree, &TaskTree::done,
        &taskTree, [] { qDebug() << "The copying finished successfully."; });
tasktree.start();

In the example above, the inter-task data consists of a QByteArray content variable [2] enclosed in a CopyStorage custom struct [1]. If the loader finishes successfully, it stores the data in a CopyStorage::content variable [5]. The saver then uses the variable to configure the saving task [6].

To enable a task tree to manage the CopyStorage struct, an instance of TreeStorage<CopyStorage> is created [3]. If a copy of this object is inserted as group's child task [7], an instance of CopyStorage struct is created dynamically when the task tree enters this group. When the task tree leaves this group, the existing instance of CopyStorage struct is destructed as it's no longer needed.

If several task trees that hold a copy of the common TreeStorage<CopyStorage> instance run simultaneously, each task tree contains its own copy of the CopyStorage struct.

You can access CopyStorage from any handler in the group with a storage object. This includes all handlers of all descendant tasks of the group with a storage object. To access the custom struct in a handler, pass the copy of the TreeStorage<CopyStorage> object to the handler (for example, in a lambda capture) [4].

When the task tree invokes a handler in a subtree containing the storage [7], the task tree activates its own CopyStorage instance inside the TreeStorage<CopyStorage> object. Therefore, the CopyStorage struct may be accessed only from within the handler body. To access the currently active CopyStorage from within TreeStorage<CopyStorage>, use the TreeStorage::operator->(), TreeStorage::operator*() or TreeStorage::activeStorage() method.

The following list summarizes how to employ a Storage object into the task tree:

1. Define the custom structure MyStorage with custom data [1], [2]
2. Create an instance of TreeStorage<MyStorage> storage [3]
3. Pass the TreeStorage<MyStorage> instance to handlers [4]
4. Access the MyStorage instance in handlers [5], [6]
5. Insert the TreeStorage<MyStorage> instance into a group [7]

TaskTree

TaskTree executes the tree structure of asynchronous tasks according to the recipe described by the Group root element.

As TaskTree is also an asynchronous task, it can be a part of another TaskTree. To place a nested TaskTree inside another TaskTree, insert the TaskTreeTask element into other tree's Group element.

TaskTree reports progress of completed tasks when running. The progress value is increased when a task finishes or is skipped or stopped. When TaskTree is finished and the TaskTree::done() or TaskTree::errorOccurred() signal is emitted, the current value of the progress equals the maximum progress value. Maximum progress equals the total number of tasks in a tree. A nested TaskTree is counted as a single task, and its child tasks are not counted in the top level tree. Groups themselves are not counted as tasks, but their tasks are counted.

To set additional initial data for the running tree, modify the storage instances in a tree when it creates them by installing a storage setup handler:

TreeStorage<CopyStorage> storage;
const Group root = ...; // storage placed inside root's group and inside handlers
TaskTree taskTree(root);
auto initStorage = [](CopyStorage &storage){
    storage.content = "initial content";
};
taskTree.onStorageSetup(storage, initStorage);
taskTree.start();

When the running task tree creates a CopyStorage instance, and before any handler inside a tree is called, the task tree calls the initStorage handler, to enable setting up initial data of the storage, unique to this particular run of taskTree.

Similarly, to collect some additional result data from the running tree, read it from storage instances in the tree when they are about to be destroyed. To do this, install a storage done handler:

TreeStorage<CopyStorage> storage;
const Group root = ...; // storage placed inside root's group and inside handlers
TaskTree taskTree(root);
auto collectStorage = [](const CopyStorage &storage){
    qDebug() << "final content" << storage.content;
};
taskTree.onStorageDone(storage, collectStorage);
taskTree.start();

When the running task tree is about to destroy a CopyStorage instance, the task tree calls the collectStorage handler, to enable reading the final data from the storage, unique to this particular run of taskTree.

Task Adapters

To extend a TaskTree with a new task type, implement a simple adapter class derived from the TaskAdapter class template. The following class is an adapter for a single shot timer, which may be considered as a new asynchronous task:

class TimerTaskAdapter : public TaskAdapter<QTimer>
{
public:
    TimerTaskAdapter() {
        task()->setSingleShot(true);
        task()->setInterval(1000);
        connect(task(), &QTimer::timeout, this, [this] { emit done(true); });
    }
private:
    void start() final { task()->start(); }
};

using TimerTask = CustomTask<TimerTaskAdapter>;

You must derive the custom adapter from the TaskAdapter class template instantiated with a template parameter of the class implementing a running task. The code above uses QTimer to run the task. This class appears later as an argument to the task's handlers. The instance of this class parameter automatically becomes a member of the TaskAdapter template, and is accessible through the TaskAdapter::task() method. The constructor of TimerTaskAdapter initially configures the QTimer object and connects to the QTimer::timeout signal. When the signal is triggered, TimerTaskAdapter emits the done(true) signal to inform the task tree that the task finished successfully. If it emits done(false), the task finished with an error. The TaskAdapter::start() method starts the timer.

To make QTimer accessible inside TaskTree under the TimerTask name, define TimerTask to be an alias to the Tasking::CustomTask<TimerTaskAdapter>. TimerTask becomes a new task type, using TimerTaskAdapter.

The new task type is now registered, and you can use it in TaskTree:

const auto onTimerSetup = [](QTimer &task) {
    task.setInterval(2000);
};
const auto onTimerDone = [](const QTimer &task) {
    qDebug() << "timer triggered";
};
const Group root {
    TimerTask(onTimerSetup, onTimerDone)
};

When a task tree containing the root from the above example is started, it prints a debug message within two seconds and then finishes successfully.