Sample Bindings Example

This example showcases how to generate Python bindings for a non-Qt C++ library.

The example defines a CMake project that builds two libraries:

  • libuniverse - a sample library with two C++ classes.

  • Universe - the generated Python extension module that contains bindings to the library above.

The project file is structured in such a way that a user can copy-paste in into their own project, and be able to build it with a minimal amount of modifications.

Description

The libuniverse library declares two classes: Icecream and Truck.

Icecream objects have a flavor, and an accessor for returning the flavor.

Truck instances store a vector of Icecream objects, and have various methods for adding new flavors, printing available flavors, delivering icecream, etc.

From a C++ perspective, Icecream instances are treated as object types (pointer semantics) because the class declares virtual methods.

In contrast Truck does not define virtual methods and is treated as a value type (copy semantics).

Because Truck is a value type and it stores a vector of Icecream pointers, the rule of five has to be taken into account (implement the copy constructor, assignment operator, move constructor, move assignment operator and destructor).

And due to Icecream objects being copyable, the type has to define an implementation of the clone() method, to avoid type slicing issues.

Both of these types and their methods will be exposed to Python by generating CPython code. The code is generated by shiboken and placed in separate .cpp files named after each C++ type. The code is then compiled and linked into a shared library. The shared library is a CPython extension module, which is loaded by the Python interpreter.

Beacuse the C++ language has different semantics to Python, shiboken needs help in figuring out how to generate the bindings code. This is done by specifying a special XML file called a typesystem file.

In the typesystem file you specify things like:

  • which C++ primitive types should have bindings (int, bool, float)

  • which C++ classes should have bindings (Icecream) and what kind of semantics (value / object)

  • Ownership rules (who deletes the C++ objects, C++ or Python)

  • Code injection (for various special cases that shiboken doesn’t know about)

  • Package name (name of package as imported from Python)

In this example we declare bool and std::string as primitive types, Icecream as an object type, Truck as a value type, and the clone() and addIcecreamFlavor(Icecream*) need additional info about who owns the parameter objects when passing them across language boundaries (in this case C++ will delete the objects).

The Truck has getters and setters for the string arrivalMessage. In the type system file, we declare this to be a property in Python:

<property type="std::string" name="arrivalMessage" get="getArrivalMessage" set="setArrivalMessage"/>

It can then be used in a more pythonic way:

special_truck.arrivalMessage = "A new SPECIAL icecream truck has arrived!\n"

After shiboken generates the C++ code and CMake makes an extension module from the code, the types can be accessed in Python simply by importing them using the original C++ names.

from Universe import Icecream, Truck

Constructing C++ wrapped objects is the same as in Python

icecream = Icecream("vanilla")
truck = Truck()

And actual C++ constructors are mapped to the Python __init__ method.

class VanillaChocolateIcecream(Icecream):
    def __init__(self, flavor=""):
        super().__init__(flavor)

C++ methods can be accessed as regular Python methods using the C++ names

truck.addIcecreamFlavor(icecream)

Inheritance works as with regular Python classes, and virtual C++ methods can be overridden simply by definining a method with the same name as in the C++ class.

class VanillaChocolateIcecream(Icecream):
    # ...
    def getFlavor(self):
        return "vanilla sprinked with chocolate"

The main.py script demonstrates usages of these types.

The CMake project file contains many comments explaining all the build rules for those interested in the build process.

Building the project

This example can only be built using CMake. The following requirements need to be met:

  • A PySide package is installed into the current active Python environment (system or virtualenv)

  • A new enough version of CMake (3.16+).

  • ninja

For Windows you will also need:

  • a Visual Studio environment to be active in your terminal

  • Correct visual studio architecture chosen (32 vs 64 bit)

  • Make sure that your Python intepreter and bindings project build configuration is the same (all Release, which is more likely, or all Debug).

The build uses the pyside_config.py file to configure the project using the current PySide/Shiboken installation.

Using CMake

You can build and run this example by executing the following commands (slightly adapted to your file system layout) in a terminal:

macOS/Linux:

cd ~/pyside-setup/examples/samplebinding

On Windows:

cd C:\pyside-setup\examples\samplebinding

mkdir build
cd build
cmake -H.. -B. -G Ninja -DCMAKE_BUILD_TYPE=Release
ninja
ninja install
cd ..

The final example can then be run by:

python main.py

Windows troubleshooting

It is possible that CMake can pick up the wrong compiler for a different architecture, but it can be addressed explicitly by setting the CC environment variable:

set CC=cl

passing the compiler on the command line:

cmake -H.. -B. -DCMAKE_C_COMPILER=cl.exe -DCMAKE_CXX_COMPILER=cl.exe

or by using the -G option:

cmake -H.. -B. -G "Visual Studio 14 Win64"

If the -G "Visual Studio 14 Win64" option is used, a sln file will be generated, and can be used with MSBuild instead of ninja. The easiest way to both build and install in this case, is to use the cmake executable:

cmake --build . --target install --config Release

Note that using the "Ninja" generator is preferred to the MSBuild one, because the MSBuild one generates configs for both Debug and Release, and this might lead to building errors if you accidentally build the wrong config at least once.

Virtualenv Support

If the python application is started from a terminal with an activated python virtual environment, that environment’s packages will be used for the python module import process. In this case, make sure that the bindings were built while the virtualenv was active, so that the build system picks up the correct python shared library and PySide6 / shiboken package.

Linux Shared Libraries Notes

For this example’s purpose, we link against the absolute path of the dependent shared library libshiboken because the installation of the library is done via a wheel, and there is no clean solution to include symbolic links in a wheel package (so that passing -lshiboken to the linker would work).

Windows Notes

The build config of the bindings (Debug or Release) should match the PySide build config, otherwise the application will not properly work.

In practice this means the only supported configurations are:

  1. release config build of the bindings + PySide setup.py without --debug flag + python.exe for the PySide build process + python39.dll for the linked in shared library.

  2. debug config build of the application + PySide setup.py with --debug flag + python_d.exe for the PySide build process + python39_d.dll for the linked in shared library.

This is necessary because all the shared libraries in question have to link to the same C++ runtime library (msvcrt.dll or msvcrtd.dll). To make the example as self-contained as possible, the shared libraries in use (pyside6.dll, shiboken6.dll) are hard-linked into the build folder of the application.

#ifndef BINDINGS_H
#define BINDINGS_H

#include "icecream.h"
#include "truck.h"

#endif // BINDINGS_H

#include "icecream.h"

#include <iostream>

Icecream::Icecream(const std::string &flavor) : m_flavor(flavor) {}

Icecream::~Icecream() = default;

std::string Icecream::getFlavor() const
{
    return m_flavor;
}

Icecream *Icecream::clone()
{
    return new Icecream(*this);
}

std::ostream &operator<<(std::ostream &str, const Icecream &i)
{
    str << i.getFlavor();
    return str;
}

#ifndef ICECREAM_H
#define ICECREAM_H

#include "macros.h"

#include <iosfwd>
#include <string>

class BINDINGS_API Icecream
{
public:
    explicit Icecream(const std::string &flavor);
    virtual Icecream *clone();
    virtual ~Icecream();
    virtual std::string getFlavor() const;

private:
    std::string m_flavor;
};

std::ostream &operator<<(std::ostream &str, const Icecream &i);

#endif // ICECREAM_H

#ifndef MACROS_H
#define MACROS_H

#if defined _WIN32 || defined __CYGWIN__
    // Export symbols when creating .dll and .lib, and import them when using .lib.
    #if BINDINGS_BUILD
        #define BINDINGS_API __declspec(dllexport)
    #else
        #define BINDINGS_API __declspec(dllimport)
    #endif
    // Disable warnings about exporting STL types being a bad idea. Don't use this in production
    // code.
    #pragma warning( disable : 4251 )
#else
    #define BINDINGS_API
#endif

#endif // MACROS_H

"""An example showcasing how to use bindings for a custom non-Qt C++ library"""

from Universe import Icecream, Truck


class VanillaChocolateIcecream(Icecream):
    def __init__(self, flavor=""):
        super().__init__(flavor)

    def clone(self):
        return VanillaChocolateIcecream(self.getFlavor())

    def getFlavor(self):
        return "vanilla sprinked with chocolate"


class VanillaChocolateCherryIcecream(VanillaChocolateIcecream):
    def __init__(self, flavor=""):
        super().__init__(flavor)

    def clone(self):
        return VanillaChocolateCherryIcecream(self.getFlavor())

    def getFlavor(self):
        base_flavor = super(VanillaChocolateCherryIcecream, self).getFlavor()
        return f"{base_flavor} and a cherry"


if __name__ == '__main__':
    leave_on_destruction = True
    truck = Truck(leave_on_destruction)

    flavors = ["vanilla", "chocolate", "strawberry"]
    for f in flavors:
        icecream = Icecream(f)
        truck.addIcecreamFlavor(icecream)

    truck.addIcecreamFlavor(VanillaChocolateIcecream())
    truck.addIcecreamFlavor(VanillaChocolateCherryIcecream())

    truck.arrive()
    truck.printAvailableFlavors()
    result = truck.deliver()

    if result:
        print("All the kids got some icecream!")
    else:
        print("Aww, someone didn't get the flavor they wanted...")

    if not result:
        special_truck = Truck(truck)
        del truck

        print("")
        special_truck.arrivalMessage = "A new SPECIAL icecream truck has arrived!\n"
        special_truck.arrive()
        special_truck.addIcecreamFlavor(Icecream("SPECIAL *magical* icecream"))
        special_truck.printAvailableFlavors()
        special_truck.deliver()
        print("Now everyone got the flavor they wanted!")
        special_truck.leave()

#include <iostream>
#include <random>

#include "truck.h"

Truck::Truck(bool leaveOnDestruction) : m_leaveOnDestruction(leaveOnDestruction) {}

Truck::Truck(const Truck &other)
{
    assign(other);
}

Truck &Truck::operator=(const Truck &other)
{
    if (this != &other) {
        m_flavors.clear();
        assign(other);
    }
    return *this;
}

Truck::Truck(Truck &&other) = default;

Truck& Truck::operator=(Truck &&other) = default;

Truck::~Truck()
{
    if (m_leaveOnDestruction)
        leave();
}

void Truck::addIcecreamFlavor(Icecream *icecream)
{
    m_flavors.push_back(IcecreamPtr(icecream));
}

void Truck::printAvailableFlavors() const
{
    std::cout << "It sells the following flavors: \n";
    for (const auto &flavor : m_flavors)
        std::cout << "  * "  << *flavor << '\n';
    std::cout << '\n';
}

void Truck::arrive() const
{
    std::cout << m_arrivalMessage;
}

void Truck::leave() const
{
    std::cout << "The truck left the neighborhood.\n";
}

void Truck::setLeaveOnDestruction(bool value)
{
    m_leaveOnDestruction = value;
}

void Truck::setArrivalMessage(const std::string &message)
{
    m_arrivalMessage = message;
}

std::string Truck::getArrivalMessage() const
{
    return m_arrivalMessage;
}

void Truck::assign(const Truck &other)
{
    m_flavors.reserve(other.m_flavors.size());
    for (const auto &f : other.m_flavors)
        m_flavors.push_back(IcecreamPtr(f->clone()));
}

bool Truck::deliver() const
{
    std::random_device rd;
    std::mt19937 mt(rd());
    std::uniform_int_distribution<int> dist(1, 2);

    std::cout << "The truck started delivering icecream to all the kids in the neighborhood.\n";
    bool result = false;

    if (dist(mt) == 2)
        result = true;

    return result;
}

#ifndef TRUCK_H
#define TRUCK_H

#include "icecream.h"
#include "macros.h"

#include <memory>
#include <vector>

class BINDINGS_API Truck
{
public:
    explicit Truck(bool leaveOnDestruction = false);
    Truck(const Truck &other);
    Truck& operator=(const Truck &other);
    Truck(Truck &&other);
    Truck& operator=(Truck &&other);

    ~Truck();

    void addIcecreamFlavor(Icecream *icecream);
    void printAvailableFlavors() const;

    bool deliver() const;
    void arrive() const;
    void leave() const;

    void setLeaveOnDestruction(bool value);

    void setArrivalMessage(const std::string &message);
    std::string getArrivalMessage() const;

private:
    using IcecreamPtr = std::shared_ptr<Icecream>;

    void assign(const Truck &other);

    bool m_leaveOnDestruction = false;
    std::string m_arrivalMessage = "A new icecream truck has arrived!\n";
    std::vector<IcecreamPtr> m_flavors;
};

#endif // TRUCK_H

cmake_minimum_required(VERSION 3.16)
cmake_policy(VERSION 3.16)

# Enable policy to not use RPATH settings for install_name on macOS.
if(POLICY CMP0068)
  cmake_policy(SET CMP0068 NEW)
endif()

# Consider changing the project name to something relevant for you.
project(SampleBinding)

# ================================ General configuration ======================================

# Set CPP standard to C++11 minimum.
set(CMAKE_CXX_STANDARD 11)

# The sample library for which we will create bindings. You can change the name to something
# relevant for your project.
set(sample_library "libuniverse")

# The name of the generated bindings module (as imported in Python). You can change the name
# to something relevant for your project.
set(bindings_library "Universe")

# The header file with all the types and functions for which bindings will be generated.
# Usually it simply includes other headers of the library you are creating bindings for.
set(wrapped_header ${CMAKE_SOURCE_DIR}/bindings.h)

# The typesystem xml file which defines the relationships between the C++ types / functions
# and the corresponding Python equivalents.
set(typesystem_file ${CMAKE_SOURCE_DIR}/bindings.xml)

# Specify which C++ files will be generated by shiboken. This includes the module wrapper
# and a '.cpp' file per C++ type. These are needed for generating the module shared
# library.
set(generated_sources
    ${CMAKE_CURRENT_BINARY_DIR}/${bindings_library}/universe_module_wrapper.cpp
    ${CMAKE_CURRENT_BINARY_DIR}/${bindings_library}/icecream_wrapper.cpp
    ${CMAKE_CURRENT_BINARY_DIR}/${bindings_library}/truck_wrapper.cpp)


# ================================== Shiboken detection ======================================
# Use provided python interpreter if given.
if(NOT python_interpreter)
    find_program(python_interpreter "python")
endif()
message(STATUS "Using python interpreter: ${python_interpreter}")

# Macro to get various pyside / python include / link flags and paths.
# Uses the not entirely supported utils/pyside_config.py file.
macro(pyside_config option output_var)
    if(${ARGC} GREATER 2)
        set(is_list ${ARGV2})
    else()
        set(is_list "")
    endif()

    execute_process(
      COMMAND ${python_interpreter} "${CMAKE_SOURCE_DIR}/../utils/pyside_config.py"
              ${option}
      OUTPUT_VARIABLE ${output_var}
      OUTPUT_STRIP_TRAILING_WHITESPACE)

    if ("${${output_var}}" STREQUAL "")
        message(FATAL_ERROR "Error: Calling pyside_config.py ${option} returned no output.")
    endif()
    if(is_list)
        string (REPLACE " " ";" ${output_var} "${${output_var}}")
    endif()
endmacro()

# Query for the shiboken generator path, Python path, include paths and linker flags.
pyside_config(--shiboken-module-path shiboken_module_path)
pyside_config(--shiboken-generator-path shiboken_generator_path)
pyside_config(--python-include-path python_include_dir)
pyside_config(--shiboken-generator-include-path shiboken_include_dir 1)
pyside_config(--shiboken-module-shared-libraries-cmake shiboken_shared_libraries 0)
pyside_config(--python-link-flags-cmake python_linking_data 0)

set(shiboken_path "${shiboken_generator_path}/shiboken6${CMAKE_EXECUTABLE_SUFFIX}")
if(NOT EXISTS ${shiboken_path})
    message(FATAL_ERROR "Shiboken executable not found at path: ${shiboken_path}")
endif()


# ==================================== RPATH configuration ====================================


# =============================================================================================
# !!! (The section below is deployment related, so in a real world application you will want to
# take care of this properly with some custom script or tool).
# =============================================================================================
# Enable rpaths so that the built shared libraries find their dependencies.
set(CMAKE_SKIP_BUILD_RPATH FALSE)
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)
set(CMAKE_INSTALL_RPATH ${shiboken_module_path} ${CMAKE_CURRENT_SOURCE_DIR})
set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
# =============================================================================================
# !!! End of dubious section.
# =============================================================================================


# =============================== CMake target - sample_library ===============================


# Define the sample shared library for which we will create bindings.
set(${sample_library}_sources icecream.cpp truck.cpp)
add_library(${sample_library} SHARED ${${sample_library}_sources})
set_property(TARGET ${sample_library} PROPERTY PREFIX "")

# Needed mostly on Windows to export symbols, and create a .lib file, otherwise the binding
# library can't link to the sample library.
target_compile_definitions(${sample_library} PRIVATE BINDINGS_BUILD)


# ====================== Shiboken target for generating binding C++ files  ====================


# Set up the options to pass to shiboken.
set(shiboken_options --generator-set=shiboken --enable-parent-ctor-heuristic
    --enable-return-value-heuristic --use-isnull-as-nb_nonzero
    --avoid-protected-hack
    -I${CMAKE_SOURCE_DIR}
    -T${CMAKE_SOURCE_DIR}
    --output-directory=${CMAKE_CURRENT_BINARY_DIR}
    )

set(generated_sources_dependencies ${wrapped_header} ${typesystem_file})

# Add custom target to run shiboken to generate the binding cpp files.
add_custom_command(OUTPUT ${generated_sources}
                    COMMAND ${shiboken_path}
                    ${shiboken_options} ${wrapped_header} ${typesystem_file}
                    DEPENDS ${generated_sources_dependencies}
                    IMPLICIT_DEPENDS CXX ${wrapped_header}
                    WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
                    COMMENT "Running generator for ${typesystem_file}.")


# =============================== CMake target - bindings_library =============================


# Set the cpp files which will be used for the bindings library.
set(${bindings_library}_sources ${generated_sources})

# Define and build the bindings library.
add_library(${bindings_library} MODULE ${${bindings_library}_sources})

# Apply relevant include and link flags.
target_include_directories(${bindings_library} PRIVATE ${python_include_dir})
target_include_directories(${bindings_library} PRIVATE ${shiboken_include_dir})
target_include_directories(${bindings_library} PRIVATE ${CMAKE_SOURCE_DIR})

target_link_libraries(${bindings_library} PRIVATE ${shiboken_shared_libraries})
target_link_libraries(${bindings_library} PRIVATE ${sample_library})

# Adjust the name of generated module.
set_property(TARGET ${bindings_library} PROPERTY PREFIX "")
set_property(TARGET ${bindings_library} PROPERTY OUTPUT_NAME
             "${bindings_library}${PYTHON_EXTENSION_SUFFIX}")
if(WIN32)
    set_property(TARGET ${bindings_library} PROPERTY SUFFIX ".pyd")
endif()

# Make sure the linker doesn't complain about not finding Python symbols on macOS.
if(APPLE)
  set_target_properties(${bindings_library} PROPERTIES LINK_FLAGS "-undefined dynamic_lookup")
endif(APPLE)

# Find and link to the python import library only on Windows.
# On Linux and macOS, the undefined symbols will get resolved by the dynamic linker
# (the symbols will be picked up in the Python executable).
if (WIN32)
    list(GET python_linking_data 0 python_libdir)
    list(GET python_linking_data 1 python_lib)
    find_library(python_link_flags ${python_lib} PATHS ${python_libdir} HINTS ${python_libdir})
    target_link_libraries(${bindings_library} PRIVATE ${python_link_flags})
endif()


# ================================= Dubious deployment section ================================

set(windows_shiboken_shared_libraries)

if(WIN32)
    # =========================================================================================
    # !!! (The section below is deployment related, so in a real world application you will
    # want to take care of this properly (this is simply to eliminate errors that users usually
    # encounter.
    # =========================================================================================
    # Circumvent some "#pragma comment(lib)"s in "include/pyconfig.h" which might force to link
    # against a wrong python shared library.

    set(python_versions_list 3 36 37 38 39)
    set(python_additional_link_flags "")
    foreach(ver ${python_versions_list})
        set(python_additional_link_flags
            "${python_additional_link_flags} /NODEFAULTLIB:\"python${ver}_d.lib\"")
        set(python_additional_link_flags
            "${python_additional_link_flags} /NODEFAULTLIB:\"python${ver}.lib\"")
    endforeach()

    set_target_properties(${bindings_library}
                           PROPERTIES LINK_FLAGS "${python_additional_link_flags}")

    # Compile a list of shiboken shared libraries to be installed, so that
    # the user doesn't have to set the PATH manually to point to the PySide6 package.
    foreach(library_path ${shiboken_shared_libraries})
        string(REGEX REPLACE ".lib$" ".dll" library_path ${library_path})
        file(TO_CMAKE_PATH ${library_path} library_path)
        list(APPEND windows_shiboken_shared_libraries "${library_path}")
    endforeach()
    # =========================================================================================
    # !!! End of dubious section.
    # =========================================================================================
endif()

# =============================================================================================
# !!! (The section below is deployment related, so in a real world application you will want to
# take care of this properly with some custom script or tool).
# =============================================================================================
# Install the library and the bindings module into the source folder near the main.py file, so
# that the Python interpeter successfully imports the used module.
install(TARGETS ${bindings_library} ${sample_library}
        LIBRARY DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}
        RUNTIME DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}
        )
install(FILES ${windows_shiboken_shared_libraries} DESTINATION ${CMAKE_CURRENT_SOURCE_DIR})
# =============================================================================================
# !!! End of dubious section.
# =============================================================================================