OpenViBE Documentation

Tutorial: create a signal processing algorithm and use it in a box

Extending OpenViBE

There are two ways OpenViBE can be extended. One is to write new algorithms, that are only used by programmers in order to perform a specific operation. The second is to create new boxes that can be used by authors in scenarios.

Depending on the task at hand, one might have to implement new algorithms and/or boxes. While it is possible to write a box which doesn't make use of any algorithm (in the sense of OpenViBE, meaning all signal processing code is written directly in the box), it is usually desirable to encapsulate signal processing operations in algorithms. The gain is not necessarily obvious at first, but it becomes evident in the long term, allowing box developers to reuse existing algorithms and build new boxes faster. In any case, it's up to the programmer to determine what operations are generic enough to justify their encapsulation in an algorithm.

This tutorial demonstrates how to add a new signal processing algorithm to OpenViBE, and how to create a new box which makes use of it. It comprises the following four sections :

After reading this tutorial, you should be able to start building your own plugins.

Algorithm definition

Here is the file containing the algorithm definition, we will detail each line of the file later on.

#ifndef __OpenViBEPlugins_Algorithm_SignalProcessingAlgorithm_H__
#define __OpenViBEPlugins_Algorithm_SignalProcessingAlgorithm_H__

#include "../../ovp_defines.h"
#include <openvibe/ov_all.h>
#include <openvibe-toolkit/ovtk_all.h>

namespace OpenViBEPlugins
{
        namespace Samples
        {
                class CAlgorithmSignalProcessingAlgorithm : public OpenViBEToolkit::TAlgorithm < OpenViBE::Plugins::IAlgorithm >
                {
                public:

                        virtual void release(void) { delete this; }

                        virtual OpenViBE::boolean initialize(void);
                        virtual OpenViBE::boolean uninitialize(void);
                        virtual OpenViBE::boolean process(void);

                        _IsDerivedFromClass_Final_(OpenViBEToolkit::TAlgorithm < OpenViBE::Plugins::IAlgorithm >, OVP_ClassId_Algorithm_SignalProcessingAlgorithm);

                protected:

                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > ip_pMatrix;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > op_pMatrix;
                };

                class CAlgorithmSignalProcessingAlgorithmDesc : public OpenViBE::Plugins::IAlgorithmDesc
                {
                public:

                        virtual void release(void) { }

                        virtual OpenViBE::CString getName(void) const                { return OpenViBE::CString("Signal processing algorithm"); }
                        virtual OpenViBE::CString getAuthorName(void) const          { return OpenViBE::CString("Yann Renard"); }
                        virtual OpenViBE::CString getAuthorCompanyName(void) const   { return OpenViBE::CString("INRIA"); }
                        virtual OpenViBE::CString getShortDescription(void) const    { return OpenViBE::CString("This is a sample signal processing algorithm that does nothing but can be used as a quick start for creating new signal processing algorithms"); }
                        virtual OpenViBE::CString getDetailedDescription(void) const { return OpenViBE::CString(""); }
                        virtual OpenViBE::CString getCategory(void) const            { return OpenViBE::CString("Samples"); }
                        virtual OpenViBE::CString getVersion(void) const             { return OpenViBE::CString("1.0"); }
                        virtual OpenViBE::CString getStockItemName(void) const       { return OpenViBE::CString("gtk-execute"); }

                        virtual OpenViBE::CIdentifier getCreatedClass(void) const    { return OVP_ClassId_Algorithm_SignalProcessingAlgorithm; }
                        virtual OpenViBE::Plugins::IPluginObject* create(void)       { return new OpenViBEPlugins::Samples::CAlgorithmSignalProcessingAlgorithm; }

                        virtual OpenViBE::boolean getAlgorithmPrototype(
                                OpenViBE::Kernel::IAlgorithmProto& rAlgorithmPrototype) const
                        {
                                rAlgorithmPrototype.addInputParameter (OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix,     "Matrix", OpenViBE::Kernel::ParameterType_Matrix);
                                rAlgorithmPrototype.addOutputParameter(OVP_Algorithm_SignalProcessingAlgorithm_OutputParameterId_Matrix,    "Matrix", OpenViBE::Kernel::ParameterType_Matrix);

                                rAlgorithmPrototype.addInputTrigger   (OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize,   "Initialize");
                                rAlgorithmPrototype.addInputTrigger   (OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process,      "Process");
                                rAlgorithmPrototype.addOutputTrigger  (OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone, "Process done");

                                return true;
                        }

                        _IsDerivedFromClass_Final_(OpenViBE::Plugins::IAlgorithmDesc, OVP_ClassId_Algorithm_SignalProcessingAlgorithmDesc);
                };
        };
};

#endif // __OpenViBEPlugins_Algorithm_SignalProcessingAlgorithm_H__

First of all, we will include every identifier / define needed for this plugin to work. The OpenViBE toolkit will help in the implementation so we include it as well.

#ifndef __OpenViBEPlugins_Algorithm_SignalProcessingAlgorithm_H__
#define __OpenViBEPlugins_Algorithm_SignalProcessingAlgorithm_H__

#include "../../ovp_defines.h"
#include <openvibe/ov_all.h>
#include <openvibe-toolkit/ovtk_all.h>

In order to avoid name collisions, it is a safe practice to include all classes in a namespace. All standard OpenViBE plugins are defined in sub namespaces of OpenViBEPlugins. Here we are going to build a new box for example purposes, so we narrow down the scope to the Samples sub namespace.

namespace OpenViBEPlugins
{
        namespace Samples
        {

Now that we are in the OpenViBEPlugins::Samples namespace, we have to define the classes making up the algorithm :

  • the algorithm itself, and
  • its associated descriptor.

The descriptor is retrieved by the kernel upon startup. It provides information about the algorithm and also allows to create instances of this algorithm.

We start by declaring the algorithm class.

First off, we must choose an appropriate class name. Since this example algorithm is meant to be a general signal processing example, we use a general name. In practice, the name of an algorithm should always convey a rough idea of what kind of signal processing it performs.

The interface common to all algorithms is IAlgorithm. Our algorithm must therefore inherit from it to be identified as a proper algorithm. However, algorithms implementation is made easier by the OpenViBEToolkit::TAlgorithm template, which implements several functionalities common to all algorithms (e.g. direct access to managers). Therefore, we inherit from this template, using the required IAlgorithm interface as the template argument.

The first method to implement is release, which deletes the algorithm.

                class CAlgorithmSignalProcessingAlgorithm : public OpenViBEToolkit::TAlgorithm < OpenViBE::Plugins::IAlgorithm >
                {
                public:

                        virtual void release(void) { delete this; }

The gist of an algorithm implementation lies in three methods : initialize, uninitialize and process. Here is the life cycle of an algorithm :

  • the algorithm is created (using the algorithm descriptor)
  • initialize is called once
  • process is called an indefinite number of times
  • uninitialize is called once
  • the algorithm is deleted

Each of these methods returns a boolean reflecting whether they operated successfully. If any of them returns false, the caller should assume the algorithm failed and should stop calling this algorithm any further.

                        virtual OpenViBE::boolean initialize(void);
                        virtual OpenViBE::boolean uninitialize(void);
                        virtual OpenViBE::boolean process(void);

All OpenViBE classes have their own class identifier. Once a new identifier has been generated, one should associate it to the algorithm class using the _IsDerivedFromClass_Final_ macro. The first argument of the macro is the name of the parent class, the second is the algorithm identifier.

                        _IsDerivedFromClass_Final_(OpenViBEToolkit::TAlgorithm < OpenViBE::Plugins::IAlgorithm >, OVP_ClassId_Algorithm_SignalProcessingAlgorithm);

                protected:

In this simple example, we are going to design an algorithm that takes a single input parameter (a matrix) and produces a single output parameter (another matrix).

In order to facilitate parameters usage, the algorithm should embed its parameters in parameter handlers. Parameter handlers provide transparent access to embedded parameters using an interface similar to the parameter interface.

The algorithm definition is now complete, and we move on to its descriptor. The algorithm descriptor provides information as to the role of its associated algorithm, its author, version, category and so on.

The interface common to all algorithm descriptors is IAlgorithmDesc, which we inherit from.

The release function can be left empty since a single instance of any given descriptor is created by the plugin which contains it, at plugin loading time, and eventually destroyed by the plugin when it is unloaded.

A number of methods return description strings which can be used to get a general idea of what the algorithm consists in.

                class CAlgorithmSignalProcessingAlgorithmDesc : public OpenViBE::Plugins::IAlgorithmDesc
                {
                public:

                        virtual void release(void) { }

                        virtual OpenViBE::CString getName(void) const                { return OpenViBE::CString("Signal processing algorithm"); }
                        virtual OpenViBE::CString getAuthorName(void) const          { return OpenViBE::CString("Yann Renard"); }
                        virtual OpenViBE::CString getAuthorCompanyName(void) const   { return OpenViBE::CString("INRIA"); }
                        virtual OpenViBE::CString getShortDescription(void) const    { return OpenViBE::CString("This is a sample signal processing algorithm that does nothing but can be used as a quick start for creating new signal processing algorithms"); }
                        virtual OpenViBE::CString getDetailedDescription(void) const { return OpenViBE::CString(""); }
                        virtual OpenViBE::CString getCategory(void) const            { return OpenViBE::CString("Samples"); }
                        virtual OpenViBE::CString getVersion(void) const             { return OpenViBE::CString("1.0"); }
                        virtual OpenViBE::CString getStockItemName(void) const       { return OpenViBE::CString("gtk-execute"); }

The descriptor specifies what kind of algorithm it is able to create thanks to the getCreatedClass method, which returns the name of the algorithm class. Actual instances of the algorithm may be created using the create method.

                        virtual OpenViBE::CIdentifier getCreatedClass(void) const    { return OVP_ClassId_Algorithm_SignalProcessingAlgorithm; }
                        virtual OpenViBE::Plugins::IPluginObject* create(void)       { return new OpenViBEPlugins::Samples::CAlgorithmSignalProcessingAlgorithm; }

The algorithm prototype may be retrieved using getAlgorithmPrototype. The prototype gives information about inputs, outputs and triggers.

Inputs and outputs are parameters and can easily be manipulated with parameter handlers. Triggers are related to the algorithm state : input triggers specify how the algorithm should execute when called, while output triggers give an indication of what state the algorithm ended in when done with processing.

                        virtual OpenViBE::boolean getAlgorithmPrototype(
                                OpenViBE::Kernel::IAlgorithmProto& rAlgorithmPrototype) const
                        {

This signal processing algorithm has two parameters : one input containing the signal matrix to process and one output containing the processed signal. Each of these parameters is identified by an identifier, a name and a type.

                                rAlgorithmPrototype.addInputParameter (OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix,     "Matrix", OpenViBE::Kernel::ParameterType_Matrix);
                                rAlgorithmPrototype.addOutputParameter(OVP_Algorithm_SignalProcessingAlgorithm_OutputParameterId_Matrix,    "Matrix", OpenViBE::Kernel::ParameterType_Matrix);

This signal processing algorithm also has three triggers.

The first one is an input trigger and is used to notify the algorithm that the initialization step could be performed. This initialization is different from the one done in initialize : when the latter method is called, the algorithm itself is initialized, whereas the former method initializes data the algorithm will process later on. Preconditions that must be validated before activating this trigger include :

  • The algorithm initialize method returned successfully

  • The input matrix description was filled

The second trigger also is an input trigger. When set, it requests the actual processing of the input matrix and results in the production of an output matrix.

The last trigger is an output trigger used to tell the caller about the outcome of the signal processing stage. When and only when an output matrix is successfully produced, the algorithm activates this output trigger.

                                rAlgorithmPrototype.addInputTrigger   (OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize,   "Initialize");
                                rAlgorithmPrototype.addInputTrigger   (OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process,      "Process");
                                rAlgorithmPrototype.addOutputTrigger  (OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone, "Process done");

Finally, the method returns true, notifying the caller (the kernel) that the box prototype was successfully retrieved.

                                return true;
                        }

Similarly to how we associated an identifier to the algorithm class, we specify the identifier of the algorithm descriptor using the _IsDerivedFromClass_Final_ macro. The first argument is algorithm descriptor's parent interface and the second is the actual descriptor identifier.

                        _IsDerivedFromClass_Final_(OpenViBE::Plugins::IAlgorithmDesc, OVP_ClassId_Algorithm_SignalProcessingAlgorithmDesc);
                };
        };
};

#endif // __OpenViBEPlugins_Algorithm_SignalProcessingAlgorithm_H__

Algorithm implementation

Here is the file containing the algorithm implementation, we will detail each line of the file later on.

#include "ovpCAlgorithmSignalProcessingAlgorithm.h"

using namespace OpenViBE;
using namespace OpenViBE::Kernel;
using namespace OpenViBE::Plugins;

using namespace OpenViBEPlugins;
using namespace OpenViBEPlugins::Samples;

boolean CAlgorithmSignalProcessingAlgorithm::initialize(void)
{
        ip_pMatrix.initialize(this->getInputParameter(OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix));
        op_pMatrix.initialize(this->getOutputParameter(OVP_Algorithm_SignalProcessingAlgorithm_OutputParameterId_Matrix));
        return true;
}

boolean CAlgorithmSignalProcessingAlgorithm::uninitialize(void)
{
        op_pMatrix.uninitialize();
        ip_pMatrix.uninitialize();
        return true;
}

boolean CAlgorithmSignalProcessingAlgorithm::process(void)
{
        IMatrix* l_pInputMatrix=ip_pMatrix;
        IMatrix* l_pOutputMatrix=op_pMatrix;

        if(this->isInputTriggerActive(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize))
        {
                OpenViBEToolkit::Tools::Matrix::copyDescription(*l_pOutputMatrix, *l_pInputMatrix);
        }

        if(this->isInputTriggerActive(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process))
        {
                OpenViBEToolkit::Tools::Matrix::copyContent(*l_pOutputMatrix, *l_pInputMatrix);
                for(uint32 i=0; i<l_pInputMatrix->getDimensionSize(0); i++)
                {
                        l_pOutputMatrix->getBuffer()[i*l_pInputMatrix->getDimensionSize(1)]=0;
                }

                this->activateOutputTrigger(OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone, true);
        }

        return true;
}

First off, for easier development, it is common practice to declare using several common OpenViBE namespaces at the beginning of the file, after including the required header files. A good rule of thumb is to never use "using" directives in header files, and to use them sparsely in implementation files. Considering OpenViBE does not contain conflicting classes (no two classes are named identically among all its files), it is acceptable to declare using OpenViBE namespaces.

#include "ovpCAlgorithmSignalProcessingAlgorithm.h"

using namespace OpenViBE;
using namespace OpenViBE::Kernel;
using namespace OpenViBE::Plugins;

Similarly, the OpenViBEPlugins project does not contain conflicting classes and makes it acceptable to declare using some of its namespaces.

using namespace OpenViBEPlugins;
using namespace OpenViBEPlugins::Samples;

On to the real meat of the implementation : there are three algorithm methods two implement,and the good news is that two of them are simple :)

In this example, initialize and uninitialize simply consist in connecting or disconnecting our parameter handlers to the actual parameters. These handlers will be used later to ease parameter value manipulation.

Let's initialize our parameters.

boolean CAlgorithmSignalProcessingAlgorithm::initialize(void)
{

ip_pMatrix is the input matrix pointer used by this algorithm. This parameter was declared in the algorithm descriptor as being of type OV_TypeId_Matrix and identified as OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix. The parameter can be retrieved with the getInputParameter function and passed to the handler using its initialize method.

Past this point, ip_pMatrix can be used as the object it encapsulates, that is, as an IMatrix* pointer using the -> operator.

        ip_pMatrix.initialize(this->getInputParameter(OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix));

Similarly, op_pMatrix is the output matrix pointer of this algorithm. The parameter can be retrieved with the getOutputParameter function and given to the handler. Past this point, ip_pMatrix can be used as an IMatrix* pointer using the -> operator.

        op_pMatrix.initialize(this->getOutputParameter(OVP_Algorithm_SignalProcessingAlgorithm_OutputParameterId_Matrix));

Returning true at the end of this initialize function tells the kernel everything went on nicely and no error occurred. If false is returned here, the kernel will consider this algorithm failed to initialize and won't call it anymore in the future.

        return true;
}

Uninitialize should be called to terminate an algorithm which won't be used anymore. Thus, every initialized member should be freed and the whole environment should be left as it was before initialize was called. In this simple example, all that is needed is to disconnect our parameter handlers from their respective parameters.

boolean CAlgorithmSignalProcessingAlgorithm::uninitialize(void)
{

op_pMatrix can be disconnected from its parameter thanks to the uninitialize method. Past this point, using operator -> will throw an exception and cause a crash.

        op_pMatrix.uninitialize();

The same can be done with ip_pMatrix.

        ip_pMatrix.uninitialize();

As in initialize, returning true notifies the kernel everything ran smoothly.

        return true;
}

Now all that is left to implement is the core of the algorithm, the process method which does the actual signal processing. This method is called each time operations should be performed by this algorithm.

What operations the algorithm should execute may be specified using input triggers. Every time process is called, the algorithm should check the state of its input triggers and deduce what computations to perform. At the end of the processing, the algorithm can reflect its status by setting the appropriate output triggers.

boolean CAlgorithmSignalProcessingAlgorithm::process(void)
{

First of all, we will retrieve input and output matrix pointers from the parameter handlers. Since our handler instances were templated with IMatrix*, there is an automatic cast operator that returns a pointer to the actual IMatrix* value.

        IMatrix* l_pInputMatrix=ip_pMatrix;
        IMatrix* l_pOutputMatrix=op_pMatrix;

Now, we have to examine each of the input triggers to know what to process. Those input triggers are declared in the algorithm prototype. In this case, there are two of them, namely OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize and OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process. Let's check whether the first one is activated

        if(this->isInputTriggerActive(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize))
        {

When the initialize trigger is active, we know that the input matrix contains a complete description of the input matrices that will be forwarded to this algorithm (number of dimensions, dimension sizes, labels...).

Since our processing function only changes some matrix values, the description of its output matrix is identical to that of its input matrix. Therefore, we copy it using the copyDescription helper function of the toolkit. Past this point, the output matrix will have the same aspect as the input matrix. However, its contents are undefined for now.

                OpenViBEToolkit::Tools::Matrix::copyDescription(*l_pOutputMatrix, *l_pInputMatrix);
        }

Let's now test whether the process input trigger is set.

        if(this->isInputTriggerActive(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process))
        {

When this particular trigger is active, we know that the input matrix is completely filled, including its buffer (the part of the matrix that contains values). This is where the actual processing can take place. As an example, we demonstrate how to copy the input signal buffer, slightly modifying it by changing only the first sample of each channel by setting it to 0.

Input matrices are assumed to be two dimensional, and the samples of a given channel are listed in a row before moving on to the next channel.

  • the first dimension (index 0) of the matrix represents the number of channels
  • the second dimension (index 1) is made up of the samples of any given channel

The getBuffer method returns a pointer to the first value of the matrix, that is, the first sample of the first channel. To access the jth sample of the ith channel, one must offset this pointer by the following formula :

i * sampleCount + j

where sampleCount is the number of samples per channel, given by the size of the second dimension (index 1).

In this example, we iterate over channels (i) and set their first sample (j == 0) to 0.

                OpenViBEToolkit::Tools::Matrix::copyContent(*l_pOutputMatrix, *l_pInputMatrix);
                for(uint32 i=0; i<l_pInputMatrix->getDimensionSize(0); i++)
                {
                        l_pOutputMatrix->getBuffer()[i*l_pInputMatrix->getDimensionSize(1)]=0;
                }

Now that the processing is done, this can be reflected to the caller by activating the "process done" output trigger, whose state will be checked by the caller. Therefore, the OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone output trigger is activated.

                this->activateOutputTrigger(OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone, true);
        }

Finally, we should return true to notify the kernel that no error occurred during this processing step. If false is returned, the kernel won't ever call this algorithm anymore.

        return true;
}

Box algorithm definition

Here is the file containing the box algorithm definition, we will detail each line of the file later on.

#ifndef __OpenViBEPlugins_BoxAlgorithm_SignalProcessingBoxAlgorithm_H__
#define __OpenViBEPlugins_BoxAlgorithm_SignalProcessingBoxAlgorithm_H__

#include "../../ovp_defines.h"
#include <openvibe/ov_all.h>
#include <openvibe-toolkit/ovtk_all.h>

namespace OpenViBEPlugins
{
        namespace Samples
        {
                class CBoxAlgorithmSignalProcessingBoxAlgorithm : virtual public OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >
                {
                public:

                        virtual void release(void) { delete this; }

                        virtual OpenViBE::boolean initialize(void);
                        virtual OpenViBE::boolean uninitialize(void);
                        virtual OpenViBE::boolean processInput(OpenViBE::uint32 ui32InputIndex);
                        virtual OpenViBE::boolean process(void);

                        _IsDerivedFromClass_Final_(OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >, OVP_ClassId_BoxAlgorithm_SignalProcessingBoxAlgorithm);

                protected:

                        OpenViBE::boolean m_bActive;

                        OpenViBE::Kernel::IAlgorithmProxy* m_pSignalDecoder;
                        OpenViBE::Kernel::TParameterHandler < const OpenViBE::IMemoryBuffer* > ip_pMemoryBufferToDecode;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::uint64 > op_ui64SamplingRate;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > op_pDecodedMatrix;

                        OpenViBE::Kernel::IAlgorithmProxy* m_pSignalProcessingAlgorithm;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > ip_pSignalProcessingAlgorithmMatrix;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > op_pSignalProcessingAlgorithmMatrix;

                        OpenViBE::Kernel::IAlgorithmProxy* m_pSignalEncoder;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::uint64 > ip_ui64SamplingRate;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > ip_pMatrixToEncode;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMemoryBuffer* > op_pEncodedMemoryBuffer;
                };

                class CBoxAlgorithmSignalProcessingBoxAlgorithmDesc : virtual public OpenViBE::Plugins::IBoxAlgorithmDesc
                {
                public:

                        virtual void release(void) { }

                        virtual OpenViBE::CString getName(void) const                { return OpenViBE::CString("Signal processing box algorithm"); }
                        virtual OpenViBE::CString getAuthorName(void) const          { return OpenViBE::CString("Yann Renard"); }
                        virtual OpenViBE::CString getAuthorCompanyName(void) const   { return OpenViBE::CString("INRIA"); }
                        virtual OpenViBE::CString getShortDescription(void) const    { return OpenViBE::CString("This is a sample signal processing box algorithm that uses the sample signal processing algorithm in order to demonstrate how to build a signal processing box algorithm"); }
                        virtual OpenViBE::CString getDetailedDescription(void) const { return OpenViBE::CString(""); }
                        virtual OpenViBE::CString getCategory(void) const            { return OpenViBE::CString("Samples"); }
                        virtual OpenViBE::CString getVersion(void) const             { return OpenViBE::CString("1.0"); }
                        virtual OpenViBE::CString getStockItemName(void) const       { return OpenViBE::CString("gtk-execute"); }

                        virtual OpenViBE::CIdentifier getCreatedClass(void) const    { return OVP_ClassId_BoxAlgorithm_SignalProcessingBoxAlgorithm; }
                        virtual OpenViBE::Plugins::IPluginObject* create(void)       { return new OpenViBEPlugins::Samples::CBoxAlgorithmSignalProcessingBoxAlgorithm; }

                        virtual OpenViBE::boolean getBoxPrototype(
                                OpenViBE::Kernel::IBoxProto& rBoxAlgorithmPrototype) const
                        {
                                rBoxAlgorithmPrototype.addInput  ("Input signal",  OV_TypeId_Signal);
                                rBoxAlgorithmPrototype.addOutput ("Output signal", OV_TypeId_Signal);
                                rBoxAlgorithmPrototype.addSetting("Active", OV_TypeId_Boolean, "true");
                                rBoxAlgorithmPrototype.addFlag   (OpenViBE::Kernel::BoxFlag_IsUnstable);
                                return true;
                        }

                        _IsDerivedFromClass_Final_(OpenViBE::Plugins::IBoxAlgorithmDesc, OVP_ClassId_BoxAlgorithm_SignalProcessingBoxAlgorithmDesc);
                };
        };
};

#endif // __OpenViBEPlugins_BoxAlgorithm_SignalProcessingBoxAlgorithm_H__

First of all, we will include every identifier / define needed for this plugin to work. The OpenViBE toolkit will help in the implementation so we include it also.

#ifndef __OpenViBEPlugins_BoxAlgorithm_SignalProcessingBoxAlgorithm_H__
#define __OpenViBEPlugins_BoxAlgorithm_SignalProcessingBoxAlgorithm_H__

#include "../../ovp_defines.h"
#include <openvibe/ov_all.h>
#include <openvibe-toolkit/ovtk_all.h>

In order to avoid name collisions, it is desirable to define all classes within namespaces. All standard OpenViBE plugins are defined in sub namespaces of OpenViBEPlugins.

namespace OpenViBEPlugins
{
        namespace Samples
        {

We are now working within the OpenViBEPlugins::Samples namespace. We have to define two classes : the box descriptor and the box algorithm itself. The descriptor is retrieved by the kernel at startup to get general information about the box algorithm. It is this descriptor which allows to create actual instances of this box algorithm.

The interface common to all box algorithms is IBoxAlgorithm. However, box algorithm implementation is made easier thanks to the OpenViBEToolkit::TBoxAlgorithm template, which implements operations common to all box algorithms such as direct access to managers. We therfore inherit from this template, using the IBoxAlgorithm interface as template argument.

                class CBoxAlgorithmSignalProcessingBoxAlgorithm : virtual public OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >
                {

We start by declaring the release method which deletes the box algorithm.

                public:

                        virtual void release(void) { delete this; }

The core of a box algorithm is split into three methods : initialize, uninitialize and process. Additionally, a box algorithm can use several notification callbacks for input reception, clock ticks, message reception...

Its life cycle looks like the following :

  • one initialize call,
  • several notification/process calls
  • one uninitialize call.

Each of these methods returns a boolean reflecting whether it operated successfully. If any of these methods returns false, the kernel will stop using the box-algorithm.

                        virtual OpenViBE::boolean initialize(void);
                        virtual OpenViBE::boolean uninitialize(void);
                        virtual OpenViBE::boolean processInput(OpenViBE::uint32 ui32InputIndex);
                        virtual OpenViBE::boolean process(void);

As with any other OpenViBE class, the box algorithm should be given a class identifier. This is easily done using the _IsDerivedFromClass_Final_ macro.

                        _IsDerivedFromClass_Final_(OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >, OVP_ClassId_BoxAlgorithm_SignalProcessingBoxAlgorithm);

The m_bActive member will be used to activate/deactivate the box from the Designer.

                protected:

                        OpenViBE::boolean m_bActive;

The box implementation heavily relies on algorithms to do its job. This signal processing box algorithm will use three algorithms :

  • the first is responsible for decoding the input stream,
  • the second does the actual signal processing, and
  • the last one encodes the output stream.

Each of these algorithms has input and/or output parameters. In order to facilite access to these parameters, our box algorithm embeds them in parameter handlers.

First, we define the signal decoder and its associated parameter handlers. The signal decoder takes a memory buffer to decode and produces a signal matrix and an unsigned integer holding the sampling rate.

Second, we define the signal processing algorithm and its associated parameter handlers. This is the algorithm that was designed earlier in this tutorial. It takes a signal matrix as input and produces a new signal matrix as output.

                        OpenViBE::Kernel::IAlgorithmProxy* m_pSignalProcessingAlgorithm;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > ip_pSignalProcessingAlgorithmMatrix;
                        OpenViBE::Kernel::TParameterHandler < OpenViBE::IMatrix* > op_pSignalProcessingAlgorithmMatrix;

Lastly, we define the signal encoder and its associated parameter handlers. The signal encoder takes a signal matrix and an unsigned integer for the sampling rate as inputs and produces a memory buffer as output.

We're done with the declaration of the box algorithm, and move on to the descriptor. It provides information about the box algorithm, including its name, author, version and so on. The 'category' it should appear get listed under in the Designer may be specified here as well.

                class CBoxAlgorithmSignalProcessingBoxAlgorithmDesc : virtual public OpenViBE::Plugins::IBoxAlgorithmDesc
                {
                public:

                        virtual void release(void) { }

                        virtual OpenViBE::CString getName(void) const                { return OpenViBE::CString("Signal processing box algorithm"); }
                        virtual OpenViBE::CString getAuthorName(void) const          { return OpenViBE::CString("Yann Renard"); }
                        virtual OpenViBE::CString getAuthorCompanyName(void) const   { return OpenViBE::CString("INRIA"); }
                        virtual OpenViBE::CString getShortDescription(void) const    { return OpenViBE::CString("This is a sample signal processing box algorithm that uses the sample signal processing algorithm in order to demonstrate how to build a signal processing box algorithm"); }
                        virtual OpenViBE::CString getDetailedDescription(void) const { return OpenViBE::CString(""); }
                        virtual OpenViBE::CString getCategory(void) const            { return OpenViBE::CString("Samples"); }
                        virtual OpenViBE::CString getVersion(void) const             { return OpenViBE::CString("1.0"); }
                        virtual OpenViBE::CString getStockItemName(void) const       { return OpenViBE::CString("gtk-execute"); }

The descriptor also allows to create actual instances of the box algorithm. To that end, it tells the kernel what kind of box it can create and then the create method performs the instanciation.

                        virtual OpenViBE::CIdentifier getCreatedClass(void) const    { return OVP_ClassId_BoxAlgorithm_SignalProcessingBoxAlgorithm; }
                        virtual OpenViBE::Plugins::IPluginObject* create(void)       { return new OpenViBEPlugins::Samples::CBoxAlgorithmSignalProcessingBoxAlgorithm; }

The getBoxPrototype method retrieves the prototype of the box, i.e. a description of its inputs and outputs as well as its settings and flags.

  • Inputs and outputs are streams and will have to be decoded / encoded by specific algorithms.
  • Settings offer a way to preconfigure a box before using it.
  • Flags are used to remind the user about some unusual properties of a box, such as its being under development ('unstable' flag) or deprecated.

                        virtual OpenViBE::boolean getBoxPrototype(
                                OpenViBE::Kernel::IBoxProto& rBoxAlgorithmPrototype) const
                        {

Our signal processing box will have one input and one output, both of 'signal' type.

                                rBoxAlgorithmPrototype.addInput  ("Input signal",  OV_TypeId_Signal);
                                rBoxAlgorithmPrototype.addOutput ("Output signal", OV_TypeId_Signal);

Here a single boolean setting is declared and meant to hold the initial activation state.

                                rBoxAlgorithmPrototype.addSetting("Active", OV_TypeId_Boolean, "true");

Since this box is only meant to be used as a testbed, it is safer to flag it as unstable so as not to be mistaken with more robust boxes!

                                rBoxAlgorithmPrototype.addFlag   (OpenViBE::Kernel::BoxFlag_IsUnstable);

Finally, the descriptor returns true, notifying the kernel the box prototype was successfully retrieved.

                                return true;
                        }

Of course, the descriptor also has to be given an identifier using the _IsDerivedFromClass_Final_ macro.

                        _IsDerivedFromClass_Final_(OpenViBE::Plugins::IBoxAlgorithmDesc, OVP_ClassId_BoxAlgorithm_SignalProcessingBoxAlgorithmDesc);
                };
        };
};

#endif // __OpenViBEPlugins_BoxAlgorithm_SignalProcessingBoxAlgorithm_H__

Box algorithm implementation

Here is the file containing the box algorithm implementation, we will detail each line of the file later on.

#include "ovpCBoxAlgorithmSignalProcessingBoxAlgorithm.h"

using namespace OpenViBE;
using namespace OpenViBE::Kernel;
using namespace OpenViBE::Plugins;

using namespace OpenViBEPlugins;
using namespace OpenViBEPlugins::Samples;

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::initialize(void)
{
        IBox& l_rStaticBoxContext=this->getStaticBoxContext();

        CString l_sActive;
        l_rStaticBoxContext.getSettingValue(0, l_sActive);
        m_bActive=this->getConfigurationManager().expandAsBoolean(l_sActive);

        m_pSignalDecoder=&this->getAlgorithmManager().getAlgorithm(this->getAlgorithmManager().createAlgorithm(OVP_GD_ClassId_Algorithm_SignalStreamDecoder));
        m_pSignalDecoder->initialize();
        ip_pMemoryBufferToDecode.initialize(m_pSignalDecoder->getInputParameter(OVP_GD_Algorithm_SignalStreamDecoder_InputParameterId_MemoryBufferToDecode));
        op_ui64SamplingRate.initialize(m_pSignalDecoder->getOutputParameter(OVP_GD_Algorithm_SignalStreamDecoder_OutputParameterId_SamplingRate));
        op_pDecodedMatrix.initialize(m_pSignalDecoder->getOutputParameter(OVP_GD_Algorithm_SignalStreamDecoder_OutputParameterId_Matrix));

        m_pSignalProcessingAlgorithm=&this->getAlgorithmManager().getAlgorithm(this->getAlgorithmManager().createAlgorithm(OVP_ClassId_Algorithm_SignalProcessingAlgorithm));
        m_pSignalProcessingAlgorithm->initialize();
        ip_pSignalProcessingAlgorithmMatrix.initialize(m_pSignalProcessingAlgorithm->getInputParameter(OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix));
        op_pSignalProcessingAlgorithmMatrix.initialize(m_pSignalProcessingAlgorithm->getOutputParameter(OVP_Algorithm_SignalProcessingAlgorithm_OutputParameterId_Matrix));

        m_pSignalEncoder=&this->getAlgorithmManager().getAlgorithm(this->getAlgorithmManager().createAlgorithm(OVP_GD_ClassId_Algorithm_SignalStreamEncoder));
        m_pSignalEncoder->initialize();
        ip_ui64SamplingRate.initialize(m_pSignalEncoder->getInputParameter(OVP_GD_Algorithm_SignalStreamEncoder_InputParameterId_SamplingRate));
        ip_pMatrixToEncode.initialize(m_pSignalEncoder->getInputParameter(OVP_GD_Algorithm_SignalStreamEncoder_InputParameterId_Matrix));
        op_pEncodedMemoryBuffer.initialize(m_pSignalEncoder->getOutputParameter(OVP_GD_Algorithm_SignalStreamEncoder_OutputParameterId_EncodedMemoryBuffer));

        ip_ui64SamplingRate.setReferenceTarget(op_ui64SamplingRate);
        ip_pSignalProcessingAlgorithmMatrix.setReferenceTarget(op_pDecodedMatrix);
        ip_pMatrixToEncode.setReferenceTarget(op_pSignalProcessingAlgorithmMatrix);

        return true;
}

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::uninitialize(void)
{
        ip_pMatrixToEncode.uninitialize();
        ip_ui64SamplingRate.uninitialize();
        m_pSignalEncoder->uninitialize();
        this->getAlgorithmManager().releaseAlgorithm(*m_pSignalEncoder);

        op_pSignalProcessingAlgorithmMatrix.uninitialize();
        ip_pSignalProcessingAlgorithmMatrix.uninitialize();
        m_pSignalProcessingAlgorithm->uninitialize();
        this->getAlgorithmManager().releaseAlgorithm(*m_pSignalProcessingAlgorithm);

        op_pDecodedMatrix.uninitialize();
        op_ui64SamplingRate.uninitialize();
        m_pSignalDecoder->uninitialize();
        this->getAlgorithmManager().releaseAlgorithm(*m_pSignalDecoder);

        return true;
}

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::processInput(uint32 ui32InputIndex)
{
        this->getBoxAlgorithmContext()->markAlgorithmAsReadyToProcess();

        return true;
}

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::process(void)
{
        IBoxIO& l_rDynamicBoxContext=this->getDynamicBoxContext();

        for(uint32 i=0; i<l_rDynamicBoxContext.getInputChunkCount(0); i++)
        {
                ip_pMemoryBufferToDecode=l_rDynamicBoxContext.getInputChunk(0, i);
                op_pEncodedMemoryBuffer=l_rDynamicBoxContext.getOutputChunk(0);

                m_pSignalDecoder->process();

                if(m_pSignalDecoder->isOutputTriggerActive(OVP_GD_Algorithm_SignalStreamDecoder_OutputTriggerId_ReceivedHeader))
                {
                        m_pSignalProcessingAlgorithm->process(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize);
                        m_pSignalEncoder->process(OVP_GD_Algorithm_SignalStreamEncoder_InputTriggerId_EncodeHeader);

                        l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
                }
                if(m_pSignalDecoder->isOutputTriggerActive(OVP_GD_Algorithm_SignalStreamDecoder_OutputTriggerId_ReceivedBuffer))
                {
                        m_pSignalProcessingAlgorithm->process(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process);

                        if(m_pSignalProcessingAlgorithm->isOutputTriggerActive(OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone))
                        {
                                m_pSignalEncoder->process(OVP_GD_Algorithm_SignalStreamEncoder_InputTriggerId_EncodeBuffer);

                                l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
                        }
                }
                if(m_pSignalDecoder->isOutputTriggerActive(OVP_GD_Algorithm_SignalStreamDecoder_OutputTriggerId_ReceivedEnd))
                {
                        m_pSignalEncoder->process(OVP_GD_Algorithm_SignalStreamEncoder_InputTriggerId_EncodeEnd);

                        l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
                }

                l_rDynamicBoxContext.markInputAsDeprecated(0, i);
        }

        return true;
}

First off, for easier development, it is convenient to declare using some common OpenViBE namespaces so that they don't have to be explicitly typed every time. Again, such statements should never appear in in header files, but are acceptable in implementation files given that OpenViBE does not have conflicting classes among its namespaces.

#include "ovpCBoxAlgorithmSignalProcessingBoxAlgorithm.h"

using namespace OpenViBE;
using namespace OpenViBE::Kernel;
using namespace OpenViBE::Plugins;

The same can be done for the OpenViBEPlugins project namespaces.

using namespace OpenViBEPlugins;
using namespace OpenViBEPlugins::Samples;

Now we have two simple functions to implement, namely initialize and uninitialize. In our case, they consist in retrieving setting values, creating or releasing the necessary algorithms and connecting or disconnecting our parameter handlers to the actual parameters. These handlers will be used later to ease parameter value manipulation.

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::initialize(void)
{

The static box context allows to programmatically retrieve information about the box prototype, including the number of inputs / outputs / settings and their type.

        IBox& l_rStaticBoxContext=this->getStaticBoxContext();

Here we will use this static box context to retrieve the value of the "Active" setting. Settings are accessed with their 0-based index thanks to the getSettingValue function. The returned string may be parsed by the configuration manager. For example, in this case, we know the setting string is a boolean. So the expandAsBoolean function will return the boolean value corresponding to the setting string.

        CString l_sActive;
        l_rStaticBoxContext.getSettingValue(0, l_sActive);
        m_bActive=this->getConfigurationManager().expandAsBoolean(l_sActive);

Now we use the algorithm manager to create the three algorithms used by this box, then initialize these algorithms, and finally connect them together.

The stream decoder and encoder algorithms are common algorithms owned by another project in the platform. Their identifiers may be found in the ov_global_defines.h file which can be generated thanks to the plugin inspector tool.

Let's start with the signal decoder. It is created using the algorithm manager. Next, we initialize it.

Then we connect the parameter handlers of the box to their corresponding parameters. For example, ip_pMemoryBufferToDecode is the input memory buffer the decoder will work on. This parameter was declared in the decoder algorithm descriptor as being of type OV_TypeId_MemoryBuffer and with identifier OVP_GD_Algorithm_SignalStreamDecoder_InputParameterId_MemoryBufferToDecode. The parameter can be retrieved with the getInputParameter function and given to the handler. Past this point, ip_pMemoryBufferToDecode can be used as an IMemoryBuffer* pointer using the -> operator.

The same is done to initialize the sampling rate output parameter of the box with the output parameter of the decoder. And the decoded matrix parameter handler is connected to the matrix output parameter of the decoder.

        m_pSignalDecoder=&this->getAlgorithmManager().getAlgorithm(this->getAlgorithmManager().createAlgorithm(OVP_GD_ClassId_Algorithm_SignalStreamDecoder));
        m_pSignalDecoder->initialize();
        ip_pMemoryBufferToDecode.initialize(m_pSignalDecoder->getInputParameter(OVP_GD_Algorithm_SignalStreamDecoder_InputParameterId_MemoryBufferToDecode));
        op_ui64SamplingRate.initialize(m_pSignalDecoder->getOutputParameter(OVP_GD_Algorithm_SignalStreamDecoder_OutputParameterId_SamplingRate));
        op_pDecodedMatrix.initialize(m_pSignalDecoder->getOutputParameter(OVP_GD_Algorithm_SignalStreamDecoder_OutputParameterId_Matrix));

Now we create and initialize the signal processing algorithm. This algorithm is defined in this project, so we can directly use the class identifier from the ovp_defines.h file. Then we connect our parameter handlers to their corresponding parameters.

        m_pSignalProcessingAlgorithm=&this->getAlgorithmManager().getAlgorithm(this->getAlgorithmManager().createAlgorithm(OVP_ClassId_Algorithm_SignalProcessingAlgorithm));
        m_pSignalProcessingAlgorithm->initialize();
        ip_pSignalProcessingAlgorithmMatrix.initialize(m_pSignalProcessingAlgorithm->getInputParameter(OVP_Algorithm_SignalProcessingAlgorithm_InputParameterId_Matrix));
        op_pSignalProcessingAlgorithmMatrix.initialize(m_pSignalProcessingAlgorithm->getOutputParameter(OVP_Algorithm_SignalProcessingAlgorithm_OutputParameterId_Matrix));

the last algorithm to create and initialize is the signal encoder. As for the decoder, we have to use the class identifier and the parameter identifiers from the ovp_global_defines.h file.

        m_pSignalEncoder=&this->getAlgorithmManager().getAlgorithm(this->getAlgorithmManager().createAlgorithm(OVP_GD_ClassId_Algorithm_SignalStreamEncoder));
        m_pSignalEncoder->initialize();
        ip_ui64SamplingRate.initialize(m_pSignalEncoder->getInputParameter(OVP_GD_Algorithm_SignalStreamEncoder_InputParameterId_SamplingRate));
        ip_pMatrixToEncode.initialize(m_pSignalEncoder->getInputParameter(OVP_GD_Algorithm_SignalStreamEncoder_InputParameterId_Matrix));
        op_pEncodedMemoryBuffer.initialize(m_pSignalEncoder->getOutputParameter(OVP_GD_Algorithm_SignalStreamEncoder_OutputParameterId_EncodedMemoryBuffer));

Now that algorithms and parameter handlers are initialized, we must forward decoded data to the signal processing algorithm, and then forward processed data to the encoding algorithm.

To facilite this, it is convenient to connect parameters together. This minimizes the efforts needed to pass data from one algorithm to the next.

For example, the signal processing algorithm does not change the sampling rate value. This means this parameter can be connected from the decoder straight to the encoder.

However, this is not the true of the decoded matrix, which must go through the signal processing algorithm first. This is where the setReferenceTarget method comes in handy. This method allows to link one parameter to another.

Past the following line, any modification on the value of the sampling rate parameter of the decoder will immediately affect the sampling rate parameter of the encoder.

        ip_ui64SamplingRate.setReferenceTarget(op_ui64SamplingRate);

The following code block also makes use of setReferenceTarget to forward the signal matrix from one algorithm to another, and then to a third one.

The first statement links the decoder output matrix to the signal processing algorithm input matrix. The second statement links the processed matrix to the encoder input matrix. All parameters used here will actually share the same object value.

        ip_pSignalProcessingAlgorithmMatrix.setReferenceTarget(op_pDecodedMatrix);
        ip_pMatrixToEncode.setReferenceTarget(op_pSignalProcessingAlgorithmMatrix);

Again, returning true tells the kernel the box was correctly initialized. Returning false would have the kernel assume this box failed to initialize and it would stop calling it in the future.

        return true;
}

Uninitialization notifies the box algorithm that it won't be used anymore. Thus every initialized member should be freed and the whole environment should be left as it was before initialize was called.

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::uninitialize(void)
{

To release an algorithm, three steps have to be followed :

  • disconnect parameter handlers from their embedded parameter by calling uninitialize.
  • call uninitialize on the algorithm itself
  • have the algorithm manager release the algorithm

In the following code block, ip_pMatrixToEncode is disconnected from its parameter thanks to the uninitialize method. Past this point, using operator -> will throw an exception and cause a crash. The same is done on each of the parameter handlers of the signal encoder. Then the signal encoder itself can be uninitialized and a request can be sent to the algorithm manager to release this algorithm that won't be used anymore.

        ip_pMatrixToEncode.uninitialize();
        ip_ui64SamplingRate.uninitialize();
        m_pSignalEncoder->uninitialize();
        this->getAlgorithmManager().releaseAlgorithm(*m_pSignalEncoder);

The signal processing algorithm is released next.

        op_pSignalProcessingAlgorithmMatrix.uninitialize();
        ip_pSignalProcessingAlgorithmMatrix.uninitialize();
        m_pSignalProcessingAlgorithm->uninitialize();
        this->getAlgorithmManager().releaseAlgorithm(*m_pSignalProcessingAlgorithm);

And finally, the signal decoder is released.

        op_pDecodedMatrix.uninitialize();
        op_ui64SamplingRate.uninitialize();
        m_pSignalDecoder->uninitialize();
        this->getAlgorithmManager().releaseAlgorithm(*m_pSignalDecoder);

Again, this function should return true to notify the kernel everything went on fine.

        return true;
}

All that is left to code now concerns event notification and data processing.

Let's start with notifications. A box can ask to be notified of different types of events, such as message arrival or clock ticks. This particular box only cares about input data arrival, an event upon which the box will trigger a data processing procedure. This is why a single event handler (processInput) is implemented here.

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::processInput(uint32 ui32InputIndex)
{

Entering this method means there is pending input data. In this simple example, the box is ready to process such data as soon as it arrives. The following statement marks the box as candidate for such processing, which will lead to its process method to be called. on the input.

        this->getBoxAlgorithmContext()->markAlgorithmAsReadyToProcess();

        return true;
}

On to the heart of the box implementation : the processing part. This is where input data chunks are retrieved in order to be decoded, processed and encoded before being forwarded to the next box as output chunks.

boolean CBoxAlgorithmSignalProcessingBoxAlgorithm::process(void)
{

The dynamic box context contains box communication information. Part of such information is made up of pending input chunks. Keeping a reference on this context allows to access directly input and output chunks.

        IBoxIO& l_rDynamicBoxContext=this->getDynamicBoxContext();

Notifying the kernel that a box algorithm is ready to process input data does not necessarily trigger an immediate process call. Therefore, multiple input chunks may be pending when process gets called eventually. This is why one should always iterate on input chunks to be sure none is left unprocessed.

Since our box has only one input, a single loop iterates over chunks of this input (index 0).

        for(uint32 i=0; i<l_rDynamicBoxContext.getInputChunkCount(0); i++)
        {

Each chunk retrieved from this input is going to be decoded, processed and encoded again.

Here, parameter handlers initialized earlier are told where to fetch input data and where to store output data. The getInputChunk method retreives input chunks from a given input (first argument) at a given index (second argument). Similarly, getOutputChunk retrieves output chunks from a given output at a given index.

                ip_pMemoryBufferToDecode=l_rDynamicBoxContext.getInputChunk(0, i);
                op_pEncodedMemoryBuffer=l_rDynamicBoxContext.getOutputChunk(0);

At this point, the box is ready to start processing the chunk. First, we have the decoder decode it.

                m_pSignalDecoder->process();

The decoder has several output triggers telling us what was just decoded. There are three chunk categories :

  • headers (received once per input)
  • buffers (received an undeterminate number of times, depending on how much data there is to process)
  • end nodes (received once per input).

Depending on what was decoded, different actions will be undertaken.

                if(m_pSignalDecoder->isOutputTriggerActive(OVP_GD_Algorithm_SignalStreamDecoder_OutputTriggerId_ReceivedHeader))
                {

In case a header was received, we simply request the initialization of the signal processing algorithm by activating its initialize trigger. When its process method is called next, this trigger activation status evaluates to true, causing the algorithm to initialize its output matrix.

                        m_pSignalProcessingAlgorithm->process(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Initialize);

At this point, the header part of the output stream can be encoded.

                        m_pSignalEncoder->process(OVP_GD_Algorithm_SignalStreamEncoder_InputTriggerId_EncodeHeader);

Lastly, the output chunk can be marked as ready to be sent. This will let the kernel send it to the boxes that are connected to this output. One thing that we did not notice is that each stream buffer corresponds to a given time period. This time period is retrieved thanks to the getInputChunkStartTime and getInputChunkEndTime functions of the dynamic context. Similarly, when marking an output chunk as ready to send, the box must specify the time period that this chunk corresponds to.

                        l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
                }

In case a buffer is received, we can request the signal processing algorithm to process it. This is where the first sample of each channel is reset to 0, while any other sample is left untouched.

                if(m_pSignalDecoder->isOutputTriggerActive(OVP_GD_Algorithm_SignalStreamDecoder_OutputTriggerId_ReceivedBuffer))
                {
                        m_pSignalProcessingAlgorithm->process(OVP_Algorithm_SignalProcessingAlgorithm_InputTriggerId_Process);

Depending on the outcome of the actual processing, we may be ready to request a buffer encoding. The processing status is checked using the "process done" output trigger : when set to true, a buffer was successfully processed.

                        if(m_pSignalProcessingAlgorithm->isOutputTriggerActive(OVP_Algorithm_SignalProcessingAlgorithm_OutputTriggerId_ProcessDone))
                        {

It is now time to encode the processed matrix.

                                m_pSignalEncoder->process(OVP_GD_Algorithm_SignalStreamEncoder_InputTriggerId_EncodeBuffer);

As was done for the header chunk, this new output chunk is flagged as ready to send with the corresponding time period.

                                l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
                        }
                }

Lastly, in case the decoder decoded an end node, we do not have to process anything. This just means that the input stream is closed and that no more buffers will be received from it. A new header would have to be sent first for new buffers to be received again.

Here, we just have to notify the following boxes that an end node was decoded. This is done by encoding and sending such a node.

                if(m_pSignalDecoder->isOutputTriggerActive(OVP_GD_Algorithm_SignalStreamDecoder_OutputTriggerId_ReceivedEnd))
                {
                        m_pSignalEncoder->process(OVP_GD_Algorithm_SignalStreamEncoder_InputTriggerId_EncodeEnd);

As for header and buffer chunks, we mark this new output chunk as ready to send with the corresponding time period.

                        l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
                }

Finally, since the input chunk was processed, we can notify the kernel that it can now be released. The chunk object won't be effectively released until the process function returns. This call can rather be seen as 'flagging' the chunk for deletion.

                l_rDynamicBoxContext.markInputAsDeprecated(0, i);
        }

        return true;
}

Conclusion

Now that both plugins are created, we have to register them to the kernel at plugin loading time. For this reason, in ovp_main.cpp we use the OVP_Declare_New macro as follows :
 OVP_Declare_New(OpenViBEPlugins::Samples::CAlgorithmSignalProcessingAlgorithmDesc);
 OVP_Declare_New(OpenViBEPlugins::Samples::CBoxAlgorithmSignalProcessingBoxAlgorithmDesc);