Tutorial 1: Implementing a signal processing box

  • NB: Document updated for OpenViBE 1.0.0 (07.May.2015)
n.b. This tutorial describes how to make a signal processing box starting from nothing, using the Skeleton Generator tool. If you’re ultimately interested in implementing your own box, then depending on your preference and skills, a quicker and more pragmatic way to make a new box could be to modify the code of an existing OpenViBE box. Boxes Hello World and Signal Average are examples of very simple boxes that can work as elementary starting points. However, following this tutorial can be instructive for learning about OpenViBE.

Preliminary

From this point we assume that you have downloaded the sources of OpenViBE and already built the whole software using the provided scripts at least once.

For those who want to catch up, download the sources or checkout the code, and then follow the build instructions.

Designing a new box

Before implementing a new OpenViBE box, consider taking some time to think about the design you want to achieve. Think of the following questions:

  • Does the box need settings specified by the user ?
  • What are the inputs of my box ? Do I need Streamed Matrix ? Signal ? Stimulations ?
  • What output my box is producing?
  • When do I need to process? When a new chunk comes (data-driven box) or at a specific frequency (time-driven box) ?
  • What are the conditions I need to check before processing ? Should the box handle the errors ?
  • What are the information, warning or error messages I need to log ?
  • etc.

Lets take a concrete example. In this tutorial we will implement a signal processing box that will flatten every channel of the incoming signal, upon receiving a specific stimulation. The same stimulation will switch this behavior off. This may sound totally useless – in fact it may be, but with this example we will cover all the basics of box development. Here is the design we want to achieve:

  • 1 Signal input and 1 Signal output
  • 1 Stimulation input
  • 2 settings : the value used for the flat signal and the stimulation that activates the processing
  • the box is data-driven : processing has to be done on every chunk of input signal to produce a new signal chunk
  • if an unknown stimulation is received, a warning message will be printed in the console

Using the Skeleton-Generator

The skeleton-generator is a developer tool we strongly advise you to use when developing new boxes or acquisition driver. Take a look at the dedicated documentation page for a description of the application. Anyway, press the ‘Help’ buttons next to each entry if any question remains.

Launch the generator and fill the information needed as described below.

General information

Let’s name our box “Signal Flattener“. The box will appear in the Designer under the category ‘Signal processing’.
Of course you can change the settings and see what happens.


Note that the actual Skeleton Generator may look a bit different than in the pictures below depending on the OpenViBE version,
but all the necessary functionality should be present.

 Completing the General information tab in the skeleton-generator(click here to show content)
 Completing the General information tab in the skeleton-generator(click here to hide content)


Inputs

We need 2 inputs : the signal and the triggers. The user won’t be able to add, remove or change any input.

 Completing the Inputs information tab in the skeleton-generator(click here to show content)
 Completing the Inputs information tab in the skeleton-generator(click here to hide content)

Outputs

We only need 1 signal output, that will be either the same as the input signal or a flattened signal.

 Completing the Outputs information tab in the skeleton-generator(click here to show content)
 Completing the Outputs information tab in the skeleton-generator(click here to hide content)

Settings

Let’s add 2 settings : the level value used to flatten the signal, and the stimulation used as a trigger. Let’s take the stimulation OVTK_StimulationId_Label_01 as a default trigger.

 Completing the Settings information tab in the skeleton-generator(click here to show content)
 Completing the Settings information tab in the skeleton-generator(click here to hide content)

Codecs

We will need a Signal Decoder, a Stimulation Decoder and a Signal Encoder. These are used to process input and output streams when they are passed between the box and the kernel.

We will use the codec toolkit to simplify our implementation.

Processing Method

Select only the first method, in order to call “process” every time a new chunk of data is received on the inputs.

Box Listener

As we don’t allow the user to modify, add or remove any input,output or setting we don’t need any Box Listener callback (mechanisms that react to box property changes). Just leave this option unchecked.

Check & Generate !

Now that everything we need is provided, the Skeleton-Generator will have to check for any incoherency that could interfere with the generation process. Click on ‘Check...’ You should get the following result in the console:

 (click here to show content)
 (click here to hide content)

----- STATUS -----
[   OK   ] Valid box name.
[   OK   ] Valid class name.
[   OK   ] Valid category.
[   OK   ] Valid box version.
[   OK   ] Valid short description.
[   OK   ] Valid detailed description.
Checking inputs...
>>[   OK   ] Valid input 0 [Signal]
>>[   OK   ] Valid input 1 [Trigger]
Checking outputs...
>>[   OK   ] Valid output 0 [Processed Signal]
Checking settings...
>>[   OK   ] Valid setting 0 [Level]
>>[   OK   ] Valid setting 1 [Trigger]
Checking algorithm...
>>[   OK   ] Valid algorithm 0 [Signal stream decoder]
>>[   OK   ] Valid algorithm 1 [Stimulation stream decoder]
>>[   OK   ] Valid algorithm 2 [Signal stream encoder]
----- SUCCESS -----
Press 'Generate!' to generate the files. If you want to modify your choices, please press the 'Check' button again.

If an error occurred, modify the failing items and check again. When everything is fine, press ‘Generate!‘. you will be asked where to put the generated files. Open your OpenViBE source root and select the folder contrib/plugins/processing/signal-processing/src/.

The generation process may take a few seconds. Upon success your file explorer should pop out in the designated folder. 3 files are generated :

  • ovpCBoxAlgorithmSignalFlattener.h : the C++ header of the box class
  • ovpCBoxAlgorithmSignalFlattener.cpp : the implementation of the class
  • README.txt : contains useful information… don’t miss it !
The skeletons generated are highly commented. If you think information are incomplete or missing, please contact us and we will do our best to improve it.

As stated in the README.txt file, you have to add the box to the main file in order to register it into the Designer later on.

Open ovp_main.cpp and include the box header:

...
#include "ovpCBoxAlgorithmSignalFlattener.h" // including the code
...

Then declare the new box by adding its descriptor in the declare section, using the OVP_Declare_New macro :

...
OVP_Declare_New(OpenViBEPlugins::SignalProcessing::CBoxAlgorithmSignalFlattenerDesc) // declaring my box
...

Build the empty box

The skeleton produced by the generator is an empty shell for your box. Inputs, outputs, settings are declared but not used. At this point, you should be able to build the box without writing a single line of code in it!

The code should now compile as part of the OpenViBE build. You may have to ‘touch’ (== save with new filesystem timestamp) the CMakeLists.txt file in the parent directory for the build process to include the new files.

Once the box is built you should see it in the Designer. The skeleton-generator flags the box as ‘unstable’ by default, thus the box is not visible for users. You can make Designer show the unstable boxes by ticking the little box at the top of the box list (not visible in the old picture below).

This is what you should have at this point in the Designer:

 (click here to show content)
 (click here to hide content)

When pressing ‘Play’ and ‘Stop’, the scenario should start without any problem.

Now that the skeleton is in place and running, we move on to the processing part.

Implementing the box

Open the .h and .cpp files in your favorite editor. Here is a summary of what you will find in the code:

Header

Please take some time to read the comments given in the header, you will find most of what you need

 File: ovpCBoxAlgorithmSignalFlattener.h(click here to show content)
 File: ovpCBoxAlgorithmSignalFlattener.h(click here to hide content)

#ifndef __OpenViBEPlugins_BoxAlgorithm_SignalFlattener_H__
#define __OpenViBEPlugins_BoxAlgorithm_SignalFlattener_H__

//You may have to change this path to match your folder organisation
#include "ovp_defines.h"

#include <openvibe/ov_all.h>
#include <toolkit/ovtk_all.h>

// The unique identifiers for the box and its descriptor.
// Identifier are randomly chosen by the skeleton-generator.
#define OVP_ClassId_BoxAlgorithm_SignalFlattener OpenViBE::CIdentifier(0x6AAE706A, 0x67976660)
#define OVP_ClassId_BoxAlgorithm_SignalFlattenerDesc OpenViBE::CIdentifier(0x6AAE706A, 0x67976660)

namespace OpenViBEPlugins
{
    namespace SignalProcessing
    {
        /**
         * \class CBoxAlgorithmSignalFlattener
         * \author Laurent Bonnet (INRIA)
         * \date Tue Oct 18 18:20:32 2011
         * \brief The class CBoxAlgorithmSignalFlattener describes the box Signal Flattener.
         *
         */
        class CBoxAlgorithmSignalFlattener : virtual public OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >
        {
        public:
            virtual void release(void) { delete this; }

            virtual OpenViBE::boolean initialize(void);
            virtual OpenViBE::boolean uninitialize(void);

            //Here is the different process callbacks possible
            // - On clock ticks :
            //virtual OpenViBE::boolean processClock(OpenViBE::CMessageClock& rMessageClock);
            // - On new input received (the most common behaviour for signal processing) :
            virtual OpenViBE::boolean processInput(OpenViBE::uint32 ui32InputIndex);

            // If you want to use processClock, you must provide the clock frequency.
            //virtual OpenViBE::uint64 getClockFrequency(void);

            virtual OpenViBE::boolean process(void);

            // As we do with any class in openvibe, we use the macro below
            // to associate this box to an unique identifier.
            // The inheritance information is also made available,
            // as we provide the superclass OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >
            _IsDerivedFromClass_Final_(OpenViBEToolkit::TBoxAlgorithm < OpenViBE::Plugins::IBoxAlgorithm >, OVP_ClassId_BoxAlgorithm_SignalFlattener);

        protected:
            // Codec algorithms specified in the skeleton-generator:
            // Signal stream decoder
            OpenViBEToolkit::TSignalDecoder < CBoxAlgorithmSignalFlattener > m_oAlgo0_SignalDecoder;
            // Stimulation stream decoder
            OpenViBEToolkit::TStimulationDecoder < CBoxAlgorithmSignalFlattener > m_oAlgo1_StimulationDecoder;
            // Signal stream encoder
            OpenViBEToolkit::TSignalEncoder < CBoxAlgorithmSignalFlattener > m_oAlgo2_SignalEncoder;

        private:
            OpenViBE::boolean m_bFlatSignalRequested;
            OpenViBE::float64 m_f64LevelValue;
            OpenViBE::uint64  m_ui64Trigger;

        };

        // If you need to implement a box Listener, here is a sekeleton for you.
        // Use only the callbacks you need.
        // For example, if your box has a variable number of input, but all of them must be stimulation inputs.
        // The following listener callback will ensure that any newly added input is stimulations :
        /*        
        virtual OpenViBE::boolean onInputAdded(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index)
        {
            rBox.setInputType(ui32Index, OV_TypeId_Stimulations);
        };
        */

        /*
        // The box listener can be used to call specific callbacks whenever the box structure changes : input added, name changed, etc.
        // Please uncomment below the callbacks you want to use.
        class CBoxAlgorithmSignalFlattenerListener : public OpenViBEToolkit::TBoxListener < OpenViBE::Plugins::IBoxListener >
        {
        public:

            //virtual OpenViBE::boolean onInitialized(OpenViBE::Kernel::IBox& rBox) { return true; };
            //virtual OpenViBE::boolean onNameChanged(OpenViBE::Kernel::IBox& rBox) { return true; };
            //virtual OpenViBE::boolean onInputConnected(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onInputDisconnected(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onInputAdded(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onInputRemoved(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onInputTypeChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onInputNameChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onOutputConnected(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onOutputDisconnected(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onOutputAdded(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onOutputRemoved(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onOutputTypeChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onOutputNameChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onSettingAdded(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onSettingRemoved(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onSettingTypeChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onSettingNameChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onSettingDefaultValueChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };
            //virtual OpenViBE::boolean onSettingValueChanged(OpenViBE::Kernel::IBox& rBox, const OpenViBE::uint32 ui32Index) { return true; };

            _IsDerivedFromClass_Final_(OpenViBEToolkit::TBoxListener < OpenViBE::Plugins::IBoxListener >, OV_UndefinedIdentifier);
        };
        */

        /**
         * \class CBoxAlgorithmSignalFlattenerDesc
         * \author Laurent Bonnet (INRIA)
         * \date Tue Oct 18 18:20:32 2011
         * \brief Descriptor of the box Signal Flattener.
         *
         */
        class CBoxAlgorithmSignalFlattenerDesc : virtual public OpenViBE::Plugins::IBoxAlgorithmDesc
        {
        public:

            virtual void release(void) { }

            virtual OpenViBE::CString getName(void) const                { return OpenViBE::CString("Signal Flattener"); }
            virtual OpenViBE::CString getAuthorName(void) const          { return OpenViBE::CString("Laurent Bonnet"); }
            virtual OpenViBE::CString getAuthorCompanyName(void) const   { return OpenViBE::CString("INRIA"); }
            virtual OpenViBE::CString getShortDescription(void) const    { return OpenViBE::CString("Flatten the incoming signal"); }
            virtual OpenViBE::CString getDetailedDescription(void) const { return OpenViBE::CString("When the stimulation specified by the user is received, all channels are flattened to the same value, specified in the box settings. The stimulation triggers off this behavior."); }
            virtual OpenViBE::CString getCategory(void) const            { return OpenViBE::CString("Signal processing"); }
            virtual OpenViBE::CString getVersion(void) const             { return OpenViBE::CString("1.0"); }
            virtual OpenViBE::CString getStockItemName(void) const       { return OpenViBE::CString("gtk-remove"); }

            virtual OpenViBE::CIdentifier getCreatedClass(void) const    { return OVP_ClassId_BoxAlgorithm_SignalFlattener; }
            virtual OpenViBE::Plugins::IPluginObject* create(void)       { return new OpenViBEPlugins::SignalProcessing::CBoxAlgorithmSignalFlattener; }

            /*
            virtual OpenViBE::Plugins::IBoxListener* createBoxListener(void) const               { return new CBoxAlgorithmSignalFlattenerListener; }
            virtual void releaseBoxListener(OpenViBE::Plugins::IBoxListener* pBoxListener) const { delete pBoxListener; }
            */
            virtual OpenViBE::boolean getBoxPrototype(
                OpenViBE::Kernel::IBoxProto& rBoxAlgorithmPrototype) const
            {
                rBoxAlgorithmPrototype.addInput("Signal",OV_TypeId_Signal);
                rBoxAlgorithmPrototype.addInput("Trigger",OV_TypeId_Stimulations);

                //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanModifyInput);
                //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanAddInput);

                rBoxAlgorithmPrototype.addOutput("Processed Signal",OV_TypeId_Signal);

                //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanModifyOutput);
                //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanAddOutput);

                rBoxAlgorithmPrototype.addSetting("Level",OV_TypeId_Float,"0");
                rBoxAlgorithmPrototype.addSetting("Trigger",OV_TypeId_Stimulation,"OVTK_StimulationId_Label_01");

                //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanModifySetting);
                //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanAddSetting);

                rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_IsUnstable);

                return true;
            }
            _IsDerivedFromClass_Final_(OpenViBE::Plugins::IBoxAlgorithmDesc, OVP_ClassId_BoxAlgorithm_SignalFlattenerDesc);
        };
    };
};

#endif // __OpenViBEPlugins_BoxAlgorithm_SignalFlattener_H__

The header file contains 3 classes

  • The box, that will be instantiated when playing a scenario
  • The box descriptor, from which the Designer knows the box structure (inputs, outputs, etc.)
  • An optional box listener, used to “monitor” the box structure. We won’t use it in this tutorial.

Box class

The box header already declares the functions you will have to implement:

  • boolean initialize(void) : gets the settings from the box, prepare the codecs, etc.
  • boolean uninitialize(void) : cleanup everything
  • boolean processInput(OpenViBE::uint32 ui32InputIndex) : called by kernel every time a new chunk is available on the inputs
  • boolean process(void) : the function called in the processing loop; it’s where the job is actually done
All these functions return a boolean. If it’s true, then the process may go on. If at some point false is returned by any of these functions, the box is automatically deactivated by the kernel.

The codec algorithms are also declared for you:

// Signal stream decoder
OpenViBEToolkit::TSignalDecoder < CBoxAlgorithmSignalFlattener > m_oAlgo0_SignalDecoder;
// Stimulation stream decoder
OpenViBEToolkit::TStimulationDecoder < CBoxAlgorithmSignalFlattener > m_oAlgo1_StimulationDecoder;
// Signal stream encoder
OpenViBEToolkit::TSignalEncoder < CBoxAlgorithmSignalFlattener > m_oAlgo2_SignalEncoder;

We will need few more variables. We add a private boolean member m_bFlatSignalRequested that will save the current state of our box, telling the process function whether to flatten the signal or not. Even if it is not mandatory, we will also save the settings in dedicated variables.

private:
    OpenViBE::boolean m_bFlatSignalRequested;
    OpenViBE::float64 m_f64LevelValue;
    OpenViBE::uint64  m_ui64Trigger;

Box descriptor class

As you may already noticed, the box descriptor CBoxAlgorithmSignalFlattenerDesc contains all the information you specified in the generator: name, description, category, etc. It also gives the kernel the way to actually create the box itself with:

virtual OpenViBE::Plugins::IPluginObject* create(void)
{
    return new OpenViBEPlugins::SignalProcessing::CBoxAlgorithmSignalFlattener;
}

Let’s take a closer look at the major function getBoxPrototype(OpenViBE::Kernel::IBoxProto& rBoxAlgorithmPrototype).

This function is used to construct the box in the Designer when drag’n'dropping from the category to the editing pane.

virtual OpenViBE::boolean getBoxPrototype(
                OpenViBE::Kernel::IBoxProto& rBoxAlgorithmPrototype) const
{

It describes the inputs of the box, and tells if the user can add/remove or modify the inputs. Uncomment the box flags and check what happens in the Designer (right click on the box to add inputs for example).

    rBoxAlgorithmPrototype.addInput("Signal",OV_TypeId_Signal);
    rBoxAlgorithmPrototype.addInput("Trigger",OV_TypeId_Stimulations);

    //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanModifyInput);
    //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanAddInput);

The outputs are also declared, with the same flags as for the inputs.

    rBoxAlgorithmPrototype.addOutput("Processed Signal",OV_TypeId_Signal);

    //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanModifyOutput);
    //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanAddOutput);

And then the settings are declared, with name, type and default value:

    rBoxAlgorithmPrototype.addSetting("Level",OV_TypeId_Integer,"0");
    rBoxAlgorithmPrototype.addSetting("Trigger",OV_TypeId_Stimulation,"OVTK_StimulationId_Label01");

    //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanModifySetting);
    //rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_CanAddSetting);

Adding inputs, outputs and settings is done in sequence. As a result the indexation follows the order used in the descriptor. In our example, the first setting (index 0) is the “Level”. “Trigger” is at index 1.

Finally, we have box-level flags declared, e.g. for unstable or deprecated box.

    rBoxAlgorithmPrototype.addFlag(OpenViBE::Kernel::BoxFlag_IsUnstable);

    return true;
}

The Implementation

 File: ovpCBoxAlgorithmSignalFlattener.cpp(click here to show content)
 File: ovpCBoxAlgorithmSignalFlattener.cpp(click here to hide content)

#include "ovpCBoxAlgorithmSignalFlattener.h"

using namespace OpenViBE;
using namespace OpenViBE::Kernel;
using namespace OpenViBE::Plugins;

using namespace OpenViBEPlugins;
using namespace OpenViBEPlugins::SignalProcessing;

boolean CBoxAlgorithmSignalFlattener::initialize(void)
{
    // Signal stream decoder, connect to input stream 0
    m_oAlgo0_SignalDecoder.initialize(*this,0);
    // Stimulation stream decoder, connect to input stream 1
    m_oAlgo1_StimulationDecoder.initialize(*this, 1);
    // Signal stream encoder, connect to output stream 0
    m_oAlgo2_SignalEncoder.initialize(*this, 0);

    // We connect the Signal Input with the Signal Output so every chunk on the input will be copied to the output automatically.
    m_oAlgo2_SignalEncoder.getInputMatrix().setReferenceTarget(m_oAlgo0_SignalDecoder.getOutputMatrix());
    m_oAlgo2_SignalEncoder.getInputSamplingRate().setReferenceTarget(m_oAlgo0_SignalDecoder.getOutputSamplingRate());

    // Then we save the settings in variables:
    // the "Level" setting is at index 0, we can auto cast it from CString to float64
    m_f64LevelValue = FSettingValueAutoCast(*this->getBoxAlgorithmContext(), 0);
    // the "Trigger" setting is at index 1, we can auto cast it from CString to uint64
    m_ui64Trigger = FSettingValueAutoCast(*this->getBoxAlgorithmContext(), 1);

    // Finally, we disable the flat mode at start
    m_bFlatSignalRequested = false;

    return true;
}
/*******************************************************************************/

boolean CBoxAlgorithmSignalFlattener::uninitialize(void)
{
    m_oAlgo0_SignalDecoder.uninitialize();
    m_oAlgo1_StimulationDecoder.uninitialize();
    m_oAlgo2_SignalEncoder.uninitialize();

    return true;
}
/*******************************************************************************/

boolean CBoxAlgorithmSignalFlattener::processInput(uint32 ui32InputIndex)
{
    // tell the kernel we are ready to process !
    getBoxAlgorithmContext()->markAlgorithmAsReadyToProcess();

    return true;
}
/*******************************************************************************/

boolean CBoxAlgorithmSignalFlattener::process(void)
{

    // the static box context describes the box inputs, outputs, settings structures
    // IBox& l_rStaticBoxContext=this->getStaticBoxContext();
    // the dynamic box context describes the current state of the box inputs and outputs (i.e. the chunks)
    IBoxIO& l_rDynamicBoxContext=this->getDynamicBoxContext();

    // we will first check the pendoing stimulations to check if the trigger has been received.
    //iterate over all chunk on input 1 (Stimulations)
    for(uint32 i=0; i<l_rDynamicBoxContext.getInputChunkCount(1); i++)
    {
        //we decode the i:th chunk
        m_oAlgo1_StimulationDecoder.decode(i);
        if(m_oAlgo1_StimulationDecoder.isHeaderReceived())
        {
            // Header received. This happens only once when pressing "play".
            // nothing to do...
        }
        if(m_oAlgo1_StimulationDecoder.isBufferReceived())
        {
            // A buffer has been received, lets' check the stimulations inside
            IStimulationSet* l_pStimulations = m_oAlgo1_StimulationDecoder.getOutputStimulationSet();
            for(uint32 j=0; j<l_pStimulations->getStimulationCount(); j++)
            {
                uint64 l_ui64StimulationCode = l_pStimulations->getStimulationIdentifier(j);
                uint64 l_ui64StimulationDate = l_pStimulations->getStimulationDate(j);
                CString l_sStimulationName   = this->getTypeManager().getEnumerationEntryNameFromValue(OV_TypeId_Stimulation, l_ui64StimulationCode);
                //If the trigger is received, we switch the mode
                if(l_pStimulations->getStimulationIdentifier(j) == m_ui64Trigger)
                {
                    m_bFlatSignalRequested = !m_bFlatSignalRequested;
                    this->getLogManager() << LogLevel_Info << "Flat mode is now ["
                                                           << (m_bFlatSignalRequested ? "ENABLED" : "DISABLED")
                                                           << "]\n";
                }
                else
                {
                    this->getLogManager() << LogLevel_Warning << "Received an unrecognized trigger, with code ["
                                                              << l_ui64StimulationCode
                                                              << "], name ["
                                                              << l_sStimulationName
                                                              << "] and date ["
                                                              << time64(l_ui64StimulationDate)
                                                              << "]\n";
                }
            }
        }
        if(m_oAlgo1_StimulationDecoder.isEndReceived())
        {
            // End received. This happens only once when pressing "stop".
            // nothing to do...
        }
    }

    //now lets process the signal according to current mode
    //iterate over all chunk on input 0 (signal)
    for(uint32 i=0; i<l_rDynamicBoxContext.getInputChunkCount(0); i++)
    {
        // decode the chunk i
        m_oAlgo0_SignalDecoder.decode(i);
        // the decoder may have decoded 3 different parts : the header, a buffer or the end of stream.
        if(m_oAlgo0_SignalDecoder.isHeaderReceived())
        {
            // Header received. This happens only once when pressing "play".
            // Now we know the sampling rate of the signal, and we can get it thanks to:
            //uint64 l_uiSamplingFrequency = m_oAlgo0_SignalDecoder.getOutputSamplingRate();

            // Pass the header to the next boxes, by encoding a header 
            m_oAlgo2_SignalEncoder.encodeHeader();
            // send the output chunk containing the header. The dates are the same as the input chunk:
            l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
        }
        if(m_oAlgo0_SignalDecoder.isBufferReceived())
        {
            // Buffer received.
            IMatrix* l_pMatrix = m_oAlgo0_SignalDecoder.getOutputMatrix(); // the StreamedMatrix of samples.
            uint32 l_ui32ChannelCount = l_pMatrix->getDimensionSize(0);
            uint32 l_ui32SamplesPerChannel = l_pMatrix->getDimensionSize(1);
            float64* l_pBuffer = l_pMatrix->getBuffer();
            // according to current mode, we modify or not the buffers
            if(m_bFlatSignalRequested)
            {
                // we can access the sample i from channel j with: l_pBuffer[i+j*l_ui32SamplesPerChannel]
                // in our case we modify every samples, so we only do:
                for(uint32 j=0; j<l_pMatrix->getBufferElementCount(); j++)
                {
                    l_pBuffer[j] = m_f64LevelValue;
                }
            }

            // Encode the output buffer (modified or not) :
            m_oAlgo2_SignalEncoder.encodeBuffer();
            // and send it to the next boxes :
            l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));

        }
        if(m_oAlgo0_SignalDecoder.isEndReceived())
        {
            // End of stream received. This happens only once when pressing "stop". Just pass it to the next boxes so they receive the message :
            m_oAlgo2_SignalEncoder.encodeEnd();
            l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i), l_rDynamicBoxContext.getInputChunkEndTime(0, i));
        }

        // The current input chunk has been processed, and automatically discarded thanks to the codec toolkit.
        // you don't need to call "l_rDynamicBoxContext.markInputAsDeprecated(0, i);"
    }

    return true;
}

Initialize & Uninitialize

The function initialize is called once, when pressing “play” in a scenario. The uninitialize function is called once when pressing “stop” afterward.

The initialization phase starts with the algorithms used in the box. Thanks to the codec toolkit the decoders and encoders are initialized simply with one line:

 // Signal stream decoder, connect to input stream 0
 m_oAlgo0_SignalDecoder.initialize(*this,0);
 // Stimulation stream decoder, connect to input stream 1
 m_oAlgo1_StimulationDecoder.initialize(*this,1);
 // Signal stream encoder, connect to output stream 0
 m_oAlgo2_SignalEncoder.initialize(*this,0);

We need to provide a reference on the box object in which the codec is running (*this) as we will need access to the box dynamic context (i.e. the chunks).

When the Flat mode is not enabled, our box outputs the signal it received on its input. Let’s simplify this process by connecting directly the signal decoder to the signal encoder thanks to the setReferenceTarget method. The signal stream needs matrices of samples and a sampling rate, thus we do:

 m_oAlgo2_SignalEncoder.getInputMatrix().setReferenceTarget(m_oAlgo0_SignalDecoder.getOutputMatrix());
 m_oAlgo2_SignalEncoder.getInputSamplingRate().setReferenceTarget(m_oAlgo0_SignalDecoder.getOutputSamplingRate());

The inputs take an output as its unique reference target, and not the opposite. As a result one input can only be connected to one output, and one output can be connected to multiple inputs.

The last step on initialization is to save the settings in our class members, using the auto cast functions to simplify the job:

// the "Level" setting is at index 0, we can auto cast it from CString to float64
m_f64LevelValue = FSettingValueAutoCast(*this->getBoxAlgorithmContext(), 0);
// the "Trigger" setting is at index 1, we can auto cast it from CString to uint64
m_ui64Trigger = FSettingValueAutoCast(*this->getBoxAlgorithmContext(), 1);

Finally, we disable the Flat mode at start:

m_bFlatSignalRequested = false;

The uninitialization phase destroys the algorithms. Using the codec toolkit we simply need to do:

m_oAlgo0_SignalDecoder.uninitialize();
m_oAlgo1_StimulationDecoder.uninitialize();
m_oAlgo2_SignalEncoder.uninitialize();

Process

As we requested, the process method we use will be data-driven. The callback processInput is implemented, and marks the box as ready to process when a new chunk is received on the inputs:

boolean CBoxAlgorithmSignalFlattener::processInput(uint32 ui32InputIndex)
{
    // tell the kernel we are ready to process !
    getBoxAlgorithmContext()->markAlgorithmAsReadyToProcess();

    return true;
}

It’s then up to the kernel to decide when to call the process function, with respect to the scheduling of the whole scenario.

The process will now parse the chunks and process the signal according to the triggers received. First, we iterate over stimulation chunks on input 1 in order to check if the Flat mode is requested:

for(uint32 i=0; i<l_rDynamicBoxContext.getInputChunkCount(1); i++)
{
    //we decode the i:th chunk
    m_oAlgo1_StimulationDecoder.decode(i);

At this point the chunk can be of 3 different natures: Header, Buffer or End. In our case only the buffers are important as we only want to check the stimulation they may contain:

if(m_oAlgo1_StimulationDecoder.isBufferReceived())
{
    // A buffer has been received, lets' check the stimulations inside
    IStimulationSet* l_pStimulations = m_oAlgo1_StimulationDecoder.getOutputStimulationSet();
    for(uint32 j=0; j<l_pStimulations->getStimulationCount(); j++)
    {

The buffers in a stimulation stream must be sent, even if there is no stimulation inside. Even if it may not be obvious at first sight, the reason is simple: if we know there is no stimulation in a defined time range, we need to forward that information in the stream. As a result don’t be surprised if the codec decodes many empty chunks.

The stimulation information (identifier, name, date and duration) can be retrieved through the following calls:

        uint64 l_ui64StimulationCode = l_pStimulations->getStimulationIdentifier(j);
        uint64 l_ui64StimulationDate = l_pStimulations->getStimulationDate(j);
        CString l_sStimulationName   = this->getTypeManager().getEnumerationEntryNameFromValue(OV_TypeId_Stimulation, l_ui64StimulationCode);

For fixed precision purpose, the time in OpenViBE is encoded on 64 bits unsigned integers, with fixed point at 32 bits (i.e. 32 bits for the seconds, 32 bits for the fractional part).

The stimulation code can then be compared to the trigger defined by the user. We will print a message with log level “Information” that will appear in the Designer console to know in which mode we’re currently in:

        //If the trigger is received, we switch the mode
        if(l_pStimulations->getStimulationIdentifier(j) == m_ui64Trigger)
        {
            m_bFlatSignalRequested = !m_bFlatSignalRequested;
            this->getLogManager() << LogLevel_Info << "Flat mode is now ["
                                                   << (m_bFlatSignalRequested ? "ENABLED" : "DISABLED")
                                                   << "]\n";
        }

If a stimulation has been received but is not recognized, we print a warning message:

        else
        {
        this->getLogManager() << LogLevel_Warning << "Received an unrecognized trigger, with code ["
                                                  << l_ui64StimulationCode
                                                  << "], name ["
                                                  << l_sStimulationName
                                                  << "] and date ["
                                                  << time64(l_ui64StimulationDate)
                                                  << "]\n";
        }
    }
}

Since OpenViBE 0.12.0 it’s possible to log human-readable time values, using the time64 type. The OpenViBE log manager will automatically handle the translation from 64bits (32:32) values to formatted text.

Now we move on to the processing of signal chunks. We decode th chunks on the signal input at index 0:

    for(uint32 i=0; i<l_rDynamicBoxContext.getInputChunkCount(0); i++)
    {
        // decode the chunk i 
        m_oAlgo0_SignalDecoder.decode(i);

The decoder may have decoded 3 different parts : the header, a buffer or the end of stream.

If a Header is received (user just pressed “play”), we need to prepare the signal output stream with its own header. In our case the output stream has the same structure as the input stream, and thanks to the reference targets we set the header is ready to be sent “as is”. We encode the EBML header and tell the kernel that this new output chunk is ready to be sent.

if(m_oAlgo0_SignalDecoder.isHeaderReceived())
{
    // Pass the header to the next boxes, by encoding a header:
    m_oAlgo2_SignalEncoder.encodeHeader();
    // send the output chunk containing the header. The dates are the same as the input chunk:
    l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i),
                                                    l_rDynamicBoxContext.getInputChunkEndTime(0, i));
}

If a buffer is received, we can extract the matrix of samples. In the case of a signal stream the streamed matrix has 2 dimensions: the channels, and the samples. The dimension sizes will give you the number of channel and the number of samples per channel per buffer.

if(m_oAlgo0_SignalDecoder.isBufferReceived())
{
    IMatrix* l_pMatrix = m_oAlgo0_SignalDecoder.getOutputMatrix(); // the StreamedMatrix of samples.
    uint32 l_ui32ChannelCount = l_pMatrix->getDimensionSize(0);
    uint32 l_ui32SamplesPerChannel = l_pMatrix->getDimensionSize(1);
    float64* l_pBuffer = l_pMatrix->getBuffer();

You could also retrieve the channel names by calling l_pMatrix->getDimensionLabel(0, channel_index)

When a Buffer is received on the stream, two cases are possible, with respect to the current mode. If the Flat mode is enabled, we need to modify the buffer of samples by changing all values to the requested Level. Otherwise we just need to forward the signal buffer untouched to the signal encoder as we did with the header:

    if(m_bFlatSignalRequested)
    {
        for(uint32 j=0; j<l_pMatrix->getBufferElementCount(); j++)
        {
            l_pBuffer[j] = m_f64LevelValue;
        }
    }

    m_oAlgo2_SignalEncoder.encodeBuffer();
    l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i),
                                                    l_rDynamicBoxContext.getInputChunkEndTime(0, i));

}

The matrix buffer is linear. For example with a signal matrix,  you can access the sample i from channel j with: l_pBuffer[i+j*l_ui32SamplesPerChannel]

Finally if a End of stream has been received (i.e. user pressed “stop”), we can directly encode its twin on the signal output:

    if(m_oAlgo0_SignalDecoder.isEndReceived())
    {
        m_oAlgo2_SignalEncoder.encodeEnd();
        l_rDynamicBoxContext.markOutputAsReadyToSend(0, l_rDynamicBoxContext.getInputChunkStartTime(0, i),
                                                        l_rDynamicBoxContext.getInputChunkEndTime(0, i));
    }
}

return true;

The codec toolkit masks some of the mechanics behind stream encoding and decoding. One of them is the input chunk management: you don’t have to mark the input chunk as deprecated as it is automatically done, telling the kernel to discard this chunk after process(). If you need to keep the chunk in the input stack, decode it using decode(input_index, chunk_index, false) 

Testing the Box

Once everything is compiled and running, you can test your new box in a dedicated scenario. Connect the Signal Flattener box to a sinus oscillator to produce a dummy signal, and a keyboard stimulator to trigger the stimulations at will. A signal display will print the signal coming from the Signal Flattener.

Testing the box in a simple scenario

Leave the settings untouched and play the scenario. Two windows open: the signal display showing the unmodified signal and the keyboard stimulator dialog. As you can see the default trigger OVTK_StimulationId_Label_01 is sent when pressing the ‘a’ on your keyboard. Click on the keyboard stimulator dialog and press ‘a’ to see the signal becoming flat 0 on every channel.

Releasing the key ‘a’ will send another stimulation OVTK_StimulationId_Label_00 which is not recognized by the signal Flattener, thus a warning message is printed in the console.

Using the Signal Flattener box

Double-click on the Signal Flattener to edit the settings. Choose a level at 2.5 and a trigger OVTK_StimulationId_Label_02 (key ‘z’ with the keyboard stimulator). Check if you get what you expect…

Conclusion

This first tutorial tried to be as complete as possible in order to give you all the basis to develop new signal processing boxes in OpenViBE.

As a conclusion we would like to remind you some of the important aspects we discussed here :

  • Stream logic: the hierarchy, the timing problematic, the codecs
  • Design thinking: take your time to design the box structure and processing methods before digging the code
  • Tools: the developer tools are here to help, don’t forget them

And finally… Community !
Experienced developers can be found on IRC channel or forum to discuss developments!
And if you find your box really useful and you think the community could benefit from it, don’t hesitate to contribute.

The following tutorials will be focused on more specific aspects of the plugin implementation. In tutorial 2 we will build 1 matrix processing algorithm, embedded in a box. This will enlighten the modularity principle of the platform.

This entry was posted in Box plugins and tagged , . Bookmark the permalink.