Tutorial – Level 2 – Using Matlab with OpenViBE

  • NB: cursory update for OpenViBE 1.2.1 (doc updated 07.jul.2016)

Note from the author : the Matlab Scripting box plugin for OpenViBE have been developed for the researchers more experienced with Matlab scripting than C++ programming. As the OpenViBE developers generally do not use Matlab, we are happy to receive any comments about it. Please send us your feedback and suggestion of improvement: bug reports, usability feedback, documentation requests, etc.

Note: V0.16.x has a bug that Matlab working directory path can not contain spaces, preventing running Matlab tutorial scenarios from their default install location on Windows in “Program Files (x86)”. To use the scenarios, please move them to a folder with no spaces in the path. (This issue has been fixed in the git version.)

Note: On Linux, the presence of matlab plugin .so can cause crashes in seemingly unrelated Simple DSP box as the two may use different versions of boost which are then wrongly exposed to the other party due to the dlopen() flag RTLD_GLOBAL. Unfortunately this flag is needed or we get other issues with third-party libraries, for example that the imports done inside Python plugin scripts no longer work. The compromise solution is that Matlab plugin compilation is currently by default DISABLED in the main CMakeLists.txt. You may enable the Matlab plugin compilation and possibly set the flag to RTLD_LOCAL to avoid issues in ovCKernelLoader.cpp, but be aware of the consequences.

Introduction

OpenViBE is currently compatible with MATLAB through a box:

  • Matlab Scripting: a general-purpose box with unlimited inputs, outputs and settings. User writes the Matlab functions that will be called by the box.

This box is developed and supported by the Inria team. It relies on the Matlab Engine, and requires a valid installation of Matlab to be compiled and to run. To be precise, the Matlab documentation states :

 

The MATLAB engine library contains routines that allow you to call MATLAB software from your own programs, thereby employing MATLAB as a computation engine. You must use an installed version of MATLAB; you cannot run the MATLAB engine on a machine that only has the MATLAB Compiler Runtime (MCR).

The Matlab Scripting box is included in the official release packages, and thus can be used directly in any scenario, as long as you have a valid Matlab installation on your computer.

Requirements

The Matlab box has mostly been tested on Windows 7 32 bits + MATLAB R2011a 32 bits. On Linuxes, we cannot give recommendations what versions work or not: try with the one you have, first.

The Matlab box is not compatible with MATLAB 64 bits version on Windows. If you have multiple Matlab installations on the computer, make sure OpenViBE picks the 32bit one. Having multiple Matlabs may lead to the issue “Failed to load the Matlab engine.”
Please contact us to inform us of any compatibility issue. We will update this section with your feedback !

Compiling the ‘plugins/processing/matlab’ project

A valid installation of MATLAB includes all the libraries and exe required to compile and use the boxes.

To be precise, CMake looks for the following libraries when compiling OpenViBE:

  • On Windows : libmex.lib, libmx.lib, libeng.lib under  ${Matlab_ROOT}/extern/lib/win32/microsoft/
  • On Linux : mex, mx, eng, under  ${Matlab_ROOT}/bin/glnx86 or ${Matlab_ROOT}/bin/glnxa64

When building the software, the process should find all these libraries, as showed in the following build log example:

 A successful build log on Windows(click here to show content)
 A successful build log on Windows(click here to hide content)

Configuring and building plugins\processing\matlab …

–   Found OpenViBE…
– …
–   Found Matlab…
–     [  OK  ] Third party lib C:/Program Files/MATLAB/R2011a/extern/lib/win32/microsoft/libmex.lib
–     [  OK  ] Third party lib C:/Program Files/MATLAB/R2011a/extern/lib/win32/microsoft/libmx.lib
–     [  OK  ] Third party lib C:/Program Files/MATLAB/R2011a/extern/lib/win32/microsoft/libeng.lib
– Configuring done
– Generating done
– Build files have been written to: C:/openvibe/local-tmp/plugins/processing/matlab/

If the build process does not find a valid MATLAB installation, the boxes won’t be built with the project. An empty dynamic library will be produced, which will result in the following message in the console when executing the OpenViBE Designer :

[WARNING] No 'plugin object descriptor' found from [../bin/openvibe-plugins-matlab.dll]
          even if it looked like a plugin module

When building OpenViBE with an invalid MATLAB installation (i.e. MATLAB 64 bits on Windows) the build process will find MATLAB but not the libs resulting in the following build log :

 Build fails on Windows with MATLAB 64 bits(click here to show content)
 Build fails on Windows with MATLAB 64 bits(click here to hide content)

Configuring and building openvibe\plugins\processing\matlab\…

–   Found OpenViBE…
– …
–   Found Matlab…
–     [FAILED] Third party lib libmex
–     [FAILED] Third party lib libmx
–     [FAILED] Third party lib libeng
–  FAILED to find Matlab Libs, the plugins won’t be built. Please ensure you have a valid MATLAB installation (32 bits only).
–  Configuring done
–  Generating done
–  Build files have been written to: C:/openvibe/local-tmp/plugins/processing/matlab/

The Matlab Scripting box

The Matlab Scripting box is available in the Designer in the category Scripting.

Here is the different settings and how to set it correctly :

  • Box clock frequency (in Hertz) : defines the frequency on which the box will be called by OpenViBE, from 1 to 128 Hz. Default value is 64Hz, you can lower it if you experience bad performance when executing the box.
  • Path to Matlab executables (Windows) / Matlab launch command (Linux): tells OpenViBE how to execute the Matlab engine. Set it to the directory where is matlab.exe on Windows or type the command you use to launch Matlab on Linux.
  • Matlab working directory : the directory where are located all your function files. This path must be Matlab compliant (i.e. some special characters are not recognized)
  • Initialize function : the name of the Matlab function called by the box when pressing ‘play’ in the scenario.
  • Process function : the name of the Matlab function called by the box on every box clock tick.
  • Uninitialize function : the name of the Matlab function called by the box when pressing ‘stop’ in the scenario.

You can add as many input and output as you need. All the stream types are compatible. You can also add new settings in the box, they will be available in the Matlab structure as well.

You can use more than one Matlab Scripting box in the same scenario. Each box has its own matlab workspace defined in the box setting.

If you are using the same workspace for several boxes, they will share this workspace. For example a global variable could be modified by all the boxes of the same workspace.

The OpenViBE toolbox for the Matlab Scripting box

The Matlab Scripting box relies on a set of Matlab functions to work. This toolbox is installed with openvibe in share/openvibe/plugins/matlab.
It contains 2 types of functions: those called by the box itself internally, and those who are meant to be called by the user in his script. In this section we describe these functions and the structures manipulated by Matlab and OpenViBE.

Setting up the environment

The box automatically sets up the Matlab environment to work properly. For your information, here is the procedure:

The box first creates a representation of the openvibe box in matlab:

box = OV_createBoxInstance(input_count, output_count) : 
        out STRUCT box : a structure representing the box in Matlab:
        in  DOUBLE input_count  : the number of input of the box, to create the structures accordingly.
        in  DOUBLE output_count : the number of output of the box, to create the structures accordingly.

Then it calls a function that will create several global variables for all the OpenViBE stimulation codes:

OV_defines()

The name of the variables and their values are strictly the same as in OpenViBE. For example:

global OVTK_StimulationId_Label_00; 
OVTK_StimulationId_Label_00 = uint64(hex2dec('00008100')); // 33024

Thus you can directly compare a stimulation identifier received on an input to these global variables.

Matlab structures and functions description

One instance of the Matlab Scripting box creates its own box in Matlab. This box is passed as an argument of every functions. For the functions that outputs a new box instance, the same variable is used and thus is updated.

The box structure is defined as follow:

STRUCT box:
> box.clock           : DOUBLE the current openvibe time, updated at every clock tick.
> box.clock_frequency : DOUBLE the box clock frequency.
> box.settings        : an array of setting structure(s).
> box.inputs          : a cell array of input structure(s).
> box.outputs         : a cell array of output structure(s).
> box.user_data       : empty user-space structure.

The box clock and clock frequency are very useful when your script needs to generate an output, but lacks an input to synchronize with.

The setting structures are filled according to the openvibe box settings defined in the scenario. Default settings are not stored in the structure.

STRUCT setting :
> setting.name  : STRING (name of the setting in the openvibe box).
> setting.type  : UINT64 openvibe code of the setting type.
> setting.value : the value of the setting. Its type can be DOUBLE (for the openvibe types integer and float),
                  LOGICAL (boolean), UINT64 (stimulation code) or STRING (anything else).

The input and output structures are defined by particular headers, and are meant to be filled at run time. When the Matlab Scripting box receives the headers on its inputs it forwards the information to Matlab, and modify the box representation accordingly.

The output headers structure must be set by the user in the Matlab script. Once the header is set, the Matlab Scripting box is able to get this information from the matlab structure and construct the corresponding OpenViBE header.

The IO structures also contains a buffer for the incoming chunk of data.

STRUCT input/output :
> io.header : STRUCT containing the stream description (e.g. for a signal: channel names, sampling rate, etc).
> io.buffer : CELL ARRAY of chunk structure(s).

The different header structures are listed below, along with the corresponding functions to set it in a convenient way (i.e. you won’t miss a field). Manually setting the header structures of your output is of course also possible (e.g. if your output stream has the same structure as an input, input.header = output.header is the simplest way to set it).

NB: A corresponding Getter function also exists for each header type. These functions are called by the Matlab Scripting box to get all the information in one call to the Matlab engine.

STRUCT header : STREAMED MATRIX
> header.type             : STRING = 'Streamed Matrix Stream'
> header.nb_dimensions    : DOUBLE number of dimensions of the matrix (N)
> header.dimension_sizes  : 1xN DOUBLE ARRAY of dimension sizes (one size per dimension)
> header.dimension_labels : 1xN CELL ARRAY of STRING

Setter Function (optional):
[box_out] = OV_setStreamedMatrixOutputHeader(box_in, output_index, nb_dim, dim_sizes, dim_labels)
STRUCT header : SIGNAL
> header.type                  : STRING = 'Signal Stream'
> header.nb_channels           : DOUBLE number of channels
> header.nb_samples_per_buffer : DOUBLE number of samples per buffer block
> header.channel_names         : CELL ARRAY of STRING
> header.sampling_rate         : DOUBLE sampling frequency

Setter Function (optional):
[box_out] = OV_setSignalOutputHeader(box_in, output_index, nb_channel, nb_samples_per_buffer, channel_names, sampling_rate)
STRUCT header : FEATURE VECTOR
> header.type        : STRING = 'Feature Vector Stream'
> header.nb_features : DOUBLE number of feature in the vector
> header.labels      : CELL ARRAY of STRING

Setter Function (optional):
[box_out] = OV_setFeatureVectorOutputHeader(box_in, output_index, nb_features, labels)
STRUCT header : SPECTRUM
> header.type          : STRING = 'Spectrum Stream'
> header.nb_channels   : DOUBLE number of channels
> header.channel_names : CELL ARRAY of STRING
> header.nb_bands      : DOUBLE number of frequency bands in the spectrum
> header.band_names    : CELL ARRAY of STRING the name of the bands (e.g. '10-15Hz')
> header.bands         : [nb_bands x 2 DOUBLE] the pairs of frequencies (band start and stop)

Setter Function (optional):
[box_out] = OV_setSpectrumOutputHeader(box_in, output_index, nb_channels, channel_names, nb_bands, band_names, bands)
STRUCT header : CHANNEL LOCALISATION
> header.type          : STRING = 'Channel Localisation Stream'
> header.nb_channels   : DOUBLE number of channels
> header.channel_names : CELL ARRAY of STRING
> header.dynamic       : LOGICAL tells if coordinates are dynamic

Setter Function (optional):
[box_out] = OV_setChannelLocalisationOutputHeader(box_in, output_index, nb_channels, channel_names, dynamic)
STRUCT header : STIMULATIONS
> header.type          : STRING = 'Stimulation Stream'

Setter Function (optional):
[box_out] = OV_setStimulationsOutputHeader(box_in)

As you can see all the setter functions take a box structure as input, and output the modified box structure. A typical call is thus:

box = OV_setXXXXXOutputHeader(box, ...)
% ... continue using the 'box' variable ...

The chunk structure stored in the IO buffers can be either a matrix (for Stramed Matrix stream and its child streams) or a stimulation set.

STRUCT chunk : MATRIX CHUNK
> chunk.start_time  : DOUBLE translation of the openvibe timestamp in seconds, precision 0.0001 sec.
> chunk.end_time    : DOUBLE
> chunk.matrix_data : DOUBLE ARRAY the matrix of data.
                      The structure of the matrix is the same as in OpenViBE.
                      For example, a signal matrix has N(channels) lines and N(samples) columns.
STRUCT chunk : SIMULATION CHUNK
> chunk.start_time  : DOUBLE translation of the openvibe timestamp in seconds, precision 0.0001 sec.
> chunk.end_time    : DOUBLE
> chunk.matrix_data : DOUBLE ARRAY the stimulation set containing N stimulation(s).
                      N stimulations are stored in a 3xN matrix, ie. 1 stimulation per column.
                      Each stimulation is defined by its code, date and duration.

The buffers may contains any number of chunks. The buffers can be manipulated through convenient matlab functions:

% To get the current count of pending chunk in buffer:
nb_pending = OV_getNbPendingInputChunk(box_in, input_index)
             out DOUBLE nb_pending : the number of pending chunk in the buffer.
             in STRUCT box_in      : the box structure.
             in DOUBLE input_index : index of the input.
% To pop the oldest chunk in a buffer (i.e. the first chunk to process)
% Note that the chunk is removed from the buffer after this call.
[box_out, start_time, end_time, matrix_data] = OV_popInputBuffer(box_in, input_index)
        out STRUCT box_out     : the Matlab box after modification.
        out DOUBLE start_time  : the chunk start time in seconds.
        out DOUBLE end_time    : the chunk end time in seconds.
        out ARRAY  matrix_data : the data, type and structure depends on the stream type.
        in  STRUCT box_in      : the Matlab box from which we pop a new chunk.
        in  DOUBLE input_index : the input index in the box.

If there is no chunk in the buffer the function returns:
        > start_time  = -1
        > end_time    = -1
        > matrix_data = []

When you need to output a new buffer, you can do so by calling the following function:

% To add a new chunk in an output buffer:
[box_out] = OV_addOutputBuffer(box_in,output_index,start_time,end_time,matrix_data);
             out DOUBLE box_out     : the box after modification.
             in STRUCT box_in       : the box structure.
             in DOUBLE output_index : index of the input.
             in DOUBLE start_time   : the chunk start time in seconds.
             in DOUBLE end_time     : the chunk end time in seconds.
             in ARRAY  matrix_data  : the data, type and structure depends on the stream type

Tutorial 1 : signal filter

Open the scenario share/openvibe/scenarios/box-tutorials/matlab/tuto1-signal-filter.xml.

This scenario is very simple : it filters a sinus signal and sets all the samples to 0 when receiving a specific stimulation. If the same stimulation is received once again, the output signal goes back to normal.

The Matlab Scripting box takes 2 inputs : the signal and the stimulation stream. It outputs a signal which has the same structure (channel count, etc.) as the input.

First step is to configure the Matlab Scripting box. You need to set your local Matlab directory and the working directory to match your local installation of Matlab and OpenViBE. The working directory is [openvibe]/share/openvibe/scenarios/box-tutorials/matlab/. It must be an absolute path. The three matlab functions are in this directory.

Finally, as you can notice we added a new setting named “zero trigger” of type Stimulation. This defines which stimulation is used to switch the behavior of the box.

Configuring the Matlab Scripting Box

Initialization script

Open the Matlab function file tuto1_signal_filter_Initialize.m.

 tuto1_signal_filter_Initialize.m(click here to show content)
 tuto1_signal_filter_Initialize.m(click here to hide content)

% tuto1_signal_filter_Initialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto1_signal_filter_Initialize is called when pressing 'play' in the scenario.
%
function box_out = tuto1_signal_filter_Initialize(box_in)
    disp('Matlab initialize function has been called.')
    % we add a field to save the trigger state
    box_in.user_data.trigger_state = false;
    % Don't forget to pass the modified box as output.
    box_out = box_in;
end

This function add a field in the user_data structure in the box : a boolean indicating the current state of the trigger. Please note that we must output the modified box to be able to use it afterwards. This function will be called once, when pressing “play” in the scenario.

Process script

Open the Matlab function file tuto1_signal_filter_Process.m.

 tuto1_signal_filter_Process.m(click here to show content)
 tuto1_signal_filter_Process.m(click here to hide content)

% tuto1_signal_filter_Process.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto1_signal_filter_Process is called at the clock frequency.
%
function box_out = tuto1_signal_filter_Process(box_in)

    % we iterate over the pending chunks on input 2 (STIMULATIONS)
     for i = 1: OV_getNbPendingInputChunk(box_in,2)

         % we pop the first chunk to be processed, note that box_in is used as
         % the output variable to continue processing
         [box_in, start_time, end_time, stim_set] = OV_popInputBuffer(box_in,2);

         % the stimulation set is a 3xN matrix.
         % The openvibe stimulation stream even sends empty stimulation sets
         % so the following boxes know there is no stimulation to expect in
         % the latest time range. These empty chunk are also in the matlab buffer.
         if(numel(stim_set) >= 3) % at least one stim in the set.
             fprintf('Received stimulation code %i at time %f\n', stim_set(1), stim_set(2))
             if stim_set(1) == box_in.settings(1).value % I'm lazy... I only check the first stim in the set.
                 box_in.user_data.trigger_state = ~box_in.user_data.trigger_state;
                 disp('Trigger is switched.')
             end
         end
     end

     % we iterate over the pending chunks on input 1 (SIGNAL)
     for i = 1: OV_getNbPendingInputChunk(box_in,1)

         % we pop the first chunk to be processed, note that box_in is used as the output
         % variable to continue processing
         [box_in, start_time, end_time, matrix_data] = OV_popInputBuffer(box_in,1);

         % if requested, all samples are set to 0
         if(box_in.user_data.trigger_state)
             matrix_data(1:end) = 0;
         end

         % we add the chunk to the signal output buffer. Note that we use box_in as the output variable.
         % The header of the output must be set prior to sending the first buffer.
         % In our case the headers are the same.
         box_in.outputs{1}.header = box_in.inputs{1}.header;
         % The chunk dates are the same as the input chunk
         box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,matrix_data);

     end

     % Pass the modified box as output to continue processing
     box_out = box_in;
end

As we set the box clock frequency to 64 Hz, this function will be called 64 time per second. It works in two steps. First we check the stimulation input for new triggers, and switch the box field accordingly. Note that every time we pop a new input chunk, we use the current box instance as output, to continue the processing on an updated box structure. In this case, the “pop” functions remove the last input chunk from the buffer.

As we only modify the samples values of the input signal, we set the output header as a copy of the input header. This will be done with every process but it is only needed once (prior to the first buffer ever sent).

Uninitialization script

Open the Matlab function file tuto1_signal_filter_Uninitialize.m.

 tuto1_signal_filter_Uninitialize.m(click here to show content)
 tuto1_signal_filter_Uninitialize.m(click here to hide content)

% tuto1_signal_filter_Uninitialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto1_signal_filter_Uninitialize is called when pressing 'stop' in the scenario.
%
function box_out = tuto1_signal_filter_Uninitialize(box_in)
    disp('Matlab uninitialize function has been called.')
    box_out = box_in;
end

We don’t need to do anything when stopping the scenario, we just display a message.

Execution

The result can be found below. The signal is set to 0 when the stimulation OVTK_StimulationId_Label_01 (33024) is received.

In the console, you can see the corresponding output :

 Console output(click here to show content)
 Console output(click here to hide content)

[ INF ] At time 0.000 sec <Box algorithm::Signal filter>
---- MATLAB BUFFER - INFO ----
Matlab initialize function has been called.
[ INF ] At time 1.719 sec <Box algorithm::Signal filter>
---- MATLAB BUFFER - INFO ----
Received stimulation code 33025 at time 1.687500
Trigger is switched.
[ INF ] At time 2.156 sec <Box algorithm::Signal filter>
---- MATLAB BUFFER - INFO ----
Received stimulation code 33024 at time 2.125000
[ INF ] At time 4.094 sec <Box algorithm::Signal filter>
---- MATLAB BUFFER - INFO ----
Received stimulation code 33025 at time 4.062500
Trigger is switched.
[ INF ] At time 4.938 sec <Box algorithm::Signal filter>
---- MATLAB BUFFER - INFO ----
Received stimulation code 33024 at time 4.906250
[ INF ] At time 9.172 sec <Box algorithm::Signal filter>
---- MATLAB BUFFER - INFO ----
Matlab uninitialize function has been called.

Tutorial 2 : FFT and Matlab plot

Open the scenario share/openvibe/scenarios/box-tutorials/matlab/tuto2-FFT-filter.xml.

In this scenario we add some noise in the upcoming signal, and plot the spectrum amplitude (FFT) in Matlab to check if this noise appears in the signal.
Please note that we only do this process on the first channel of the signal, thus the output signal has only one channel. At the end, we will plot the mean spectrum before stopping the scenario.

First step is to configure the Matlab Scripting box. You need to set your local Matlab directory and the working directory to match your local installation of Matlab and OpenViBE. The working directory is [openvibe]/share/openvibe/scenarios/box-tutorials/matlab/. It must be an absolute path. The three matlab functions are in this directory.

We added 4 FLOAT settings : the noise frequency and amplitude, that will be added in the signal, and the frequency range used for plotting the spectrum.

Initialization script

Open the Matlab function file tuto2_FFT_filter_Initialize.m.

 tuto2_FFT_filter_Initialize.m(click here to show content)
 tuto2_FFT_filter_Initialize.m(click here to hide content)

% tuto2_FFT_filter_Initialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 28/07/2011
%
% This file contains the function called once by the matlab box on its initialization (when playing the scenario).

function box_out = tuto2_FFT_filter_Initialize(box_in)

    % we display the setting values
    disp('Box settings are:')
    for i=1:size(box_in.settings,2)
        fprintf('\t%s : %s\n',box_in.settings(i).name, num2str(box_in.settings(i).value));
    end

    % let's add a user-defined indicator to know if the output header is set
    box_in.user_data.is_headerset = false;

    % We also add some statistics
    box_in.user_data.nb_matrix_processed = 0;
    box_in.user_data.mean_fft_matrix = 0;

    box_out = box_in;

end

This function prints the box settings and add fields in the user_data structure in the box : a boolean indicating if we need to set the output header, and the variables we need to compute the mean spectrum. Please note that we must output the modified box to be able to use it afterwards. This function will be called once, when pressing “play” in the scenario.

Process script

Open the Matlab function file tuto2_FFT_filter_Process.m.

 tuto2_FFT_filter_Process.m(click here to show content)
 tuto2_FFT_filter_Process.m(click here to hide content)

% tuto2-FFT-filter_Process.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 28/07/2011
%
% This function adds a noise in the input signal and plot the resulting FFT
function box_out = tuto2_FFT_filter_Process(box_in)

    for i = 1: OV_getNbPendingInputChunk(box_in,1)

        if(~box_in.user_data.is_headerset)
            % The output is the input + noise, only on first channel
            box_in.outputs{1}.header = box_in.inputs{1}.header;
            box_in.outputs{1}.header.nb_channels = 1;
            box_in.user_data.is_headerset = 1;
            % We print the header in the console
           disp('Input header is :')
           box_in.inputs{1}.header
        end

        % we increment the matrix count
        box_in.user_data.nb_matrix_processed = box_in.user_data.nb_matrix_processed + 1;

        [box_in, start_time, end_time, matrix_data] = OV_popInputBuffer(box_in,1);

        Fs = box_in.inputs{1}.header.sampling_rate; % Sampling frequency
        T = 1/Fs; % Sample time
        L = box_in.inputs{1}.header.nb_samples_per_buffer; % Length of signal
        t = (0:L-1)*T; % Time vector

        % We generate the requested sinusoid noise
        noise_amplitude = box_in.settings(2).value;
        noise_frequency = box_in.settings(1).value;
        sinusoid = noise_amplitude * sin(2*pi* noise_frequency *t);

        % We add this sinus to the original signal on first channel only
        sig = sinusoid + matrix_data(1,1:L);
        subplot(3,1,1);
        plot(Fs*t,sinusoid)
        title('Noise')
        xlabel('time (seconds)')

        subplot(3,1,2);
        plot(Fs*t,sig)
        title('Signal Corrupted with the noise (channel 1)')
        xlabel('time (seconds)')

        % we compute the FFT
        NFFT = 2^nextpow2(L); % Next power of 2 from length of y
        Y = fft(sig,NFFT)/L;
        f = Fs/2*linspace(0,1,NFFT/2+1);
        % Plot single-sided amplitude spectrum.
        plot_range_fmin = box_in.settings(3).value;
        plot_range_fmax = box_in.settings(4).value;
        subplot(3,1,3)
        plot(f(plot_range_fmin*2:plot_range_fmax*2),2*abs(Y(plot_range_fmin*2:plot_range_fmax*2)))
        title('Single-Sided Amplitude Spectrum of the corrupted signal (channel 1)')
        xlabel('Frequency (Hz)')
        ylabel('Amplitude')

        % we sum the spectrums for later mean computation
        if box_in.user_data.nb_matrix_processed == 1
           box_in.user_data.mean_fft_matrix = Y;
        else
           box_in.user_data.mean_fft_matrix = box_in.user_data.mean_fft_matrix + Y;
        end

        box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,sig);
    end

    box_out = box_in;
end

The header is set once, and he’s just the input header with one constraint : one channel only.

The FFT is plotted on every input chunk. If your input stream has a high frequency, you won’t be able to see something. In our scenario, we compute a 2 second window before passing it to Matlab, using Time Based Epoching. Therefore you will have one figure updated every 2 seconds of signal.

The figure plotted has 3 subfigures : the pure noise signal, the signal corrupted with this noise, and the amplitude spectrum.

Uninitialization script

Open the Matlab function file tuto2_FFT_filter_Uninitialize.m.

 tuto2_FFT_filter_Uninitialize.m(click here to show content)
 tuto2_FFT_filter_Uninitialize.m(click here to hide content)

% tuto2_FFT_filter_Uninitialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto2_FFT_filter_Uninitialize is called when pressing 'stop' in the scenario.
% We compute the mean FFT and plot it for 10 seconds.
%
function box_out = tuto2_FFT_filter_Uninitialize(box_in)
    disp('Uninitializing the box...')

    Fs = box_in.inputs{1}.header.sampling_rate; % Sampling frequency
    L = box_in.inputs{1}.header.nb_samples_per_buffer; % Length of signal

    NFFT = 2^nextpow2(L);
    f = Fs/2*linspace(0,1,NFFT/2+1);

    box_in.user_data.mean_fft_matrix = box_in.user_data.mean_fft_matrix / box_in.user_data.nb_matrix_processed;

    %% we close the previous figure window and plot the mean FFT between 5 and 50Hz.
    close(gcf);
    plot_range_fmin = box_in.settings(3).value;
    plot_range_fmax = box_in.settings(4).value;
    plot(f(plot_range_fmin*2:plot_range_fmax*2),2*abs(box_in.user_data.mean_fft_matrix(plot_range_fmin*2:plot_range_fmax*2)))
    title('MEAN Single-Sided Amplitude Spectrum of the corrupted signal (channel 1)')
    xlabel('Frequency (Hz)')
    ylabel('Amplitude')

    % We pause the execution for 10 seconds (to be able to see the figure before
    % the scenario is stopped and the Matlab engine closed)
    pause(10);

    box_out = box_in;
end

The Uninitialize function plots the mean spectrum amplitude matrix and pauses the execution for 10 seconds. After the pause, the scenario is stopped and the Matlab engine closed (so it closes any figure automatically).

Execution

Signal display in OpenViBE
Plotting the results in Matlab
Mean spectrum amplitudes

In the console, you can see the corresponding output :

 Console output(click here to show content)
 Console output(click here to hide content)

[ INF ] At time 0.000 sec <Box algorithm::Tuto2 : FFT>
---- MATLAB BUFFER - INFO ----
Box settings are:
 Noise frequency : 20
 Noise amplitude : 10
 Plot range : min frequency : 5
 Plot range : max frequency : 50
[ INF ] At time 0.094 sec <Box algorithm::Tuto2 : FFT>
---- MATLAB BUFFER - INFO ----
Input header is :
ans =
                  type: 'Signal Stream'
           nb_channels: 1
 nb_samples_per_buffer: 1024
         channel_names: {'CPz'}
         sampling_rate: 512
[ INF ] At time 7.688 sec <Box algorithm::Tuto2 : FFT>
---- MATLAB BUFFER - INFO ----
Uninitializing the box...

Tutorial 3 : Signal and Stimulation generator

Open the scenario share/openvibe/scenarios/box-tutorials/matlab/tuto3-sinus-generator.xml.

The Matlab Scripting box is configured to produce a sinusoidal signal and stimulations.

We added 4 new settings in the box : the channel count (integer), the samples per buffer count (integer), the amplitude of signal (float), and the stimulation sent every second.

In the current configuration, the box clock frequency is set to 32Hz. With 16 samples per buffer, we thus have a sampling frequency of 32*16 = 512Hz.

The signal is simply a different sinusoid for each channel, the frequency being a multiple of 10Hz (10Hz for first channel, 20Hz for second, etc.).

Initialization script

Open the Matlab function file tuto3_sinus_generator_Initialize.m.

 tuto3_sinus_generator_Initialize.m(click here to show content)
 tuto3_sinus_generator_Initialize.m(click here to hide content)

% tuto3_sinus_generator_Initialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
function box_out = tuto3_sinus_generator_Initialize(box_in)

   % we display the setting values
   disp('Box settings are:')
   for i=1:size(box_in.settings,2)
      fprintf('\t%s : %s\n',box_in.settings(i).name, num2str(box_in.settings(i).value));
   end

   % As we produce the signal ourselves, we already know the output header

   nb_channels = box_in.settings(1).value;
   channel_names = cell(1,nb_channels);
   for chan = 1 : nb_channels
      channel_names{chan} = sprintf('channel %i',chan);
   end

   nb_samples_per_buffer = box_in.settings(2).value;
   sampling_rate = box_in.clock_frequency * nb_samples_per_buffer;

   box_in = OV_setSignalOutputHeader(box_in, 1, nb_channels, nb_samples_per_buffer, channel_names, sampling_rate);
   box_in = OV_setStimulationOutputHeader(box_in,2);

   % end time of the last chunk sent
   box_in.user_data.last_end_time = 0;
   % number of call to the process function
   box_in.user_data.call_count = 0;
   % sample count
   box_in.user_data.nb_samples = 0;

   box_out = box_in;

end

This function prints the box settings and add fields in the user_data structure in the box : the end time of the last data chunk sent, a sample and a chunk counters. As we produce ourselves the signal and stimulation streams, we can set the headers in the initialization phase. For this purpose we use the functions of the openvibe toolbox.

Please note that we use the same variable box_in as output of these functions so we keep working on an updated box structure.

Process script

Open the Matlab function file tuto3_sinus_generator_Process.m.

 tuto3_sinus_generator_Process.m(click here to show content)
 tuto3_sinus_generator_Process.m(click here to hide content)

% tuto3_sinus_generator_Process.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
function box_out = tuto3_sinus_generator_Process(box_in)

   channel_count = box_in.settings(1).value;
   samples_per_buffer = box_in.settings(2).value;
   amplitude = box_in.settings(3).value;
   stimulation = box_in.settings(4).value;

   start_time = box_in.user_data.last_end_time;
   end_time = box_in.clock;
   % We produce data of the past step (e.g. at t2 we produce the data of [t1,t2]), thus we skip the first call
   if start_time ~= 0
      T = 1/box_in.outputs{1}.header.sampling_rate;
      t = (box_in.user_data.nb_samples:box_in.user_data.nb_samples+samples_per_buffer-1)*T;
      signal = zeros(channel_count, samples_per_buffer);
      freq = 10;
      for chan = 1 : channel_count
         sinus = amplitude * sin(2*pi* freq *t);
         signal(chan,1:samples_per_buffer) = sinus;
         freq = freq + 10;
      end

      if mod(box_in.user_data.chunk_count, box_in.clock_frequency) == 0
         stim_set = [stimulation; box_in.clock; 0];
         disp('Sending a stimulation...');
      else
         stim_set = [];
      end

      box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,signal);
      box_in = OV_addOutputBuffer(box_in,2,start_time,end_time,stim_set);

      box_in.user_data.nb_samples = box_in.user_data.nb_samples + samples_per_buffer;
   end

   box_in.user_data.last_end_time = end_time;
   box_in.user_data.call_count = box_in.user_data.call_count +1;

   box_out = box_in;
end

We produce a N-by-M signal matrix. One channel per line, samples_per_buffer columns. Along with each chunk of signal, we also produce a stimulation set. Most of the time it’s an empty set, but it’s still important as the other box after our generator may need to know if there is no stimulation is the last time range. As you notice, a stimulation set is made of N columns, one stimulation per column (stim identifier, date then duration). This matrix structure allows the box to parse all the stimulations in a set directly, as the matrix is indexed column after column.

Uninitialization script

Open the Matlab function file tuto3_sinus_generator_Uninitialize.m.

 tuto3_sinus_generator_Uninitialize.m(click here to show content)
 tuto3_sinus_generator_Uninitialize.m(click here to hide content)

% tuto3_sinus_generator_Uninitialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto3_sinus_generator_Uninitialize is called when pressing 'stop' in the scenario.
%
function box_out = tuto3_sinus_generator_Uninitialize(box_in)
   disp('Uninitializing the box...')

   box_out = box_in;
end

As you can see the Uninitialize function does nothing.

Execution

The box generates some sinusoidal signal automatically, along with stimulations (one every second – see the red lines):

Sinus Generator

In the console, you can see the corresponding output :

 Console output(click here to show content)
 Console output(click here to hide content)

[ INF ] At time 0.000 sec <Box algorithm::Tuto3 : sinus generator>
---- MATLAB BUFFER - INFO ----
Box settings are:
 Channel count : 4
 Samples per buffer : 16
 Amplitude : 5
 Stimulation : 33025
[ INF ] At time 1.000 sec <Box algorithm::Tuto3 : sinus generator>
---- MATLAB BUFFER - INFO ----
Sending a stimulation...
[ INF ] At time 2.000 sec <Box algorithm::Tuto3 : sinus generator>
---- MATLAB BUFFER - INFO ----
Sending a stimulation...
[ INF ] At time 3.000 sec <Box algorithm::Tuto3 : sinus generator>
---- MATLAB BUFFER - INFO ----
Sending a stimulation...
[ INF ] At time 4.000 sec <Box algorithm::Tuto3 : sinus generator>
---- MATLAB BUFFER - INFO ----
Sending a stimulation...
[ INF ] At time 4.602 sec <Box algorithm::Tuto3 : sinus generator>
---- MATLAB BUFFER - INFO ----
Uninitializing the box...

Troubleshooting

In this section, we review the most common errors and mistakes and what they can cause.

Please contact us on the regular channels if you find any unexpected behavior or bugs so we can either correct the Matlab boxes or add new items in this section.

Failed to Load Matlab Engine

Sometimes the Matlab engine fails to open because there is not an available Matlab license. Another reason can be because the operation gets confused with multiple Matlab installations on the computer. It can also fail due to the found Matlab being 64bit instead of 32bit as required. Unfortunately, the Matlab API call engOpen() used to connect to the Matlab engine does not return any information helping to resolve the problem. One way to find out if the problem is in OpenViBE, is to locate C++ tutorial examples from the internet that use engOpen(), and see if you can get these tutorials to work.

Failed to Load Matlab Engine at first try

With the Windows Installer version, we identified an incorrect behavior of the box : when pressing play for the first try, the scenario won’t find Matlab and the box will fail to execute correctly. When re-playing the scenario by pressing play once again, the Matlab Engine (if everything is correctly set) should successfully run.

This bug does not happen with an OpenViBE built from sources. Investigations are in progress.

On Linux, MATLAB box keeps saying ‘Can’t start MATLAB engine’, what do I do ?

In order to get MATLAB box to work properly on Linux, you may need csh to be installed. This is not always the case, notably on Ubuntu distributions.

On Linux, loading the Matlab plugin says ‘error:libmx.so: cannot open shared object file: No such file or directory’?

Before launching Designer, you must export LD_LIBRARY_PATH to include the path where the Matlab .so files are installed.

Matlab License

The Matlab Engine can be launched in background only if it can access a valid Matlab License.

If the Matlab License Manager can’t get a valid License, it will output an error window. The OpenViBE Designer would then be still waiting for the Matlab Engine to respond: you can expect the application to freeze for about 30 seconds. After this timeout, the call to Matlab engine fails and the following message would be printed in the console:

[ ERROR ] At time 0.000 sec <Box algorithm::Matlab Scripting> Could not open Matlab engine
[WARNING] Box algorithm <Matlab Scripting> has been deactivated because initialization phase returned bad status

The box being deactivated, it does not process any data but the scenario would keep running without it.

Output Headers : be sure to set it right !

If you try to send a new buffer on an output and you get this message for example :

[ ERROR ] At time 0.094 sec <Box algorithm::Signal filter> Error calling [OV_getSignalOutputHeader]. Did you correctly set
          the output header in the matlab structure ?

It means that you didn’t set the header (here it’s a signal output) correctly. The box is not able to construct the OpenViBE output stream.

Solution : set the header before the very first buffer sent. You can either use an existing header, as it is done in tutorial 1, or the dedicated Matlab functions as in tutorial 3.

If you construct a somehow corrupted header or a header that does not match the theoretical stream structure, the application has a good chance of crash.

For example, if you omit a field or use a field incorrectly (e.g. header.sampling_rate = 'plop') the Designer can just randomly crash during execution.  We are aware of this problem and we will try to improve error handling in the following updates.

Be sure to save your OpenViBE scenarios often.

Manually closing the Matlab engine

The Matlab Scripting box opens the Matlab Engine on initialization (play) and closes it when uninitializing (stop).  If an error occurs the Matlab Engine may not be closed correctly. This means that when you will run a scenario with a Matlab Scripting box once again, it will use the previous engine, still in the state you left it ! This can lead to unexpected behavior (e.g. if you have some global variables you are using in your script, they will still be there).

The simplest way to check the state of the Matlab Engine is by looking at the matlab window running in background. When opening the engine, Matlab launch a specific window called Matlab Command Window. This window is a basic Matlab interpreter in which you can type Matlab commands. When closing the Engine, this window should be closed automatically. If such window is still alive when there is no Matlab scenario running in the OpenViBE Designer, close it manually before playing such scenario.

Performance issues

Depending on the processing you implemented the Matlab scripting boxes may take some time to do the process. If a box spends too much time its process loop, a message would be printed in the console:

[WARNING] <Player::can not reach realtime> 1 second(s) late...
[WARNING] <Player::can not reach realtime> 2 second(s) late...
[WARNING] <Player::can not reach realtime> 3 second(s) late...
[WARNING] <Player::can not reach realtime> 4 second(s) late...
[WARNING] <Player::can not reach realtime> 5 second(s) late...

Meaning that the OpenViBE Player, that schedules the process of every boxes, is not able to process the data in real-time. For example, if you set the Matlab scripting box clock frequency to 32Hz, one process call should be less than 1/32 th second long. Otherwise, the process loop accumulates a delay.

Try to lower the clock frequency to extend the period of time between 2 process calls. The Matlab process function may have to process more chunk of data at a time (internal buffers are filled continuously), but you will save the “interface time”, between the C++ code and the Matlab engine.

 

This entry was posted in Tutorials and tagged , . Bookmark the permalink.