- NB: cursory update for OpenViBE 2.0.0 (doc updated 22.Nov.2018)
Note : the Matlab Scripting box plugin for OpenViBE has been developed for researchers that are more experienced with Matlab scripting than C++ programming. However, at Inria Rennes where the box has been developed, we rarely use OpenViBE and Matlab together. As a result, our experience with how the box works in practice is limited. We are happy to receive any comments about it. Please send us your feedback and suggestion of improvement. Bug reports, usability feedback, documentation requests, etc. are all welcome.
Introduction
OpenViBE is currently compatible with MATLAB through a box called Matlab Scripting. This is a general-purpose box with unlimited number of inputs, outputs and settings. User writes the Matlab functions that will be called by the box: Initialize function will be run once when user presses play, Process will be run repeatedly, and Uninitialize when the scenario stops.
This box is developed and supported by the Inria team.
Requirements
The box relies on the Matlab Engine and requires a valid installation of Matlab to be used. To be precise, the Matlab documentation states :
The Matlab Scripting box is included in the official OpenViBE release packages, and thus can be used directly in any scenario, as long as you have a valid Matlab installation on your computer.
OpenViBE runs on the operating systems and architectures listed here. However, there are requirements regarding the suitable versions of Matlab.
- On Windows, if you downloaded 32bit OpenViBE (2.1.0 and earlier),
- The box requires 32 bit Matlab components installed. For example, Matlab 2014a still has the 32bit support. In this case, the path to Matlab will look like “
C:/Program Files (x86)/MATLAB/R2014a/bin/win32/MATLAB.exe“. If either (x86) or ‘win32’ are not part of the path, that may be sign of a potential issue. We have successfully used the Matlab Scripting box with 32bit OpenViBEs from Matlab 2011 to Matlab 2014a at least.
- The box requires 32 bit Matlab components installed. For example, Matlab 2014a still has the 32bit support. In this case, the path to Matlab will look like “
- On Windows, if you downloaded 64bit OpenViBE (option since 2.2.0),
- The box requires 64 bit Matlab components installed. In this case, the path to Matlab will look like “
C:/Program Files/MATLAB/R2015b/bin/win64/MATLAB.exe“.
- The box requires 64 bit Matlab components installed. In this case, the path to Matlab will look like “
- On Linuxes, OpenViBE is compiled with the same ‘bitness’ as the operating system. We cannot give recommendations what versions of Matlab work or not. Start by trying the one you have.
Please inform us of any compatibility issue. We will update this section with your feedback!
The Matlab Scripting box
The Matlab Scripting box is available in the Designer in the category Scripting.
Here are the different settings and how to set them correctly :
- Box clock frequency (in Hertz) : defines the frequency on which the box will be called by OpenViBE, from 1 to 128 Hz. Default value is 64Hz. You can lower the value if you experience bad performance when executing the box.
- Path to Matlab executables (Windows) / Matlab launch command (Linux): tells OpenViBE how to execute the Matlab engine. Set it to the directory where is matlab.exe on Windows or type the command you use to launch Matlab on Linux. Note that in OpenViBE all paths must be given with forward slashes (/).
- Matlab working directory : the directory where all your .m function files are located. This path must be Matlab compliant (i.e. some special characters are not recognized)
- Initialize function : the name of the Matlab function called by the box when pressing ‘play’ in the scenario.
- Process function : the name of the Matlab function called by the box on every box clock tick.
- Uninitialize function : the name of the Matlab function called by the box when pressing ‘stop’ in the scenario.
The most important thing is to get the Matlab path right. On Windows, this path will look something like “C:/Program Files (x86)/MATLAB/R2014a/bin/win32/MATLAB.exe” (for Matlab 2014a). These paths can change a little from Matlab version to another.
You can add as many inputs and outputs as you need. These connections can be configured with the usual stream types (especially Signal, Streamed Matrix, Stimulations, Feature Vector). You can also add new settings to the box. The values of these settings will be available on the Matlab side as well.
You can use more than one Matlab Scripting box in the same scenario. Each box has its own matlab workspace defined in the box setting.
If you are using the same workspace for several boxes, they will share this workspace. For example a global variable could be modified by all the boxes of the same workspace.
The OpenViBE toolbox for the Matlab Scripting box
Note: This section is very technical and may be skipped on the first read. The reader is instead recommend to take a look at the tutorial Matlab script implementations below and then refer to this section for explanations where needed.
The Matlab Scripting box relies on a set of Matlab functions to work. These functions are installed with openvibe in share/openvibe/plugins/matlab.
The folder contains 2 types of functions: those called by the box itself internally, and those who are meant to be called by the user in his script. In this section we describe these functions and the structures manipulated by Matlab and OpenViBE.
Setting up the environment
The box automatically sets up the Matlab environment to work properly. For your information, here is the procedure:
The box first creates a representation of the openvibe box in matlab:
box = OV_createBoxInstance(input_count, output_count) :
out STRUCT box : a structure representing the box in Matlab:
in DOUBLE input_count : the number of input of the box, to create the structures accordingly.
in DOUBLE output_count : the number of output of the box, to create the structures accordingly.
Then it calls a function that will create several global variables for all the OpenViBE stimulation codes:
OV_defines()
The name of the variables and their values are strictly the same as in OpenViBE. For example:
global OVTK_StimulationId_Label_00;
OVTK_StimulationId_Label_00 = uint64(hex2dec('00008100')); // 33024
Thus you can directly compare a stimulation identifier received on an input to these global variables.
Matlab structures and functions description
One instance of the Matlab Scripting box creates its own box in Matlab. This box is passed as an argument of every functions. For the functions that outputs a new box instance, the same variable is used and thus is updated.
The box structure is defined as follow:
STRUCT box: > box.clock : DOUBLE the current openvibe time, updated at every clock tick. > box.clock_frequency : DOUBLE the box clock frequency. > box.settings : an array of setting structure(s). > box.inputs : a cell array of input structure(s). > box.outputs : a cell array of output structure(s). > box.user_data : empty user-space structure.
The box clock and clock frequency are very useful when your script needs to generate an output, but lacks an input to synchronize with.
The setting structures are filled according to the openvibe box settings defined in the scenario. Default settings are not stored in the structure.
STRUCT setting :
> setting.name : STRING (name of the setting in the openvibe box).
> setting.type : UINT64 openvibe code of the setting type.
> setting.value : the value of the setting. Its type can be DOUBLE (for the openvibe types integer and float),
LOGICAL (boolean), UINT64 (stimulation code) or STRING (anything else).
The input and output structures are defined by particular headers, and are meant to be filled at run time. When the Matlab Scripting box receives the headers on its inputs it forwards the information to Matlab, and modify the box representation accordingly.
The output headers structure must be set by the user in the Matlab script. Once the header is set, the Matlab Scripting box is able to get this information from the matlab structure and construct the corresponding OpenViBE header.
The IO structures also contains a buffer for the incoming chunk of data.
STRUCT input/output : > io.header : STRUCT containing the stream description (e.g. for a signal: channel names, sampling rate, etc). > io.buffer : CELL ARRAY of chunk structure(s).
The different header structures are listed below, along with the corresponding functions to set it in a convenient way (i.e. you won’t miss a field). Manually setting the header structures of your output is of course also possible (e.g. if your output stream has the same structure as an input, input.header = output.header is the simplest way to set it).
NB: A corresponding Getter function also exists for each header type. These functions are called by the Matlab Scripting box to get all the information in one call to the Matlab engine.
STRUCT header : STREAMED MATRIX > header.type : STRING = 'Streamed Matrix Stream' > header.nb_dimensions : DOUBLE number of dimensions of the matrix (N) > header.dimension_sizes : 1xN DOUBLE ARRAY of dimension sizes (one size per dimension) > header.dimension_labels : 1xN CELL ARRAY of STRING Setter Function (optional): [box_out] = OV_setStreamedMatrixOutputHeader(box_in, output_index, nb_dim, dim_sizes, dim_labels)
STRUCT header : SIGNAL > header.type : STRING = 'Signal Stream' > header.nb_channels : DOUBLE number of channels > header.nb_samples_per_buffer : DOUBLE number of samples per buffer block > header.channel_names : CELL ARRAY of STRING > header.sampling_rate : DOUBLE sampling frequency Setter Function (optional): [box_out] = OV_setSignalOutputHeader(box_in, output_index, nb_channel, nb_samples_per_buffer, channel_names, sampling_rate)
STRUCT header : FEATURE VECTOR > header.type : STRING = 'Feature Vector Stream' > header.nb_features : DOUBLE number of feature in the vector > header.labels : CELL ARRAY of STRING Setter Function (optional): [box_out] = OV_setFeatureVectorOutputHeader(box_in, output_index, nb_features, labels)
STRUCT header : SPECTRUM > header.type : STRING = 'Spectrum Stream' > header.nb_channels : DOUBLE number of channels > header.channel_names : CELL ARRAY of STRING > header.nb_bands : DOUBLE number of frequency bands in the spectrum > header.band_names : CELL ARRAY of STRING the name of the bands (e.g. '10-15Hz') > header.bands : [nb_bands x 2 DOUBLE] the pairs of frequencies (band start and stop) Setter Function (optional): [box_out] = OV_setSpectrumOutputHeader(box_in, output_index, nb_channels, channel_names, nb_bands, band_names, bands)
STRUCT header : CHANNEL LOCALISATION > header.type : STRING = 'Channel Localisation Stream' > header.nb_channels : DOUBLE number of channels > header.channel_names : CELL ARRAY of STRING > header.dynamic : LOGICAL tells if coordinates are dynamic Setter Function (optional): [box_out] = OV_setChannelLocalisationOutputHeader(box_in, output_index, nb_channels, channel_names, dynamic)
STRUCT header : STIMULATIONS > header.type : STRING = 'Stimulation Stream' Setter Function (optional): [box_out] = OV_setStimulationsOutputHeader(box_in)
As you can see all the setter functions take a box structure as input, and output the modified box structure. A typical call is thus:
box = OV_setXXXXXOutputHeader(box, ...) % ... continue using the 'box' variable ...
The chunk structure stored in the IO buffers can be either a matrix (for Stramed Matrix stream and its child streams) or a stimulation set.
STRUCT chunk : MATRIX CHUNK
> chunk.start_time : DOUBLE translation of the openvibe timestamp in seconds, precision 0.0001 sec.
> chunk.end_time : DOUBLE
> chunk.matrix_data : DOUBLE ARRAY the matrix of data.
The structure of the matrix is the same as in OpenViBE.
For example, a signal matrix has N(channels) lines and N(samples) columns.
STRUCT chunk : SIMULATION CHUNK
> chunk.start_time : DOUBLE translation of the openvibe timestamp in seconds, precision 0.0001 sec.
> chunk.end_time : DOUBLE
> chunk.matrix_data : DOUBLE ARRAY the stimulation set containing N stimulation(s).
N stimulations are stored in a 3xN matrix, ie. 1 stimulation per column.
Each stimulation is defined by its code, date and duration.
The buffers may contains any number of chunks. The buffers can be manipulated through convenient matlab functions:
% To get the current count of pending chunk in buffer:
nb_pending = OV_getNbPendingInputChunk(box_in, input_index)
out DOUBLE nb_pending : the number of pending chunk in the buffer.
in STRUCT box_in : the box structure.
in DOUBLE input_index : index of the input.
% To pop the oldest chunk in a buffer (i.e. the first chunk to process)
% Note that the chunk is removed from the buffer after this call.
[box_out, start_time, end_time, matrix_data] = OV_popInputBuffer(box_in, input_index)
out STRUCT box_out : the Matlab box after modification.
out DOUBLE start_time : the chunk start time in seconds.
out DOUBLE end_time : the chunk end time in seconds.
out ARRAY matrix_data : the data, type and structure depends on the stream type.
in STRUCT box_in : the Matlab box from which we pop a new chunk.
in DOUBLE input_index : the input index in the box.
If there is no chunk in the buffer the function returns:
> start_time = -1
> end_time = -1
> matrix_data = []
When you need to output a new buffer, you can do so by calling the following function:
% To add a new chunk in an output buffer:
[box_out] = OV_addOutputBuffer(box_in,output_index,start_time,end_time,matrix_data);
out DOUBLE box_out : the box after modification.
in STRUCT box_in : the box structure.
in DOUBLE output_index : index of the input.
in DOUBLE start_time : the chunk start time in seconds.
in DOUBLE end_time : the chunk end time in seconds.
in ARRAY matrix_data : the data, type and structure depends on the stream type
Tutorial 1 : signal filter
Open the scenario share/openvibe/scenarios/box-tutorials/matlab/tuto1-signal-filter.xml.
This scenario is very simple : it filters a sinus signal and sets all the samples to 0 when receiving a specific stimulation. If the same stimulation is received once again, the output signal goes back to normal.
The Matlab Scripting box takes 2 inputs : the signal and the stimulation stream. It outputs a signal which has the same structure (channel count, etc.) as the input.
First step is to configure the Matlab Scripting box. You need to set your local Matlab directory and the working directory to match your local installation of Matlab and OpenViBE. The working directory is [openvibe]/share/openvibe/scenarios/box-tutorials/matlab/. It must be an absolute path. The three matlab functions are in this directory.
Finally, as you can notice we added a new setting named “zero trigger” of type Stimulation. This defines which stimulation is used to switch the behavior of the box.
Configuring the Matlab Scripting BoxInitialization script
Open the Matlab function file tuto1_signal_filter_Initialize.m.
% tuto1_signal_filter_Initialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto1_signal_filter_Initialize is called when pressing 'play' in the scenario.
%
function box_out = tuto1_signal_filter_Initialize(box_in)
disp('Matlab initialize function has been called.')
% we add a field to save the trigger state
box_in.user_data.trigger_state = false;
% Don't forget to pass the modified box as output.
box_out = box_in;
end
This function add a field in the user_data structure in the box : a boolean indicating the current state of the trigger. Please note that we must output the modified box to be able to use it afterwards. This function will be called once, when pressing “play” in the scenario.
Process script
Open the Matlab function file tuto1_signal_filter_Process.m.
% tuto1_signal_filter_Process.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto1_signal_filter_Process is called at the clock frequency.
%
function box_out = tuto1_signal_filter_Process(box_in)
% we iterate over the pending chunks on input 2 (STIMULATIONS)
for i = 1: OV_getNbPendingInputChunk(box_in,2)
% we pop the first chunk to be processed, note that box_in is used as
% the output variable to continue processing
[box_in, start_time, end_time, stim_set] = OV_popInputBuffer(box_in,2);
% the stimulation set is a 3xN matrix.
% The openvibe stimulation stream even sends empty stimulation sets
% so the following boxes know there is no stimulation to expect in
% the latest time range. These empty chunk are also in the matlab buffer.
if(numel(stim_set) >= 3) % at least one stim in the set.
fprintf('Received stimulation code %i at time %f\n', stim_set(1), stim_set(2))
if stim_set(1) == box_in.settings(1).value % I'm lazy... I only check the first stim in the set.
box_in.user_data.trigger_state = ~box_in.user_data.trigger_state;
disp('Trigger is switched.')
end
end
end
% we iterate over the pending chunks on input 1 (SIGNAL)
for i = 1: OV_getNbPendingInputChunk(box_in,1)
% we pop the first chunk to be processed, note that box_in is used as the output
% variable to continue processing
[box_in, start_time, end_time, matrix_data] = OV_popInputBuffer(box_in,1);
% if requested, all samples are set to 0
if(box_in.user_data.trigger_state)
matrix_data(1:end) = 0;
end
% we add the chunk to the signal output buffer. Note that we use box_in as the output variable.
% The header of the output must be set prior to sending the first buffer.
% In our case the headers are the same.
box_in.outputs{1}.header = box_in.inputs{1}.header;
% The chunk dates are the same as the input chunk
box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,matrix_data);
end
% Pass the modified box as output to continue processing
box_out = box_in;
end
As we set the box clock frequency to 64 Hz, this function will be called 64 time per second. It works in two steps. First we check the stimulation input for new triggers, and switch the box field accordingly. Note that every time we pop a new input chunk, we use the current box instance as output, to continue the processing on an updated box structure. In this case, the “pop” functions remove the last input chunk from the buffer.
As we only modify the samples values of the input signal, we set the output header as a copy of the input header. This will be done with every process but it is only needed once (prior to the first buffer ever sent).
Uninitialization script
Open the Matlab function file tuto1_signal_filter_Uninitialize.m.
% tuto1_signal_filter_Uninitialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto1_signal_filter_Uninitialize is called when pressing 'stop' in the scenario.
%
function box_out = tuto1_signal_filter_Uninitialize(box_in)
disp('Matlab uninitialize function has been called.')
box_out = box_in;
end
We don’t need to do anything when stopping the scenario, we just display a message.
Execution
The result can be found below. The signal is set to 0 when the stimulation OVTK_StimulationId_Label_01 (33024) is received.
In the console, you can see the corresponding output :
[ INF ] At time 0.000 sec <Box algorithm::Signal filter> ---- MATLAB BUFFER - INFO ---- Matlab initialize function has been called.
[ INF ] At time 1.719 sec <Box algorithm::Signal filter> ---- MATLAB BUFFER - INFO ---- Received stimulation code 33025 at time 1.687500 Trigger is switched.
[ INF ] At time 2.156 sec <Box algorithm::Signal filter> ---- MATLAB BUFFER - INFO ---- Received stimulation code 33024 at time 2.125000
[ INF ] At time 4.094 sec <Box algorithm::Signal filter> ---- MATLAB BUFFER - INFO ---- Received stimulation code 33025 at time 4.062500 Trigger is switched.
[ INF ] At time 4.938 sec <Box algorithm::Signal filter> ---- MATLAB BUFFER - INFO ---- Received stimulation code 33024 at time 4.906250
[ INF ] At time 9.172 sec <Box algorithm::Signal filter> ---- MATLAB BUFFER - INFO ---- Matlab uninitialize function has been called.
Tutorial 2 : FFT and Matlab plot
Open the scenario share/openvibe/scenarios/box-tutorials/matlab/tuto2-FFT-filter.xml.
In this scenario we add some noise in the upcoming signal, and plot the spectrum amplitude (FFT) in Matlab to check if this noise appears in the signal.
Please note that we only do this process on the first channel of the signal, thus the output signal has only one channel. At the end, we will plot the mean spectrum before stopping the scenario.
First step is to configure the Matlab Scripting box. You need to set your local Matlab directory and the working directory to match your local installation of Matlab and OpenViBE. The working directory is [openvibe]/share/openvibe/scenarios/box-tutorials/matlab/. It must be an absolute path. The three matlab functions are in this directory.
We added 4 FLOAT settings : the noise frequency and amplitude, that will be added in the signal, and the frequency range used for plotting the spectrum.
Initialization script
Open the Matlab function file tuto2_FFT_filter_Initialize.m.
% tuto2_FFT_filter_Initialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 28/07/2011
%
% This file contains the function called once by the matlab box on its initialization (when playing the scenario).
function box_out = tuto2_FFT_filter_Initialize(box_in)
% we display the setting values
disp('Box settings are:')
for i=1:size(box_in.settings,2)
fprintf('\t%s : %s\n',box_in.settings(i).name, num2str(box_in.settings(i).value));
end
% let's add a user-defined indicator to know if the output header is set
box_in.user_data.is_headerset = false;
% We also add some statistics
box_in.user_data.nb_matrix_processed = 0;
box_in.user_data.mean_fft_matrix = 0;
box_out = box_in;
end
This function prints the box settings and add fields in the user_data structure in the box : a boolean indicating if we need to set the output header, and the variables we need to compute the mean spectrum. Please note that we must output the modified box to be able to use it afterwards. This function will be called once, when pressing “play” in the scenario.
Process script
Open the Matlab function file tuto2_FFT_filter_Process.m.
% tuto2-FFT-filter_Process.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 28/07/2011
%
% This function adds a noise in the input signal and plot the resulting FFT
function box_out = tuto2_FFT_filter_Process(box_in)
for i = 1: OV_getNbPendingInputChunk(box_in,1)
if(~box_in.user_data.is_headerset)
% The output is the input + noise, only on first channel
box_in.outputs{1}.header = box_in.inputs{1}.header;
box_in.outputs{1}.header.nb_channels = 1;
box_in.user_data.is_headerset = 1;
% We print the header in the console
disp('Input header is :')
box_in.inputs{1}.header
end
% we increment the matrix count
box_in.user_data.nb_matrix_processed = box_in.user_data.nb_matrix_processed + 1;
[box_in, start_time, end_time, matrix_data] = OV_popInputBuffer(box_in,1);
Fs = box_in.inputs{1}.header.sampling_rate; % Sampling frequency
T = 1/Fs; % Sample time
L = box_in.inputs{1}.header.nb_samples_per_buffer; % Length of signal
t = (0:L-1)*T; % Time vector
% We generate the requested sinusoid noise
noise_amplitude = box_in.settings(2).value;
noise_frequency = box_in.settings(1).value;
sinusoid = noise_amplitude * sin(2*pi* noise_frequency *t);
% We add this sinus to the original signal on first channel only
sig = sinusoid + matrix_data(1,1:L);
subplot(3,1,1);
plot(Fs*t,sinusoid)
title('Noise')
xlabel('time (seconds)')
subplot(3,1,2);
plot(Fs*t,sig)
title('Signal Corrupted with the noise (channel 1)')
xlabel('time (seconds)')
% we compute the FFT
NFFT = 2^nextpow2(L); % Next power of 2 from length of y
Y = fft(sig,NFFT)/L;
f = Fs/2*linspace(0,1,NFFT/2+1);
% Plot single-sided amplitude spectrum.
plot_range_fmin = box_in.settings(3).value;
plot_range_fmax = box_in.settings(4).value;
subplot(3,1,3)
plot(f(plot_range_fmin*2:plot_range_fmax*2),2*abs(Y(plot_range_fmin*2:plot_range_fmax*2)))
title('Single-Sided Amplitude Spectrum of the corrupted signal (channel 1)')
xlabel('Frequency (Hz)')
ylabel('Amplitude')
% we sum the spectrums for later mean computation
if box_in.user_data.nb_matrix_processed == 1
box_in.user_data.mean_fft_matrix = Y;
else
box_in.user_data.mean_fft_matrix = box_in.user_data.mean_fft_matrix + Y;
end
box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,sig);
end
box_out = box_in;
end
The header is set once, and he’s just the input header with one constraint : one channel only.
The FFT is plotted on every input chunk. If your input stream has a high frequency, you won’t be able to see something. In our scenario, we compute a 2 second window before passing it to Matlab, using Time Based Epoching. Therefore you will have one figure updated every 2 seconds of signal.
The figure plotted has 3 subfigures : the pure noise signal, the signal corrupted with this noise, and the amplitude spectrum.
Uninitialization script
Open the Matlab function file tuto2_FFT_filter_Uninitialize.m.
% tuto2_FFT_filter_Uninitialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto2_FFT_filter_Uninitialize is called when pressing 'stop' in the scenario.
% We compute the mean FFT and plot it for 10 seconds.
%
function box_out = tuto2_FFT_filter_Uninitialize(box_in)
disp('Uninitializing the box...')
Fs = box_in.inputs{1}.header.sampling_rate; % Sampling frequency
L = box_in.inputs{1}.header.nb_samples_per_buffer; % Length of signal
NFFT = 2^nextpow2(L);
f = Fs/2*linspace(0,1,NFFT/2+1);
box_in.user_data.mean_fft_matrix = box_in.user_data.mean_fft_matrix / box_in.user_data.nb_matrix_processed;
%% we close the previous figure window and plot the mean FFT between 5 and 50Hz.
close(gcf);
plot_range_fmin = box_in.settings(3).value;
plot_range_fmax = box_in.settings(4).value;
plot(f(plot_range_fmin*2:plot_range_fmax*2),2*abs(box_in.user_data.mean_fft_matrix(plot_range_fmin*2:plot_range_fmax*2)))
title('MEAN Single-Sided Amplitude Spectrum of the corrupted signal (channel 1)')
xlabel('Frequency (Hz)')
ylabel('Amplitude')
% We pause the execution for 10 seconds (to be able to see the figure before
% the scenario is stopped and the Matlab engine closed)
pause(10);
box_out = box_in;
end
The Uninitialize function plots the mean spectrum amplitude matrix and pauses the execution for 10 seconds. After the pause, the scenario is stopped and the Matlab engine closed (so it closes any figure automatically).
Execution
Signal display in OpenViBE
Plotting the results in Matlab
Mean spectrum amplitudesIn the console, you can see the corresponding output :
[ INF ] At time 0.000 sec <Box algorithm::Tuto2 : FFT> ---- MATLAB BUFFER - INFO ---- Box settings are: Noise frequency : 20 Noise amplitude : 10 Plot range : min frequency : 5 Plot range : max frequency : 50
[ INF ] At time 0.094 sec <Box algorithm::Tuto2 : FFT>
---- MATLAB BUFFER - INFO ----
Input header is :
ans =
type: 'Signal Stream'
nb_channels: 1
nb_samples_per_buffer: 1024
channel_names: {'CPz'}
sampling_rate: 512
[ INF ] At time 7.688 sec <Box algorithm::Tuto2 : FFT> ---- MATLAB BUFFER - INFO ---- Uninitializing the box...
Tutorial 3 : Signal and Stimulation generator
Open the scenario share/openvibe/scenarios/box-tutorials/matlab/tuto3-sinus-generator.xml.
The Matlab Scripting box is configured to produce a sinusoidal signal and stimulations.
We added 4 new settings in the box : the channel count (integer), the samples per buffer count (integer), the amplitude of signal (float), and the stimulation sent every second.
In the current configuration, the box clock frequency is set to 32Hz. With 16 samples per buffer, we thus have a sampling frequency of 32*16 = 512Hz.
The signal is simply a different sinusoid for each channel, the frequency being a multiple of 10Hz (10Hz for first channel, 20Hz for second, etc.).
Initialization script
Open the Matlab function file tuto3_sinus_generator_Initialize.m.
% tuto3_sinus_generator_Initialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
function box_out = tuto3_sinus_generator_Initialize(box_in)
% we display the setting values
disp('Box settings are:')
for i=1:size(box_in.settings,2)
fprintf('\t%s : %s\n',box_in.settings(i).name, num2str(box_in.settings(i).value));
end
% As we produce the signal ourselves, we already know the output header
nb_channels = box_in.settings(1).value;
channel_names = cell(1,nb_channels);
for chan = 1 : nb_channels
channel_names{chan} = sprintf('channel %i',chan);
end
nb_samples_per_buffer = box_in.settings(2).value;
sampling_rate = box_in.clock_frequency * nb_samples_per_buffer;
box_in = OV_setSignalOutputHeader(box_in, 1, nb_channels, nb_samples_per_buffer, channel_names, sampling_rate);
box_in = OV_setStimulationOutputHeader(box_in,2);
% end time of the last chunk sent
box_in.user_data.last_end_time = 0;
% number of call to the process function
box_in.user_data.call_count = 0;
% sample count
box_in.user_data.nb_samples = 0;
box_out = box_in;
end
This function prints the box settings and add fields in the user_data structure in the box : the end time of the last data chunk sent, a sample and a chunk counters. As we produce ourselves the signal and stimulation streams, we can set the headers in the initialization phase. For this purpose we use the functions of the openvibe toolbox.
Please note that we use the same variable box_in as output of these functions so we keep working on an updated box structure.
Process script
Open the Matlab function file tuto3_sinus_generator_Process.m.
% tuto3_sinus_generator_Process.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
function box_out = tuto3_sinus_generator_Process(box_in)
channel_count = box_in.settings(1).value;
samples_per_buffer = box_in.settings(2).value;
amplitude = box_in.settings(3).value;
stimulation = box_in.settings(4).value;
start_time = box_in.user_data.last_end_time;
end_time = box_in.clock;
% We produce data of the past step (e.g. at t2 we produce the data of [t1,t2]), thus we skip the first call
if start_time ~= 0
T = 1/box_in.outputs{1}.header.sampling_rate;
t = (box_in.user_data.nb_samples:box_in.user_data.nb_samples+samples_per_buffer-1)*T;
signal = zeros(channel_count, samples_per_buffer);
freq = 10;
for chan = 1 : channel_count
sinus = amplitude * sin(2*pi* freq *t);
signal(chan,1:samples_per_buffer) = sinus;
freq = freq + 10;
end
if mod(box_in.user_data.chunk_count, box_in.clock_frequency) == 0
stim_set = [stimulation; box_in.clock; 0];
disp('Sending a stimulation...');
else
stim_set = [];
end
box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,signal);
box_in = OV_addOutputBuffer(box_in,2,start_time,end_time,stim_set);
box_in.user_data.nb_samples = box_in.user_data.nb_samples + samples_per_buffer;
end
box_in.user_data.last_end_time = end_time;
box_in.user_data.call_count = box_in.user_data.call_count +1;
box_out = box_in;
end
We produce a N-by-M signal matrix. One channel per line, samples_per_buffer columns. Along with each chunk of signal, we also produce a stimulation set. Most of the time it’s an empty set, but it’s still important as the other box after our generator may need to know if there is no stimulation is the last time range. As you notice, a stimulation set is made of N columns, one stimulation per column (stim identifier, date then duration). This matrix structure allows the box to parse all the stimulations in a set directly, as the matrix is indexed column after column.
Uninitialization script
Open the Matlab function file tuto3_sinus_generator_Uninitialize.m.
% tuto3_sinus_generator_Uninitialize.m
% -------------------------------
% Author : Laurent Bonnet (INRIA)
% Date : 25 May 2012
%
% The function tuto3_sinus_generator_Uninitialize is called when pressing 'stop' in the scenario.
%
function box_out = tuto3_sinus_generator_Uninitialize(box_in)
disp('Uninitializing the box...')
box_out = box_in;
end
As you can see the Uninitialize function does nothing.
Execution
The box generates some sinusoidal signal automatically, along with stimulations (one every second – see the red lines):
Sinus GeneratorIn the console, you can see the corresponding output :
[ INF ] At time 0.000 sec <Box algorithm::Tuto3 : sinus generator> ---- MATLAB BUFFER - INFO ---- Box settings are: Channel count : 4 Samples per buffer : 16 Amplitude : 5 Stimulation : 33025
[ INF ] At time 1.000 sec <Box algorithm::Tuto3 : sinus generator> ---- MATLAB BUFFER - INFO ---- Sending a stimulation...
[ INF ] At time 2.000 sec <Box algorithm::Tuto3 : sinus generator> ---- MATLAB BUFFER - INFO ---- Sending a stimulation...
[ INF ] At time 3.000 sec <Box algorithm::Tuto3 : sinus generator> ---- MATLAB BUFFER - INFO ---- Sending a stimulation...
[ INF ] At time 4.000 sec <Box algorithm::Tuto3 : sinus generator> ---- MATLAB BUFFER - INFO ---- Sending a stimulation...
[ INF ] At time 4.602 sec <Box algorithm::Tuto3 : sinus generator> ---- MATLAB BUFFER - INFO ---- Uninitializing the box...
Compiling the ‘plugins/processing/matlab’ project
These instructions may be helpful if you intend to compile OpenViBE yourself and want Matlab box to be built as well.
A valid installation of MATLAB includes all the libraries and exe required to compile and use the boxes.
To be precise, CMake looks for the following libraries when compiling OpenViBE:
- On Windows : libmex.lib, libmx.lib, libeng.lib under ${Matlab_ROOT}/extern/lib/win32/microsoft/
- On Linux : mex, mx, eng, under ${Matlab_ROOT}/bin/glnx86 or ${Matlab_ROOT}/bin/glnxa64
When building the software, the process should find all these libraries, as showed in the following build log example:
Configuring and building plugins\processing\matlab …
— Found OpenViBE…
— …
— Found Matlab…
— [ OK ] Third party lib C:/Program Files/MATLAB/R2011a/extern/lib/win32/microsoft/libmex.lib
— [ OK ] Third party lib C:/Program Files/MATLAB/R2011a/extern/lib/win32/microsoft/libmx.lib
— [ OK ] Third party lib C:/Program Files/MATLAB/R2011a/extern/lib/win32/microsoft/libeng.lib
— Configuring done
— Generating done
— Build files have been written to: C:/openvibe/local-tmp/plugins/processing/matlab/
If the build process does not find a valid MATLAB installation, the boxes won’t be built with the project. An empty dynamic library will be produced, which will result in the following message in the console when executing the OpenViBE Designer :
[WARNING] No 'plugin object descriptor' found from [../bin/openvibe-plugins-matlab.dll]
even if it looked like a plugin module
When building OpenViBE with an invalid MATLAB installation (i.e. MATLAB 64 bits on Windows) the build process will find MATLAB but not the libs resulting in the following build log :
Configuring and building openvibe\plugins\processing\matlab\…
— Found OpenViBE…
— …
— Found Matlab…
— [FAILED] Third party lib libmex
— [FAILED] Third party lib libmx
— [FAILED] Third party lib libeng
— FAILED to find Matlab Libs, the plugins won’t be built. Please ensure you have a valid MATLAB installation (32 bits only).
— Configuring done
— Generating done
— Build files have been written to: C:/openvibe/local-tmp/plugins/processing/matlab/
Known issues
Here are some issues that we know exist with the Matlab box. If you find others, let us know.
Troubleshooting
In this section, we review the most common errors and mistakes and what they can cause.
Please contact us if you find any unexpected behavior or bugs so we can either correct the Matlab boxes or add new items in this section.
Failed to Load Matlab Engine
Sometimes the Matlab engine fails to open because a Matlab license is not available. Another reason can be because the code gets confused with multiple Matlab installations on the computer. It can also fail due to the found Matlab being 64bit instead of 32bit. Unfortunately, the Matlab API call engOpen() used to connect to the Matlab engine does not return any information that would help to resolve the problem. One way to find out if the problem is in OpenViBE or a more general one is to locate some C++ tutorial examples from the internet that use engOpen(), and see if you can get these tutorials to work.
Launched matlab seems to have wrong version or bitness
If you have Matlabs of several bitness or versions installed in Windows, the engOpen() call can return one of them, possibly based on registry. To specify which one you want to use, open the administration command prompt, cd to the folder where matlab.exe is located, and run “matlab /regserver". That may fix the problem.
To see which version of Matlab it launched, you can type version to the Matlab console.
Failed to Load Matlab Engine at first try
Sometime in the past, when using OpenViBE from the Windows installer package, we identified an incorrect behavior of the box : when pressing play for the first try, the scenario won’t find Matlab and the box will fail to execute correctly. When re-playing the scenario by pressing play once again, the Matlab Engine (if everything is correctly set) runs successfully.
We have not encountered this problem lately. If you run into it, let us know.
On Linux, MATLAB box keeps saying ‘Can’t start MATLAB engine’, what do I do ?
In order to get MATLAB box to work properly on Linux, you may need csh to be installed. This is not always the case, notably on Ubuntu distributions.
On Linux, loading the Matlab plugin says ‘error:libmx.so: cannot open shared object file: No such file or directory’?
Before launching Designer, you must export LD_LIBRARY_PATH to include the path where the Matlab .so files are installed.
Matlab License
The Matlab Engine can be launched in background only if it can access a valid Matlab License.
If the Matlab License Manager can’t get a valid License, it will output an error window. The OpenViBE Designer would then be still waiting for the Matlab Engine to respond: you can expect the application to freeze for about 30 seconds. After this timeout, the call to Matlab engine fails and the following message would be printed in the console:
[ ERROR ] At time 0.000 sec <Box algorithm::Matlab Scripting> Could not open Matlab engine [WARNING] Box algorithm <Matlab Scripting> has been deactivated because initialization phase returned bad status
As a result, the box is deactivated and does not process any data.
Output Headers : be sure to set it right !
If you try to send a new buffer on an output and you get this message for example :
[ ERROR ] At time 0.094 sec <Box algorithm::Signal filter> Error calling [OV_getSignalOutputHeader]. Did you correctly set
the output header in the matlab structure ?
It means that you didn’t set the header (here it’s a signal output) correctly. The box is not able to construct the OpenViBE output stream.
Solution : set the header before the very first buffer sent. You can either use an existing header, as it is done in tutorial 1, or the dedicated Matlab functions as in tutorial 3.
If you construct a somehow corrupted header or a header that does not match the theoretical stream structure, the application has a good chance of crash.
For example, if you omit a field or use a field incorrectly (e.g. header.sampling_rate = 'plop') the Designer can just randomly crash during execution. We are aware of this problem and we will try to improve error handling in the following updates.
Be sure to save your OpenViBE scenarios often.
Manually closing the Matlab engine
The Matlab Scripting box opens the Matlab Engine on initialization (play) and closes it when uninitializing (stop). If an error occurs the Matlab Engine may not be closed correctly. This means that when you will run a scenario with a Matlab Scripting box once again, it will use the previous engine, still in the state you left it ! This can lead to unexpected behavior (e.g. if you have some global variables you are using in your script, they will still be there).
The simplest way to check the state of the Matlab Engine is by looking at the matlab window running in background. When opening the engine, Matlab launch a specific window called Matlab Command Window. This window is a basic Matlab interpreter in which you can type Matlab commands. When closing the Engine, this window should be closed automatically. If such window is still alive when there is no Matlab scenario running in the OpenViBE Designer, close it manually before playing such scenario.
Performance issues
Depending on the processing you implemented the Matlab scripting boxes may take some time to return from the process() call. If a box spends too much time in its process loop, a message is printed in the console:
[WARNING] <Player::can not reach realtime> 1 second(s) late... [WARNING] <Player::can not reach realtime> 2 second(s) late... [WARNING] <Player::can not reach realtime> 3 second(s) late... [WARNING] <Player::can not reach realtime> 4 second(s) late... [WARNING] <Player::can not reach realtime> 5 second(s) late...
This meanis that the OpenViBE scheduler is not able to process the data in real-time due to the latency introduced. For example, if you set the Matlab scripting box clock frequency to 32Hz, one process call should be less than 1/32 th second long. Otherwise, the process loop accumulates a delay.
Try to lower the clock frequency to extend the period of time between 2 process calls. The Matlab process function may have to process more chunk of data at a time (internal buffers are filled continuously), but you will save the “interface time”, between the C++ code and the Matlab engine.
