How to trigger sound event based on eeg?

[beatsme]

Forgive me if this is the wrong forum, or if this is a stupid question, but I am genuinely interested in learning to use OpenVibe and to work with EEG data-driven experimentation.

My goal is to design a simple experiment in which auditory or visual stimuli will respond to EEG data. In other words, simple neuro-feedback stuff. I've noticed that there is a Sound Player box and a Clock Stimulator box. Using these two, I can trigger a "beep" sound to play based on a timer (e.g., every 1000ms). The Sound Player box takes a Stimulator as input. I can obtain real-time EEG data using the Acquisition Client box, but how can I transform this into a Stimulator to drive the Sound Player with?

Clock Stimulator ----> Sound Player
Acquisition Client ----> [ ? ] ----> Sound Player

With some exploration of the other boxes, I've noticed that there is a Feature Aggregator which can take a Signal as input, and output a Feature Vector. There is then a Classifier Processor box that can take a Feature Vector and output a Stimulator.

Acquisition Client ----> Feature Aggregator ----> Classifier Processor ----> Sound Player

Is this my only option? Is this my best option? I feel like going this route would be like using a chainsaw to slice bread. Really, what I would like to do is this:

Every 1000ms play a Sound
IF brain wave is Alpha (7-13Hz), THEN play Sound_1
IF brain wave is Not Alpha, THEN play Sound_2

How can I do this?

[lbonnet]

Hi beatsme, welcome on the forum!

I've noticed that there is a Sound Player box and a Clock Stimulator box. Using these two, I can trigger a "beep" sound to play based on a timer (e.g., every 1000ms).

Exactly.

I can obtain real-time EEG data using the Acquisition Client box, but how can I transform this into a Stimulator to drive the Sound Player with?

Look at the box documentation to know what their inputs/outputs are about (click on a box and press F1). For example the acquisition client gives you signal (EEG) stimulation and other informations it takes from a server on the network. Some EEG amplifiers have additional functions such as trigger/tagging of the signal; then the stimulation can be sent and received in the client.

IF brain wave is Alpha (7-13Hz), THEN play Sound_1
IF brain wave is Not Alpha, THEN play Sound_2

OpenViBE cannot detect "brain wave is alpha or not alpha". Alpha is a component of the signal, always present. What you can do is measuring the power of that frequency band, and if it goes above or below a threshold, trigger a stimulation to produce a sound for example.
Look at the neurofeedbackor tie-fighter scenarios given with openvibe in share/openvibe-scenarios. They compute the band power on beta frequency.

The threshold part cannot be done directly with available boxes (except if you use the Matlab box, but it can be tricky)
This functionality is pretty simple though. Developingsuch box is not hard if you have some C++ knowledge.

Using a classifier is necessary if you want to discriminate several signal "patterns" like "movement of the right hand" versus "movement of the left hand". To do so you need training examples for each class.

Hope this helps

Laurent

[beatsme]

Thank you! Very informative response. I will look into developing a new box.

[beatsme]

Progress so far:
I can now build from the source.
I can now create new boxes using the skeleton generator, and include them in the build.

I am not sure how to implement my algorithm in C++ using the OpenViBE source.

The basic algorithm I would like to create is simple:
Every 1000ms check input against threshold (constant)
If input > threshold, send output "A"
If input < threshold, send output "B"

In OpenViBE terms, it might be stated like this:
Take as input a Signal.
Check this input against a pre-defined "threshold" value. For simplicity's sake, let's say the threshold value is "1".
IF the input signal is GREATER THAN the threshold, send as output a Stimulation corresponding to "OVTK_StimulationId_Label_00".
IF the input signal is LESS THAN the threshold, send as output a Stimulation corresponding to "OVTK_StimulationId_Label_01".

Here is the skeleton I generated:
Input: Signal
Output: Stimulation
Codecs: Signal Decoder, Stimulation Encoder
Processing method: Call the 'process' at a specific frequency

Now:
As I understand it, the Signal that I will be receiving via the Signal Decoder would ordinarily be a matrix of values, corresponding to the number of channels in the signal stream. But I would like that Signal to already be transformed into a power value for a single channel. If I follow the neurofeedback example, this seems pretty simple. So the Signal I will actually be receiving as input will be a matrix with a single channel and a single value?

Is that right? If so, which method do I need to refer to that value in order to perform a comparison?

[lbonnet]

Hi beatsme !

It's good to see that you understood quiet well the main ideas when it comes to box development!

The basic algorithm I would like to create is simple:
Every 1000ms check input against threshold (constant)
If input > threshold, send output "A"
If input < threshold, send output "B"

The idea is good, but it's not the job of a single box (in the "openvibe way of thinking").
Your box should be simpler:
- every time a new input is received, check it against a threshold
- If input > threshold, send output "Stimulation A"
- If input < threshold, send output "Stimulation B"

Here is the skeleton I generated:
Input: Signal
Output: Stimulation
Codecs: Signal Decoder, Stimulation Encoder
Processing method: Call the 'process' at a specific frequency

In the case I give you above, you need:
Input: Signal
Output: Stimulation
Codecs: Signal Decoder, Stimulation Encoder
Processing method: Call the 'process' on every new input chunk

I also advise you to add settings:
- Stimulation to be sent if below threshold
- Stimulation to be sent if above threshold

The part "Every 1000ms check input against threshold" is possible using other boxes already in openvibe.
The Neurofeedback scenario computes the band power of beta frequency using :
- Temporal filter : to extract only the required frequency range
- Time Based Epoching : produce a signal window of 1 sec every 100 msec
- DSP : to get the square of the signal (one simple way to get the power of the frequency)
- Signal average : to average the whole epoch, it produces 1 value per channel per epoch

Your Threshold box can be very simple : it takes an input signal matrix, and triggers the stimulation A or B with respect to the first value in the matrix. It's up to the upper stream (coming into your box) to process the signal in order to have one value at a defined frequency (using the epoching box).

Of course you can add more complexity, for example, triggers A or B if at least one channel has a value in the matrix greater than the threshold.

Hope this helps !

Laurent

[beatsme]

Hi lbonnet,

Thanks so much for your explanations! You've definitely made it sound simple enough.
I will take another crack at it when I return after the holidays!

Cheers.

[Aizat89]

hye, do you already success trigger the sound event based on eeg?
if yes, can you please guide me on how to develop the threshold box..? i have no idea in how to implement algorithm in C++ using openVIBE source.

i'm really appreciate your kindness if you can provide me the solution with an image of the design (since i'm newbie in openvibe, example in image could rather help me for better understanding the concept.)

[jlegeny]

Hello,

you can use the Sign Change Detector combined with the DSP to get the threshold box functionality. After that simply connect the Sound Player box to it.