Synchronization in OpenViBE

[MattDarling]

I'm working on setting up an experiment and using OpenViBE to record information from our NeuroSky MindWave (the original white model). When we were using more complicated recording hardware, we used some software by Neurobehavioural Systems called Presentation to write our experiments. We used a parallel port in order to send event codes from Presentation to our recording software, possibly through the amplifier. Then in our recorded data, we would have specific milliseconds tagged with an event code representing which stimulus had been presented at that time.

My question is, how could I set up something similar with OpenViBE? I'm happy to continue using the Presentation software for development, but if OpenViBE can offer similar timing accuracy for the presentation of images and so on, I would be just as happy to switch. I'd prefer to avoid having to re-implement my own version of the Neurobs software in C or C++, however. Assuming I continue to use Presentation, I need some way to receive input in OpenViBE of various event codes as output by the software (some fairly sparse documentation of the output options provided in Presentation can be found here).

I've been considering running both programs on the same computer and creating virtual a serial/parallel port to connect the two. I've read a blog post Yann Renard wrote about hardware tagging, but this seems to be a slightly different situation. At any rate, it's not clear to me how the IDriverCallback documentation linked in his post would be used in order to receive these events and incorporate them into my recorded data.

Any guidance you could provide would be appreciated!

-Matthew

[toncho11]

Hello MattDarling,

We do hardware tagging in my lab.

Yes, you can continue using the Presentation to perform the hardware tags, OpenVibe will do the data acquisition. The driver NeuroSky MindWave that is available in OpenVibe must start to deliver the tags you need (if not already enabled). Usually an extra channel is added in OpenVibe. This channel is like a regular EEG channel with exception that the data is hardware tags. This means that you can select this channel with box "channel selector" in OpenVibe and then process it. For example we have a box that detects a rising edge (current value is changed to a bigger value) that outputs an OpenVibe stimulation which has the exact timing as the hardware tag (in terms of the signal). For every value that we expect we assign a stimulation. Then we continue our processing using these simulations. I can provide you with this box if this is what you are interested in.

And yes we also tag on a virtual serial port over USB cabel, then we convert the USB to parallel port (via Arduino) and plug it in the amplifier. Also today I implemented in our box [hardware tag -> OpenVibe stimulation] a fix for a hardware glitch when Arduino is used. The fix is performing well, so I would recommend this method.

Cheers,
Anton

[yrenard]

Dear Anton, dear MattDarling,

it is not exactly true that data tagging is to be done with an additional EEG channel. Actually, OpenViBE tags data with what's called stimulations. It's the responsability of each acquisition driver to generate stimulations when it has to, usually depending on input triggers. Adding an additional channel, while doing the work and probably working around a number of specific issues you (or your team) may have (had), is not the right way to do it. Indeed, scenarios are hardware agnostic. A scenario author should not have any assumption whether a channel or family of channels is EEG or not. They are all supposed to be EEG. And stimulations carry the tagging. This is the way it is done in OpenViBE !

This said, it is open, you have source code, you can do what you want for your own needs

Hope this helps,
Best regards,
Yann

[MattDarling]

Yann, Anton, thank you both - I think I'm starting to figure this out, at least a little bit.

Unfortunately, anything that depends on functionality of the acquisition driver isn't likely to work for us, because the NeuroSky hardware doesn't support things like that (as far as I know). To the best of my knowledge, there's no way to send triggers to the NeuroSky, so having the acquisition driver generate stimulations probably isn't possible.

In the meantime, is there any conceivable way that OpenViBE could receive stimulation info with a sort of "stimulation acquisition server"? In other words, a box that waits for signals to come in, via a parallel/serial port/other processes on the same computer, and then generates the appropriate tags for OpenViBE to process. If this is already possible, I'd love to hear about it Yann, I know you've written in the past about hardware tagging vs software tagging, but I've yet to figure out how to receive tags generated by software other than OpenViBE.

Our hope was that by running everything on one computer, we would avoid some of the major synchronization issues. Of course, our tagged data might be off by some (hopefully very small) amount from the actual location, but we can handle that afterwards when we process the data.

-Matthew

[yrenard]

MattDarling,

You could use a Button VRPN Client box for instance and generate the tags from a VRPN server of your own.

I think other ways of doing this are currently being implemented by some users but this is not yet available in the public release.

Best regards,
Yann

[toncho11]

Hello Matthew,

You can try our software tagging developed in my lab. You will receive the EEG signal and stimulations in the acquisition server. It works well, but it is not perfect. I do not want to fall into implementation details now. We will appreciate if you can test it, so that one day it can be integrated in OpenVibe mainstream.

To contact me further use my OpenVibe forum user name and gmail.

You will need:

1) a way to execute our C++ library that does the tagging in your external application. We also have a .NET (C#) wrapper

2) replace your acquisition server to be able to receive the stimulations. This can be done in two ways:

2.1) simply download our modified acquisition server "acquisition-server-gipsa-1.0.zip" from: http://code.google.com/p/openvibe-gipsa ... loads/list

2.2) or alternatively compile our source code modification of the acquisition server yourself, but you do not need that if 2.1) works fine for you

Also note that the AS and your external application (Presentation) must be on the same computer.

There is a possibility to print statistics in the end - how many stimulations came a bit late (so they were adjusted to current time) and how much exactly, but this must be configured.

Cheers,
Anton

[toncho11]

1) Maybe I could write a Python wrapper for our communication library that you could use in Presentation. I see Presentation supports Python.

[MattDarling]

1) Maybe I could write a Python wrapper for our communication library that you could use in Presentation. I see Presentation supports Python.

Actually, this would be incredible, because I happen to have just found out about an open-source Python-based alternative called PsychoPy. For the record, I've found it to be far superior so far, and am looking forward to converting to their system.

Does the acquisition-server zip file also contain the C++/C# libraries? It doesn't seem like it, but perhaps I'm missing something.

-Matt

[toncho11]

Hello Matt,

No, the library is not supplied in a way you can use it yourself.

So, you going to use the software tagging, right?

For C++ and C# I just need to make a package.

For Python I will need to devote some time.

Cheers,
Anton

[benry]

I have a problem similar to the ones discussed and I think it to be quite common, but since I do not have much experience in this field, I am asking for some suggestions/comments...

We are planning some experiments where we need to show to the subjects some images/videos and we would like to acquire EEG signals using openvibe to later do the analysis off-line. Since we may perform also ERP and ERS/ERD analysis, we were thinking about timing issues related to the stimuli presentation. Maybe we are overthinking about this, but I would like to have your opinions about:

Is there in OpenVibe a box usable to present images, in order to save in the EEG recording a stimulus-timestamp (stimulation) associated to the visualization of the image? Could it be used to present also videos? (the same box used for images, I guess, could be used to save a stimulus-timestamp for every frame of a video). Does operating system latencies due to processes/threads scheduling may influence enough the stimuli-presentation and stimuli-timestamp-recording in order to compromise ERP/ERS/ERD analysis? Does the use of real-time OS like Linux/RTAI should be taken into consideration to achieve better results?

Since I think that to present images and to save the correct timestamp of the image visualizations has to be quite common, how is it usually done in the OpenVibe environment?


Thanks in advance,

Enrico

[jlegeny]

Hello Enrico

welcome to the forums.

There is indeed a box in OpenViBE used to present the images, called Display Cue Image. This box displays an image upon receiving a stimulation, you can define multiple images and map them to different input stimulations (by adding additional settings to the box via the context menu).

This box has also a stimulation output which sends stimulation at the time the images are actually displayed. This synchronization uses Gtk event and does its best to be precise, however due to the Gtk functionality we can not be 100% sure to have really precise timing.

Cheers
Jozef

[cmuehl]

Hello,

I have also a problem sending event info from Presentation (neurobs.com) to OpenVibe. I hope you can help me to find a solution.

I am considering to implement software tagging to transfer event info from Presentation to OpenVibe. Before doing so, I would just like to see if this makes sense, i.e. if the software trigger timing is reliable enough. I am planning experiments that last about 15 - 30 minutes (15 if I make a break in the middle) and I want to use OpenVibe as recording software. Since my event blocks are long (2 minutes each), drifts of the range of 1 second should be acceptable. However, on a trial level within single blocks I have a much faster pace (2 seconds each trial) and would profit from a higher precision. Is it possible to say if software triggering is working for block tagging (i.e. does not deviate much more than a second) or even for much more precise trial tagging?

I followed this forum discussion so far, but it unfortunately stopped before offering specifics. Is there currently some API that I can use to add software tags? In which format do I have to deliver tags?

Thanks in advance,
Christian

[jlegeny]

Hello cmuehl,

openvibe acquisition server has the possibility to receive software tags through the Interprocess Messaging API of Boost library. There is a short documentation and an example implementation here:

http://openvibe.inria.fr/send-stimulati ... on-server/

We did not make long term drift tests ourselves but our partners have reported that it should work, as long as the drift correction of the acquisition server remains enabled.

Cheers
Jozef

[Ddesire143]

Hello Enrico

welcome to the forums.

There is indeed a box in OpenViBE used to present the images, called Display Cue Image. This box displays an image upon receiving a stimulation, you can define multiple images and map them to different input stimulations (by adding additional settings to the box via the context menu).

This box has also a stimulation output which sends stimulation at the time the images are actually displayed. This synchronization uses Gtk event and does its best to be precise, however due to the Gtk functionality we can not be 100% sure to have really precise timing.

Cheers
Jozef

Hello there,
I am also trying to do the same procedure for my research. I have to display multiple images while recording the eeg signal. Could you please send me the example of this kind of scenario because I am not able to understand the display cue image box. I am using motor-imagery lua script to send the stimulation
thanks

[jtlindgren]

Hi Ddesire143,

there is a tutorial file cue-image.xml in box-tutorials/ folder. Does it clear up things any?


Happy hacking,
Jussi