How to reproduce P300 Speller using Emotiv EPOC headset

[davidchung]

Hi all,

I’m a newbie in BCI and I would like to reproduce P300 speller using Emotiv EPOC Headset (Education version) and OpenViBE (v0.8.0).

The description in OpenViBE website explains P300-speller demo (http://openvibe.inria.fr/documentation/ ... eller.html) using actiCap, Brain Products GmbH. Also, it seems that the instruction is not up-to-date. For example, I cannot find “RecordingData2GDF.xml” in my openvibe source code (v0.80). However, I do see p300-speller and p300-speller-xDAWN directories under openvibe/dist/share/openvibe-scenarios/bci/

Can you please guide me how I may test P300-speller using Emotiv headset? Any other pointers to better understand P300 speller using OpenViBE would be very helpful as well.

Thanks!

[bpayan]

Dear Davidchung,

Thank you for your interest in OpenViBE and welcome on this forum.

I suggest you to use the p300-speller-xDAWN, it has better result than the p300-speller.
First, for the acquisition, it's better to use this process:
Learn one letter with the “p300-speller-1-acquisition.xml” scenarios, for this, in the line 8 of the file “p300-speller-stimulation.cfg” changed the SettingValue by 1.
Next, in the scenario “p300-speller-2-train-xDAWN.xml” load the file “${Path_Samples}/signals/p300-speller-xdawn-1-[Record_Date].ov” and start the player in fast mode.
For the scenario “p300-speller-3-train-classifier.xml” load the same file and start the player in fast mode.
This process initializes your configurations files.

For learning, used the “p300-speller-4-online.xml” with 10 letters (in the line 8 of the file “p300-speller-stimulation.cfg” changed the SettingValue by 10).
Next, used scenarios “p300-speller-2-train-xDAWN.xml” and “p300-speller-3-train-classifier.xml” with the newest record in the "signals" folder. You can see in the console the classification percent. For a good result this value need to be better than 83,3%.
To finish you can make your test with the “p300-speller-4-online.xml” scenarios.

I hope this helps,

Baptiste

[davidchung]

Hi Baptiste,

Thank you so much for your kind explanation about how to reproduce P300-speller-xDAWN, however, I need to rollback a little to verify my settings for OpenViBE acquisition server. I apologize if I am asking basic questions here.

Here is my settings for "OpenViBE acquistion server".
+ Driver: Emotiv EPOC
+ Connection port: 1024
+ Sample count per sent block: 4

After setting the parameters as above, the messages at the bottom of the acquisition server are described below (e.g. "0 host connected..."):

After pressing "Connect", the Windows console messages are:
[ INF ] Connection succeeded!
[ INF ] Connecting to device [Emotiv EPOC]...
(=> The acquisition server's message is "Connected! Ready... 0 host connected...")

After pressing "Play", the Windows console messages are:
[ INF ] Starting the acquisition...
[ INF ] Now acquiring...
(=> The acquisition server's message is "Sending... 0 host connected...")

Question 1. How do you compute the "sample count per sent block"? Emotiv EPOC is 128 Hz sampling rate.
Question 2. What is the value of "connection port" for Emotiv headset?
Question 3. Do you have to run "test-acquisition server" in background BEFORE launching the "test-designer"?


With the abovementioned settings, when I run "p300-speller-1-acquisition.xml in "test-designer" (for SettingValue = 1), the message at the bottom of "P300 Speller Visualisation" indicates the "Result: ".

Question 4. How do I verify whether it has correctly generated p300-speller-xdawn-1-[Record_Date].ov file in "signals" folder.

Question 5. Am I getting null (e.g. "Result: ") because Emotiv EPOC was not properly connected?


Please let me know what I am doing wrong here. Sorry for the long message, I'm trying to minimize the impact from time zone difference between France and CA, USA.^^

Again much thanks for your help!
David

[lbonnet]

Hi David !

After setting the parameters as above, the messages at the bottom of the acquisition server are described below (e.g. "0 host connected..."):

The "host connected" count represents the number of acquisition clients (i.e. Acquisition Client Boxes in designer scenarios) connected to this server.
This number will increment when you play a scenario with an Acquisition Client box inside.

Question 1. How do you compute the "sample count per sent block"? Emotiv EPOC is 128 Hz sampling rate.

In fact we don't compute it. You choose it
The Acquisition Server sends blocks to the clients on network connection. This represents the number of sample the acquisition driver will send with each block.
With 128Hz sampling frequency and 4 samples per block, one block represents 1/32 second ~31 ms.

Question 2. What is the value of "connection port" for Emotiv headset?

This is not a value for emotiv headset. It is a value for Acquisition Server (for network communication you need host name and port number).

Question 3. Do you have to run "test-acquisition server" in background BEFORE launching the "test-designer"

I guess I already answered that
In OpenViBE you have to run the server before the clients.

Question 4. How do I verify whether it has correctly generated p300-speller-xdawn-1-[Record_Date].ov file in "signals" folder.

The file should be in the folder share\openvibe-scenarios\signals.

Question 5. Am I getting null (e.g. "Result: ") because Emotiv EPOC was not properly connected?

Nope. The acquisition scenario is just recording data for the classifier and filter to be trained. You won't have "results" because no classification is made so far.
The visualization panel just displays the random flashing sequences. You will have to train the xDAWN spatial filter and LDA classifier with these data afterward.

I must warn you the P300 speller is not the simplest scenario to start with... especially with the Emotiv headset.
The basic scenario works with medical EEG we use in labs, but for the EPOC it needs some modifications to work; modifications you can do if you are familiar with both the software and the default scenario.


Hope this helps..

Laurent

[davidchung]

For learning, used the “p300-speller-4-online.xml” with 10 letters (in the line 8 of the file “p300-speller-stimulation.cfg” changed the SettingValue by 10).
Next, used scenarios “p300-speller-2-train-xDAWN.xml” and “p300-speller-3-train-classifier.xml” with the newest record in the "signals" folder. You can see in the console the classification percent. For a good result this value need to be better than 83,3%.
To finish you can make your test with the “p300-speller-4-online.xml” scenarios.

Baptiste

Thank you for your help. I appreciate for your help. I think I was able to initialize the spatial filter and classifier according your your description (i.e. the 1st paragraph posted by Baptiste).

However, I am not clear about the final step: Online use of the speller (i.e. the 2nd paragraph ).

Do I only have to run “p300-speller-4-online.xml”?

What do you mean by ==> Next, used scenarios “p300-speller-2-train-xDAWN.xml” and “p300-speller-3-train-classifier.xml” with the newest record in the "signals" folder.

Thanks,
David

[bpayan]

Dear davidchung,

If we cut the process in 3 part:

• You have the initialize phase: this phase initializes the spatial filter and the classifier configs files. This phase uses 3 scenarios "p300-speller-1-acquisition.xml", "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml". For this phase, it's not necessary to have a good classifier so you can just learn with 1 letter for a faster phase.
• You have the training phase: For this phase, you need more P300s' examples, so you can learn with 10 letters. This phase uses 3 scenarios "p300-speller-4-online.xml" (for acquire data, and save their in a file. You will need to use this file in the next 2 scenarios), "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml".
• You have the online phase: this is the phase where the system try to find the good letters. This phase uses the scenario "p300-speller-4-online.xml".

P.S: In the training phase, the scenario "p300-speller-4-online.xml" gives results' letters. It's normal these letters are different to the targets.

Next, used scenarios "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml" with the newest record in the "signals" folder.

I mean, you are in the training phase and the box "Generic stream writer" in the scenario "p300-speller-4-online.xml" saves your data in a file, "p300-speller-xdawn-4-[$core{date}-$core{time}].ov" ( $core{date}-$core{time}= the date and time of your system when you start the scenario). In the scenarios "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml" you have a box "Generic stream reader" to load your data. This is in these boxes you need change the parameter and specify your data's file.

I hope these explications are clearer.

Baptiste

[davidchung]

P.S: In the training phase, the scenario "p300-speller-4-online.xml" gives results' letters. It's normal these letters are different to the targets.

Hi Baptiste,

Thank you for your clarification. I was able to run the steps, and was able to get the following result after the online phase (step3): The success ratio is pretty bad here. I will try it again from the scratch after your feedback.

Target: C8T6D8TIEL
Result: X8F6D9TIFL

Question 1. Why are some of the characters displayed in orange? Aren't all incorrect results should be displayed in black?

Question 2. I noticed that two different characters are highlighted - one in blue and the other in green during the training phase when I am running "p300-speller-4-online.xml". I've fired P300 when the blue characters are flashed, not at the green characters, however, the instruction in the p300-speller-4-online.xml (in the "Overview" box) explains that I need to focus on a green letter.

FYI, here is the instruction:
When you start this scenario, just press
the <b>"a"</b> key to start the flash sequence.

You will then be presented a green letter that
you have to focus on, followed by a 12 times
flashing sequence of the whole grid. The detected
letter will be presented in red. This will be repeated
10 times.

Thanks a lot again for your help.

Happy Thanksgiving!
David

[bpayan]

Dear davidchung,

Question 1. Why are some of the characters displayed in orange? Aren't all incorrect results should be displayed in black?

The letter proposed by the speller correspond to the crossing of the line and the column who contains a P300. So it's possible to have a good column and a wrong line or inversely. To show clearly this case compared as a wrong column and wrong line proposed, a system of color is used.

Green for a good letter, orange for a good column and a wrong line or inversely and black for a wrong column and a wrong line.

Question 2. I noticed that two different characters are highlighted - one in blue and the other in green during the training phase when I am running "p300-speller-4-online.xml". I've fired P300 when the blue characters are flashed, not at the green characters, however, the instruction in the p300-speller-4-online.xml (in the "Overview" box) explains that I need to focus on a green letter.

Yes, you have two different letters highlighted. The green letter is the target. This is this letter you need to focus when the speller flash lines and column.

The Result proposed by the speller is the letter display in blue.

Before, the Result letter was displayed in red, but generally for people this color is interpreted as a wrong result, so the color was changed by a blue color, but the documentation in the scenario wasn't changed. We are sorry for the inconvenient and thank you for having report this bug.

I hope these helps and you will have a good result.

Baptiste

[davidchung]

Yes, you have two different letters highlighted. The green letter is the target. This is this letter you need to focus when the speller flash lines and column.

The Result proposed by the speller is the letter display in blue.
Baptiste

Sorry, I am still confused at your explanation.

When I am running the "p300-speller-4-online.xml" in the TRAINING PHASE, the "Target" letter is randomly highlighted with "BLUE" not GREEN in the application. However, the Overview box says that I need to focus on green letters. This discrepancy is what I am confused.

Why do you say that I need to focus on "GREEN" letter?

Also, the first letter which gets highlighted is BLUE, not GREEN. If you say that I need to focus on GREEN, then what do I do when this first BLUE letter is highlighted?

Shouldn't I focus on BLUE letters? And the corresponding results are displayed in GREEN.

Thanks,
David

[bpayan]

Dear davidchung,

You're right the target is highlighted in blue and the result in green. I'm working in a branch of the simple-visualisation plugin that's why I use an old speller visualisation version in which colours are inversed.

I'm sorry for the inconvenient,

Baptiste

[davidchung]

• You have the initialize phase: this phase initializes the spatial filter and the classifier configs files. This phase uses 3 scenarios "p300-speller-1-acquisition.xml", "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml". For this phase, it's not necessary to have a good classifier so you can just learn with 1 letter for a faster phase.
• You have the training phase: For this phase, you need more P300s' examples, so you can learn with 10 letters. This phase uses 3 scenarios "p300-speller-4-online.xml" (for acquire data, and save their in a file. You will need to use this file in the next 2 scenarios), "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml".
• You have the online phase: this is the phase where the system try to find the good letters. This phase uses the scenario "p300-speller-4-online.xml".

P.S: In the training phase, the scenario "p300-speller-4-online.xml" gives results' letters. It's normal these letters are different to the targets.

Hi Baptiste,

Again, thank you so much for your kind and prompt help.

After completing the TRAINING PHASE, my Classifier performance on whole set is 82.2878% (see my console message below), however, when I run the ONLINE PHASE my RESULT do not match my TARGET (0%).

QUESTION 1.
Do you know any performance results using OpenViBE's P300-Speller with Emotiv headset?
Do you have any advice about what might have gone wrong here?

QUESTION 2.
In the INITIALISE PHASE, why do we have to run "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml" in the "fast" mode?

What difference does it make if I run them in the "normal" mode?

QUESTION 3.
I have used the "normal" mode when I was running "p300-speller-2-train-xDAWN.xml" and "p300-speller-3-train-classifier.xml" in the TRAINING MODE. Is this correct?


FYI, here is my console message (from the TRAINING PHASE):

[ INF ] Exporting scenario...
[ INF ] Adding configuration file(s) [../share/openvibe-delayed.conf]
[ INF ] Adding configuration file(s) [C:\Documents and Settings\david1.chung/openvibe-delayed.conf]
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Received train stimulation
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> k-fold test could take quite a long time, be patient
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 1 (0x1) / 20 (0x14) (performance : 87.3239%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 2 (0x2) / 20 (0x14) (performance : 91.6667%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 3 (0x3) / 20 (0x14) (performance : 83.3333%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 4 (0x4) / 20 (0x14) (performance : 81.6901%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 5 (0x5) / 20 (0x14) (performance : 87.5%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 6 (0x6) / 20 (0x14) (performance : 80.5556%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 7 (0x7) / 20 (0x14) (performance : 85.9155%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 8 (0x8) / 20 (0x14) (performance : 83.3333%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 9 (0x9) / 20 (0x14) (performance : 80.5556%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 10 (0xa) / 20 (0x14) (performance : 86.1111%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 11 (0xb) / 20 (0x14) (performance : 80.2817%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 12 (0xc) / 20 (0x14) (performance : 77.7778%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 13 (0xd) / 20 (0x14) (performance : 81.9444%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 14 (0xe) / 20 (0x14) (performance : 78.8732%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 15 (0xf) / 20 (0x14) (performance : 79.1667%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 16 (0x10) / 20 (0x14) (performance : 83.3333%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 17 (0x11) / 20 (0x14) (performance : 80.2817%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 18 (0x12) / 20 (0x14) (performance : 80.5556%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 19 (0x13) / 20 (0x14) (performance : 79.1667%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Finished with partition 20 (0x14) / 20 (0x14) (performance : 76.3889%)
[ INF ] At time 1195879956480 (0x11670000000)<Box algorithm::Classifier trainer> Classifier performance on whole set is 82.2878%

[yrenard]

Dear davidchung,

indeed, there are some mistakes in the comments of the scenarios, sorry for the inconvenience. The letter you should focus on is definitely the one with the first color you see on screen

As regarding to the performances, can you please tell us if you are working on Linux or Windows ?

Yann

[bpayan]

Dear DavidChung,

Your result on the training phase looks good, but your Online result is surprising. I've no performance results on the Emotiv headset, so I can't help you for this.

For the scenarios 2, 3 and 5 of the xDAWN speller, you have no difference between the fast mode and the normal mode because this is just a replay of your acquisition's data. I use the fast mode because as it name says, it's faster.

Baptiste

[davidchung]

As regarding to the performances, can you please tell us if you are working on Linux or Windows ?
Yann

Hi Yann,

I am running OpenViBE (0.8.0) on Windows XP.

If you know anyone who has worked on (or any results about) OpenViBE's P300 speller using Emotiv headset, please inform me on this forum. I would like to know what their results are.

Thanks,
David

[yrenard]

Dear davidchung,

I did some tests yesterday with the latest SVN and achieved some results such as 6 correct letters out of 10 plus 4 with either the line or column wrong. I believe better results could be achieved tunning the processing pipeline but I didn't have time to adjust the settings in the best way yet.

I will post a message as soon as I have better results.

Regards,
Yann