Qs about artifact removal filters and P300 speller scenario

[stefan]

Hello everyone,

I’ve some questions regarding the p300 speller scenario:
•K-fold testing costs long training time, are there other means to use for P300 speller?
•How can I remove artifact from my data? I'd like to investigate artifact removal filters on openvibe if any.
•Why the 'channel selector' box is set to choose only 3 channels? Note that I use Emotive EEG recording system, and channel Cz is not available.
•Can I reach the accuracy of each letter of the speller individually?
•When I do the experiment this message appears: comparison flag is not used yet and the file will be stored as it is, although it is marked as true on the writing box!
•How can I improve the performance for untrained people? How much training is sufficient?
•When I use the online scenario for the p300 speller after I train xDAWN filter and the classifier I get this warning ”<Algorithm:: LDA classifier-GPL> Feature vector size 57(0x93) and hyperlane parameter size 0(0x0) does not match”

Any help is appreciated
Regards,

[lbonnet]

Hi stefan,

K-fold testing costs long training time, are there other means to use for P300 speller?

The k-fold test gives you an estimation of the classifier accuracy. You can lower the number of partitions to reduce the computation time.
Default value is 20 partitions. Make it 1 and you have no more k-fold test (the estimation may then be unprecise).

How can I remove artifact from my data? I'd like to investigate artifact removal filters on openvibe if any.

There is no "artifact removal" box in openvibe so far. However you can use several boxes to build a signal processing pipeline that will detect artifacts.
One simple example could be a simple threshold (using DSP box) that detects if the signal has an unusual spike. The temporal or spatial filters can also help removing some noisy component of the signal.
Removing artifacts is not so simple though. Writing your own signal processing box is a possibility if you have in mind a particular artifact removal algorithm.

Why the 'channel selector' box is set to choose only 3 channels? Note that I use Emotive EEG recording system, and channel Cz is not available.

The P300-Speller scenarios are made to be used with a 3 channels setup (Cz;O1;O2). As stated in the documentation page of the box, the Channel Selector can select as many channel as you need.
The Emotiv EPOC has 14 channels, and the electrode placement is not really reliable as there is always a variability when putting the headset on. You should definitely consider using the P300-Speller-xDAWN scenarios instead. xDAWN is an algorithm that computes the "best" spatial filter for P300 detection. With the xDAWN scenarios, the number and position of electrode does not matter.

Can I reach the accuracy of each letter of the speller individually?

There is no accuracy per letter so to speak. The classifier is trained to detect P300 response on a given signal part. It's trained with many example of P300 and no-P300 examples.
The accuracy you get after training is an estimation when testing the classifier on the training data. These data contains many more no-P300 examples than P300 example (on 6 row flashes only one should trigger a P300 - same for column). Thus the chance level is 5/6 = 83.3%.

When I do the experiment this message appears: comparison flag is not used yet and the file will be stored as it is, although it is marked as true on the writing box!

This warning message is not important. The Generic File Writer box does not implement compression mechanism, yet the setting is set to true in the scenarios. This should be corrected asap.

How can I improve the performance for untrained people? How much training is sufficient?

The P300 speller can be improved with training, that's for sure. However I don't really know how much... It depends on the user (some of them seem to have a very clear P300 response naturally, some need several sessions before showing good results).
As you are using the Emotiv EPOC, the challenge is a bit more complex than with "medical" EEG of course (less precision) but I guess you can reach good performances with some tweaking in the scenarios.

When I use the online scenario for the p300 speller after I train xDAWN filter and the classifier I get this warning ”<Algorithm:: LDA classifier-GPL> Feature vector size 57(0x93) and hyperlane parameter size 0(0x0) does not match”

It means that the classifier found incoherency between the data it processes (the feature vectors) and the LDA hyperplane of the classifier (what has been trained by the Classifier Trainer).
Are you sure you trained the classifier (and xDAWN of course) on the data you acquired ? (I mean did you change the file reader boxes in the training scenarios with the file you acquired ?)

Hope this helps.

Laurent-

[stefan]

Thank you Laurent for your help
I’ve concerns about the accuracy of the P300 speller with xDAWN filter. SVM classifier was trained for 12 sessions and the average accuracy was 83.33%. However, when I used the online scenario, I didn’t get any letter right all targets were wrong. Did you find the same problems? How can I improve the performance?
Also, how can I get the selected channels by xDAWN filter/ how can I visualize its results?
Your help is much appreciated.
Cheers

[stefan]

The previous error was because I trained SVM classifier and then used LDA online by mistake

Unfortunately, I have this error now “Plugin code cause crash 1(0x1) time(s)
[name: LDA classifier-GPL]
[class identifier: 0xd7183fe7, 0xbd74f298]
[location::process callback]”???

Another question is: it seems that the data collected using the acquisition scenario can’t be used on the replay scenario, what the reason behind this?

Cheers

[ddvlamin]

SVM classifier was trained for 12 sessions and the average accuracy was 83.33%. However, when I used the online scenario, I didn’t get any letter right all targets were wrong. Did you find the same problems? How can I improve the performance?

The P300 experiment records unbalanced data, meaning there are less target trials than non-target trials, so basically 83.3% accuracy is probably around chance level.

Also you do not need to use the SVM, you can simply use the LDA classifier. To my knowledge the SVM fixes the hyperparameters and does not determine them through cross-validation, meaning that you will have to that yourself which makes it hard to use.

For the LDA algorithm there is no need to determine hyperparameters so I do not see the need to perform cross-validation. I would use only 1 fold and thus all the data to compute the classifier's hyperplane and bias. Of course as lbonnet says, then there is no way to trust the estimation of the accuracy, but usually it does work in the test phase.

Also, how can I get the selected channels by xDAWN filter/ how can I visualize its results?
Your help is much appreciated.
Cheers

xDAWN does not select channels, instead it takes a linear combination of the channels. They are saved in the p300-spatial-filter.cfg file which you can find in the p300-speller-xDAWN directory with bci scenarios. These coefficients can be used for example in matlab to visualize the filters with the topoplot function of EEGLab.

For the other questions I'll have to refer you to the developers.

Best regards,
Dieter

[stefan]

Thank you so much Dieter

One last qustion is: what is the reason for this error which I got when I use any online scinario

“Plugin code cause crash 1(0x1) time(s)
[name: LDA classifier-GPL]
[class identifier: 0xd7183fe7, 0xbd74f298]
[location::process callback]”???

Your help is approciated.

Cheers

[ddvlamin]

“Plugin code cause crash 1(0x1) time(s)
[name: LDA classifier-GPL]
[class identifier: 0xd7183fe7, 0xbd74f298]
[location::process callback]”???

I do not actually know what caused this, but maybe you can post your log files in the dist\log directory so the developers can take a look (if it contains more details?). The only error I came across when using the classifier is the one that complains about the fact that the feature vector size and the hyperplane parameter size do not match (which you also mentioned in your previous post). So are you sure that you used both in training and testing/online scenarios the same classifier? Maybe you can also check if the file that contains the classifier parameters really contains the updated parameters after training (looking at the date of the file) and check if it is really that file that is being used in the classifier processor box in the online scenario.

Best regards,
Dieter

[lbonnet]

Hi stefan,

This crash of the LDA classifier algorithm can happen for various reasons.

If I understood well, you are :
- using the acquisition scenario, you manage to get some data
- you train the xDAWN filter with these data (with "train successfull" message at the end)
- you train the LDA classifier with these data (with "train successfull" message at the end)
- you try the online scenario

... and the LDA crashes.

The message you copy-pasted to us is not very meaningfull as-is. Can you give us the full log ?
Maybe we can find something interesting in it
you can find the text log in the /log folder (openvibe-designer.log) of openvibe.

Cheers
Laurent

[lbonnet]

thx Dieter !
sorry for cross-posting

[stefan]

Yes, I do the steps just as you mentioned but when I do the online testing, I get this error. This is the information from the log folder

[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[ ERROR ] Plugin code caused crash 1 (0x1) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match
[ ERROR ] Plugin code caused crash 2 (0x2) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match
[ ERROR ] Plugin code caused crash 2 (0x2) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[WARNING] <Player::can not reach realtime> 3 (0x3) second(s) late...
[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match
[ ERROR ] Plugin code caused crash 2 (0x2) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[WARNING] <Player::can not reach realtime> 6 (0x6) second(s) late...
[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match
[ ERROR ] Plugin code caused crash 2 (0x2) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[WARNING] <Player::can not reach realtime> 10 (0xa) second(s) late...
[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match
[ ERROR ] Plugin code caused crash 2 (0x2) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]
[WARNING] <Player::can not reach realtime> 13 (0xd) second(s) late...
[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match
[ ERROR ] Plugin code caused crash 2 (0x2) time(s)
[ ERROR ] [name:LDA classifier - GPL]
[ ERROR ] [class identifier:(0xd7183fc7, 0xbd74f298)]
[ ERROR ] [location:process callback]

It is repeated in loops

[stefan]

I do not get a "train successfull" message at the end of training the LDA classifier??

[lbonnet]

Hi stefan,
Sorry for late reply.

Code: Select all

[WARNING] <Algorithm::LDA classifier - GPL> Feature vector size 152 (0x98) and hyperplane parameter size 0 (0x0) does not match

Means that the data used to train the LDA and the data used online to classify do not have the same structure, the feature space is not the same.
Your LDA hyperplane has parameter size 0, this is not normal at all.

I do not get a "train successfull" message at the end of training the LDA classifier??

A regular training session should end with an estimation of the classifier accuracy. With k-fold test you get several performance measures on different subset of the training data. These messages you can't possibly miss.
If you don't have anything at all after all the file is read, maybe the train trigger was not received by the trainer box.

As we previously asked, please send us the full log file of the designer, after you did all the steps (train xDAWN/train LDA/online and crash).

Laurent