xDAWN P300 speller training xDAWN + Classifier.

Working with OpenViBE signal processing scenarios and doing scenario/BCI design
Post Reply
User
Posts: 16
Joined: Wed May 18, 2016 8:43 pm

xDAWN P300 speller training xDAWN + Classifier.

Post by User »

Hello,

I've ran these steps and I have some questions, I'd be glad to find the answer for.

1- What are the eigen values? how are they calculated and used in this algorithm?

2- What is the difference between the cross validation accuracy and the training set accuracy?
2.1- which one do I choose to describe the accuracy of the classifier?
2.2- I've noticed that the relation between cross validation accuracy and the training set accuracy is an inverse relation, such that if the lower the cross validation accuracy the higher the training set accuracy, why?

3- I didn't fully understand this, can someone explain?

<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Cls vs cls 1 2
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 1: 51.2 48.8 %, 240 examples
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 2: 1.5 98.5 %, 1200 examples
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Training set accuracy is 93.3333% (optimistic)
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Cls vs cls 1 2
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 1: 61.3 38.8 %, 240 examples
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 2: 0.3 99.8 %, 1200 examples

Thank you for any help you might provide.

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: xDAWN P300 speller training xDAWN + Classifier.

Post by jtlindgren »

Hi User, these are questions mainly related to machine learning practices. I'd recommend you to read up on the subject
For example the famous textbook by Hastie, Tibshirani and Friedman is freely available and can be found by google.

The eigenvalues illustrate how the selected xdawn filters contribute to reconstructing the matrix decomposed by the eigenvector decomposition. Basically a high eigenvalue means that the corresponding filter is important. A quickly dropping eigenvalue distribution can suggest how many components to keep.

If training error is higher than the crossvalidation error, this means the classifier likely overfits the data. This is tricky to evaluate with eeg, see the doc on OV documentation page on how to crossvalidate better. Most practical way though might be to record separate training and test session (but use the test session only once - if you tweak pipeline to do well on prerecorded test set, the results will again be optimistic.

The thing you print there is the confusion matrix, it shows how much each class was predicted as itself or some other task. Here the number is per flash. However, since p300 combines many predictions to spell a letter, this will not exactly correspond to spelling accuracy. Nevertheless, the matrix can be used for diagnosis, e.g. to see if flash or no flash is never predicted. In a good situation you'll want the percentages on the diagonal in the matrix.

Cheers,
Jussi

Post Reply