OpenViBE forum

The OpenViBE community
It is currently Fri Feb 21, 2020 1:08 am

All times are UTC




Post new topic Reply to topic  [ 2 posts ] 
Author Message
PostPosted: Wed Oct 12, 2016 9:38 pm 
Offline

Joined: Wed May 18, 2016 8:43 pm
Posts: 16
Hello,

I've ran these steps and I have some questions, I'd be glad to find the answer for.

1- What are the eigen values? how are they calculated and used in this algorithm?

2- What is the difference between the cross validation accuracy and the training set accuracy?
2.1- which one do I choose to describe the accuracy of the classifier?
2.2- I've noticed that the relation between cross validation accuracy and the training set accuracy is an inverse relation, such that if the lower the cross validation accuracy the higher the training set accuracy, why?

3- I didn't fully understand this, can someone explain?

<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Cls vs cls 1 2
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 1: 51.2 48.8 %, 240 examples
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 2: 1.5 98.5 %, 1200 examples
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Training set accuracy is 93.3333% (optimistic)
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Cls vs cls 1 2
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 1: 61.3 38.8 %, 240 examples
<Box algorithm::(0x0a5a6a4a, 0x1d92a778) aka Classifier trainer> Target 2: 0.3 99.8 %, 1200 examples

Thank you for any help you might provide.


Top
 Profile  
Reply with quote  
PostPosted: Thu Dec 15, 2016 10:59 am 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 778
Location: INRIA Rennes, FRANCE
Hi User, these are questions mainly related to machine learning practices. I'd recommend you to read up on the subject
For example the famous textbook by Hastie, Tibshirani and Friedman is freely available and can be found by google.

The eigenvalues illustrate how the selected xdawn filters contribute to reconstructing the matrix decomposed by the eigenvector decomposition. Basically a high eigenvalue means that the corresponding filter is important. A quickly dropping eigenvalue distribution can suggest how many components to keep.

If training error is higher than the crossvalidation error, this means the classifier likely overfits the data. This is tricky to evaluate with eeg, see the doc on OV documentation page on how to crossvalidate better. Most practical way though might be to record separate training and test session (but use the test session only once - if you tweak pipeline to do well on prerecorded test set, the results will again be optimistic.

The thing you print there is the confusion matrix, it shows how much each class was predicted as itself or some other task. Here the number is per flash. However, since p300 combines many predictions to spell a letter, this will not exactly correspond to spelling accuracy. Nevertheless, the matrix can be used for diagnosis, e.g. to see if flash or no flash is never predicted. In a good situation you'll want the percentages on the diagonal in the matrix.

Cheers,
Jussi


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Americanized by Maƫl Soucaze.