Page 1 of 1

high cross-valid accuracy poor prediction in online scenario

Posted: Tue Dec 11, 2018 2:03 pm
by umar
Hello,
Could someone explain why cross validation of accuracy of about 80% on xDawn p300 speller, yet it could not not correctly predict any single letter in an online scenario? Although, the class wise accuracy is very poor as it is between 4-5 % for the target, and 90-95 % for the non-target. Is this normal? Could there be any known cause of this? if not how can one improve on it? Can connectivity measure when training CSP spatial filter trainer help?
Thanks!

Re: high cross-valid accuracy poor prediction in online scen

Posted: Wed Dec 19, 2018 6:29 am
by jtlindgren
Hi, thats often asked why the xval accuracy might exceed onlibe accuracy. Have you read this?

http://openvibe.inria.fr/tutorial-how-t ... te-better/

There could also be a bug, its not impossible. You are also not telling which amp and electrode layout you are using.

We should put these points to the FAQ or even write a separate document about what to do in case of poor perf. Maybe I'll sketch one during christmas.

Cheers,
Jussi

Re: high cross-valid accuracy poor prediction in online scen

Posted: Sun Dec 23, 2018 9:51 pm
by umar
Hi Jussi,
Many thanks for the reply. Yes, I later came across the cross -validation doc.
Best
Umar