Poor performance on motorimagery and P300speller with Emotiv

Working with OpenViBE signal processing scenarios and doing scenario/BCI design
Post Reply
lionkinglc
Posts: 7
Joined: Wed Aug 09, 2017 2:51 am

Poor performance on motorimagery and P300speller with Emotiv

Post by lionkinglc »

Hi everyone

This is my first post in this forum. I just started to use Openvibe to build BCI using Emotiv EPOC+.

Having read related posts in this forum, I chose the scenario with CSP to run motor imagery. Since Emotiv cannot cover C3 and C4, during the testing, I have the headset tilted.

In the first testing, I used the original CSP example. The classification performance is only 60%. During the online session, the classification result is not stable which makes it hard to calculate the performance. It is always like, the classifier first gives a left bar and then shifts to a right bar, or the opposite.

So in the second testing, I added a Channel Selector which selects the two nodes above C3 and C4 before the Temporal Filter. But the classification performance during the training and online session is almost the same.

My question is: am I doing something wrong here? Is there anything I can do to improve the accuracy, or at least makes the prediction from the trained classifier stable?

Previous posts recommend Emotiv may work with ssvep and P300. So I also tried the P300 speller. But again the results were frustrating. The prediction is always A in the end. Could someone give me some advice on this as well?

I will appreciate any light into this.

Thanks,
Chandler

tgaugry
Posts: 68
Joined: Thu Feb 09, 2017 10:17 am

Re: Poor performance on motorimagery and P300speller with Em

Post by tgaugry »

Hi,

After discussing with some colleagues, it seems that they had problems in the past due to the amount of noise on the signal with this headset.
Could you test your signal quality using emotiv software, to see if this is a signal quality problem or not ?

Cheers,

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Poor performance on motorimagery and P300speller with Em

Post by jtlindgren »

Agree with tgaugry, though it might be difficult to say much about signal quality without an expert eye. You can check that the impedances are ok though, with the emotiv panel.

My three cents are in this old thread,

viewtopic.php?f=17&t=9610

Also, xDAWN P300 should work better on random, non-experienced users than motor imagery. I don't have personal experience with either paradigm on Emotiv though.


Cheers,
Jussi

lionkinglc
Posts: 7
Joined: Wed Aug 09, 2017 2:51 am

Re: Poor performance on motorimagery and P300speller with Em

Post by lionkinglc »

Thank you for your replies.

It has been a long time. I have a master student who is working on this, but the progress is still frustrating.

The emotiv panel shows the impedances are okay, and in the signal-monitoring program, eye blinks and jaw clenching are visible.

I tried both motor-imagery and motor-imagery-csp. In both scenarios, I tilted the headset to make sure two nodes cover C3 and C4. In motor-imagery, I removed the surface laplacian and only selected C3 and C4 to analyze. In the one with csp, I also selected C3 and C4 to analyze.

Can anyone give me some more ideas to improve the results? Thank you so much!

Sincerely,
Chandler

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Poor performance on motorimagery and P300speller with Em

Post by jtlindgren »

Limiting the number of electrodes to just C3 and C4 probably makes the classification more difficult. Laplacian and CSP approaches are precisely used because they allow to extract more discriminable features compared to the raw electrode data; single electrode readings are rather noisy aggregates of the potentials propagating to the surface from the myriad different locations inside the brain volume. The results of such feature extraction techniques somewhat correspond to source localization, i.e. in motor imagery they try to isolate the part of the signal thats coming from left and right motor cortices, respectively. This isolation may subsequently make the classification more easy. Intuitively, a band power computer from a single electrode contains more interference than a band power computed only from data originating from relevant cortical areas. The latter is an ideal situation, in practice it is impossible to perfectly localize (isolate) the sources.


Cheers,
Jussi

lionkinglc
Posts: 7
Joined: Wed Aug 09, 2017 2:51 am

Re: Poor performance on motorimagery and P300speller with Em

Post by lionkinglc »

Hi Jussi,

Thank you for your fast response and good advice.

This time I included 8 nodes in motor-imagery-csp. They are AF3, AF4, F3, F4, F7, F7, FC5, FC6 on emotiv, but again I tilted it so two nodes cover C3 and C4, with the others around them.

I have to say the cross-validation accuracy is surprisingly high at 80%. However, when I used the trained classifier to test its online performance, the results only show left even though I was moving my right hand. and of course, I kept the headset in the same position during the training process.

I wonder how to get a good online result, and I appreciate any light into this.

Thanks and cheers,
Chandler

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Poor performance on motorimagery and P300speller with Em

Post by jtlindgren »

Some introspection and speculation here. Personally to get it better, I needed to practice myself several sessions, though even with that it was difficult and I was easily distracted from the task or by the feedback. Especially the classification not giving the result I wanted was something that easily spoiled my mood, dropped me out of the 'flow' and made it even more difficult. Something like a year ago we tried motor imagery with some naive subjects (they had no previous experience) using gtec usbamp as well as a high-class EGI amplifier. Out of the 5 subjects we had, 1 could reach non-random accuracy in the online session. If two records were made, both of them offline (i.e. two training sessions, one used as train, the other as test), in offline analysis the subjects actually performed better (better classification accuracy) than the online suggested. I suspect this was because they were not distracted by the feedback, internal pressure to succeed and then annoyance from failure -- all these may be reflected in the EEG.

The reason that the classifier tilts to one side is likely due to nonstationarity, that is, the EEG has changed its nature from the train to the test session and the decision surface that was good during training is no longer good. I suppose subjects experienced in motor imagery may be able keep the EEG more steady, but I can't presently remember if there are papers showing this.

Afaik, as far as our current capability to process the EEG stands, Motor Imagery demands more skill from the user than P300 or SSVEP which are more like 'reflexes'.


Best,
Jussi

kiyarash
Posts: 27
Joined: Tue Apr 11, 2017 10:44 am

Re: Poor performance on motorimagery and P300speller with Em

Post by kiyarash »

lionkinglc wrote:Hi Jussi,

Thank you for your fast response and good advice.

This time I included 8 nodes in motor-imagery-csp. They are AF3, AF4, F3, F4, F7, F7, FC5, FC6 on emotiv, but again I tilted it so two nodes cover C3 and C4, with the others around them.

I have to say the cross-validation accuracy is surprisingly high at 80%. However, when I used the trained classifier to test its online performance, the results only show left even though I was moving my right hand. and of course, I kept the headset in the same position during the training process.

I wonder how to get a good online result, and I appreciate any light into this.

Thanks and cheers,
Chandler

Hi I have the same low online accuracy as you. Were you eventually able to get a good online accuracy using the emotive epoc+ in the MI CSP scenario ?

if yes would you share your experience. (how did you tilt the headset or any other change that made things better)

Best regards,
Kiarash

Post Reply