OpenViBE forum

The OpenViBE community
It is currently Fri Aug 17, 2018 7:11 pm

All times are UTC




Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Thu Oct 06, 2016 3:55 pm 
Offline

Joined: Mon Oct 03, 2016 5:54 pm
Posts: 1
Hi everyone,

I am trying to use an OpenBCI headset with the xDawn P300 speller in OpenViBE, but upon running the "online" scenario the only result is appears to predict is 'A'.

Here are some notes about my particular configuration and current debugging state:
[*]I'm using OpenViBE 1.2.2
[*]I'm using the "OpenBCI" acquisition server driver
[*]In the signal monitoring scenario, I can definitely see EEG readings
[*]I'm currently using the following tutorial: http://openvibe.inria.fr/p300-speller-xdawn/
[*]I'm running OpenViBE on a Ubuntu virtual machine (with USB-passthrough for the OpenBCI dongle) contained on an OSX host
[*]The training scenario reports an 80-90% accuracy (I've tried this a few times)
[*]The online scenario never returns anything other than the letter 'A' for predictions
[*]The signal decimation has always returned an error (since sampling rate and sample count per buffer are not compatible). I have fixed this by changing the signal decimation factor from the default (4 I believe) to 2 in the three scenarios where it is relevant. I have a suspicion this may be related to the problem, but I'm not sure what the proper fix for it is.

Does anyone have any recommendations?

My current attempt at improving this is to use a native Windows 7 host machine with OpenViBE to try and reduce the number of moving parts (as maybe latency is a problem). While I'm setting that up, I thought I would see if anyone in the community has any advice they would give.


Top
 Profile  
Reply with quote  
PostPosted: Fri Oct 14, 2016 12:03 pm 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 733
Location: INRIA Rennes, FRANCE
Hi Chris,

P300 like all BCI, is tricky to diagnose. P300 is made more difficult by the fact that the alignment of the markers (stimulations) needs generally to be quite precise. Random ideas,

1) What does the 'replay' scenario give you, does it work with the data you recorded for the training? It should.

2) get the P300 test dataset from the page

http://openvibe.inria.fr/datasets/

and run the scenarios with that. If it works, this suggests that the problem might be somewhere in the data: alignment of the stimulations or the content of the signal itself. If it doesn't, something has broken in OV or the scenarios.

3) You can also take a look at the individual flash predictions which get summed up by the .lua script. Do they make sense? Are the predictions indicating A is the correct answer?

VM might affect the timing somehow. I haven't run BCI over a VM myself. Usually I try to minimize all possible complications.


Three cents,
Jussi


Top
 Profile  
Reply with quote  
PostPosted: Wed Oct 19, 2016 4:02 pm 
Offline

Joined: Tue Oct 18, 2016 4:00 pm
Posts: 1
jtlindgren wrote:
Hi Chris,

P300 like all BCI, is tricky to diagnose. P300 is made more difficult by the fact that the alignment of the markers (stimulations) needs generally to be quite precise. Random ideas,

1) What does the 'replay' scenario give you, does it work with the data you recorded for the training? It should.

2) get the P300 test dataset from the page

http://openvibe.inria.fr/datasets/

and run the scenarios with that. If it works, this suggests that the problem might be somewhere in the data: alignment of the stimulations or the content of the signal itself. If it doesn't, something has broken in OV or the scenarios.

3) You can also take a look at the individual flash predictions which get summed up by the .lua script. Do they make sense? Are the predictions indicating A is the correct answer?

VM might affect the timing somehow. I haven't run BCI over a VM myself. Usually I try to minimize all possible complications.


Three cents,
Jussi


I'm also having the same problem.

1) Replay doesn't give me any results either from my online run.
2) I tried using those XMLs and still only gives me "A," however when I use the dataset to run a replay, I can see that it works.
3) It says "A" is the wrong answer.


Top
 Profile  
Reply with quote  
PostPosted: Thu Oct 20, 2016 9:49 am 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 733
Location: INRIA Rennes, FRANCE
Hi, one way to see what the classifier is predicting is to set

Code:
   do_debug = true


in p300-speller-accumulator.lua .

In P300, there are only two classes underneath corresponding to the state 'did the P300 phenomenon happen in the brain' with the classes being 'yes' and 'no'. The phenomena happens when the flash hits the letter the user is attending to. However, most of the flashes are on letters that the user is *not* attending. Hence, when the classifier is estimated, there are much more 'no' classes, so if the classifier can't do better, it will predict 'no'. This means that its best dumb guess is to say 'no' always. So if your classifier is just predicting the majority class, it will predict that the P300 did *not* happen ever. Further, if on every flash the answer is 'no p300' from the classifier, the result will be integrated to 'A'.

With that debug enabled, If you see only lines like

91 sec <Box algorithm::(0x0000445b, 0x000068e3) aka P300 accumulator> <In Script> Push col target 5
[ INF ] At time 165.078 sec <Box algorithm::(0x0000445b, 0x000068e3) aka P300 accumulator> <In Script> Pred fifo row 3 is nontarget

the key is the 'nontarget' there, you have essentially a situation that your classifier is always predicting the same 'no', so no row or column will get any 'votes', and finding the max over an array of zeroes in .lua returns the first index, hence A.

Anyway, thats for the explanation. Why this happens is another question. How I'd test if you have reasonable data is to do the following,

1) Do two separate recordings with p300-xdawn-1-acquisition.xml
2) Use the first recording to train the xdawn and the classifier (scenarios 2 and 3)
3) Use the first recording to run 'replay'
4) Use the second recording to run 'replay'

Now the expectation should be that if you have decent data, both steps should give decent accuracy, and step 3 should be better than step 4. If both break badly, either your data is terrible or there is a bug. If 3 works but 4 doesn't, your datasets don't resemble each other (some drift of amp or brainstate) or the classifier overfits badly. If both work, the online scenario should as well, you can try this by replacing the Acquisition Client box with the Generic Stream reader reading from a file. Note that in this case you wont get the correct predictions, but you shouldn't see 'A's either.

In any case, if you can gather a more specific understanding of what works and what doesn't, it'd be easier to home in on the problem.

ps. If anybody is reading this who has successfully used P300 in OV 1.2.x, let us know! So far I only know that it worked for me in the spring 2016, using gtec gusbamp with 16 channels placed more or less equidistant around the head.


Best,
Jussi


Top
 Profile  
Reply with quote  
PostPosted: Tue Feb 07, 2017 10:29 am 
Offline

Joined: Wed Aug 10, 2016 8:02 pm
Posts: 2
jtlindgren wrote:

ps. If anybody is reading this who has successfully used P300 in OV 1.2.x, let us know! So far I only know that it worked for me in the spring 2016, using gtec gusbamp with 16 channels placed more or less equidistant around the head.


Best,
Jussi


I had successfully, run the P300 scenario quite esay (the standard one without xDAWN), using gtec's gusbamp with 8 channels in both online modes (copy spelling and free spelling), I run the experiments with 10 sequences and reached a good accuracy (90 %).

by the way, my OV version is 1.2.2


Top
 Profile  
Reply with quote  
PostPosted: Fri Jun 30, 2017 7:41 am 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
I am having exactly the same issue, did anyone manage to find a solution ?

Thanks
Patrick


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 11:52 am 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
Hi Jussi,
I am a PhD student doing my studies on BCI. I am currently using OpenBCI and was evaluating the P300 on OpenVibe, however like the post says I am always getting an A.

These are to answer your comments:
The dataset download from the website ran perfectly.
I am not running any VM. I am running a Windows 7 machine.
I have tried the first and second recording as you suggested, however both failed.

I am attaching a number of images of my setup. If you check after running the classifier *everything* is labelled as "nontarget".

Could it be the acquisition client? I am reverting to Jeremy's python script and running it on a generic streamer to see the results. Will let you know how it goes.

In the meantime if you have any other ideas, please do let me know; stuck on it at the moment.

Thanks
Patrick

Image
Image


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 11:56 am 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
I also forgot to mention, if I run the acquisition recording on the reply, it shows only A as result as well. So the problem is not actually within the train and classifier scenario.


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 12:09 pm 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 733
Location: INRIA Rennes, FRANCE
Hi,

the output of the classifier trainer box suggests that it has produced a classifier which always predicts the majority label. With this classifier it is not surprising you will always see the same outcome, since it will always vote 'no flash'. As to why the classifier would be like this, I don't know. One reason could be if there is absolutely no p300 effect discernible in the data.


Best,
Jussi


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 1:07 pm 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
Hi Jussi,
thanks for your prompt reply.
I can visually see the P300 effect when I put a Signal Display box when running the replay. I can send you the data if you want to check (however need your email address).

Today and tomorrow I will try to run Jeremy's python script and the Generic Streamer acquisition. He made it work with the previous version using OpenBCI.
My thought is that maybe there is something wrong with the OpenBCI acquisition?

Will let you know how it goes.

Thanks
Patrick


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 1:28 pm 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 733
Location: INRIA Rennes, FRANCE
Hi Patrick,

although it'd be great to get some more test data from different systems (mine is based on gtec's gusbamp), I don't have time at the moment to investigate the issue. Basically I'd do two things,

1) check that after each processing step, the output is what you expect
2) if so, try to see if the classifier can be tweaked somehow to produce better models (you want to get away from 100% in one class kind of predictions)

What does Jeremy's script exactly do that you mention?

Cheers,
Jussi


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 1:41 pm 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
Hi Jussi,
I can understand and relate :)

1) will do
2) tried to tweak the classifier, I set it from 3 output channels to 2 and to 1 and changed countless settings.
If I understand correctly if I run the acquisition (scenario 1) and I replay it in scenario 5, it should give me the correct output letters and not A, right (say target = S, results should be = S)? Or since it has not been processed, it will not display anything useful and will just always show A in this case?

Jeremy had written a python script for OpenBCI to stream data ( https://github.com/jfrey-xx/OpenBCI_Pyt ... /README.md ) and used a Generic Raw Telnet Reader acquisition to successfully make the P300: CoAdapt P300 Speller (OpenViBE v0.18 only) work. Now my thought was to use his script and generic reader to try and see if the xDawn P300 works. If it does, it might be something with the OpenBCI acquisition? If not, I am at a loss at the moment.
The other weird thing is (and don't take my word on it, I just tried it for an hour), i tried the SSVEP scenario and it did not work as well, hence why I suspected its something to do with the acquisition server?

I have used the combo of OpenBCI with ElectroCap for 2 years and managed to publish a paper and several other results, so it is not an issue with the equipment.

Thanks
Patrick


Top
 Profile  
Reply with quote  
PostPosted: Tue Jul 04, 2017 2:45 pm 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
One last thing, I had to eliminate the "Signal Decimation" due to OpenBCI 250 sampling rate (or set it to 2 which is to only multiplier that works for both sampling rate and sample count per buffer). That shouldn't affect the results, am I correct?

Will report my results by tomorrow on the generic stream by tomorrow.

Patrick


Top
 Profile  
Reply with quote  
PostPosted: Wed Jul 19, 2017 1:23 pm 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
Hey Jussi,
sorry for my late reply, been hectic the past weeks.

I have tried Jeremy's python (with generic acquisition) instead of OpenBCI acquisition, but it still gave me all as no hits and display only 'A' both in replay of the train and online.

Tomorrow I will be trying to revert back to Openvibe 0.1.8 and try with the "p300-coadapt-stimulator", so at least I am able to replicate Jeremy's success. If this works fine, it must be something related to the OpenVibe new version or xdawn compatibility with OpenBCI ?

Honestly I am at a loss, apart from reverting to the old version I have no idea what else to try. If you have any more ideas for me to try, please let me know.

Thanks
Patrick


Top
 Profile  
Reply with quote  
PostPosted: Fri Aug 04, 2017 12:17 pm 
Offline

Joined: Thu Jun 29, 2017 9:32 pm
Posts: 14
Hey Jussi,
as per my previous post, I downgraded to Openvibe 0.1.8 with the "p300-codapt-stimulator" and after a couple of tries and some settings changed, it worked.

In the next few days I will reinstall the new version and try again from scratch, maybe I missed something.
If you have any other ideas, please do let me know.

Thanks
Patrick


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Americanized by Maël Soucaze.