Hello, i'm Florian LARRUE, cogntive sciences student phd in interfaces and virtual reality in laboratory EA 487 Cognition et Facteurs Humains, University Bordeaux II, France.
I work actually on BCi and displacement in large space and i would like to separate cognitive and motor activities using openvibe.
I'm totally newbie to configure openvibe and in signal theory. I don't find electrodes for laplacian surface, and can't define best frequencies of signal ( i use generic frequencies for all subjects).
Openvibe server is connected with deltamed driver on port 1024, and actually i use 32 packets by samples ( i think i could put 8 or 16 but i don't know influences on classification).
The goal of this study is to make subjects learn a way in a virtual environment with thought and after, examine wether subject can reproduce the same way in real life.
Driver for deltamed has been developed and functions correctly. For information, frequency acquisition of the system is 256 hz and the system writes informations in an EEG file by packets of height ( for the twenty three EEG channels).
The system is correctly connected to our virtual environment with VRPN communication, but the problem is that classification is not very good (about 65 %).
I would like that, when people think left in virtual environment, the system thinks left (idem for right ) too, but actually the system always turns towards left or right without no non control command ( i need an asynchronous system).
Learning scenarios are already created, but the classification percentage is not very good, i need knowledge on parameters and signal theory!!!
Thanks a lot for your help and sorry for my bad English.
F.LARRUE
Deltamed System and real time acquisition
-
fabien.lotte
- Posts: 112
- Joined: Sun Mar 14, 2010 12:58 pm
Re: Deltamed System and real time acquisition
Hi Florian,
Welcome on the OpenViBE forum! Here are some suggestions for some of your problems.
Low classification performance: Generally this can be improved by using appropriate and highly subject-specific spatial and temporal filtering.
- For spatial filtering, the best approach would be to use something like CSP (Common Spatial Pattern), but if I am not wrong it is not implemented yet in OpenViBE. Surface Laplacian would be another (although not as good) solution. It is achieved by mutiplying the signal of a given channel by N, where N is the number of its neighboring channels and then subtracting from it the signals from these neighboring channels multiplied by 1/N. For instance if you have channels C5, C3, C1, CP3 and FC3 and you want a laplacian channel over C3, you will build a new channel whose signal is equal to 4*C3 - 1/4 * (C5 + C1 + CP3 + FC3). The spatial filter box can be used to design this laplacian channel. Typically for left/right hand motor imagery-based BCI, we would use laplacian channels over C3 and C4.
- For temporal filtering, it is also better to use subject-specific frequency bands when designing a motor imagery based BCI. We know that the mu (~8-12 Hz) and beta (~16-24 Hz) rhythms are involved, but these rhythms are usually slightly different for each person, and its better to take this specificity into account. Some machine learning/statistical approaches have been proposed in the BCI literature to do that, or alternatively you can also use feature selection techniques to do the job. In openvibe, there is a box to do that, but it is only available for the linux version, in my branch (flotte), not in the trunc. This method is roughly what is described in this paper: M. Zhong, F. Lotte, M. Girolami, A. Lécuyer, "Classifying EEG for Brain Computer Interfaces Using Gaussian Processes", Pattern Recognition Letters, vol. 29, no. 3, pp. 354-359, 2008 (available here: http://hal.inria.fr/docs/00/26/55/11/PD ... _draft.pdf). Another possibility to find these frequency bands, which is highly unconvenient but still works, is to look for them by hand! So typically you would change the settings of the temporal filters in openvibe (by slightly changing the width and position of the mu and beta bands - changing only one band at a time would be of course more efficient), and by looking at the cross-validation performance returned by the training of the classifier, you would know which frequency bands are (roughly) the best.
Asynchronous BCI design: This is a tricky part, as most current BCI are still synchronous. Typically, to design an asynchronous (also known as self-paced) BCI, you need to be able to identify the 'non-control state', i.e., when the user does not want to send any command. This can be achieved in different ways, by using a separate classifier discriminating non-control from intentional control states, by learning the non-control state as an additional class in your problem or by using thresholds on the output of your classifier (see, for instance, F. Lotte, H. Mouchère, A. Lécuyer, "Pattern rejection strategies for the design of self-paced EEG-based Brain-Computer Interfaces", International Conference on Pattern Recognition (ICPR), pp.1-5, 2008 - available here: http://hal.inria.fr/docs/00/31/08/78/PDF/icpr08.pdf). If you're not familiar with signal processing or machine learning, the last approach may be the easiest to do although it is generally not the most efficient. Typically, with this approach, you will issue a command only if the output of your classifier (e.g., absolute value of the output of the LDA classifier, which corresponds to the distance to the separating hyperplane) is higher than a given threshold (to be empirically identified). You could also use temporal threshold, a.k.a, dwell time, which means that a given command must be issued continuously for a minimum given duration before being really executed. See this interesting paper for more discussion on these threshold topics: G. Townsend, B. Graimann, and G. Pfurtscheller.
"Continuous EEG classification during motor imagerysimulation of an asynchronous BCI". IEEE Trans. on Neural
Sys. and Rehab., 12(2):258265, 2004.
I hope it helps!
Fabien
Welcome on the OpenViBE forum! Here are some suggestions for some of your problems.
Low classification performance: Generally this can be improved by using appropriate and highly subject-specific spatial and temporal filtering.
- For spatial filtering, the best approach would be to use something like CSP (Common Spatial Pattern), but if I am not wrong it is not implemented yet in OpenViBE. Surface Laplacian would be another (although not as good) solution. It is achieved by mutiplying the signal of a given channel by N, where N is the number of its neighboring channels and then subtracting from it the signals from these neighboring channels multiplied by 1/N. For instance if you have channels C5, C3, C1, CP3 and FC3 and you want a laplacian channel over C3, you will build a new channel whose signal is equal to 4*C3 - 1/4 * (C5 + C1 + CP3 + FC3). The spatial filter box can be used to design this laplacian channel. Typically for left/right hand motor imagery-based BCI, we would use laplacian channels over C3 and C4.
- For temporal filtering, it is also better to use subject-specific frequency bands when designing a motor imagery based BCI. We know that the mu (~8-12 Hz) and beta (~16-24 Hz) rhythms are involved, but these rhythms are usually slightly different for each person, and its better to take this specificity into account. Some machine learning/statistical approaches have been proposed in the BCI literature to do that, or alternatively you can also use feature selection techniques to do the job. In openvibe, there is a box to do that, but it is only available for the linux version, in my branch (flotte), not in the trunc. This method is roughly what is described in this paper: M. Zhong, F. Lotte, M. Girolami, A. Lécuyer, "Classifying EEG for Brain Computer Interfaces Using Gaussian Processes", Pattern Recognition Letters, vol. 29, no. 3, pp. 354-359, 2008 (available here: http://hal.inria.fr/docs/00/26/55/11/PD ... _draft.pdf). Another possibility to find these frequency bands, which is highly unconvenient but still works, is to look for them by hand! So typically you would change the settings of the temporal filters in openvibe (by slightly changing the width and position of the mu and beta bands - changing only one band at a time would be of course more efficient), and by looking at the cross-validation performance returned by the training of the classifier, you would know which frequency bands are (roughly) the best.
Asynchronous BCI design: This is a tricky part, as most current BCI are still synchronous. Typically, to design an asynchronous (also known as self-paced) BCI, you need to be able to identify the 'non-control state', i.e., when the user does not want to send any command. This can be achieved in different ways, by using a separate classifier discriminating non-control from intentional control states, by learning the non-control state as an additional class in your problem or by using thresholds on the output of your classifier (see, for instance, F. Lotte, H. Mouchère, A. Lécuyer, "Pattern rejection strategies for the design of self-paced EEG-based Brain-Computer Interfaces", International Conference on Pattern Recognition (ICPR), pp.1-5, 2008 - available here: http://hal.inria.fr/docs/00/31/08/78/PDF/icpr08.pdf). If you're not familiar with signal processing or machine learning, the last approach may be the easiest to do although it is generally not the most efficient. Typically, with this approach, you will issue a command only if the output of your classifier (e.g., absolute value of the output of the LDA classifier, which corresponds to the distance to the separating hyperplane) is higher than a given threshold (to be empirically identified). You could also use temporal threshold, a.k.a, dwell time, which means that a given command must be issued continuously for a minimum given duration before being really executed. See this interesting paper for more discussion on these threshold topics: G. Townsend, B. Graimann, and G. Pfurtscheller.
"Continuous EEG classification during motor imagerysimulation of an asynchronous BCI". IEEE Trans. on Neural
Sys. and Rehab., 12(2):258265, 2004.
I hope it helps!
Fabien
Re: Deltamed System and real time acquisition
Hello
thanks a lot for all your informations.
I've read articles that you gave me, very interesting but hard to understand for a newbie.
For Laplace filtering, my spatial filter box is apparently ok (there is no CSP in OpenVibe as you said me).
For temporal filtering, i read paper, i don't understand all... but for classification; i tried to change mu and beta signals and classification is better than generic bands; i go to try to use openvibe box with linux system but i don't know how to do for compiling openvibe with linux... A virtual machine is set up on my computer with ubuntu 10.10.
For asynchronous BCI, i think i will use SVM algorithm because the article saids that non linear classifier is better for non control state command. But in openvibe, must i put a new class detection for non control command because i have in my box "situmlation based epoching" OVTK_GDF_LEFT and in another box OVTK_GDF_RIGHT. I seen that they had a parameter OVTK_GDF_REJECTION; must i class with this new parameter? Must I class three classes or only two (left and right command) because its said that a rejection class is better than a specialized classifier and thresholds are the less efficient? (In the last article "Continuous EEG classification during motor imagery simulation of an asynchronous BCI", it is said also that there are three classes: right,left, and rest).
I don't know how to put a threshold in openvibe to forbid data when data are not superior to threshold.
After reading the last paper, i don't arrive to know if we have to use threshold and dwell... I go to try with and without, with dwell of 0.25 s and refractory period between 1.0 and 1.25 seconds (like in the last paper).
Thanks for your help.
Syncerely.
Florian
thanks a lot for all your informations.
I've read articles that you gave me, very interesting but hard to understand for a newbie.
For Laplace filtering, my spatial filter box is apparently ok (there is no CSP in OpenVibe as you said me).
For temporal filtering, i read paper, i don't understand all... but for classification; i tried to change mu and beta signals and classification is better than generic bands; i go to try to use openvibe box with linux system but i don't know how to do for compiling openvibe with linux... A virtual machine is set up on my computer with ubuntu 10.10.
For asynchronous BCI, i think i will use SVM algorithm because the article saids that non linear classifier is better for non control state command. But in openvibe, must i put a new class detection for non control command because i have in my box "situmlation based epoching" OVTK_GDF_LEFT and in another box OVTK_GDF_RIGHT. I seen that they had a parameter OVTK_GDF_REJECTION; must i class with this new parameter? Must I class three classes or only two (left and right command) because its said that a rejection class is better than a specialized classifier and thresholds are the less efficient? (In the last article "Continuous EEG classification during motor imagery simulation of an asynchronous BCI", it is said also that there are three classes: right,left, and rest).
I don't know how to put a threshold in openvibe to forbid data when data are not superior to threshold.
After reading the last paper, i don't arrive to know if we have to use threshold and dwell... I go to try with and without, with dwell of 0.25 s and refractory period between 1.0 and 1.25 seconds (like in the last paper).
Thanks for your help.
Syncerely.
Florian
-
fabien.lotte
- Posts: 112
- Joined: Sun Mar 14, 2010 12:58 pm
Re: Deltamed System and real time acquisition
If I am not wrong, I think that there is a crop or threshold box in openvibe (Yann, Laurent?) that can be used to threshold the classifier output. Anyway, whether you use a threshold, an additional class or a specialized classifier, the particular class label you give to the "non-control state" is not really important. It is probably clearer to use as label something as "OVTK_GDF_REJECTION" as you mention, but what is really important is how you use this label in your application to which openvibe sends the commands. Typically, if your application receives "OVTK_GDF_LEFT" or "OVTK_GDF_RIGHT", it will trigger a camera rotation towards the left or the right in the virtual environment, whereas if it receives "OVTK_GDF_REJECTION", no action will be executed. By the way, you can also deal with the thresholds (if you choose to use threshold) in your VR application rather than in openvibe, if it is more convenient for you. In this case, you will use openvibe tol send analog commands to your application with VRPN (rather than switch/button commands), and an action will be executed in your application only if this analog command is higher than a given threshold.
Re: Deltamed System and real time acquisition
Dear Fabien & Florian,
indeed, there is a Crop box but I recommand the use of the Simple DSP box that will probably be more flexible to achieve what Florian wants to do. Just use the <, > and ?: operators in order to transform the values
Hope this helps,
Yann
indeed, there is a Crop box but I recommand the use of the Simple DSP box that will probably be more flexible to achieve what Florian wants to do. Just use the <, > and ?: operators in order to transform the values
Hope this helps,
Yann
my blog
my tweets
my linkedinRe: Deltamed System and real time acquisition
hello
thanks for your informations and your help, in fact problem was just my DSP Box was not the last compiled...
I use now SVM Machine, it works better than LDA algorithm, and i go to try to use BLIFF for having best performances on signals but i 've problems to compile OpenVibe with Linux; "dist" folder is not created when i compile openvibe with "linux-build" file, so i can't use "linux-test" file.
Thanks for your help.
F.LARRUE
thanks for your informations and your help, in fact problem was just my DSP Box was not the last compiled...
I use now SVM Machine, it works better than LDA algorithm, and i go to try to use BLIFF for having best performances on signals but i 've problems to compile OpenVibe with Linux; "dist" folder is not created when i compile openvibe with "linux-build" file, so i can't use "linux-test" file.
Thanks for your help.
F.LARRUE
Re: Deltamed System and real time acquisition
Dear flobx,
the dist folder should be created at then end of the build process. I suppose it fails to build the software. Can you send us the output of the console when it actually fails ? Also please tell us the distribution & version you are using.
Regards,
Yann
the dist folder should be created at then end of the build process. I suppose it fails to build the software. Can you send us the output of the console when it actually fails ? Also please tell us the distribution & version you are using.
Regards,
Yann
Re: Deltamed System and real time acquisition
hello
sorry for my late response.
Happy new year to you!
i 've Oracle VM Virtual Box to emulate Ubuntu Version 10.10.
Openvibe installation is in folder usr\openvibe\. When i click on file "linux-build" in usr\openvibe\scripts\, compilation starts and when it's finished, console closes by itself; so I have no message error; but "dist" folder is not created and so, I can't start file "Linux-test" (situated in usr\openvibe\scripts)
Thanks you again for your help.
Regards,
Florian
sorry for my late response.
Happy new year to you!
i 've Oracle VM Virtual Box to emulate Ubuntu Version 10.10.
Openvibe installation is in folder usr\openvibe\. When i click on file "linux-build" in usr\openvibe\scripts\, compilation starts and when it's finished, console closes by itself; so I have no message error; but "dist" folder is not created and so, I can't start file "Linux-test" (situated in usr\openvibe\scripts)
Thanks you again for your help.
Regards,
Florian
Re: Deltamed System and real time acquisition
Dear flobx,
you should probably open up a console and run linux-build from there
Please send us the output of the build process or even a snapshot of the your console at the end of the build process.
Regards,
yann
you should probably open up a console and run linux-build from there
Please send us the output of the build process or even a snapshot of the your console at the end of the build process.
Regards,
yann
Re: Deltamed System and real time acquisition
hello sorry, i m very bad under linux, too long time that i did not use this system...
i've found for having linux console and to start linux-build under a terminal console...
This is the screenshot of error
thanks for your help!
Florian
i've found for having linux console and to start linux-build under a terminal console...
This is the screenshot of error
thanks for your help!
Florian
- screenshot error
- screenshot.jpg (161.64 KiB) Viewed 14395 times
Re: Deltamed System and real time acquisition
Dear flobx,
based on the messages here, I see that Ogre3D / OIS could not be found. Did you correctly install the dependencies using the linux-install_dependencies script ?
Yann
based on the messages here, I see that Ogre3D / OIS could not be found. Did you correctly install the dependencies using the linux-install_dependencies script ?
Yann
Re: Deltamed System and real time acquisition
hello yann, yes i installed linux dependencies, i 've configured ogre manually, and i 've another problem now with boost. I think my computer doesn't detect linux dependencies... I give you anoter snapshot of the new error.
Regards.
Florian
Regards.
Florian
- Attachments
-
- screenshot2.JPG (131.12 KiB) Viewed 14383 times
Re: Deltamed System and real time acquisition
Dear flobx,
if you use a supported Linux distro + you have root access in order to install native packages, then the dependency installation + the build process should be straight forward.
In order to sort this out, please provide the following :
Then just send us the txt file instead of splash screens
Yann
if you use a supported Linux distro + you have root access in order to install native packages, then the dependency installation + the build process should be straight forward.
In order to sort this out, please provide the following :
- The output of the linux-install_dependencies
- The content of scripts/dependencies/log/ogre.log
- The complete output of the linux-build
Code: Select all
$ whatever_command > dumpfile.txt
Yann
Re: Deltamed System and real time acquisition
Hello Fabien,
Can you please elaborate more on your box? Please provide a name and path to your code.
I read the paper you have provided. It is about Gaussian Process (GP) classification.
This box is suppose to implement GP and provide extra info based on the incoming feature vectors. Is that correct?
This extra info can be used for:
- rejecting some wrong feature vectors
- removing noise and non-events
Something else? I am talking about the general picture and trying to summarize your publication in respect to what is coded and what can be coded.
Cheers,
Anton
Can you please elaborate more on your box? Please provide a name and path to your code.
I read the paper you have provided. It is about Gaussian Process (GP) classification.
This box is suppose to implement GP and provide extra info based on the incoming feature vectors. Is that correct?
This extra info can be used for:
- rejecting some wrong feature vectors
- removing noise and non-events
Something else? I am talking about the general picture and trying to summarize your publication in respect to what is coded and what can be coded.
Cheers,
Anton