Application requirements advice

[terry]

Hi, I am new to OpenViBE and what it is capable of but it looks really exciting and maybe just what I am looking for. I could do with some help and advice from any willing experienced users. So, do you think that standard, out of the box, OpenViBE could be used to develop an application to support the following functions? (I know many of the detailed functionality below is standard however it makes it far easier to follow if I include the full requirements).

a. Capture data from a NeuroSky headset (think this is standard).
b. Training data would then be captured for several patterns/classes of brainwaves to obtain a profile for certain actions or tasks (such as clicking a mouse, deep thought, agitation, secondary conflicting stimulus, etc). Each class would then be stored for the individual to build up their own unique profile.
c. Participants would then be asked to perform a series of tasks and real time data would be collected to providing feedback against each one of the profile/classes.
d. Then the system would provide an analysis of each of the classes such as total cognitive load, timings etc.
e. The system would also display a graphical indication of each of the classes – possibly a bar graph with columns for the classes plus one for total cognitive load (possibly live plus accumulated).
f. Keep a database of the profiles/individuals for future use by the user.
g. Allow parameters to be input that could alter the maximum cognitive load available (not feeling too good, distracted by other factors, tired from other work). These constraining factors could be requested prior to the session, and/or at the end to gather stats. This would be stored against the user’s profile.
h. Provide statistics/graphs at the end of the session that gave a view of the user’s overall performance.

And, if it is possible, do you know any institutions or individuals that would be interested in helping developing this new application?

Many thanks in anticipation of your assistance.

Terry Wilcox
http://www.theoryofhumanexcellence.org

[lbonnet]

Hello Terry and welcome on board !
Let's take your questions one by one...

a. Capture data from a NeuroSky headset (think this is standard).

Indeed it is standard now. You juste have to turn on the MindSet, connect it through the bluetooth manager (configured with lowest COM port available), and run the Neurosky driver in the acquisition server. More information in this old news post.

b. Training data would then be captured for several patterns/classes of brainwaves to obtain a profile for certain actions or tasks (such as clicking a mouse, deep thought, agitation, secondary conflicting stimulus, etc). Each class would then be stored for the individual to build up their own unique profile.

This sounds like a complete signal processing chain with feature extraction and classification.
The standard OpenViBE platform can handle some of your needs, however you must be aware of what you are trying to do, and its complexity.
Acquiring data and storing it in files for further use is OK and can be done directly with the standard OpenViBE without being a BCI specialist.
Building several classifiers to discriminate many mental states is a much more complicated task. We provide the LDA classifier (linear discriminant analysis) that try to separate the incoming features in 2 classes. If you need multi-class classification, I'm afraid It will require another classification algorithm, not included in OpenViBE. The included boxes can be used to implement some feature extraction techniques, but you need to know what you are doing, what signal processing concepts are behind (such as temporal filtering, FFT or epoching).

c. Participants would then be asked to perform a series of tasks and real time data would be collected to providing feedback against each one of the profile/classes.

You can save the EEG+events data with OpenViBE. However, the "series of tasks" must be handled through a dedicated application I suppose. The software platform provides a convenient framework to prototype some simple applications that can be connected with OpenViBE. See this tutorial for example.

d. Then the system would provide an analysis of each of the classes such as total cognitive load, timings etc.

OpenViBE does not provide all-in-one analysis bricks, but it is still possible to use the included boxes to process the signal and find what you want.
For example you can monitor some precise frequency bands related to cognitive load using spectral analysis. Anyway building a signal processing requires some knowledge of what you are doing.

e. The system would also display a graphical indication of each of the classes – possibly a bar graph with columns for the classes plus one for total cognitive load (possibly live plus accumulated).

The software provides some visualization widgets for signal, classifier accuracies (bar graph), matrix of values... For example you can display the accuracy of each classifier at the end of a training session. However more specialized visualizations will definitely require the implementation of dedicated boxes.

f. Keep a database of the profiles/individuals for future use by the user.

Not standard
I must say that with that kind of functionality, your application is drifting from the original purpose of OpenViBE (a software for real-time neurosciences).

g. Allow parameters to be input that could alter the maximum cognitive load available (not feeling too good, distracted by other factors, tired from other work). These constraining factors could be requested prior to the session, and/or at the end to gather stats. This would be stored against the user’s profile.

Adapting the system to current user's state sounds like a calibration (?)
You can do some calibration phase before each session, but we don't provide an automatic way to do it.

h. Provide statistics/graphs at the end of the session that gave a view of the user’s overall performance.

This analysis is going out of the software area of expertise (real-time processing). It is an offline analysis, that can be done outside openvibe with dedicated tools.
Matlab with the EEGlab toolbox is a widely used example for offline analysis.

As a conclusion I will say that OpenViBE provides the basic bricks to build, test, improve the BCI signal processing. It also provides convenient way to build your own treatment boxes (for example new classifiers) or application prototypes. The software platform does not provide advanced visualizations or end-user GUI, because it's not its purpose.
Moreover, OpenViBE is designed for real-time practices. For offline analysis, other tools will handle the job more efficiently (Matlab+eeglab or python for example).

As regards the Neurosky Mindset, I suggest you further investigate what is possible to do with it. Try their applications, make your own idea.
You should keep in mind that simple fact : the MindSet has one electrode, on an unprecise frontal position. A medical certified EEG like g.Tec g.USBAmp has 16 electrodes on a precise cap. An EGI sensor array has 250+ channels. The hardware naturally limits what you can do, what precision (spatial and temporal) you can reach.
In my opinion you can build a BCI with Neurosky MindSet, if you only focus on what it is possible to detect with 1 electrode on the frontal site (which will be naturally not so much).
Moreover you should know that even with state-of-the-art EEG amplifiers, specialists often focus on only one or two mental tasks.

You seem really excited by your project and I'm glad you are !
BCI have thrilling applications, but they require some prior knowledge that can't be skipped.
I hope you have a more clear view of OpenViBE and what it can do for you.
If you want to contribute in the project it will require some programming skills, and we will do our best to help you if you choose this path!

Best regards,

Laurent-

[terry]

Hi Laurent

Many, many thanks for your answer. Yes, I am very, very excited regarding my thoughts on OpenVibe and what it can do. Watched the videos and read a lot of the info on the site. Sorry about posting message twice but did not get any response for several days so thought I had put it in the wrong place.

So, I understood it all except the very last statement. Could you please clarify this bit:

If you want to contribute in the project it will require some programming skills, and we will do our best to help you if you choose this path!

So, although I am computer literate, it is a long time since I did any real techie stuff like programming (probably in my degree 30 years ago in FORTRAN). Still OK at reading manuals and making things work! But the programming bit is important.

So, to clarify a few things that may help you give a little more advice:

The some of the major classes will be very similar (types of decisions - automatic, intuitive, deliberate plus day dreaming type thought). If I could start with these, that would be great. Then there would be other classes which I guess are along the same lines as driving a game .. clicks, mouse control, but these will be actually performed rather than imagined.

So, given these are the types of classes for the analysis, any more thoughts.

Hopefully, if you think it is do-able .. then I am mega excited.

Cheers & thanks for the help,

Terry

[lbonnet]

Hi Terry,

So, I understood it all except the very last statement. Could you please clarify this bit:

If you want to contribute in the project it will require some programming skills, and we will do our best to help you if you choose this path!

I meant that if you want to write your own signal processing or visualization boxes, it will require knowledge of C++ language. If you want to take a look at the implementation, check the developer tutorial pages for code examples.
If your last experience of coding is Fortran 30 years ago, I guess it will be very hard for you unless you take intensive C++ courses
We can help you designing and implementing a new box, but we won't help for purely C++ problems, it would be very time-consuming for everyone.
If you really need such development I suggest you find someone with practical experience of C++. (s)He could use the website tutorials and our occasional help here on the forum (or IRC).

The some of the major classes will be very similar (types of decisions - automatic, intuitive, deliberate plus day dreaming type thought). If I could start with these, that would be great.

I won't say so fast that these "classes" will be similar in terms of BCI. Are they triggering identifiable brainwave patterns ? are they spatially located at the same area of the brain ?
I guess what you could do is recording the EEG and do some signal processing to discriminate these mental states, but I'm not sure which method will be efficient (it is not really my area of expertise, I'm a software engineer )
Anyway if you have some literature about EEG processing for these types of decision, I think you could do it with OpenViBE.

Then there would be other classes which I guess are along the same lines as driving a game... clicks, mouse control, but these will be actually performed rather than imagined.

I don't clearly understand. You want to detect in the EEG if the user clicked/moved a mouse ? It is possible to find some patterns related to hand movement in the motor cortices, but definitely not with a Neurosky MindSet.

Cheers

Laurent