- NB: Document based on OpenViBE 0.18.0
CoAdapt P300 protocol & application contributed by Dieter Devlaminck, Loic Mahe & Maureen Clerc @ Inria Sophia-Antipolis.
Document sketched by Loic Mahe @ Inria.
This document presents an overall sketch of the design of the CoAdapt P300 stimulator and the related components (used by the scenarios in
p300-coadapt-stimulator/). For the usage instructions, see here.
Use of OpenGL : The glFinish function block the thread until the current buffer is displayed. By calling the tagging method just after the call to glFinish, we can reliably mark the moment the flashes appear on the screen.
Creation of a window and the OpenGL context : originally done with SDL but SDL is a heavy dependency for such a light use. Other options were SFML and GLFW. Gtk3 allow OpenGL context but we can not upgrade because of lack of support on windows. GLFW was lighter thus was chosen.
Font : originally used FTGL, but we saw we could reuse OpenViBE pango and cairo.
The interaction of the classes that implement the P300 are illustrated in the following figure.
The sequence generator decides which letters are to be flashed together and in what order. Modes are row/column (legacy from original p300 application), riprand (permutation of the groups, change group at each repetition) or file (to read the group from a file, mainly to replay data as it was displayed).
The application receives classification results from openvibe via shared memory. Those are used by the evidence accumulation module to determine which symbol the user was attending to.
The evidence accumulation was originally done in the OpenViBE designer. It was moved inside because this will eventually allow the stimulator to make ‘smarter’ flashes. If two letters are highly probable, the stimulator, now having access to this information, will be able to tell the sequence generator to put these two letters in different groups to discriminate them better.
The stimulator contains the main loop of the program. Based on stimulated time steps, it will go from one state to another (e.g. if the duration of flash is over then it will go to the noflash state). The stimulator then orders the visualiser to update the display according to the state it is in by sending it the necessary informations (namely the state e.g. flash, flash stop, feedback, target, etc … and if necessary the group of letter being flashed).
The visualiser is the main class. It contains the main function. It is in charge of creating the GLFW window and OpenGL context. It delegates the actual drawing to the MainContainer class. The MainContainer class creates the keyboard. The synchronisation is done with this class drawAndSync function which redraws the area (with eventual changes) and tags the moment (by ordering the tagger to send the stimulus it received from the stimulator).
In the keyboard area, a key is created for each different states (flash, noflash, target, …) of a symbol and stored in graphical memory (OpenGL display list, will eventually be changed to VBO buffers).
The word prediction module is optional. It uses what is present in the result area (since the last space) to ask presage (the prediction library) for the predicted words. It displays the seven first answers (number will eventually be configurable) in the slots reserved for predictions if the keyboard layout has them.