Incremental classifier learning

Concerning processing components: filters, file load/save, visualizations, communication ...
Post Reply
xskola
Posts: 12
Joined: Tue Mar 13, 2018 12:39 pm

Incremental classifier learning

Post by xskola »

Hi, I am trying to implement incremental learning for an MI scenario. I set up a simple scenario (following the example in OV stock scenario files) that re-trains my classifier after the stimulation from clock stimulator is sent. This works, however, I noticed that after the training is triggered, the classifier is trained on the set of new feature vectors only, replacing the old classifier settings completely. This has an issue - the insufficient number of vectors generated during the currently ongoing scenario cannot actually train the classifier very well.

My idea would be to have an incremental learning that blends the new feature vectors with the original set of features, as they come into the classifier. So the original classifier configuration is not forgotten when the new set of features arrives.

Is there a way how to achieve this? For example somehow "pre-load" the classifier trainer with the original features (from a file), and then let it add more features on the fly? Or somehow concatenate the saved signal file with the newly recorded data incoming from the acquisition server?

Thanks,
Filip

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Incremental classifier learning

Post by jtlindgren »

Hi,

You mean incremental between sessions? In single session the example bundled in box-tutorials/ should retrain the classifier with all the data so far. Perhaps you're doing something differently?

If you wish to do incremental between sessions the most lightweight might be to save each feature vector set to the disk, catenate these and use stream switch box or some other trickery to bootstrap the classifier first before feeding it the current sessions features. There might be an additional challenge from the timestamping of the previous features, possibly making the kernel delay them. This could be hackable around by saving the vectors to csv and making a script to set all timestamps to 0.

With C++ it'd be relatively simple to add a feature to the training box that it takes an initial set of training examples from the disk.



Hope this helps,
Jussi

xskola
Posts: 12
Joined: Tue Mar 13, 2018 12:39 pm

Re: Incremental classifier learning

Post by xskola »

Thank you for the reply! Yes, i actually meant between-session incremental learning. I was able to read the necessary feature vectors (modified to be starting at 0.1s each) via CSV file reader, and the only problem I am facing now is how to insert data from multiple sources (CSV reader and the new features from the aggregator) into one box (classifier trainer). I understood that stream matrix multiplexer is the tool for that (please correct me if I am wrong here).

So I set up stream multiplexer with two feature vector inputs; however, if I send the CSV file there, nothing is outputted. Is this a bug or am I on the wrong path here?

To reproduce my issue, the following is sufficient: if I plug CSV reader output to CSV writer input, the feature set file is copied, as expected. If I put the multiplexer box between those two CSV boxes, nothing is copied (the output file contains only a header).

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Incremental classifier learning

Post by jtlindgren »

Its probably that the multiplexer expects both streams to have identical structure in the time sense or something. Not sure. I haven't often used that box. You could try to use signal concatenating instead while the data is still raw signal. Make the concatenation box read the previous recording and feed the current to the other input. And then store the catenated eeg + stims as a new file. But actually even this probably doesnt work as then designer would spend a lot of time reading the old file.

All in all its trickier than I thought, without resorting to C++ level hacks. I'll get back to you if I get a good idea. Meanwhile, anybody else is of course welcome to chime in.

Cheers,
Jussi

xskola
Posts: 12
Joined: Tue Mar 13, 2018 12:39 pm

Re: Incremental classifier learning

Post by xskola »

Well, you were right :) The issue was in the timing of the feature vectors. I didn't care enough and created all the fake timestamps with the same starting and ending time. Although this is enough to feed into the classifier, the multiplexer box did not forward these features to the output. I just changed all the features' timings to 0.01, 0.02 (Time, End Time) and now it gets through the multiplexer without issues and even works with the classification as expected!

Thanks a lot for your time!
Filip

Post Reply