robotarm - can not reach realtime

Making & changing box plugins and external apps
Post Reply
ddvlamin
Posts: 160
Joined: Thu Aug 13, 2009 8:39 am
Location: Ghent University
Contact:

robotarm - can not reach realtime

Post by ddvlamin »

Hello,

We wanted to use the P300 speller to do some simple tasks with a robot arm so I wrote an algorithm to read an xml file which specified a list of commands for a number of tasks. Depending on the chosen P300 speller character, these commands are then written to the serial port, which connects the pc to the robot arm.

However it takes a while, e.g. 18 seconds, to complete the task. Then I get the warning that openvibe can not reach realtime which is off course not surprising. So here are my questions:
  • All the EEG signals present during the completion of the task are off course irrelevant as the speller visualisation is also halted. But when I want to spell the next character does openvibe then first read the very old buffered EEG signals or does it only look at the most recent data when the P300 speller visualization resumes? I ask this because openvibe reports something like: 18 seconds late, 17 seconds late and so forth. So it seems openvibe still tries to process the passed 18 seconds of data?
  • If that is the case, can I do something to throw away the old buffered data?
  • Maybe a cleaner solution: The task could be completed while the user is already focussing on the next character. This would require a thread that executes the robot arm movement in parallel to the P300 speller visualization. Is this possible and how should I do that?
Hopefully this is somewhat clear...

Thanks in advance,
Dieter

yrenard
Site Admin
Posts: 645
Joined: Fri Sep 01, 2006 3:39 pm
Contact:

Re: robotarm - can not reach realtime

Post by yrenard »

Dear ddvlamin,

this sounds like an exciting application !!

Concerning your problem, I guess that you are sending orders and controlling your robot in a single call of the process function. This call is blocking so it should return as quickly as possible. I see two possible solutions to your problem. Either you can split the robot control in different steps, thus you send the commands to it through multiple calls to the process function. Either it is not easy to split the robot control, then you could trigger a thread or even an external process to control the robot. In any of those two cases, the process call should not take too much time and the leading boxes will be able to do their job correctly.

I hope this is clear enough, tell me if it's not.

However, could you share some photos / videos of your robot controlled by OpenViBE ? We plan to add a specific category about the applications the community made of OpenViBE. This is a good opportunity to start this section soon ;)

Yann

ddvlamin
Posts: 160
Joined: Thu Aug 13, 2009 8:39 am
Location: Ghent University
Contact:

Re: robotarm - can not reach realtime

Post by ddvlamin »

this sounds like an exciting application !!
Nice to hear you find it interesting. We use a simple, not too expensive robotic arm of lynxmotion:

Image

We will use it for demos starting from february 2010. Then I will upload a video as we will record it.
Concerning your problem, I guess that you are sending orders and controlling your robot in a single call of the process function. This call is blocking so it should return as quickly as possible.
This is indeed the problem.
Either you can split the robot control in different steps, thus you send the commands to it through multiple calls to the process function.
This will probably be impossible as even an basic movement, for example the positioning of one servo motor, takes a while. On top of that I have to periodically ask for the status of the robot, in order to know whether it finished the task or not.
I programmed a seperate algorithm box for now. So, I thought separate boxes somehow executed parallel. For example if you place two signal displays, both are executed in parallel. Why is this not the case for my box, it is a terminal one. Why does everything else wait for it?
Either it is not easy to split the robot control, then you could trigger a thread or even an external process to control the robot. In any of those two cases, the process call should not take too much time and the leading boxes will be able to do their job correctly.
Ok, so I created a thread with the help of the boost package, included in OpenViBE: Boost.Thread. Indeed, now, I see the robot completing it first task when the first P300 character is selected, while the main program keeps on executing (rows and columns keep flashing). However for some strange region when the first task is completed and the thread ends, then OpenViBE crashes. Do you have any idea what could cause this because I'm out of ideas for the moment. It just crashes which does not tell me anything. The only thing I know is that the code worked before I included the thread concept, so I gues it has something to do with my knowledge about threading and boost.

I also had to do something strange to get it compiled. I compiled the the project plugins-file-io in visual studio with the option /MTd and not with /MDd as it was originally. So it's being statically linked as the dynamic linking did not work for me (again my knowledge here is very limited). Then I copy pasted the OpenViBE-plugins-file-io-dynamic.lib and OpenViBE-plugins-file-io-dynamic.dll from D:\openvibe\local-tmp\visual\trunc\Debug to the respective OpenViBE directories bin and lib.
Attachments
file-io.zip
(13.94 KiB) Downloaded 274 times

yrenard
Site Admin
Posts: 645
Joined: Fri Sep 01, 2006 3:39 pm
Contact:

Re: robotarm - can not reach realtime

Post by yrenard »

Dear ddvlamin,
ddvlamin wrote:I programmed a seperate algorithm box for now. So, I thought separate boxes somehow executed parallel. For example if you place two signal displays, both are executed in parallel. Why is this not the case for my box, it is a terminal one. Why does everything else wait for it?
In fact the boxes are not executed in parallel. They must all take a small amount of time to process the reasonably small amount of data they receive. This gives the impression of being executed in parallel but they aren't. This is part of future work ;)
ddvlamin wrote:Ok, so I created a thread with the help of the boost package, included in OpenViBE: Boost.Thread. Indeed, now, I see the robot completing it first task when the first P300 character is selected, while the main program keeps on executing (rows and columns keep flashing). However for some strange reason when the first task is completed and the thread ends, then OpenViBE crashes. Do you have any idea what could cause this because I'm out of ideas for the moment. It just crashes which does not tell me anything. The only thing I know is that the code worked before I included the thread concept, so I guess it has something to do with my knowledge about threading and boost.
Well at this stage it's quite hard to tell you what went wrong. If you are not familiar with threads, maybe you should grab some documentation on google in order to learn the most common mistakes... One important thing you could check is that the main thread (i.e. the one calling your process function) never reads or writes memory in the same place as your robot controller thread (and vice versa ;) ). You should copy the data you want to give to your controller thread before it is actually triggered and then it must work "standalone". Also, you should take care that a second controller thread is not triggered when the second letter is selected (with the first task not being completely terminated).
ddvlamin wrote:I also had to do something strange to get it compiled. I compiled the the project plugins-file-io in visual studio with the option /MTd and not with /MDd as it was originally. So it's being statically linked as the dynamic linking did not work for me (again my knowledge here is very limited). Then I copy pasted the OpenViBE-plugins-file-io-dynamic.lib and OpenViBE-plugins-file-io-dynamic.dll from D:\openvibe\local-tmp\visual\trunc\Debug to the respective OpenViBE directories bin and lib.
This is rather strange you have to do this... From the MSDN documentation, I don't understand what difference can be made between those two switches and how why that is needed for the file-io project only... I'll play around with those switches as soon as possible.

I hope it helps, keep us informed
Yann

ddvlamin
Posts: 160
Joined: Thu Aug 13, 2009 8:39 am
Location: Ghent University
Contact:

Re: robotarm - can not reach realtime

Post by ddvlamin »

For now I implemented it by writing a seperate program and handling communication between openvibe and that program through shared memory (-->boost). Off course, I did not know about the VRPN, so it is probably better to use that. However I still want to use the threads so that I can use OpenViBE itself to handle communication with the serial port. I will look into that.

I don't really remember why I had to link it that way, but it had something to do with the fact that I used the latest boost package (as I needed a specific feature)... but I can't seem to find it anymore...

Anyhow, here's a small movie of the whole setup

http://www.thewired.be/?p=254

yrenard
Site Admin
Posts: 645
Joined: Fri Sep 01, 2006 3:39 pm
Contact:

Re: robotarm - can not reach realtime

Post by yrenard »

Dear ddvlamin,

thank you for the video you just sent, it's amazing ! And congratulations for managing OpenViBE that way. It makes me really happy to see our software used this way !

However, I just noticed that you can control your robot from command line ? If so, please consider using the Run Command box ! I posted a discussion about this box lately on my blog.

Best regards,
Yann

Post Reply