Box behaving differently in different situations

Concerning processing components: filters, file load/save, visualizations, communication ...
karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Box behaving differently in different situations

Post by karthiks »

Hey,

Wish you all the best for 2012.

I have coded a box which generates a series of data after getting a trigger. This box has been showing weird behaviour depending on what data is being used to generate the trigger signal.

If I use a file reader, the speed of data generation is normal. But when I switch to using it online with acquisition box, the speed increases to a huge amount and the application crashes.

Could someone point me towards any resources which could explain this and also help control it?
Regards,

Karthik

lbonnet
Site Admin
Posts: 417
Joined: Wed Oct 07, 2009 12:11 pm

Re: Box behaving differently in different situations

Post by lbonnet »

Hi karthiks,

... and happy new year !

Could you provide us more information such has log of the console, error messages, log files, etc.

If i may, what is performing you box ?

Also, you can check the memory consumption when running your scenario (with task manager on windows for example).
If the memory consumption goes crazy after a while, maybe there is a problem in a box (yours?) that not discard the data after use.
Such "memory leak" may happen, causing crash at some point.

What you call "speed of generation" is driven by the number of sample per block and the sampling frequency.
With a file reader, the frequency is given in the file, and the number of sample per block is a setting of the file reader box.
Online, it depends on the driver used.

You may check how the stream is structured using an EBML Stream spy in category Tools. You will have a message printed in the console/log file when a block is received.
The first block, thus the first message, contains all header information. On a signal stream you will get the sampling frequency. You can also deduce from these messages the size of the further blocks (size of the matrix containing the samples).
This way you can check if the stream is well-formed.

Hope this helps,

Laurent
Follow us on twitter >> openvibebci

Checkout my (old) blog for some OpenViBE tips & tricks : here !

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

lbonnet wrote:
Could you provide us more information such has log of the console, error messages, log files, etc.
There is no error message on console. It just stops working, which I suppose is because it gets overwhelmed by the amount of data being generated.
lbonnet wrote: If i may, what is performing you box ?
It is a PCPG which generates kinematics.
lbonnet wrote:Also, you can check the memory consumption when running your scenario (with task manager on windows for example).
If the memory consumption goes crazy after a while, maybe there is a problem in a box (yours?) that not discard the data after use.
Such "memory leak" may happen, causing crash at some point.
I shall take a look at this too. But I do not see these crashes every time or frequently. Mostly it is just that the frequency of data generation is too high.
lbonnet wrote:What you call "speed of generation" is driven by the number of sample per block and the sampling frequency.
With a file reader, the frequency is given in the file, and the number of sample per block is a setting of the file reader box.
Online, it depends on the driver used.
Is there a way I could control the speed of an individual box? Because the frequency of data generation from this PCPG box is important (as you can understand).
Regards,

Karthik

yrenard
Site Admin
Posts: 645
Joined: Fri Sep 01, 2006 3:39 pm
Contact:

Re: Box behaving differently in different situations

Post by yrenard »

Dear karthiks,

as Laurent stated, the EBML Stream Spy box will probably help in sorting this out. This box is designed to dump the stream structure in the console log. Using this, we can easily observe bad chunk dating, abnormal number of chunks per scheduler steps etc. Please put such box in your scenario, connect it to your box (or whatever box you suspect to cause the issue you are facing), change the box's log level to something that will actually be logged and send us a log file including the output :)

Hope that helps,
regards,
Yann

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

yrenard wrote:Dear karthiks,

as Laurent stated, the EBML Stream Spy box will probably help in sorting this out. This box is designed to dump the stream structure in the console log. Using this, we can easily observe bad chunk dating, abnormal number of chunks per scheduler steps etc. Please put such box in your scenario, connect it to your box (or whatever box you suspect to cause the issue you are facing), change the box's log level to something that will actually be logged and send us a log file including the output :)

Hope that helps,
regards,
Yann
Dear Yann,

Thank you. I did take a look and I found that the speed changes according to acquisition frequency. So, I was wondering if there was a way to have the PCPG running at a different speed or do all the boxes run at the same speed?
Regards,

Karthik

yrenard
Site Admin
Posts: 645
Joined: Fri Sep 01, 2006 3:39 pm
Contact:

Re: Box behaving differently in different situations

Post by yrenard »

Dear karthiks,

do you mean when the sampling frequency / sample count per block ratio changes ?

If this is what you mean, you would probably be interested in re-epoching the signal stream with a Time Based Epoching box.

Hope this helps,
Yann

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

yrenard wrote:Dear karthiks,

do you mean when the sampling frequency / sample count per block ratio changes ?

If this is what you mean, you would probably be interested in re-epoching the signal stream with a Time Based Epoching box.

Hope this helps,
Yann
Dear Yann,

Let me make the work flow a little clearer. I acquire signals from the amplifier at 2048 kHz. I process these signals to extract relevant SSVEP spike and use that as the trigger for my Central Pattern Generator box. This CPG box is independent after that. The problem is - the generation of signals from the CPG box is dependant on the acquisition frequency. If I read from a file, the pattern of kinematics is fine. If I read from the amplifier, the generation becomes so fast that it is unusable.

I was wondering if I could determine the frequency of generation of the CPG box while maintaining the 2048kHz acquisition frequency?
Regards,

Karthik

lbonnet
Site Admin
Posts: 417
Joined: Wed Oct 07, 2009 12:11 pm

Re: Box behaving differently in different situations

Post by lbonnet »

Hi karthiks,

I assume your PCPG box is data-driven (you implement the processInput function and not the processClock).
Thus the faster you give it data, the faster it produces output.
I am not familiar with this algorithm... what are its inputs ? When do you need to process it ?

As Yann stated, you can use Time Based Epoching box.
The idea is to process less but bigger blocks of data. This box can produce large windows of signal (e.g. 0.5 second window every 0.5 seconds). The pipeline afterward will receive these big blocks and process them one by one.
If your algorithm is capable of processing large chunk of data (and it is still relevant) it should be ok, and you naturally lower the process frequency of your box.

For testing purpose, you can also perform a downsampling of the signal using the Downsampling box. (box still marked as unstable).


Laurent
Follow us on twitter >> openvibebci

Checkout my (old) blog for some OpenViBE tips & tricks : here !

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

lbonnet wrote:Hi karthiks,

I assume your PCPG box is data-driven (you implement the processInput function and not the processClock).
Thus the faster you give it data, the faster it produces output.
I am not familiar with this algorithm... what are its inputs ? When do you need to process it ?

As Yann stated, you can use Time Based Epoching box.
The idea is to process less but bigger blocks of data. This box can produce large windows of signal (e.g. 0.5 second window every 0.5 seconds). The pipeline afterward will receive these big blocks and process them one by one.
If your algorithm is capable of processing large chunk of data (and it is still relevant) it should be ok, and you naturally lower the process frequency of your box.

For testing purpose, you can also perform a downsampling of the signal using the Downsampling box. (box still marked as unstable).


Laurent
Dear Laurent,

Thank you. This algorithm is mine and part of a project that I am doing. It is a data generation box, all it does is take a stimulation and then generate data. Once it gets the stimulation, it does not process any inputs, rather it just outputs data.

I am considering keeping it as an independent program and drive it from OpenViBE using VRPN.
Regards,

Karthik

yrenard
Site Admin
Posts: 645
Joined: Fri Sep 01, 2006 3:39 pm
Contact:

Re: Box behaving differently in different situations

Post by yrenard »

lbonnet wrote:For testing purpose, you can also perform a downsampling of the signal using the Downsampling box. (box still marked as unstable).
The Signal Decimation box is stable and perfectly matches the needs here :) just keep in mind that you sould low pass filter before using this box to avoid aliasing.

Hoep this helps,
Yann

lbonnet
Site Admin
Posts: 417
Joined: Wed Oct 07, 2009 12:11 pm

Re: Box behaving differently in different situations

Post by lbonnet »

Indeed... Forget about the Downsampling box. The Signal Decimation is the one I had in mind when previously speaking. I need some sleep :|
It is a data generation box, all it does is take a stimulation and then generate data. Once it gets the stimulation, it does not process any inputs, rather it just outputs data.
Ok... But with too high sampling frequency, the stimulation triggers are generated too fast I believe.
You can still decimate the signal or use epoching boxes to generate less stimulations, no ?

L-
Follow us on twitter >> openvibebci

Checkout my (old) blog for some OpenViBE tips & tricks : here !

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

lbonnet wrote:Indeed... Forget about the Downsampling box. The Signal Decimation is the one I had in mind when previously speaking. I need some sleep :|
It is a data generation box, all it does is take a stimulation and then generate data. Once it gets the stimulation, it does not process any inputs, rather it just outputs data.
Ok... But with too high sampling frequency, the stimulation triggers are generated too fast I believe.
You can still decimate the signal or use epoching boxes to generate less stimulations, no ?

L-
The thing is after it takes the first stimulation, it does not take anymore. It just gets switched on and then it is automatic. But I guess I could try writing a slower sampling rate in the stimulation output stream.

Thank you both. :)
Regards,

Karthik

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

This is the code of the trigger box which sends the stimulations. If I change the frequency here, will it reduce the generation frequency in the PCPG box too? Or it will still depend on the initial frequency of acquisition?

Code: Select all

#include "ovpCBoxAlgorithmPCPGTrigger.h"

using namespace OpenViBE;
using namespace OpenViBE::Kernel;
using namespace OpenViBE::Plugins;

using namespace OpenViBEPlugins;
using namespace OpenViBEPlugins::Tools;

using namespace OpenViBEToolkit;

boolean CBoxAlgorithmPCPGTrigger::initialize(void)
{
	m_f64Threshold = FSettingValueAutoCast(*this->getBoxAlgorithmContext(),0);
	m_bTriggerActive = false;

	m_oStreamDecoder.initialize(*this);
	m_oStimEncoder.initialize(*this);
	m_vDecoders.push_back(&m_oStreamDecoder);
	m_vEncoders.push_back(&m_oStimEncoder);

	return true;
}

boolean CBoxAlgorithmPCPGTrigger::uninitialize(void)
{
	for (uint32 i=0; i<m_vDecoders.size(); i++)
	{
		m_vDecoders[i]->uninitialize();
	}

	for (uint32 i=0; i<m_vEncoders.size(); i++)
	{
		m_vEncoders[i]->uninitialize();
	}

	return true;
}

boolean CBoxAlgorithmPCPGTrigger::processInput(uint32 ui32InputIndex)
{
	this->getBoxAlgorithmContext()->markAlgorithmAsReadyToProcess();

	return true;

}
boolean CBoxAlgorithmPCPGTrigger::process(void)
{
	OpenViBE::Kernel::TParameterHandler <IStimulationSet*> l_oStimulationSet = m_oStimEncoder.getInputStimulationSet();

	IBoxIO& l_rDynamicBoxContext=this->getDynamicBoxContext();
	IBox& l_rStaticBoxContext=this->getStaticBoxContext();

	for(uint32 i=0; i<l_rStaticBoxContext.getInputCount(); i++)
	{
		for(uint32 j=0; j<l_rDynamicBoxContext.getInputChunkCount(i); j++)
		{
			m_vDecoders[i]->decode(i,j);

			if(m_vDecoders[i]->isHeaderReceived())
			{
				m_vEncoders[i]->encodeHeader(i);
			}

			if(m_vDecoders[i]->isBufferReceived())
			{
				uint64 l_ui64CurrentTime = getPlayerContext().getCurrentTime();
				IMatrix* l_pMatrix = m_oStreamDecoder.getOutputMatrix();
				float64* l_pBuffer = l_pMatrix->getBuffer();
				l_oStimulationSet->setStimulationCount(0);
				if(*l_pBuffer > m_f64Threshold && !m_bTriggerActive)
				{
					uint64 l_ui32Stimulationflag = this->getTypeManager().getEnumerationEntryValueFromName(OV_TypeId_Stimulation,"OVTK_StimulationId_Label_01");
					l_oStimulationSet->appendStimulation(l_ui32Stimulationflag, l_ui64CurrentTime,0);
					m_bTriggerActive = true;
				}
				/*if(*l_pBuffer < m_f64Threshold)
				{
					m_bTriggerActive = false;
				}*/
				m_vEncoders[i]->encodeBuffer(i);
			}

			if(m_vDecoders[i]->isEndReceived())
			{
				m_vEncoders[i]->encodeEnd(i);
			}
			l_rDynamicBoxContext.markOutputAsReadyToSend(i, l_rDynamicBoxContext.getInputChunkStartTime(i,j), l_rDynamicBoxContext.getInputChunkEndTime(i,j));
			l_rDynamicBoxContext.markInputAsDeprecated(i, j);
		}
	}

	return true;
}
Regards,

Karthik

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

Any ideas? :)
Regards,

Karthik

karthiks
Posts: 78
Joined: Thu Sep 02, 2010 2:43 pm
Location: Université Libre de Bruxelles

Re: Box behaving differently in different situations

Post by karthiks »

yrenard wrote:Dear karthiks,

as Laurent stated, the EBML Stream Spy box will probably help in sorting this out. This box is designed to dump the stream structure in the console log. Using this, we can easily observe bad chunk dating, abnormal number of chunks per scheduler steps etc. Please put such box in your scenario, connect it to your box (or whatever box you suspect to cause the issue you are facing), change the box's log level to something that will actually be logged and send us a log file including the output :)

Hope that helps,
regards,
Yann
Dear Yann,

I have attached the txt file with the EBML stream spy messages. The log level is information

Thanks for the help.
Attachments
PCPG.txt
(196.53 KiB) Downloaded 215 times
Regards,

Karthik

Post Reply