Generic Raw Telnet Reader - Format of Raw Buffer

Making & changing box plugins and external apps
Post Reply
Warren
Posts: 4
Joined: Thu Feb 12, 2015 10:10 pm

Generic Raw Telnet Reader - Format of Raw Buffer

Post by Warren »

I am trying to send an EEG stream over the acquisition server through the 'Generic Raw Telnet Reader'

My python servero sends 14 channels of data on one line.
(i modified the acquisition ui to accept my 1 line sample count)

My question is, what format should this be sent in?
I was told in binary, but should there be a delimiter in between data pieces? (ie 0x00)
or is it one complete chunk of binary that is read in segments determined by 'sample type'
that allows you to select the type of byte. (integer, float, etc)

The documentation for the OVTK_NodeId_Buffer_StreamedMatrix_RawBuffer says 'array of float64'
when sending an array from python, i believe it is delimited by an 0x00 byte, i was curious if the array
is in an EBML format, or can i just send it directly?

my client only sends the raw eeg data line by line, i was wondering if i needed header or supply more
details, or if i could get away with the bare minimum, as it seems to be displaying the data properly.

I've been using ebml spy to view the messages and the sample type indicates "float 64"
i remember reading that though it receives other data types, that openvibe converts to
a float64 format to maintain higher precision, but that its only localy. So my question on
this, is if I the acquisition server is reading 32-bit data is the result going to 'display' a
32-bit result, or is the displayed result actually a 64-bit float.

i just wanted to be sure, because I wanted to verify what i was sending with what it is receiving,
and when it seems that its displaying the data correctly in the visualizations, the ebml spy indicates
its receiving floats.

my data format is in the range of -8900 to 8900, i was wondering if I needed to scale this down as well,
or will openvibe scale it down on its own?

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Generic Raw Telnet Reader - Format of Raw Buffer

Post by jtlindgren »

Hi Warren,

OpenViBE uses 64bits float internally. But when you're communicating with the Acquisition Servers' Telnet reader,
there is no need for CMatrix or EBML, these are constructed later from the data that the Telnet reader gets.

From the source code of its base class (ovasCDriverGenericRawReader.cpp),

Code: Select all

 * The raw reader expects the data to be formatted as follows
 *
 * [START][BLOCK0][BLOCK1][BLOCK2]...
 *  skip   parse   parse   parse  ...
 *
 * where each block is          [===========BLOCKX=================]
 *                  is read as  [===========dataFrameSize==========]
 *                  breaks to   [header====][sample====][footer====]
 *                  equals      [headerSize][sampleSize][footerSize]
 *                  means          skip        keep         skip
 *
 * For correct parsing, user must provide the exact sizes of the skipped parts "start", "header" and "footer" in bytes. 
 *
Also, what I can see from the code is that the extra precision over 32 bits gets lost, any type it
gets, it is temporarily passed via a 32bit float buffer. :( So its no use to pass it 64 bit data.

You may also be interested in the documentation of the TCP Writer box, this box is able to generate
a Telnet stream compatible with the AS expectations.

http://openvibe.inria.fr/documentation/ ... riter.html

Hope this helps.

Happy hacking,
Jussi

Warren
Posts: 4
Joined: Thu Feb 12, 2015 10:10 pm

Re: Generic Raw Telnet Reader - Format of Raw Buffer

Post by Warren »

From what i've ascertained, the data is sent in whatever format you have chosen for your acquisition server.
in byte format (ie 2 bytes for 16-bit) (4 bytes for 32-bit)
in a string of bytes (with no delimiters or spaces) , setting the proper endian format for each byte.
with an \n terminating character at the end. ( if you use crlf, it will offset the data. )

also, using microsofts telnet was not very helpful, as it truncated the data with CRLF every 80 bytes
and i believe padded it with spaces as well.

the other part of my problem was that my bytes kept getting x00 appended to the end.
(which could be due to poor coding.)

I updated my python server, perhaps it may be of use to someone.
https://github.com/CymatiCorp/CyKit

you must pardon some of my python code, im relatively new to it. if anyone has any
suggestions, I'd gladly update it.

Thank you for your time.

Warren
Posts: 4
Joined: Thu Feb 12, 2015 10:10 pm

Re: Generic Raw Telnet Reader - Format of Raw Buffer

Post by Warren »

Actually, the \n terminating character seemed to offset my data as well.
Using no terminating characters seemed to solve this problem.

so one continuous stream of samples.

jtlindgren
Posts: 775
Joined: Tue Dec 04, 2012 3:53 pm
Location: INRIA Rennes, FRANCE

Re: Generic Raw Telnet Reader - Format of Raw Buffer

Post by jtlindgren »

Right, no extra stuffs. Padding with EOLs only makes sense for ascii data which we are not sending/receiving in this case.

Did you get it working satisfactorily now?


Cheers,
Jussi

Warren
Posts: 4
Joined: Thu Feb 12, 2015 10:10 pm

Re: Generic Raw Telnet Reader - Format of Raw Buffer

Post by Warren »

yes i believe so. thank you for the assistance.

Image

Post Reply