OpenViBE forum

The OpenViBE community
It is currently Sat May 26, 2018 11:08 pm

All times are UTC




Post new topic Reply to topic  [ 9 posts ] 
Author Message
PostPosted: Mon Nov 30, 2015 7:52 am 
Offline

Joined: Thu Oct 29, 2015 6:14 am
Posts: 3
Hey guys,

I am new here, hence this is my first post :)

For a psychology seminar at my university I wanted to create a simple feedback game using OpenVibe and Unity.
I have a EEG-SMT device (2 Channels, 4 Electrodes) so the game probably will be very simple. I might want to use alpha waves to detect when someone closes his eyes.
If this is detected I want to react to it in my Unity game.
The basic thing I will need for that is of course a communication line between OpenVibe and Unity. So I tried out the TCPWriter, receiving stimuli in Unity is no problem as those are strings; however, I am not able to decode the raw signal send from the writer ...
I thought it might be possible to process the signal in OpenVibe and then somehow create a stimulus that I can send (e.g. using the SignChangeDetector).

In general I wanted to ask you for suggestions on how to realize this little project, but it also would be nice if someone could help me with the TCPWriter and decoding the raw signal.

Any help is appreciated ;)

Cheers,

Philipp


Top
 Profile  
Reply with quote  
PostPosted: Mon Nov 30, 2015 10:53 am 
Offline

Joined: Fri Nov 14, 2014 1:06 pm
Posts: 40
Hi,
The format for raw signal output from the TCP Writer Box is described in the box documentation: http://openvibe.inria.fr/documentation/ ... riter.html
You will also find a client example in openvibe/applications/examples/openvibe-to-tcpip/src/tcpip-client.cpp that shows how to read data received from the TCP Writer box.
For you project you can get inspiration from the scenario neurofeedback.xml for instance, located in dist/share/openvibe/scenarios/bci-examples/neurofeedback.
This scenario shows how to get the beta power from the brain signals.


Top
 Profile  
Reply with quote  
PostPosted: Mon Nov 30, 2015 5:24 pm 
Offline

Joined: Thu Oct 29, 2015 6:14 am
Posts: 3
Hey,

thanks for the reply.

I've looked through those examples already and also read the description in the documentation. Maybe I was a bit unclear: I am not able to decode the raw signal data that is sent by the TCPWriter to Unity (I know it is described in the documentation, that it consists of a header of 32 bytes and so on) or in other words I am too stupid I guess. I don't know how I translate the byte data stream into the values of the signal. However, if I send a stimulus it is fine as it is a string. It would just be nice to e.g. display the signal in the Unity program.

Anyway I thought it might be best to process the EEG data in OpenVibe and e.g. use the power of beta like the in neurofeedback example and depending on the power send a stimulus to the game. But at the moment I don't see many possibilities to convert a signal into a stimulus except for the SignChange Box, maybe I am missing something.


Top
 Profile  
Reply with quote  
PostPosted: Tue May 10, 2016 12:41 pm 
Offline

Joined: Tue May 10, 2016 12:11 pm
Posts: 4
Hi to all,
we are a small research group working on a BCI based application at Politecnico di Milano University. We would like to connect the board device to Unity 3D platform for the development of a neurofeedback application. We have the same problem: have you found a solution for that?

Thanks in advance


Top
 Profile  
Reply with quote  
PostPosted: Wed May 11, 2016 7:38 am 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 696
Location: INRIA Rennes, FRANCE
Hi all,

you should be able to extract/adapt the code from the tcpip-client.cpp bundled with openvibe that reads from the TCP Writer. To convince yourself it is working correctly, you can use that small example to record data to a file. Use an understandable signal (e.g. time signal, sinusoid, ...) and then load the recorded file to a hex editor. Setting the display mode as 'double', you should see the signal correctly in the file. To do the same thing in the code, you just use the header to get the signal dimension and after that interpret your buffers as doubles.

You can also use LabStreamingLayer to get data out from OpenViBE.


Best,
Jussi


Top
 Profile  
Reply with quote  
PostPosted: Wed May 11, 2016 7:50 am 
Offline

Joined: Thu Oct 29, 2015 6:14 am
Posts: 3
Hey,

For my small application I decided to use the OSC box to send alpha and beta power to unity. But if I remember correctly you can only send single values via that box, so depending on what data you want it might not be the beat choice. I also have some code for the TCP writer that at least reads thr header correctly , but the reading of the signal matrix has not been finished. I am currently on the mobile phone so I might post it later.

Cheers

Edit: The code I have here was not tested or finished, but it should give an Idea on how to deal with the TCPWriter.

Code:
using UnityEngine;
using System.Collections;
using System;
using System.IO;
using System.Net.Sockets;
using System.Text;
using UnityEngine.UI;

// http://answers.unity3d.com/questions/15422/unity-project-and-3rd-party-apps.html
public class s_TCP : MonoBehaviour
{
    internal Boolean socketReady = false;
    TcpClient mySocket;
    NetworkStream theStream;
    StreamWriter theWriter;
    StreamReader theReader;
    String Host = "localhost";
    public Int32 Port = 5678;

    public bool testHeader = true;
    public bool testSignal = false;
    public bool isString = false;
    public int testSampleChannelSize;
    public int testSampleCount;
    public int testChannelCount;

    public double[,] lastMatrix;

    public float power;

    public RawOpenVibeSignal lastSignal;

    Rigidbody2D rb;
    public class RawOpenVibeSignal {

        public int channels;
        public int samples;

        public double[,] signalMatrix;
    }
    void Start()
    {
        setupSocket();
       
    }
    void FixedUpdate()
    {
       power = ((float)readSocket());
       
    }
    // **********************************************
    public void setupSocket()
    {
        try
        {
            mySocket = new TcpClient(Host, Port);
            theStream = mySocket.GetStream();
            theWriter = new StreamWriter(theStream);
            theReader = new StreamReader(theStream);
            socketReady = true;
        }
        catch (Exception e)
        {
            Debug.Log("Socket error: " + e);
        }
    }
    public void writeSocket(string theLine)
    {
        if (!socketReady)
            return;
        String foo = theLine + "\r\n";
        theWriter.Write(foo);
        theWriter.Flush();
    }
    public double readSocket()
    {
        if (!socketReady)
            return 0; // TODO
        if (theStream.DataAvailable)
        {



            // read header once
            if (testHeader)
            {
                readHeader();

            }

            if (testSignal)
                {
                // raw signal data
                // [nSamples x nChannels]
                // all channels for one sample are sent in a sequence, then all channels of the next sample

                // create a signal object to send it to another
                RawOpenVibeSignal newSignal = new RawOpenVibeSignal();
                newSignal.samples = testSampleCount;
                newSignal.channels = testChannelCount;

                double[,] newMatrix = new double[testSampleCount,testChannelCount];

       
                    byte[] buffer = new byte[testSampleChannelSize];

                    theStream.Read(buffer, 0, testSampleChannelSize);


                int row = 0;
                int col = 0;
                    for (int i = 0; i < testSampleCount * testChannelCount * (sizeof(double)); i = i + (sizeof(double) * testChannelCount))
                    {
                        for (int j = 0; j < testChannelCount * sizeof(double); j = j + sizeof(double))
                        {

                            byte[] temp = new byte[8];

                            for(int k = 0; k < 8; k++)
                            {
                                temp[k] = buffer[i + j + k];
                            }

                        if (BitConverter.IsLittleEndian)
                        {
                           // Array.Reverse(temp);
                            double test = BitConverter.ToDouble(temp, 0);
                           
                            // TODO TEST THIS
                            //newMatrix[i / (8 * testChannelCount), j / 8] = test;
                            newMatrix[row, col] = test;
                        }
                        col++;

                        }
                    row++;
                    col = 0;
                    }

                newSignal.signalMatrix = newMatrix;
                lastSignal = newSignal;
                lastMatrix = newMatrix;

               

               displaySignalText();

                return newMatrix[0, 0];
            }
                else if (isString) {
                    Debug.Log(theReader.ReadLine());
                }

            }
            return 0;

    }

    private void readHeader()
    {
        // size of header is 8 * size of unit = 32 byte

        int variableSize = sizeof(UInt32);
        int variableCount = 8;

        int headerSize = variableCount * variableSize;

        byte[] buffer = new byte[headerSize];

        theStream.Read(buffer, 0, headerSize);

        // version number (in network byte order)
        // endianness of the stream (in network byte order, 0==unknown, 1==little, 2==big, 3==pdp)
        // sampling frequency of the signal,
        //  number of channels,
        // number of samples per chunk and
        // three variables of padding


        UInt32 version, endiannes, frequency, channels, samples;

        byte[] v = new byte[4] { buffer[0], buffer[1], buffer[2], buffer[3] };
        byte[] e = new byte[4] { buffer[4], buffer[5], buffer[6], buffer[7] };
        byte[] f = new byte[4] { buffer[8], buffer[9], buffer[10], buffer[11] };
        byte[] c = new byte[4] { buffer[12], buffer[13], buffer[14], buffer[15] };
        byte[] s = new byte[4] { buffer[16], buffer[17], buffer[18], buffer[19] };
        if (BitConverter.IsLittleEndian)
        {
            Array.Reverse(e);
            Array.Reverse(v);
            version = BitConverter.ToUInt32(v, 0);
            endiannes = BitConverter.ToUInt32(e, 0);
            frequency = BitConverter.ToUInt32(f, 0);
            channels = BitConverter.ToUInt32(c, 0);
            samples = BitConverter.ToUInt32(s, 0);
        }
        else
        {

            version = 999;
            endiannes = 0;
            frequency = 0;
            channels = 0;
            samples = 0;
        }

       // Debug.Log("Version: " + version + "\n" + "Endiannes: " + endiannes + "\n" + "sampling frequency of the signal: " + frequency + "\n" + "number of channels: " + channels + "\n" + "number of samples per chunk: " + samples + "\n");

        testHeader = false;
        testSampleCount = buffer[16];
        testChannelCount = buffer[12];
        testSampleChannelSize = buffer[12] * buffer[16] * sizeof(double);

        testSignal = true;

        rb = GetComponent<Rigidbody2D>();
    }

    public void closeSocket()
    {
        if (!socketReady)
            return;
        theWriter.Close();
        theReader.Close();
        mySocket.Close();
        socketReady = false;
    }

    void displaySignalText() {
        RawOpenVibeSignal s = lastSignal;

        StringBuilder sb = new StringBuilder();

        for (int i = 0; i < lastSignal.samples; i++) {
            sb.AppendLine("Sample" + i + "\t");
            for (int j = 0; j < lastSignal.channels; j++)
            {
               try
                {
                    sb.AppendLine("Channel" + j).Append(lastSignal.signalMatrix[i, j]);
                }
                catch {
                  Debug.Log("i:" + i + "-j:" + j);
                }
           }
        }

       // Debug.Log(sb.ToString());
       
    }
    void OnApplicationQuit()
    {
        closeSocket();
    }
} // end class s_TCP


Top
 Profile  
Reply with quote  
PostPosted: Tue May 17, 2016 7:53 am 
Offline

Joined: Tue May 10, 2016 12:11 pm
Posts: 4
Just two questions :
so do you reach to realize your realtime application?
And how do you make acquisition data from straits by OpenVibe?

Thanks a lot


Top
 Profile  
Reply with quote  
PostPosted: Sat Sep 02, 2017 7:36 pm 
Offline

Joined: Tue Apr 11, 2017 10:44 am
Posts: 23
I want to add that the box mentioned earlier is stated as unstable http://openvibe.inria.fr/documentation/1.0.1/Doc_BoxAlgorithm_TCPWriter.html

Quote:
WARNING : this box has been marked as UNSTABLE by the developer. It means that its implementation may be incomplete or that the box can only work under well known conditions. It may possibly crash or cause data loss. Use this box at your own risk, you've been warned.


so is using it really a viable solution?


Top
 Profile  
Reply with quote  
PostPosted: Fri Sep 08, 2017 11:42 am 
Offline

Joined: Tue Dec 04, 2012 3:53 pm
Posts: 696
Location: INRIA Rennes, FRANCE
That categorization is not a problem by itself. :)

http://openvibe.inria.fr/faq/#unstable


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Americanized by Maƫl Soucaze.