In my research project I deal with a mobile robot that perceives through stereo vision. As the stereo input data I currently use several datasets taken from a passenger vehicle that contain real world photos. The datasets are good to get started but have a limited content so I would need to model my own traffic situations to further work on the stereo vision system.

I am thinking about using some kind of synthetic graphics simulation as the input for the stereo system. What are my options? I can imagine a 3D graphics rendering engine whose output would be fed as the input for the stereo vision could probably be used.

I found there are general robotic simulators available like Gazebo but since I am all new to robotic simulation I do not really know where to begin.


I forgot to write that all my code is a pure C++. I use OpenCV and LIBELAS for stereo vision and Point Cloud Library (PCL) for visualization. All glued together into a single C++ project and compiles into single binary.


2 Answers 2


I highly advise against using synthetic image data for testing your stereo vision algorithms. What will happen is that you end up with a system that works excellent on your synthetic data, but poorly in the real-world. Synthetic images are much easier to process than real-world images, as they lack all the shortcomings of real cameras that will make stereo matching hard.

You can try it out yourself. There are some synthetic image sequences out there. Have a look at this one:



I did similar project with v-rep, python and opencv.

1- Set up an enviorement with visible objects

2- Set up your vision sensors with parameters

3- Write a python script that connects you to v-rep and do stereo vision application

Example code:

import vrep
import time
import cv2
import numpy as np


clientID = vrep.simxStart('', 19999, True, True, 5000, 5)

if clientID!=-1:
    print 'Connected to remote API server'
    print 'Vision Sensor object handling'
    res, v1 = vrep.simxGetObjectHandle(clientID, 'Vision_sensor#0', vrep.simx_opmode_oneshot_wait)
    print 'Getting first image'
    err, resolution, image = vrep.simxGetVisionSensorImage(clientID, v1, 0, vrep.simx_opmode_streaming_split)
    if err==0:
        print "Image ok"
    while (vrep.simxGetConnectionId(clientID) != -1):
        err, resolution, image = vrep.simxGetVisionSensorImage(clientID, v1, 0, vrep.simx_opmode_buffer)
        if err == vrep.simx_return_ok:
            img = np.array(image,dtype=np.uint8)
            if cv2.waitKey(1) & 0xFF == ord('q'):
        elif err == vrep.simx_return_novalue_flag:
            print "no image yet"
          print err
  print "Failed to connect to remote API Server"



OS: Windows 7 x64

IDE: PythonXY(Python version: 2.7, OpenCV version: 2.4)

Simulator: V-REP PRO EDU V3.3.0

  • $\begingroup$ Thanks for a nice answer! However, I forgot to write I use pure C++ - I just added it to the question. $\endgroup$
    – Kozuch
    Mar 6, 2016 at 20:36
  • $\begingroup$ @Kozuch There is a c++ interface of V-Rep. I think V-rep easier than gazebo. You should try this : coppeliarobotics.com/helpFiles/en/remoteApiFunctions.htm $\endgroup$
    – acs
    Mar 6, 2016 at 20:40
  • $\begingroup$ @Kozuch also you can try to use datasets like : mrpt.org/Karlsruhe_Dataset_Rawlog_Format $\endgroup$
    – acs
    Mar 7, 2016 at 8:19
  • $\begingroup$ Yes I used this dataset and also its successor the KITTI one... but I want to try a synthetic environment. $\endgroup$
    – Kozuch
    Mar 7, 2016 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.