Computer stereo vision simulator

Question

In my research project I deal with a mobile robot that perceives through stereo vision. As the stereo input data I currently use several datasets taken from a passenger vehicle that contain real world photos. The datasets are good to get started but have a limited content so I would need to model my own traffic situations to further work on the stereo vision system.

I am thinking about using some kind of synthetic graphics simulation as the input for the stereo system. What are my options? I can imagine a 3D graphics rendering engine whose output would be fed as the input for the stereo vision could probably be used.

I found there are general robotic simulators available like Gazebo but since I am all new to robotic simulation I do not really know where to begin.

EDIT:

I forgot to write that all my code is a pure C++. I use OpenCV and LIBELAS for stereo vision and Point Cloud Library (PCL) for visualization. All glued together into a single C++ project and compiles into single binary.

Answer 1

I highly advise against using synthetic image data for testing your stereo vision algorithms. What will happen is that you end up with a system that works excellent on your synthetic data, but poorly in the real-world. Synthetic images are much easier to process than real-world images, as they lack all the shortcomings of real cameras that will make stereo matching hard.

You can try it out yourself. There are some synthetic image sequences out there. Have a look at this one:

https://cerv.aut.ac.nz/set-2/

Answer 2

I did similar project with v-rep, python and opencv.

1- Set up an enviorement with visible objects

2- Set up your vision sensors with parameters

3- Write a python script that connects you to v-rep and do stereo vision application

Example code:

import vrep
import time
import cv2
import numpy as np

vrep.simxFinish(-1)

clientID = vrep.simxStart('127.0.0.1', 19999, True, True, 5000, 5)

if clientID!=-1:
    print 'Connected to remote API server'
    print 'Vision Sensor object handling'
    res, v1 = vrep.simxGetObjectHandle(clientID, 'Vision_sensor#0', vrep.simx_opmode_oneshot_wait)
    print 'Getting first image'
    err, resolution, image = vrep.simxGetVisionSensorImage(clientID, v1, 0, vrep.simx_opmode_streaming_split)
    if err==0:
        print "Image ok"
    time.sleep(2)
    while (vrep.simxGetConnectionId(clientID) != -1):
        err, resolution, image = vrep.simxGetVisionSensorImage(clientID, v1, 0, vrep.simx_opmode_buffer)
        if err == vrep.simx_return_ok:
            img = np.array(image,dtype=np.uint8)
            img.resize([resolution[0],resolution[1],3])
            cv2.imshow('image',img)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
        elif err == vrep.simx_return_novalue_flag:
            print "no image yet"
            pass
        else:
          print err
else:
  print "Failed to connect to remote API Server"
  vrep.simxFinish(clientID)

cv2.destroyAllWindows()

Configuration

OS: Windows 7 x64

IDE: PythonXY(Python version: 2.7, OpenCV version: 2.4)

Simulator: V-REP PRO EDU V3.3.0