Virtual Reality Tool for Active Vision


Introduction
Virtual reality (VR) can be used as a tool to analyze the interactions between the visual system of a robotic agent and the environment, with the aim of designing the algorithms to solve the visual tasks necessary to properly behave into the 3D world. The novelty of our approach lies in the use of the VR as a tool to simulate the behavior of vision systems. The visual system of a robot (e.g., an autonomous vehicle, an active vision system, or a driving assistance system) and its interplay with the environment can be modeled through the geometrical relationships between the virtual stereo cameras and the virtual 3D world. Differently from conventional applications, where VR is used for the perceptual rendering of the visual information to a human observer, in the proposed approach, a virtual world is rendered to simulate the actual projections on the cameras of a robotic system. In this way, machine vision algorithms can be quantitatively validated by using the ground truth data provided by the knowledge of both the structure of the environment and the vision system.

The tool developed by our laboratory is described in:

Manuela Chessa, Fabio Solari and Silvio P. Sabatini (2011). Virtual Reality to Simulate Visual Tasks for Robotic Systems, Virtual Reality, Jae-Jin Kim (Ed.), ISBN: 978-953-307-518-1, InTech, Available from: Virtual Reality to Simulate Visual Tasks for Robotic Systems

Other related papers:

M. Chessa, F. Solari and S.P. Sabatini - A virtual reality simulator for active stereo vision systems - VISAPP2009 [pdf]

The datasets used in our paper can be downloaded and used by Computer Vision researchers. We grant permission to use and publish all images, disparity and optic flow maps on this website. However, if you use our datasets, we request that you cite the appropriate paper.

Datasets for disparity and optic flow evaluation

Datasets for TTC evaluation

Datasets for disparity and optic flow evaluation

Databases for the evaluation of the performances of active stereo systems are still missing. The stereo geometry of the existing database is fixed, and characterized by parallel axis cameras. By using the software environment we developed, it is possible to collect a large number of data in different situations: e.g. vergent stereo cameras with different fixation points and orientation of the eyes, optic flow maps obtained for different ego-motion velocities, or different gaze orientation.
The true disparity and optic flow maps can be stored together with the 3D data from which they have been generated and the corresponding image sequences. These data can be used for future algorithm benchmarking also by other researchers in the Computer Vision community.

Sequence of stereo pairs with ground truth disparity and optic flow maps

Please cite: Manuela Chessa, Fabio Solari and Silvio P. Sabatini (2011). Virtual Reality to Simulate Visual Tasks for Robotic Systems, Virtual Reality, Jae-Jin Kim (Ed.), ISBN: 978-953-307-518-1, InTech. Available from Virtual Reality to Simulate Visual Tasks for Robotic Systems
  • The following sequence shows the active exploration of an indoor scene, representing a desktop and different objects at various distances, acquired by using a laser scanner. The simulator aims to mimic the behavior of a human-like robotic system acting in the peripersonal space. Accordingly, the interocular distance between the two cameras is set to 6 cm and the distance between the cameras and the center of the scene is about 80 cm. The fixation points have been chosen arbitrary, thus simulating an active exploration of the scene, and in their proximity the disparity between the left and the right projections is zero, while getting far from the fixation point.

    The dataset is composed by 10 left/right stereo pairs and the corresponding horizontal and vertical ground truth disparity maps. Click here to download the dataset.


Left Right
Horizontal disparity Vertical disparity


  • The simulator can be used to obtain sequences acquired by a moving observer. The position and the orientation of the head can be changed, in order to mimic the navigation in the virtual environment. For the sake of simplicity, the ocular movements are not considered and the visual axes are kept parallel.

    Each dataset is composed by 8 frames and the optic flow ground truth map, with respect to the central frame.

The robot has velocity along Z axis, only. Click here to download the dataset.
The robot has velocity along Z axis and along positive X axis. Click here to download the dataset.
The robot has velocity along Z axis and along negative X axis. Click here to download the dataset.
The robot has velocity along Z axis and rotates around Y axis. Click here to download the dataset.



Stereo pairs with ground truth

Please cite: M. Chessa, F. Solari and S.P. Sabatini - A virtual reality simulator for active stereo vision systems - VISAPP2009 [pdf]
  • The following stereo pair is obtained from a simple VRML model.
Anaglyph of the stereo pair Horizontal disparity Vertical disparity
  • The following stereo pair is obtained from the 3D data acquired by the Konica Minolta Vivid 910 laser scanner.
Anaglyph of the stereo pair Horizontal disparity Vertical disparity
[top]

Datasets for TTC evaluation

The virtual reality tool can be also used to simulate a standard moving camera. A set of sequences to benchmark the algorithms for motion interpretation (i.e. time-to-contact evaluation and reconstruction of the orientation of the surfaces).
Datasets for TTC evaluation:
[top]
For further information feel free to contact Manuela Chessa or Fabio Solari