0
$\begingroup$

I implemented a stereo camera in Gazebo by simply duplicating a camera and shifting it to the right. This works.

To get a GT for my stereo algorithm I created a depth camera at the same position as my left camera (reference camera).

The left camera and the depth camera show images which seems to have slightly different fovs and/or shifts.

Here is why this is weird to me: In my Gazebo SDF file (attached below) I have created the depth camera and the camera exactly the same way. What I mean by this is that from the camera intrinsics and the image resolution I calculated the fov using the formula:

FoV = 2 × arctan(w / 2f)

Which for a focal length fx = 510.88022853 pixels and an image resolution of 640x480 pixels (aka w = 640 pixels) yields a fov of 64.123581079° or 1.119167618 radians.

I put this value in the horizontal_fov tag of the depth camera which should then in my understanding create the same fov for the depth and normal camera. This is not the case however.

<visual name="camera_visual_left">
    <visibility_flags>1</visibility_flags>
    <pose relative_to="camera_reference_frame"> 0 0 0 0 0 0 </pose>
    <geometry>
      <box>
        <size>0.025 0.025 0.025</size>
      </box>
    </geometry>
  </visual>
  <sensor name="camera_sensor_left" type="camera">
    <pose relative_to="camera_reference_frame"> 0 0 0 0 0 0 </pose>
    <always_on>1</always_on>
    <update_rate>35</update_rate>
    <topic>camera/image_left</topic>

    <camera name="left_camera">
      <!-- <horizontal_fov>1.819167618</horizontal_fov> -->
      <!-- <horizontal_fov>1.119167618</horizontal_fov> -->
      
      <image>
        <width>640</width>
        <height>480</height>
        <format>R8G8B8</format>
      </image>

      <lens>
        <type>stereographic</type>
        <!-- <scale_to_hfov>1</scale_to_hfov>  -->
        <cutoff_angle>3.14</cutoff_angle>
        <env_texture_size>640</env_texture_size>

        <intrinsics>
          <!-- <fx>256</fx>  -->
          <fx>510.88022853</fx> 
          <!-- <fy>256</fy>  -->
          <fy>510.35793799</fy> 
          <cx>329.09962026</cx> 
          <cy>284.74455683</cy> 
          <s>0</s>
        </intrinsics>
      </lens>

      <clip>
        <near>0.05</near>
        <far>300</far>
      </clip>

      <distortion>
        <k1>0</k1>
        <k2>0</k2>
        <k3>0</k3>
        <p1>0</p1>
        <p2>0</p2>
        <center>0 0</center>
      </distortion>

      <noise>
        <type>gaussian</type>
        <mean>0.0</mean>
        <stddev>0.0</stddev>
      </noise>

    </camera>
  </sensor>

  <!-- Second stereo camera -->
  <visual name="camera_visual_right">
    <pose relative_to="camera_reference_frame"> 0 -0.092 0 0 0 0 </pose>
    <visibility_flags>0</visibility_flags>
    <geometry>
      <box>
        <size>0.025 0.025 0.025</size>
      </box>
    </geometry>
  </visual>
  <sensor name="camera_sensor_right" type="camera">
    <pose relative_to="camera_reference_frame"> 0 -0.092 0 0 0 0 </pose>
    <always_on>1</always_on>
    <update_rate>35</update_rate>
    <topic>camera/image_right</topic>

    <camera name="right_camera">
      <!-- <horizontal_fov>1.119167618</horizontal_fov> -->

      <image>
        <width>640</width>
        <height>480</height>
        <format>R8G8B8</format>
      </image>

      <lens>
        <type>stereographic</type> 
        <!-- <scale_to_hfov>true</scale_to_hfov>  -->

        <cutoff_angle>3.14</cutoff_angle> 
        <env_texture_size>640</env_texture_size>
        <intrinsics>
          
          <fx>510.88022853</fx> 
          <fy>510.35793799</fy> 
          <cx>329.09962026</cx> 
          <cy>284.74455683</cy> 
          <s>0</s>
        </intrinsics>
      </lens>

      <clip>
        <near>0.05</near>
        <far>300</far>
      </clip>

      <distortion>
        <k1>0</k1>
        <k2>0</k2>
        <k3>0</k3>
        <p1>0</p1>
        <p2>0</p2>
        <center>0 0</center>
      </distortion>

      <noise>
        <type>gaussian</type>
        <mean>0.0</mean>
        <stddev>0.0</stddev>
      </noise>

    </camera>
  </sensor>


  <sensor name="GT" type="depth_camera">
    <pose relative_to="camera_reference_frame"> 0 0 0 0 0 0 </pose>
    <always_on>1</always_on>
    <update_rate>35</update_rate>
    <topic>camera/depth_GT</topic>

    <camera name="GT_depth_camera">
      <horizontal_fov>1.119167618</horizontal_fov>
      
      <image>
        <width>640</width>
        <height>480</height>
        <format>L8</format>
      </image>

      <lens>
        <type>stereographic</type>
        <scale_to_hfov>true</scale_to_hfov> 

        <cutoff_angle>3.14</cutoff_angle>
        <env_texture_size>640</env_texture_size> 

        <intrinsics>
          <fx>510.88022853</fx> 
          <fy>510.35793799</fy> 
          <cx>329.09962026</cx> 
          <cy>284.74455683</cy> 
          <s>0</s>
        </intrinsics>
      </lens>

      <clip>
        <near>0.05</near>
        <far>300</far>
      </clip>

      <distortion>
        <k1>0</k1>
        <k2>0</k2>
        <k3>0</k3>
        <p1>0</p1>
        <p2>0</p2>
        <center>0 0</center>
      </distortion>

      <noise>
        <type>gaussian</type>
        <mean>0.0</mean>
        <stddev>0.0</stddev>
      </noise>

    </camera>
  </sensor>

The result image showing a fov/zoom/shift difference (just not the same): enter image description here

Any ideas why that is? Thanks a lot in advance!

Edit: I am not aware of having chosen a specific depth camera model. I just use the depth_camera sensor specification from gazebo. What I want is a depth camera that for the same fov as the normal camera and at the same position gets the depth information of every point in sight.

I am using gazebo garden.

Also it is not a map thing. I tried it with the default map and jsut spawning rocks into it which are comparatively less noisy and the result was about the same:

enter image description here (top left camera view, the other two are the same depth image)

$\endgroup$
3
  • $\begingroup$ Please edit your question to provide a link to the depth_camera model you are using, and tell us which version of gazebo you are using. My guess is that the model output has already been "corrected" to parallelize the sensor rays (i.e. it's not valid to treat it as a standard "optical" system.) $\endgroup$
    – Mike973
    Commented Nov 11, 2023 at 13:27
  • $\begingroup$ I edited the question: As written above I simply used the default depth_camera tag from Gazebo. Where would I be able to look up the invoked plugin of that or in general what model is used in that implementation? $\endgroup$
    – kilianf
    Commented Nov 13, 2023 at 16:24
  • $\begingroup$ "where would I be able to look up the invoked plugin?" The source code for gazebo-ignition is on github.com. There is a repository specifically for the built-in Sensors. $\endgroup$
    – Mike973
    Commented Nov 15, 2023 at 14:01

1 Answer 1

0
$\begingroup$

I think your assumption that these two data sets can be visually compared in 2D-form is not a good one. That depth data is a collection of 3d points which have been sampled in a particular way. The depth_camera model publishes the data in an Image message, but this is not a typical image.

To display this data as an image: 1) you must re-interpret the z-depth value as grayscale color, and 2) recognize it was created using orthographic projection.

A camera image using perspective projection does not look the same as an "image" using orthographic projection.

$\endgroup$
3
  • $\begingroup$ I'm actually not sure about your answer. The Gazebo depth camera yields already a projected image which is then just transferred to rviz no? I was able to visualize the two camera views in gazebo directly yielding the same result. That's why I doubt that it's an rviz thing. $\endgroup$
    – kilianf
    Commented Nov 15, 2023 at 18:41
  • $\begingroup$ Yes, you're right that rviz plugin gets an Image, not a PointCloud. But my higher-level point is still the same: the two look different because different projections are being used. I've updated my answer. $\endgroup$
    – Mike973
    Commented Nov 16, 2023 at 13:49
  • $\begingroup$ Thank you Mike I will further look into the source code. By now I believe the ogre rendering is the place to look at to understand exactly how the projection is done. $\endgroup$
    – kilianf
    Commented Nov 16, 2023 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.