0
$\begingroup$

Rosanswers logo

I have two ROS nodes, a "display node" and a "detector node", that both use cv_bridge and subscribe to the same image topic. The detector node publishes an array of ROI boxes around the objects it is designed to detect (e.g. faces). While I could draw these boxes on an OpenCV image window using the same node, I would like the boxes display along with potentially other markers from other detector nodes.

So I have a display node that listens on both the raw image topic and the detector topics and draws the appropriate boxes on a marker image that it then merges with the raw image topic and updates an OpenCV image display window.

The problem I am having is that the markers (e.g. boxes) significantly lag behind the raw image. For instance, if my detector node uses the Haar face detector, and I move my head, the face boxes trail my head movement by a very noticeable amount. (Maybe half of a second?)

I am guessing this is because the display node can update the raw image display much faster than the detector node can update the face detections. But I'm not sure what I can do about it or if this approach is even a good idea.

UPDATE: Well, I've run into a snag. I have no trouble with the CameraInfo Python example found here. However, I cannot get synchronization to work when trying to synchronize /camera/rgb/image_color (I'm using a Kinect and the openni_camera node) and a test publisher I wrote.

Here are the two test scripts I am using. The first script simply publishes a DetectionArray message about 30 times per second with nothing but the header.stamp field set to rospy.Time.now(). The second script subscribes to this topic as well as /camera/rgb/image_color. While the second script successfully prints out the value of header.stamp using the regular rospy.Subscriber callback, I never see the message "Synchronized" from the TimeSynchronizer callback. Looking at the timestamps manually using rostopic echo on the /camera/rgb/image_color and /detections topics shows that they are very close to being synchronized already so I thought this should work.

I must be missing something simple I can't see so any tips would be appreciated!

Here is the test_pub.py publisher script:

#!/usr/bin/env python

""" test_pub.py - Version 0.1 2011-06-10  
"""

import roslib
roslib.load_manifest('pi_rein')
import rospy
import sys
from std_msgs.msg import String
from pi_rein.msg import *
from pi_rein.srv import *
import message_filters

class TestSync:
    def __init__(self):
        rospy.init_node("test_pub")
        
        self.test_pub = rospy.Publisher("detections", DetectionArray)
        
        while not rospy.is_shutdown():
            msg = DetectionArray()
            msg.header.stamp = rospy.Time.now()
            self.test_pub.publish(msg)
            rospy.sleep(0.033)     

def main(args):   
    try:
        TestSync()
    except KeyboardInterrupt:
        print "Shutting down test pub node."

if __name__ == '__main__':
    main(sys.argv)

Here is the test_sync.py script:

#!/usr/bin/env python

""" test_sync.py - Version 0.1 2011-06-10  
"""

import roslib
roslib.load_manifest('pi_rein')
import rospy
import sys
from pi_rein.msg import *
from sensor_msgs.msg import Image, CameraInfo
import message_filters

class TestSync:
    def __init__(self):
        rospy.init_node("test_sync")
                
        """ Subscribe to the raw camera image topic and set the image processing callback """
        self.image_sub = message_filters.Subscriber("image", Image)
        
        """ Subscribe to a topic that provides markers """
        self.detections_sub = message_filters.Subscriber("detections", DetectionArray)
        
        self.nonsynced_detections_sub = rospy.Subscriber("detections", DetectionArray, self.detections_callback)
        
        tss = message_filters.TimeSynchronizer([self.image_sub, self.detections_sub], 100)
        tss.registerCallback(self.sync_callback)
        
        rospy.wait_for_message("image", Image, timeout=15)
        
    def sync_callback(self, image, detections):
        rospy.loginfo("Synchronized!")
        
    def detections_callback(self, detections):
        rospy.loginfo(detections.header.stamp)
        

def main(args):   
    try:
        TestSync()
        rospy.spin()
    except KeyboardInterrupt:
        print "Shutting down test sync node."

if __name__ == '__main__':
    main(sys.argv)

And this is the launch file I use to fire them both up:

<launch>

  <node pkg="pi_rein" name="test_pub" type="test_pub.py" output="screen">
  </node>

  <node pkg="pi_rein" name="test_sync" type="test_sync.py" output="screen">
    <remap from="image" to="/camera/rgb/image_color" />
  </node>
  
</launch>

Thanks!
patrick


Originally posted by Pi Robot on ROS Answers with karma: 4046 on 2011-06-10

Post score: 0


Original comments

Comment by Ugo on 2011-06-10:
My bad, sorry (in fact I discovered the REIN framework thanks to your post on the mailing list :) ).

Comment by Pi Robot on 2011-06-10:
Thanks Ugo--in fact I am trying to do something similar to REIN using Python. (See my posting on ros-users at http://code.ros.org/lurker/message/20110609.184757.957cc335.en.html). And even in the exiting C++/Nodelet REIN framework, I don't see a way to display markers from multiple detector nodes on the same image. If I am just missing it, hopefully someone will point it out as it might help me solve my synchronization issue.

Comment by Ugo on 2011-06-10:
Hi Patrick, do you know about this: http://www.ros.org/wiki/rein ? It seems to be a good framework for what you're doing.

$\endgroup$

1 Answer 1

0
$\begingroup$

Rosanswers logo

You should obey the timestamps. Your detector node sends out the appropriate boxes with the image timestamp.

The display node only displays image+markers from the same timestamp. A Time Synchronizer filter should be applicable for this.

Update: To make it work, try to set the timestamps exactly the same. I.e. for your application: The detector subscribes the image and sends out its detections with the image timestamp (not ros::Time::now()). The display subscribes image and detections for synchronization.


Originally posted by dornhege with karma: 31395 on 2011-06-10

This answer was ACCEPTED on the original site

Post score: 1


Original comments

Comment by Pi Robot on 2011-06-10:
That was it! With the timestamp set to the image timestamp and the synchronizer queue set to 100, it works like a charm. Thanks again!

Comment by Pi Robot on 2011-06-10:
Thanks dornhege--this looks like exactly what I need!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.