0
$\begingroup$

Rosanswers logo

I've been using OpenCV with ROS kinetic for years, and have come to rely on the functions toolbar that comes with cv::imshow for debugging.

We've successfully transitioned to noetic under Ubuntu 20.04, but now the toolbars in my OpenCV cv::imshow windows are gone, and I can't pan, zoom, or inspect pixel values. The images themselves show up just fine. I just can't do anything with them anymore.

Digging in,

  • it appears that I only get the toolbar with OpenCV installations that are built with the WITH_QT=ON CMake option. Is this correct? Is there another way to get the toolbar/pan/zoom/inspect functionality that I am missing?
  • Noetic appears to use a standard OpenCV 4.2 package installed as a ubuntu 20.04 package instead of using the built-in-to-ROS version of OpenCV 3.3.1 that kinetic under 16.04 used
  • I guess the standard Ubuntu 20.04 OpenCV 4.2 package is built without WITH_QT=ON?

Do I really have strip out the standard openCV from my installation and build my own version of OpenCV from source to get back the functionality I've had for the last 5 years out of the box with kinetic? Is there some other path that I am missing?

If I do have to build my own version of OpenCV from source to get the toolbars, any advice on how to do that for a Docker overlay without rebuilding OpenCV every time? Do I have to build my own .deb and install that as part of the docker overlay creation process, or does a version of OpenCV 4.2 that does what I want exist out there in a repository somewhere? Perhaps I just have to rebuild/load the individual libopencv-highgui4.2 package instead of the whole openCV system? Any advice or pointers on exactly how to do this before I just start flailing at things I only partially understand would be most helpful.


Originally posted by Jay Gowdy on ROS Answers with karma: 136 on 2021-12-02

Post score: 1


Original comments

Comment by Jay Gowdy on 2021-12-02:
Note, I did try to explicitly create a cv::namedWindow with the WINDOW_AUTOSIZE | WINDOW_KEEPRATIO | WINDOW_GUI_EXPANDED flags. No dice. The documentation for namedWindow on https://docs.opencv.org/3.4/d7/dfc/group__highgui.html#ga5afdf8410934fd099df85c75b2e0888b does confirm that with the Qt backend these should be the defaults anyway, so more evidence that the default noetic openCV highgui debian package is not built with the Qt backend enabled.

$\endgroup$

1 Answer 1

0
$\begingroup$

Rosanswers logo

Well, flailing seemed to get me where I needed to go, but it is not pretty. I would really appreciate someone telling me there was a better way.

First, you must get the source tarball for OpenCV 4.2.0 from the OpenCV website, https://github.com/opencv/opencv/archive/4.2.0.zip, unzip it and you will have a directory opencv-4.2.0

Then, in the same directory above opencv-4.2.0 (for the purposes of these instructions), get the source of the opencv debian package,

apt-get source libopencv-dev

which will create a directory opencv-4.2.0+dfsg

Now, I found that the source directory from the package is missing some critical files for enabling the Qt back end, so copy them over from the original OpenCV source (this is why you have to download it as well):

cp -a opencv-4.2.0/modules/highgui/src/files_Qt/Milky opencv-4.2.0+dfsg/modules/highgui/src/files_Qt

Now, cd to opencv-4.2.0+dfsg and we can forget about the OpenCV source. Edit debian/rules, and change the line that says

-DWITH_QT=OFF \

to be

-DWITH_QT=ON \

We need to make sure a new support library that is created as a result gets installed with highgui, so edit debian/libopencv-highgui-dev.install and add the lines

usr/include/opencv4/opencv2/cvv/*
usr/include/opencv4/opencv2/cvv.hpp
usr/lib/*/libopencv_cvv.a
usr/lib/*/libopencv_cvv.so

and then edit the file debian/libopencv-highgui4.2.install and add the line

usr/lib/*/libopencv_cvv.so.*

Then you have to get setup to build and actually build the .deb files

sudo apt-get update
sudo apt-get install dpkg-dev devscripts
sudo apt build-dep opencv
sudo apt-get install qt5-default
sudo debuild -b -uc -us

You should have a bunch of .deb files in .., and the two important ones are libopencv-highgui4.2_4.2.0+dfsg-5_amd64.deb and libopencv-highgui-dev_4.2.0+dfsg-5_amd64.deb. I gather those two .debs in a debs subdirectory of my developer dockerenv directory that overlays what I as a developer need on top of the images used for deploying on the robots. I add the following to dockerenv/Dockerfile

 # below is a highly risky way of enabling the Qt backend for OpenCV highgui.
 # If you are not actively debugging image processing code, you may want to
 # comment it out 
 COPY debs /home/acr/debs 
 RUN dpkg -r --force-depends libopencv-highgui-dev libopencv-highgui4.2
 RUN dpkg -i /home/acr/debs/libopencv-highgui4.2_4.2.0+dfsg-5_amd64.deb /home/acr/debs/libopencv-highgui-dev_4.2.0+dfsg-5_amd64.deb

This is a fairly alarming and risky feeling maneuver, and it leaves the apt system in my docker pretty much crippled, as everything that depends on highgui is all borked up.

Even more alarming and an indication of how risky this is, when I run with this I got a core dump in an invocation of pcl::ConvexHull::reconstruct. Googling led me to a fortunate thing: I just changed

 #include <pcl/surface/convex_hull.h>

to

#include <pcl/surface/impl/convex_hull.hpp>

That doesn't make me worried at all. Fortunately, this is only in my "debugging" code for use when I am developing the algorithms, and the weirdness doesn't extend to what is deployed to customers.

Again, I ask, was there a better way to do any of this, or do the ROS developers just really, really not want people to actually develop openCV code under noetic? Is there a similar problem in ROS2 with openCV, or is it all better there and this is another reason I should switch?


Originally posted by Jay Gowdy with karma: 136 on 2021-12-03

This answer was ACCEPTED on the original site

Post score: 1


Original comments

Comment by osilva on 2021-12-03:
Thank you for the thorough answer. I’m sure others will benefit in the future as well

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.