0
$\begingroup$

Rosanswers logo

Ubuntu 16.04 LTS Xenial ROS Kinetic

Linux novice, ROS novice.

Have a catkin workspace with some packages in it (robot_localization etc).

import rosbag

gives error

import error: no module named rosbag

so tried installing:

sudo apt-get update
sudo apt-get install python-rosbag

get error:

p@t:~$ sudo apt-get install python-rosbag
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python-rosbag : Depends: python-roslib but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

What should I do? How do i fix the import error?

Edit I'm sorry. I've been at this for months now. I didn't study for this. Frustration is making me go crazy.

$ which python returns "/usr/bin/python" now. I assume this is what it's supposed to be? $ grep -i anaconda /home/pieter/.bashrc returns "grep: /home/pieter.bashrc: No such file or directory" if that's still relevant

$ ~/folderimworkingin python file.py now returns with error ending in

File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/msg_loader.py", line 83, in get_msg_file
    % (base_type, package, search_path), base_type, package, search_path)
genmsg.msg_loader.MsgNotFound: Cannot locate message [Header]: unknown package [std_msgs] on search path [{}]

So am I missing a message package now or?

Edit

pieter@turing:~/benchmarking$ rospack find std_msgs
/opt/ros/kinetic/share/std_msgs

Seems reasonable.

>>> import std_msgs
>>>

does not complain.

Python --version

returns 2.7.12 now (instead of 2.7.14).

File.py:

#import numpy as np
#import scipy.io
from bag import get_topic_data
testfile = "/home/pieter/benchmarking/bagfiles/2018-03-03-11-13-48.bag"
testdata = get_topic_data(testfile,"/android/imu")
#size = len(testdata)
#print(size)

with bag.py:

import rosbag
def get_topic_data(bagFile, topic):
    all_msg = []        
    # Initialize rosbag object
    bag = rosbag.Bag(bagFile)        
    for topic, msg, t in bag.read_messages(topics=[topic]):            
        all_msg = np.append(all_msg, msg)    
    return all_msg

bag file I think is accessible?:

https://github.com/PieterVanB/bagfile

Originally posted by anonymous32749 on ROS Answers with karma: 62 on 2018-03-03

Post score: 1


Original comments

Comment by gvdhoorn on 2018-03-04:\

This is all greek to me.

I sympathise with you, but please refrain from making these types of comments.

Comment by gvdhoorn on 2018-03-05:
@OperationCrossbow: did fixing the bag fix the issues you were having with your script?

Comment by anonymous32749 on 2018-03-05:
@gvdhoorn installed the bagfixer, did catkin_make. source bashed, new terminal. says can't find "fix_bag_msg_def.py" even though it's executable and in the same folder as the input bag. But i'm googlin'.

Comment by gvdhoorn on 2018-03-05:
It's not a catkin package, so catkin_make is not needed.

You only need to clone the repository, go to the directory where your bag is and then do something like:

$ /path/to/where/bag_fixer/is/fix_bag_msg_def.py [...] out.bag
$ rosbag reindex out.bag

That's it.

[...] is the other args.

Comment by gvdhoorn on 2018-03-06:
Btw: to run programs/scripts in the current directory, always prefix the name of the script or program with ./. Otherwise bash will not be able to find the program (unless it's on the path).

Comment by gvdhoorn on 2018-03-08:
@OperationCrossbow: any luck?

Comment by anonymous32749 on 2018-03-09:
@gvdhoorn pieter@turing:~/benchmarking/bagfiles$ /catkin_ws/src/rosbag_fixer/fix_bag_msg_def.py --use-local-defs 2018-03-03-11-13-48.bag out.bag

returns: bash: /catkin_ws/src/rosbag_fixer/fix_bag_msg_def.py: No such file or directory

What am I missing?

Comment by gvdhoorn on 2018-03-09:
You're specifying an absolute path. That starts at the root of your file system and looks for a directory catkin_ws in the root, instead of in your home directory. That won't work.

You can either prefix with the path to ..

Comment by gvdhoorn on 2018-03-09:
.. your home directory, or use a relative path.

The following should work:

/home/pieter/catkin_ws/src/rosbag_fixer/fix_bag_msg_def.py --use-local-defs 2018-03-03-11-13-48.bag out.bag

provided you are in the /home/pieter/benchmarking/bagfiles directory, of course.

Comment by gvdhoorn on 2018-03-09:
But note again that there is no need to place the rosbag_fixer directory in your catkin_ws. It's not a Catkin package.

Be sure to have sourced the appropriate setup.bash btw. Either the one from your workspace, or the one in /opt/ros.

Comment by anonymous32749 on 2018-03-09:
@gvdhoorn d'oh. The bag fixer worked!

Trying to launch my file.py throws up "genmsg.msg_loader.MsgNotFound: Cannot locate message [Header]: unknown package [std_msgs]" again.

Comment by gvdhoorn on 2018-03-09:
That is the exact same error message as before. Are you using out.bag now? It doesn't touch the original bag, so you'll have to work with out.bag.

Comment by anonymous32749 on 2018-03-10:
@gvdhoorn I am not a smart person.. It seems to work now though. Barring a "no module named scipy.io" error.

Comment by gvdhoorn on 2018-03-10:
That would seem to be a different issue, and with your script, not with the infrastructure any more.

$\endgroup$

1 Answer 1

0
$\begingroup$

Rosanswers logo

Edit 9:

I've updated the tool and ran your bag through it as follows:

$ fix_bag_msg_def.py --use-local-defs 2018-03-03-11-13-48.bag out.bag
$ rosbag reindex out.bag

This replaced 6 message definitions for me. Afterwards the bag was usable. I have tested your script and I now get errors about np not being defined (which is not imported, so that error makes sense).

See the readme of gavanderhoorn/rosbag_fixer for some more information on how to use it.


Edit 8: ok. I believe I know what is going on.

Overview of your situation:

  1. your initial problem ("cannot install rosbag") was solved/a non-problem: it was already installed, and python-rosbag is not actually the pkg you want to install on Ubuntu if you already have ros-kinetic-rosbag.
  2. the "cannot import rosbag" problem was also solved/worked-around: you were using Anaconda, and the binary packages distributed via the official OSRF package repositories do not support that (the modules are installed in the 'regular' location, and Anaconda doesn't pick those up). You can later try and get Anaconda to work again.

Now as to your latest edit: you didn't mention it (I didn't ask, and I should've), but you're trying to load a bag with messages published by rosjava (ROS Java is used by ROS Android under the hood). There is a known issue with rosjava that prevents this from working.

For some references, see ros/common_msgs#20 and rosjava/rosjava_bootstrap#16 (and #q108956, a related question). Note the error message included in the comments on the first issue, it's virtually identical to yours. The problem is described in rosjava/rosjava_bootstrap#16:

It appears that published messages are missing the message definition for the child types which is used for message introspection

In a nutshell this means that rosbag has no way of figuring out what the format of the data is you're trying to load from your bag, and thus cannot continue processing it. Hence the exception. Looking at your script however it appears to be ok, and should work with just about any bag file that doesn't contain messages published by rosjava.

So to summarise: everything is ok now, it's just that you've run into an issue with rosjava that prevents your script from working (note: this is different from the script being syntactically ok and the runtime environment being able to load all required libraries, as was the problem before).

I've ran into this issue myself, and wrote a simple tool to try and fix bags that contain messages published by rosjava: gavanderhoorn/rosbag_fixer. I can't guarantee it will work and it's rather underdocumented (I only needed it once). If you can provide your bag I'm willing to try and fix it for you. Afterwards I'll update my answer with the steps needed to fix arbitrary bags so you can do it yourself if you need to -- I can't remember them now.

(note: I placed my latest edit at the top of my answer, instead of at the end)


Can you tell us how you installed ROS?

The reason you cannot get python-rosbag to install is most likely because it is not part of the official OSRF repositories, but is distributed as an upstream package in Debian (and then imported into Ubuntu). These packages conflict with the OSRF packages, and that is why python-roslib will not be installed (and thus neither python-rosbag).

The following command installs the package from the OSRF repositories:

sudo apt-get install ros-kinetic-rosbag

The situation with the upstreamed packages and the official packages is unfortunate, but something you will have to keep in mind. As explained on the wiki page I linked above, those upstreamed packages can be recognised by the absence of a ROS release code name, which in this case would be kinetic.


Edit:

So is this Anaconda? What is the output of which python?

returns: "home/username/anaconda2/bin/python.

Ok. Your issue is not that you can't install rosbag: it is already installed. Installing python-rosbag would not have helped either.

The issue is that your Anaconda installation does not seem to be aware of this (it uses different search paths, and rosbag is not on those). There are numerous threads on ROS Answers about exactly this issue.

It can be made to work, but for now, please try to use the regular Python 2 interpreter installed on your system. If your script works with that, search for some hints on how to get things working with Anaconda using Google: anaconda site:answers.ros.org.

Note though that the official binary packages provided by the OSRF repositories will always only target the system (ie: Ubuntu) provided Python interpreter, which for your platform is still the regular Python 2 python binary. Working with Anaconda you'll always have to patch things up.


Edit 2:

How does it more accurately describe what the problem is? I'm so novice that I can't even see that.

The problem is not that rosbag can't be installed, as it already is.

The problem is that the import rosbag is failing, as your Anaconda install doesn't have that Python module, only the regular Python has it.

Also, that older question is from before my computer wipe, and remains unsolved for me.

Could be, but that problem was most likely also caused by using Anaconda and then not setting it up correctly.

Also also, what is "the regular Python 2 interpreter" in your words? $ python file.py doesn't work.

So long as which python returns /home/username/anaconda2/bin/python things won't work. During installation of Anaconda you may have added /home/username/anaconda2/bin to your PATH, probably by editing /home/username/.bashrc. If you did, temporarily undo that change, start a fresh terminal and see whether which python returns something different.

As to "the regular Python 2 interpreter": Anaconda is not the official Python 2 runtime environment. For Ubuntu Xenial, that would be the python2.7 package, which you most likely also have installed.

Anaconda is an alternative interpreter. It's got some nice properties, but it also confuses a lot of users, especially novices. So much so, that there is some discussion in the ROS community to warn against its usage. See Should we warn (new) users about difficulties with Python 3 and alternative Python interpreters?.


Edit 3: just comment that export PATH=.. for now. Don't set PYTHONPATH either.

I would suggest to first see whether the regular Python 2 setup works. Then start getting Anaconda working.


Edit 4:

.bashrc file, commented out the "PATH" things at the bottom.

Good.

Can you tell me how I "see if python 2 works" and "get Anaconda working"? This is just shy of "You fix this by fixing it" to my brain.

Re: see if python 2 works: what is the output of which python now in a fresh shell? If it's no longer your Anaconda installation, then you're probably ok. Try to execute your script. If that doesn't complain, it's almost certainly using the regular Python 2 interpreter which has all the required modules (libraries) installed.

As to getting Anaconda working: I'm not going to help you with that. That may sound unfriendly, but I don't use it, so I'd have to go and install it myself and / or read up on it and try to replicate and fix your issue. There are a lot of Q&As about Anaconda and ROS on this site already, so those should help you.

The gist of the solutions I've seen is to make sure you either install the ROS Python pkgs (such as rosbag) into your Anaconda environment (using Anaconda's equivalent of pip, or something similar), or add the ROS dist-packages and/or site-packages directory to Anaconda's PYTHONPATH. But again, I'm not an expert, so refer to the Q&As I mentioned.


Edit 5:

As to "fix it yourself", no, that's fair, that's usually what I expect as an engineering answer. I'm not made for this profession.

Please relax, will you? We're already 5 edit deep into this question. I don't believe I'm leaving you to fix this yourself, am I? We all get frustrated every so often, but telling off the people that are trying to help you is not a constructive attitude.

As to "which python", I forgot to mention, it's still outputting /anaconda2/. I have no idea what's causing it and google is not playing nice.

Please make sure you have started a new terminal after editing your .bashrc. If you have done so, please show the output of grep -i anaconda /home/pieter/.bashrc.


Edit 6:

$ which python returns "/usr/bin/python" now. I assume this is what it's supposed to be?

yes, for now at least. You're now no longer using Anaconda.

$ grep -i anaconda /home/pieter/.bashrc returns "grep: /home/pieter.bashrc: No such file or directory" if that's still relevant

Are you sure you typed the correct path? /home/pieter.bashrc is not supposed to exist, so that error message would be correct. I would expect /home/pieter/.bashrc to exist (note the / between pieter and .bashrc, which is missing in your error message).

$ ~/folderimworkingin python file.py now returns with error ending in

File "/opt/ros/kinetic/lib/python2.7/dist-packages/genmsg/msg_loader.py", line 83, in get_msg_file
    % (base_type, package, search_path), base_type, package, search_path)
genmsg.msg_loader.MsgNotFound: Cannot locate message [Header]: unknown package [std_msgs] on search path [{}]

So am I missing a message package now or?

That is one of the causes of this error message, but seeing as std_msgs is such a basic package, I cannot believe that you would not have it. If rospack find std_msgs outputs a reasonable path, then I'm not sure what is going on.

If you just start python itself, then type import std_msgs, does that complain as well?


Edit 7: it's strange that importing manually works, and the std_msgs pkg is actually found.

Could you add the code of file.py to your question? Perhaps something is missing there.

Just occured to me btw: what is the output of python --version now?


Originally posted by gvdhoorn with karma: 86574 on 2018-03-03

This answer was ACCEPTED on the original site

Post score: 1


Original comments

Comment by anonymous32749 on 2018-03-03:
How I installed ROS and the current packages I own is in 4 hours time with several USB-stick reboots and complete wipes of my computer, with a doctorate student sitting next to me telling me what to do. Idk what upstreamed packages are.

Also: "ros-kinetic-rosbag is already the newest version".

Comment by gvdhoorn on 2018-03-03:\

How I installed ROS and the current packages I own is in 4 hours time with several USB-stick reboots and complete wipes of my computer, with a doctorate student sitting next to me telling me what to do

A link to a tutorial would have sufficed.

And Linux != ROS.

Comment by gvdhoorn on 2018-03-03:\

Also: "ros-kinetic-rosbag is already the newest version".

That is ok. It means you already have that package, which is expected.

What is the output of python --version?

Comment by gvdhoorn on 2018-03-03:\

Idk what upstreamed packages are.

that is why I linked you to the article on the wiki that explains it.

Comment by anonymous32749 on 2018-03-03:
@gvdhoorn Ah, sorry. My frustration is getting the better of me. Python is 2.7.14. The doctorate student (my thesis copromotor) told me this is probably the best to use (like Anaconda 2) because it's more stable, works better with ROS, etc, etc.

Comment by gvdhoorn on 2018-03-03:
So is this Anaconda? What is the output of which python? If you already have ros-kinetic-rosbag, there should be files under /opt/ros/kinetic/lib/python2.7/dist-packages/rosbag. You did source /opt/ros/kinetic/setup.bash?

Comment by anonymous32749 on 2018-03-03:
(1) returns: "home/username/anaconda2/bin/python." (2) yes, rosbag is there (3) Yes; there is source /opt/~~/setup.bash is in my bashrc file, so everytime I open a new terminal it should source this.

Comment by anonymous32749 on 2018-03-03:
@gvdhoorn still feel like helping me?

Comment by gvdhoorn on 2018-03-03:
I edited my answer about an hour ago.

Comment by gvdhoorn on 2018-03-03:
Btw: your deleted question (#q284265) more accurately describes what the problem is. And #q281035 seems to be a similar issue with Anaconda.

Comment by anonymous32749 on 2018-03-03:
@gvdhoorn How does it more accurately describe what the problem is? I'm so novice that I can't even see that. Also, that older question is from before my computer wipe, and remained unsolved. Also also, what is "the regular Python 2 interpreter" in your words? $ python file.py doesn't work.

Comment by gvdhoorn on 2018-03-03:
Btw: if your thesis supervisor advised / helped you to install Anaconda, he may also know how to (temporarily) undo the changes to your PATH.

Comment by Kansai on 2020-08-06:
"your initial problem ("cannot install rosbag") was solved/a non-problem:".... really where??

Comment by gvdhoorn on 2020-08-07:
ros-kinetic-rosbag was already installed. The problem of the OP was never rosbag related, it was the .bag file.

Additionally, he/she had Anaconda installed and active, which caused problems as rosbag is not installed in/for Anaconda, but for the system-default Python interpreter only.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.