0
$\begingroup$

Cross-post on ROS answer

Since ROS 2 Humble on Ubuntu 22.04, I noticed that now ROS 2 Humble python packages are split between two directories.

/opt/ros/humble/lib/python3.10/site-packages
And
/opt/ros/humble/local/lib/python3.10/dist-packages

For example, tf2_ros is installed under /opt/ros/humble/lib/python3.10/site-packages/tf2_ros and tf2_py is installed under /opt/ros/humble/local/lib/python3.10/dist-packages/tf2_py

From what I can see, it was not the case before (for Foxy and Galactic, at least). And I was wondering why is it split like this?

This brings the issue that if you source your ROS 2 Humble installation before installing any packages, installing files in local/lib/python3.10/dist-packages this subdirectory won't be added to your PYTHONPATH environment variable.

This force to resource the environment.

$\endgroup$

2 Answers 2

1
$\begingroup$

The destination of site-packages versus dist-packages depends on which installation and packaging system you use.

site-packages is the default for python native installers. dist-packages is an alternative location that is mandated by Debian to avoid conflicts. There's some discussion here

Third party Python software installed from Debian packages goes into dist-packages, not site-packages. This is to reduce conflict between the system Python, and any from-source Python build you might install manually. From the Debian Python Page

This means that it depends on which installation mechanism is used they may use the different default situation. If you look at tf2_py It's a c++ compiled package which is installed via cmake which defaults to the debian conventions. Whereas tf2_ros_py which provides the tf2_ros libraries uses the native python installer, thus defaulting to the upstream python convention.

This does look like a change from earlier. Unfortunately python installation is currently going through several changes in mechanisms I'm not sure why this behavior has changed.

This force to resource the environment.

In general when you install new software you should expect to need to re-source your setup files. In the case of python, there may be a new PYTHONPATH needed. Similarly, there may be new resources installed and available based on other configurations or other <*>_PATH variables extended.

Edit: Slightly extending this discussion. I missed that content was going into local/ in addition to being site-packages vs dist-packages difference I talked about above.

There's some unfortunate tweaks that various Linux distributions are doing to patch their Python libraries to inject local/ into the paths. It should never have local/ in it - some of our hooks don't work if packages end up there. If you find deb packages or rpms with local/ in them please file a ticket. We've been working to make sure that colcon-core can detect and avoid those cases of locally patched python libraries. If you're installing from source make sure that colcon-core is up to date.

$\endgroup$
2
$\begingroup$

Building off from what Tully said, I can add a little more context.

For reasons I won't get into here, several distributors of Python have begun patching the sysconfig module of Python itself to inject local into the paths reported by sysconfig.get_path (et.al.). In my opinion, this change was implemented with a mallet when it should have been done with a scalpel, but we are where we are. Fedora has revised their approach in a way that corrected the behavioral regression here, but I haven't seen anything similar from Debian/Ubuntu or Homebrew yet.

On the colcon side, this issue resulted in these changes:

On the ament side, less changes are necessary, but we have yet to release a fix. The way that these distributor patches were structured, they never affect our debian or RPM builds, so the remaining bug happens when we build ament_cmake_python packages from source outside of a debian or RPM build. The issue you're facing is also less critical than the ones we were facing in colcon because hooks for dealing with the unexpected path were generated to ensure PYTHONPATH was updated appropriately. It's still not the intended behavior, but at least it's functional. The fix for the issue is being tracked as ament/ament_cmake#386. Now that discussion around the issue has been reignited, I think we'll drive the PR to a conclusion soon.

All this said, I think you should be cautious about installing packages into a workspace and expecting that you won't need to source that workspace again. Packages may define any arbitrary environment hooks for a variety of reasons, so it will always be safest to re-source the workspace after installing or updating packages.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.