1
$\begingroup$

Rosanswers logo

Is it possible to create a rosbuild or catkin workspace, that overlays two or more independent catkin workspaces at the same time? I would like to keep the workspaces of different, independent projects A and B separated, but there may be a third project C which uses packages from A and B. In other words: Can I source two catkin-generated setup.sh scripts one after another, so that rospack and cmake's find_package() architecture will find packages from both workspaces A and B?

Here is a simple example setup with empty workspaces, which hopefully clarifies what I mean:

source /opt/ros/groovy/setup.bash

mkdir /tmp/catkin_overlay_test
cd /tmp/catkin_overlay_test

mkdir -p catkin_workspace_a/src     # create empty workspace A
mkdir -p catkin_workspace_b/src     # create empty workspace B

catkin_make -C catkin_workspace_a       # This will create catkin_workspace_a/devel
catkin_make -C catkin_workspace_b       # This will create catkin_workspace_b/devel

Now I would assume that I can use packages (if there were any) from both workspaces after having sourced both setup scripts or by merging them to a rosbuild workspace:

# Alternative 1: Source both setup scripts
source catkin_workspace_a/devel/setup.bash
source catkin_workspace_b/devel/setup.bash

# Alternative 2: Merge A and B in another rosbuild workspace C
rosws init rosbuild_workspace_c
cd rosbuild_workspace_c
rosws merge ../catkin_workspace_a/devel
rosws merge ../catkin_workspace_b/devel
source setup.bash

Both alternatives lead to the same result, namely that the setup script in workspace B overrides the environment variables set by A and packages in A are not visible:

$ env | grep catkin_overlay_test
ROS_PACKAGE_PATH=/tmp/catkin_overlay_test/catkin_workspace_b/src:/opt/ros/groovy/share:/opt/ros/groovy/stacks
ROS_WORKSPACE=/tmp/catkin_overlay_test/rosbuild_workspace_c
LD_LIBRARY_PATH=/tmp/catkin_overlay_test/catkin_workspace_b/devel/lib:/opt/ros/groovy/lib
CATKIN_TEST_RESULTS_DIR=/tmp/catkin_overlay_test/catkin_workspace_b/build/test_results
CPATH=/tmp/catkin_overlay_test/catkin_workspace_b/devel/include:/opt/ros/groovy/include
ROS_TEST_RESULTS_DIR=/tmp/catkin_overlay_test/catkin_workspace_b/build/test_results
PATH=/tmp/catkin_overlay_test/catkin_workspace_b/devel/bin:/opt/ros/groovy/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
PYTHONPATH=/tmp/catkin_overlay_test/catkin_workspace_b/devel/lib/python2.7/dist-packages:/opt/ros/groovy/lib/python2.7/dist-packages
PKG_CONFIG_PATH=/tmp/catkin_overlay_test/catkin_workspace_b/devel/lib/pkgconfig:/opt/ros/groovy/lib/pkgconfig
CMAKE_PREFIX_PATH=/tmp/catkin_overlay_test/catkin_workspace_b/devel:/opt/ros/groovy

Is this a bug in catkin or is it simply not possible to build upon multiple devel spaces? I did not try, but I assume there is no difference when using the install spaces instead. Does catkin only support a linear hierarchy of "parent workspaces"?


Originally posted by Johannes Meyer on ROS Answers with karma: 1266 on 2013-06-10

Post score: 10


Original comments

Comment by joq on 2013-06-10:
Why not just order the overlays hierarchically: C overlays B which overlays A?

Comment by Johannes Meyer on 2013-06-10:
B does not depend from A in my case. Of course it is possible to overlay them linearly, but I did not get the point why depending from two workspaces should be bad until now. In any case I would like to avoid having multiple working copies of the same repo in different workspaces.

Comment by William on 2013-06-10:
@Johannes Meyer, what I do some times is use symbolic links to construct workspaces, so rather than having a working copy of catkin, for instance, in every workspace, I clone it once and symbolically link it into each workspace I am using.

Comment by joq on 2013-06-10:
@Johannes Meyer, I understand that B does not depend upon A. But, unless it actually redefines packages in A, the linear approach provides an easy way for C to overlay them both without repeating anything.

$\endgroup$

1 Answer 1

0
$\begingroup$

Rosanswers logo

Short answer is "yes", longer answer is "it can be dangerous".

You can call any setup.bash/zsh/sh with the --extend option which will "extend" your current environment. If you do not do this then the setup.bash/zsh/sh will overwrite your current environment to restore the environment which existed when it was generated (the setup.bash/zsh/sh).

So you can do something like this:

  • source /opt/ros/groovy/setup.bash
  • catkin_make # In A_ws
  • open a new terminal

  • source /opt/ros/groovy/setup.bash
  • catkin_make # In B_ws
  • open a new terminal

  • source A_ws/devel/setup.bash
  • source B_ws/devel/setup.bash --extend
  • catkin_make # In C_ws (this builds on both A and B and /opt/ros/groovy)

BUT this can be dangerous, read on if you care:

catkin workspaces do not support multiple inheritance of workspaces by default in order to prevent inconsistent build and run environments. For example:

Consider that you have some binary debians installed, and you want to build workspace A, you would do so like this:

  • source /opt/ros/groovy/setup.bash
  • catkin_make # In A_ws

Then you separately build B (in a new terminal):

  • source /opt/ros/groovy/setup.bash
  • catkin_make # In B_ws

Now you have a workspace C which you want to use A and B, so you try:

  • source /opt/ros/groovy/setup.bash
  • source A_ws/devel/setup.bash
  • source B_ws/devel/setup.bash
  • catkin_make # In C_ws

The C_ws will likely fail to build if it relies on packages which only exist in workspace A. Why? because the setup.bash files overwrite each other. In fact when if I build workspace A like this:

  • ./src/catkin/bin/catkin_make # In A_ws (assuming catkin is in that workspace)

And then do this:

  • source /opt/ros/groovy/setup.bash
  • source A_ws/devel/setup.bash

Then you will not find the /opt/ros/groovy/ install space in your environment.

This is the rule you have to remember:

The setup.bash/zsh/sh file generated by catkin has the current environment plus the workspace which was just built.

That means that each time you source a setup.bash file which was generated by catkin it is reconstructing the environment in which it was created. The reason for this is to provide a consistent build and run environment for your packages. It in fact does not make sense to build packages in one environment and run them in another.

If you want to build workspace C on top of A and B, then you should follow this pattern:

  • source /opt/ros/groovy/setup.bash
  • catkin_make # In A_ws
  • open a new terminal (this isn't required, but it is good to keep a clean environment)

  • source A_ws/devel/setup.bash # This contains /opt/ros/groovy AND A_ws's packages
  • catkin_make # In B_ws
  • open a new terminal

  • source B_ws/devel/setup.bash # This contains /opt/ros/groovy, A_ws, and B_ws
  • catkin_make # In C_ws

In this way you are always building your current workspace on top of exactly one other workspace.

This is a necessary constraint to prevent other less obvious errors from arising. For example, consider this:

You build a workspace A which contains a library foo.

Next you build a workspace B, after sourcing A, which contains a library bar which is linked against foo from A.

But then you have a modified version of foo in a workspace C which you build after source A. You source A first to get the dependencies of foo from workspace A, but now the version of foo from workspace C is overlaying the version from A.

Next you want to build a package in workspace D which uses bar and foo, but you want to test your changes to foo from C. So you do this:

  • Source A
  • Source B
  • Source C
  • Build D

But workspace D fails because it cannot find bar. So you try sourcing B again:

  • Source B
  • Build D

Now D builds, finding bar in B and foo from A, but you do not get your modified version of foo from C.

Lets assume for a second that you used the --extend option to add the environment from C onto the environment from B. Now D builds and finds foo from C and bar from B. But there is still a problem, because bar from B is compiled against foo from A, so your package in D is now linking against foo from C AND foo from A via bar from B. This problem is not easy to catch and not all changes to foo would actually cause a problem. But to prevent this from occurring catkin enforces no multiple workspace inheritance. Because any time you do this multiple workspace inheritance you will be using a product from one workspace which was built without influence from the other, e.g. bar from B was not built in the presence of foo from C.

Now if you take the recommendation from above, you would do this:

  • Build A
  • Source A
  • Build C
  • Source C
  • Build B
  • Source B
  • Build D

Now D builds against a bar from B which linked against a foo from C which is overlaying the foo from A. By enforcing this single workspace inheritance model, catkin tries to prevent inconsistent build/run environments.

Hopefully this helps a little bit to explain why this is and why catkin workspaces have this limitation, and I would like to hear feedback on ways to improve this use case, because I cannot think of any.


Originally posted by William with karma: 17335 on 2013-06-10

This answer was ACCEPTED on the original site

Post score: 12


Original comments

Comment by Johannes Meyer on 2013-06-10:
Thanks, William, for your detailed response. I think I understand your caveats. On the other side I would assume that most users will not overlay workspaces in order to redefine packages/libs, but just to "use" distinct packages from A and B. In this simple case multiple inheritance would be okay?

Comment by Johannes Meyer on 2013-06-10:
One additional question: Is there a simple way to specify --extend when sourcing the catkin setup script from rosbuild workspaces via rosws merge/setup-file as in the example alternative 2?

Comment by William on 2013-06-10:
Perhaps. The danger comes in when people intend to just use packages from A and B, but end up causing weird build problems without realizing why. I didn't make the decision to setup workspaces like this, but I understand why it was designed this way.

Comment by William on 2013-06-10:
That I do not know. I would assume the answer is no, I would use the sequential chaining as a work around to that, or maybe someone else knows if that is possible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.