apt trying to install old versions of ros packages

Question

Not sure if this is an apt issue or not but so far this is only happening for ros packages.

Having not updated my packages for a long time, it seems apt is stuck trying to upgrade a ros package to a (now outdated and hence nonexistent) version.

e.g. when trying to upgrade ros-noetic-tf2-msgs

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  ros-noetic-tf2-msgs
1 upgraded, 0 newly installed, 0 to remove and 74 not upgraded.
Need to get 44.4 kB of archives.
After this operation, 0 B of additional disk space will be used.
Err:1 http://packages.ros.org/ros/ubuntu focal/main amd64 ros-noetic-tf2-msgs amd64 0.7.6-1focal.20230620.183347
  404  Not Found [IP: 64.50.233.100 80]
E: Failed to fetch http://packages.ros.org/ros/ubuntu/pool/main/r/ros-noetic-tf2-msgs/ros-noetic-tf2-msgs_0.7.6-1focal.20230620.183347_amd64.deb  404  Not Found [IP: 64.50.233.100 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Understandably, the (now outdated) 0.7.6-1focal.20230620.183347 is no longer found since the current version as per here is 0.7.7-1focal.20231013.184042.

I've tried different combinations of apt update, apt clean, sudo apt install ros-noetic-tf2-msgs=0.7.7-1focal.20231013.184042 but no luck.

Output of apt update and apt update --fix-missing shows nothing suspicious. All upgradable packages are ros packages

...
Hit:16 http://packages.osrfoundation.org/gazebo/ubuntu-stable focal InRelease                                         
Hit:18 http://packages.ros.org/ros/ubuntu focal InRelease                                                             
Hit:20 http://packages.ros.org/ros2/ubuntu focal InRelease  
...       
Fetched 3,863 kB in 3s (1,163 kB/s) 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
75 packages can be upgraded. Run 'apt list --upgradable' to see them.

I've also made sure that sources.list and the keys are set up correctly as of the ubuntu install instructions

apt-cache policy ros-noetic-tf2-msgs gives

ros-noetic-tf2-msgs:
  Installed: 0.7.6-1focal.20221014.220058
  Candidate: 0.7.6-1focal.20230620.183347
  Version table:
     0.7.6-1focal.20230620.183347 500
        500 http://packages.ros.org/ros/ubuntu focal/main amd64 Packages
 *** 0.7.6-1focal.20221014.220058 100
        100 /var/lib/dpkg/status

Any help would be greatly appreciated.

I'm leaving a complete ros reinstall as a last resort

Answer 1

There's something wrong with your apt-get update.

For a clean docker image freshly pulled rocker ros:noetic I get

root@ce76fb5b82a4:/# apt-cache policy ros-noetic-tf2-msgs
ros-noetic-tf2-msgs:
  Installed: (none)
  Candidate: 0.7.7-1focal.20231013.184042
  Version table:
     0.7.7-1focal.20231013.184042 500
        500 http://packages.ros.org/ros/ubuntu focal/main amd64 Packages

I would suggest you look deeper into your apt-get update.

You can get more information by running with extra debug flags like suggested here

sudo apt -oDebug::pkgAcquire::Worker=1 update

Look for caching (potentially a transparent proxy), gpg or other certificate signing failures. Until you get the latest versions from update the install is going to keep failing.

Answer 2

Thanks to Tully's answer, I spotted the following output seemed relevant

Hit:16 http://packages.osrfoundation.org/gazebo/ubuntu-stable focal InRelease
 <- http:201%20URI%20Done%0aIMS-Hit:%20true%0aLast-Modified:%20Thu,%2026%20Oct%202023%2017:13:26%20+0000%0aFilename:%20/var/lib/apt/lists/partial/packages.ros.org_ros_ubuntu_dists_focal_InRelease%0aURI:%20http://packages.ros.org/ros/ubuntu/dists/focal/InRelease
Hit:17 http://packages.ros.org/ros/ubuntu focal InRelease
 <- http:102%20Status%0aURI:%20http://packages.ros.org/ros2/ubuntu/dists/focal/InRelease%0aMessage:%20Waiting%20for%20headers
 <- https:200%20URI%20Start%0aLast-Modified:%20Tue,%2024%20Jan%202023%2006:39:27%20+0000%0aSize:%2017126%0aURI:%20https://dl.yarnpkg.com/debian/dists/stable/InRelease

I managed to force apt to redownload the package list by

sudo rm /var/lib/apt/lists/packages.ros*

which multiple sources say is safe.

Then apt update managed to get the updated package list and the subsequent apt upgrade succeeded.

I'm not sure how apt managed to get the bad package lists or if they were actually good and the problem occured elsewhere in the pipeline... (I should've inspected the package lists before removing them)