![Rosanswers logo](https://i.sstatic.net/yuGTR.png)
I am running the same setup and ran into the same problem but managed to fix it after creating a Debug build of the core ros libraries and analyzing the code of the XmlRpcServer
where the massive memory allocation happens.
The XmlRpcServer
in ros_comm
sets up a vector for querying all possible file descriptors to check if it has run into the limit of opened files. When running a container on Arch however, this limit is 1073741816. With each file descriptor having a size of 8 Byte, thats 8GB for every ros node. You can check the current (hard) limit of the running shell with ulimit -Hn
. If you reduce the limit with ulimit -Hn 524288
, the memory usage should be fine again.
You can make this limit persistent by adding the file /etc/security/limits.d/30-nofilelimit.conf
in the container with the following content:
#<domain> <type> <item> <value>
* soft nofile 1024
* hard nofile 524288
Originally posted by Ralino with karma: 46 on 2019-12-19
This answer was ACCEPTED on the original site
Post score: 3
Original comments
Comment by gvdhoorn on 2019-12-20:
This is some good detective work @Ralino.
I would strongly suggest you open an issue on ros/ros_comm about this. Please provide sufficient detail about how you found out and where the vector
is created, and a link to this Q&A.
Seeing as the focus of development is on ROS 2 these days I cannot guarantee it will be fixed (or implemented differently), but it would be tremendously valuable to have a record of your findings on the issue tracker regardless.
Comment by ixil on 2019-12-20:
@Ralino thanks for taking the time to post your work. I gave up in the end using the container approach. But relieved to know there is a solution
Comment by pumaking on 2023-03-22:
If anyone is having trouble with setting the hard limit: check out https://askubuntu.com/questions/1061572/unable-to-modify-ulimit, you may also need to run ulimit -Sn 524288
.
Comment by danambrosio on 2023-03-27:
You can also use docker run ... --ulimit nofile=1024:524288
to set the soft and hard nofile limits.
If you want this for all containers consider adding:
"default-ulimits": {
"nofile": {
"Name": "nofile",
"Hard": 524288,
"Soft": 1024
}
}
To your /etc/docker/daemon.json file.