I have very limited experience in operating systems and compiler, and most of my experience is with integration and control systems. Currently I am using ros kinetic with teb_local_planner for navigation. For those who are not familiar, teb uses libg2o which is a nonlinear least square solver, and libg2o does depends on cvsparse which uses BLAS and LAPACK; I am assuming all of these are optimized.
What I observe is that when I am running simulation and tries to give a navigation goal at a desired 20 hz, everything behaves as expected. However, when I tries to increase the number of loops with g2o (inner_iteration and outer_iteration for teb_local_planner), move_base returns teb is returning cmd_vel at lower rate (around 12 HZ let's say).
When I tried to check the amount of computation with htop, all my cores are running at around 70-80% while none of them maxed out. And when I tried to use sysprof to profile the computation, only around 5% of computation are related to teb and g2o.
My main question is why is teb lagging (not at desired rate), even though the cores are not maxed out, and it is only taking 5% of overall computation. My other question is whether compiling g2o from source and specify Cmake build type to be Release will be helpful to me. Could ROS not being a real time system cause this in any way? move_base itself is a single node, so even if communication is lagged in subscriber/publisher, it shouldn't cause any problems since it is a single process?
This question maybe a bit too specific, I think I need a fresh perspective on this since I am not trained on these topics at all.
P.S. I am using Jetson TX2 which is an ARM processor.