I read a bit more and realized that in RL states and rewards accept a wide variety of interpretations and this is the real complexity nowadays of this learning problem.
In case of PID values, problem can be formulated as the following:
imagine a Kp value, it represents a state. Next state could be increase or decrease 0.1. Same with the next state, and so on.
Moreover, in some research problems, when talking not about controllers, but about external efforts, they simplify the problem to three possible states: fixed positive value, fixed negative value or 0 value. It is complicated to apply this idea to controllers.
I am almost sure that my particular problem could be more easily formulated if I used as states another kind of variables, such as "advancing forward, backwards or not moving". RL allows it.