10

From what I understand of your question, you'd like to know if inverse kinematics and reinforcement learning are trying to solve the same problem in the particular case of robotic manipulation. Of course both of these techniques can be applied outside of this particular realm, but let's focus on robot manipulation for now. You're right that inverse ...


9

One of the key measures of any machine learning algorithm is it's ability to generalize (i.e. apply what it has learned to previously unsceen scenarios). Reinforcement learners (RL) can generalize well but this ability is in part a function of the state-space formulation in my experience. This means that if you can find the right setup then the RL learner ...


4

As I see it there are two main questions here. The first is, how do I model a robot? This is frequently done with a state-space formulation of the equations of motion. The exact equations depend on the physical construction of your robot. Yes, in order to model them with PWM input then you need to determine the transfer function from the PWM values you ...


3

The standard use of “rollout” (also called a “playout”) is in regard to an execution of a policy from the current state when there is some uncertainty about the next state or outcome - it is one simulation from your current state. The purpose is for an agent to evaluate many possible next actions in order to find an action that will maximize value (long-term ...


3

It's true that using RL in robotics involves many challenges, including the usually high dimensionality of problem spaces, the cost and limitations of real-world sessions, the impossibility or perfectly modelling the robot-environment system, and the complexity of reward functions that accurately reflect desired behaviors. That said, a number of approaches ...


3

Many reainforcement learning methods require descrete actions. As you indentified, increasing and decreasing the values is one option. If it is an adaptive PID, then it might take some time to incerase the parameters if you only have an increase by the factor of 0.1. I would recommend more then one increasing factor as possible action. Increase by 0.1, 1, 10,...


3

I read a bit more and realized that in RL states and rewards accept a wide variety of interpretations and this is the real complexity nowadays of this learning problem. In case of PID values, problem can be formulated as the following: imagine a Kp value, it represents a state. Next state could be increase or decrease 0.1. Same with the next state, and so ...


2

If you consider e.g. a robotic arm, the inverse kinematics tell you how to choose the arm's joint angles to move the arm to some position and orientation where you want it to be. In contrast to determining the forward kinemactics of some mechanism, determining it's inverse kinematics is usually hard and sometimes, there isn't even an analytic solution. ...


2

In short, yes, there are a number of robotics companies catering to just your needs. Specifically which apply to you depends on what type of robot you desire. Naturally cost varies with the complexity and durability. For example if you would be satisfied with a differential drive robot for indoor use then the iRobot Create may suit your needs. If you need ...


2

The definition of "rollouts" given by Planning chemical syntheses with deep neural networks and symbolic AI (Segler, Preuss & Waller ; doi: 10.1038/nature25978 ; credit to jsotola): Rollouts are Monte Carlo simulations, in which random search steps are performed without branching until a solution has been found or a maximum depth is reached. These ...


1

You will need MuJoCo if you care about robust physics predictions. For UR5 robot, we have created a model for MuJoCo here. This is, however, a planar robot. We have removed the joints from UR5 and we are using it as a planar robot for now. You can use it as a starting point, however. Another solution is to use OpenRAVE simulator. We wrote a controller for ...


1

A myopic policy is one that simply maximises the average immediate reward. It is "myopic" in the sense that it only considers the single criterion. It has the advantage of being relatively easy to implement. A fairly well-known example is the hill-climbing algorithm. However, a myopic search is particularly vulnerable to becoming trapped at a local optima, ...


1

I am currently working on a very similar project, the only difference is that I am using a simulation package (MATLAB Simmechanics) where I have modeled a mobile robot with 2 actuated wheels and a castor wheel. I have 4 sensors, as a result, I am not using the "middle position" as a reward but I can easily modify that. My model takes parameters such as ...


1

HRL has been embodied in a robot in multiple cases. In a reaching, shelving robot. In a robot learning how to stand-up. In robot navigation. However, how HRL applied in each of these cases varies. The first uses HRL to manipulate Dynamic Movement Primitives, while the second, older method focuses moreso on learning state space values.


1

Some years ago I have used the supervisor to know the position of a simulated Khepera robot in Webots 4. The main components of the C code was the following: Declarations: #define STEP 64 NodeRef robot_node; float robot_data[4]={0,0,0,0}; Getting the node named Khepera: robot_node=(NodeRef)supervisor_node_get_from_def("KHEPERA"); Getting data ...


Only top voted, non community-wiki answers of a minimum length are eligible