8
$\begingroup$

The answers I received to the question on training a line following robot using reinforcement learning techniques, got me to think on how to train a robot. I believe there are essentially two ways -

  1. Train the physical robot.
  2. Model the robot and simulate the training.
  3. Did I miss something?

Approach 2 is definitely the better approach. However, a priori knowledge of the motion (response), a certain PWM signal (stimulus) would cause when the robot is in a given state is required. The motion caused by a PWM signal may depend on the (1) current battery voltage, (2) the mass of the robot and the (3) current velocity (did I miss something?).

How do I model such a robot? And how do I model it quick? If I change the battery or add a few boards and other peripherals and change the mass of the robot, I would have to remodel and retrain the robot. Can I do this by providing some random stimulus PWMs and measuring the response?

added: My related question in dsp.SE

Update: A suggested edit to the title by Ian worth mentioning - "How do I model train a robot so that if its dynamics change, it does not need complete re-training?" I think this is a good question too but different from the one I am asking here. I am okay with re-training for now.

Improve this question
$\endgroup$
8
  • $\begingroup$ There are lots of great questions here. In fact it might be better to split this into multiple questions. $\endgroup$
    – DaemonMaker
    Nov 15, 2012 at 18:53
  • $\begingroup$ How do you suggest I do that without loosing the background? I already split a part of my question into dsp.SE $\endgroup$
    – Lord Loh.
    Nov 15, 2012 at 19:08
  • $\begingroup$ You basically have two questions going on here. The first being, how do I model a robot? The second being, how do I train a robot given a model and a learning algorithm? It seems to me these could be two separate questions. $\endgroup$
    – DaemonMaker
    Nov 15, 2012 at 19:20
  • $\begingroup$ You are right. The question that appears as how to train a robot was intended to be rhetorical. I think I will try to edit it out. $\endgroup$
    – Lord Loh.
    Nov 15, 2012 at 19:26
  • $\begingroup$ In that case I'll wait to see your edits before I attempt to answer your question. $\endgroup$
    – DaemonMaker
    Nov 15, 2012 at 20:13

2 Answers 2

Reset to default
4
$\begingroup$

As I see it there are two main questions here. The first is, how do I model a robot? This is frequently done with a state-space formulation of the equations of motion. The exact equations depend on the physical construction of your robot. Yes, in order to model them with PWM input then you need to determine the transfer function from the PWM values you supply to the output of your actuators. Then you plug that function in for the control signal in your model. Again the derivation of this function is robot specific. The current battery voltage and the mass of the robot would likely be useful but I don't know about the velocity.

The second question is, given a mathematical model of my robot, what is the best way to train a reinforcement learning (RL) algorithm to control it? In short there is no one best way. Training directly on the robot tends to be time consuming because it takes the robot longer to execute trials. Simulations however can result in policies that are less accurate because the physics of the simulation are necesssarily simplified. Another approach is to train the learner in simulation to get a reasonable approximation and then transfer the resulting policy to the robot for further refinement. This of course fails if the model is not sufficiently accurate. It also requires extra development.

Finally you ask "Can I [remodel and retrain the robot] by providing some random stimulus PWMs and measuring the response?" In the case of RL there is no reason to think the new optimal policy is anything like the previous optimal policy and as such there isn't much reason to think a few random controls will supply sufficient information to change the policy appropriately. Of course retraining only needs to occur if the changes you make to your robot affect the formulation of the state-space model and/or the action model that you use. For instance, if your action model is in terms of high level actions ("go-left", "go-right", "go-forward") then changing the dynamics requires changing how you implement these motions but the policy should still hold.

Improve this answer
$\endgroup$
0
$\begingroup$

For modeling the kinematics Denavit–Hartenberg parameters are commonly used. the course Introduction to Robotics provided by Stanford University is available on YouTube

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.