How mature is real-time programming in robotics? [closed]

Question

Edit: I don't know why, but this question seems to be confusing many people. I am aware of when/where/why/how to use real-time. I am interested in knowing whether people who have a real-time task would actually care enough to implement it in real-time or not.

There's no need to mention why real-time operations are important for a robot. My question is however, how much is it actually used in robotics?

Take this question for example. Only one answer mentions any platform with real-time capabilities, and it is far from the top too. ROS apparently, being a very popular platform which is not real-time.

In the real-time world however, RTAI¹ seems to be the only workable free real-time platform of use. It is however, limited to Linux (no problem), badly documented and slowly developed.

So, how much is real-time behavior sought after among robotics developers? The question is, how much are developers inclined to write real-time applications when real-time behavior is actually needed? If not much, why?

For example, reflexive behavior based on tactile data, cannot go through ROS because it would lose its real-time property. But do people really come up with a real-time solution or use ROS anyway, ignoring the real-time property?

¹ or similarly Xenomai

Answer 1

Remember that there's Real Time and there's Real Time.

Hard Real time is difficult to achieve without hardware support or low level software support, but you often don't need everything to be Hard Real Time capable. Soft & Firm Real Time response is much easier to achieve and is often more than adequate for many applications.

Also, different parts of a system can have very different real time requirements. If you are running software PID loops, they really should have a hard real time response (you really don't want to have to choose between soft tuning your PIDs or tuning them hard and having them occasionally go unstable, for instance). A vision system might have firm real time response requirements, performance will degrade if you can't process the image in time for the next decision but it need not prevent the system running, in this case if you can't process it in time you are better off throwing away the partial results and not loosing time starting analysis on the next frame. Finally your overall planning and coordination (probably the most complex part of your robotic system) can often remain firmly in the domain of soft real time.

Even in the realm of Windows PCs you can get hard real time performance, you just need the right software with the right hooks into the kernel. Beckhoff's TwinCat soft PLC quite happily ran a high scan rate PLC by slicing half of a Pentium's machine cycles, giving the other half to Windows NT, and this was over a decade ago. Even modern control systems like some options in Aerotech's A3200 range do the grunt work on the host PC, with the low level kernel taking as much CPU time as it needs for the hard real time requirements and leaving the rest of the CPU cycles for Windows 7 to use!

Answer 2

A real time system is not really required for many (most?) robotic control systems. As long as you have a control loop that runs fast enough, with low enough jitter, and doesn't miss too many cycles, then this is quite adequate for robotic control and servoing.

As evidence of this, let me present the PR2 and the Shadow Robot Hand:

This robot has about 20 degrees of freedom, all of which are servoed through ROS's main loop. Or how about the Shadow Robot Hand, which also has 20 DOFs, plus an array of tactile and other sensors, and is also servoed through ROS's main loop.

The ROS main loop suffers from a little jitter, sometimes as much as 100us, and even sometimes misses cycles altogether. But it doesn't matter if 99.9% of cycles are executed successfully.

The use of many cores within the host PCs means that one whole core is dedicated to running the main loop, and so it is very rarely delayed by other tasks.

The main reason for using a really real-time OS for a robotic system is for safety. If the robot is working in a safety critical situation, then it is your responsibility to guarantee 100% control up-time, and part of this is guaranteeing the real time nature of it.

Whether you use a real-time OS or not, your servos should do something safe in the event that the control loop dies entirely. This safety system would also be helpful on the rare occasion your non-real-time OS skips more than a cycle. For example, on the Shadow Hand, the motors are halted if the control loop misses more than 20 cycles (20ms). I have never seen this happen though.

---

Added

Another way to think about it is this: What update rate does your servo system actually need? If it's a largish arm, and doesn't need super high performance, high speed positioning, then 500Hz might be sufficient. For driving around, 200Hz is probably sufficient. In both of these cases, if your loop is actually running at 1000Hz, then a late or missing cycle really is no problem at all, as long as your control algorithm takes into account the actual elapsed time between loops.

Answer 3

The purpose of the software determines whether it needs to be strictly real-time.

Where the purpose is path planning or localization, often a low frequency is sufficent, for example, 10Hz. In these cases, a player/stage setup running on Linux is fine. We can see that there are few problems if one time step is a bit longer or shorter.

Strictly real-time behaviour is required if the robot dynamics are fast. For example, moving a robotic arm to track a position, or to handle/grip objects, and move them. If a time step is missed, the position may overshoot undesirably, and we may want more predictable behaviour. In this case, we may have a frequency up to 1kHz or more. If an operating system is used, we want a real-time operating system.

Real-time behaviour can be accomplished in embedded applications, by using timers and interrupts (compiled C code on a microcontroller). In this case, we must ensure that the processing load is not too high so that a regular sampling frequency can be maintained.

Robots using computers/microprocessors (because more processing power is required), will need to use a real-time operating system to guarantee high sampling rates.

---

Therefore, whether real-time behaviour is required depends on what the developer intends to use it for.

Answer 4

Our company builds robots using FreeRTOS running on PIC microcontrollers. For us, the main reasons to use FreeRTOS is the ease of rearranging priorities on tasks, handling multiple communication lines simultaneously, and easy communication between interrupt handlers and main tasks. Microcontrollers are far cheaper than putting a full linux machine into each robot we produce as well.

Answer 5

If you actually need real-time, then you use a real-time operating system. Safety monitoring, data acquisition, and constant sample rate control loops are common subsystems that use real-time scheduling.

The real-time portion of the programming is usually as small as possible, because it's more difficult to debug and less code is easier to check for correctness. Documentation on real-time OS's is usually pretty good (including RTAI/Xenomai).

I've used QNX and RTAI->Xenomai in different real robotics projects. I preferred QNX but Xenomai was just as effective.

Answer 6

Orocos is a mature real time robotics control software framework. I have seen it used to successfully control high speed robotic manipulators with hard real time requirements. It has many of the same framework level components as ROS, communications, configuration, serialization, and component based packaging.

Answer 7

Start thinking about your robot in terms of multiple CPUs and the real time question shifts. If you have an algorithm that needs high speed reliable feedback such as a two wheel balancer or a quad-copter stabilizer or a servo pulse out, real time is extremely important, but the task is also very constrained.

You can offload a control loop like this to a dedicated real time CPU such as the cheap 8 bit AVR or low end 32 bit ARMs found in the Arduino class of devices. There's nothing preventing you from adding many dozens of these small MCUs running dedicated control loops, USB device enumeration even makes this easy.

Now that you have the timing sensitive control loops handled by a dedicated CPU, you can relax the real time needs of the 'brain' of the robot which can run higher end logic using components such as ROS on Linux or Kinect for Windows.

Answer 8

In response to "when/in which case" real-time systems are used:

In my experience, motion control is the main application for real-time systems. For controlling motors a high frequency (100hz, 1000hz and more) and low jitter (time variations) are important. Safety is a big point here. Consider a robot among humans: For example, you want/need to ensure that the robot (arm) stops in a specific time frame/distance.

For other tasks such as path planning, vision processing and reasoning real-time system are not that important and often avoided due to the overhead in development time and hardware costs.

Nowadays, big robots such as the PR2 combine both worlds. In the real-time context of the RT enabled operating system (e.g. Linux + Xenomai) motion control is happening and in the non-real-time part (user land), vision processing and planning are embedded in systems like ROS. Both can run on the same computer.

I'm happy to edit this answer, once the question has been clarified. :-)

Answer 9

Yes, assuming the hardware resources can meet the timing requirements (enough processing power, low enough latency), when the scheduler can not sequence processes and threads appropriately, then one uses a real-time scheduler, usually attached to a kernel specifically optimized for the challenges of this. Hardware drivers can also be optimized for real-time conditions.

Yes, if ones software cannot be guaranteed to do it's job in the required time constraints, then one uses real-time approaches.

Answer 10

One good solution is to do BOTH. A design I use place "hard" real time functions on a small micro controller, tight servo control loops and such. Then there is another CPU that is larger, running Linux and ROS. I let ROS handle the higher level tasks and the uP handle things like controlling a stepper motor or running a PID loop.

As said above by others you CAN allow a non-real-time OS to run 1KHz control loops but in order for this to work you need a gross over-kill sized computer that spends most of it's time in an idle loop. If you run the Linux/ROS computer at near 100% CPU utilization the real-time performance is poor. Using a two tier design allows you to always get very good RT performance and also use a smaller, less power hungry computer (possibly a Pi2 higher level tasks.) My uP currently does not run any OS but I'm moving to FreeRTOS