20
$\begingroup$

Edit: I don't know why, but this question seems to be confusing many people. I am aware of when/where/why/how to use real-time. I am interested in knowing whether people who have a real-time task would actually care enough to implement it in real-time or not.

There's no need to mention why real-time operations are important for a robot. My question is however, how much is it actually used in robotics?

Take this question for example. Only one answer mentions any platform with real-time capabilities, and it is far from the top too. ROS apparently, being a very popular platform which is not real-time.

In the real-time world however, RTAI1 seems to be the only workable free real-time platform of use. It is however, limited to Linux (no problem), badly documented and slowly developed.

So, how much is real-time behavior sought after among robotics developers? The question is, how much are developers inclined to write real-time applications when real-time behavior is actually needed? If not much, why?

For example, reflexive behavior based on tactile data, cannot go through ROS because it would lose its real-time property. But do people really come up with a real-time solution or use ROS anyway, ignoring the real-time property?

1 or similarly Xenomai

$\endgroup$

closed as primarily opinion-based by holmeski, Mark Booth Jun 10 '16 at 17:17

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ I think this is a great question. Consider splitting it into two and clarifying your main question. 'Can ROS be used for real-time?' or 'Is ROS used with real-time?' (2 different questions) are separate from your main question. $\endgroup$ – hauptmech Oct 27 '12 at 9:23
  • $\begingroup$ @hauptmech, well ROS certainly can't be used for real-time, since it's not! $\endgroup$ – Shahbaz Oct 27 '12 at 11:49
  • $\begingroup$ I agree with @hauptmech. The questions is confusing. On the top your are asking, how many people/how often real-time systems are used and later on your asking when/in which case. Both are good questions, so please split it into two or reduce to one. Thanks! $\endgroup$ – bit-pirate Nov 8 '12 at 0:29
  • $\begingroup$ @bit-pirate, I don't understand why you say that I asked when/in which case. I never asked such a thing. Like I said The question is, how much are developers inclined to write real-time applications when real-time behavior is actually needed? In other words, what percentage of applications that do require real-time behavior, are actually implemented in real-time? I personally know when and in which case real-time behavior is needed and have absolutely no question on that matter. In fact, I am surprised to see answers explaining that. $\endgroup$ – Shahbaz Nov 8 '12 at 13:22
  • $\begingroup$ Thanks for the clarification! For me, the title was confusing. IMO real-time programming + implementation is mature in Robotics, but it involves more efforts (time, money, skill etc), so most people avoid it, when it is not really necessary. $\endgroup$ – bit-pirate Nov 9 '12 at 1:02

10 Answers 10

10
$\begingroup$

Remember that there's Real Time and there's Real Time.

Hard Real time is difficult to achieve without hardware support or low level software support, but you often don't need everything to be Hard Real Time capable. Soft & Firm Real Time response is much easier to achieve and is often more than adequate for many applications.

Also, different parts of a system can have very different real time requirements. If you are running software PID loops, they really should have a hard real time response (you really don't want to have to choose between soft tuning your PIDs or tuning them hard and having them occasionally go unstable, for instance). A vision system might have firm real time response requirements, performance will degrade if you can't process the image in time for the next decision but it need not prevent the system running, in this case if you can't process it in time you are better off throwing away the partial results and not loosing time starting analysis on the next frame. Finally your overall planning and coordination (probably the most complex part of your robotic system) can often remain firmly in the domain of soft real time.

Even in the realm of Windows PCs you can get hard real time performance, you just need the right software with the right hooks into the kernel. Beckhoff's TwinCat soft PLC quite happily ran a high scan rate PLC by slicing half of a Pentium's machine cycles, giving the other half to Windows NT, and this was over a decade ago. Even modern control systems like some options in Aerotech's A3200 range do the grunt work on the host PC, with the low level kernel taking as much CPU time as it needs for the hard real time requirements and leaving the rest of the CPU cycles for Windows 7 to use!

$\endgroup$
  • $\begingroup$ The distinction here is quite pertinent... even in "real world" low-level systems, the really real-time bit is quite small (based on a timer tick interrupt) whereas most of the system is nominally real-time (but +/- a few nano-seconds here and there is tolerable). I smile when I see people talking about real-time applications built on WindowsCE or Linux... $\endgroup$ – Andrew Oct 26 '12 at 19:04
  • 1
    $\begingroup$ As I say @Andrew with the right software, even Windows 7 can be made hard real time with an RTX. Not sure why you don't consider Windows CE to be real-time though, it's had real-time deterministic task scheduling since version 2 and Linux can be made realtime with a kernel like RTLinux. $\endgroup$ – Mark Booth Oct 26 '12 at 20:40
  • $\begingroup$ Hi again Mark (not sure who is stalking who here...) - I agree that you can do it, but harsh experience has shown that in many (?most?) cases users (managers) ignore the required add-ons and assume that the vanilla system will do. $\endgroup$ – Andrew Oct 26 '12 at 21:10
  • $\begingroup$ @Andrew - My experience with RTX was that it just worked. Back in the Pentium 4 days you had to be careful not to use integrated graphics or audio which saturated the PCI bus, but that shouldn't be a problem these days. $\endgroup$ – Mark Booth Oct 26 '12 at 21:25
  • $\begingroup$ It's been a long time, but reading back this page, especially with regards to windows, I'd just like to say you are mentioning only part of how a system is made realtime. Realtime scheduling is important of course, but there are all sorts of things that can generate spikes that also need to be handled to make a system realtime. Interrupts are the common example, SMI, caches and memory bandwidth are other examples. Just because there is a realtime scheduler doesn't make a system realtime. $\endgroup$ – Shahbaz May 28 '18 at 17:54
8
$\begingroup$

A real time system is not really required for many (most?) robotic control systems. As long as you have a control loop that runs fast enough, with low enough jitter, and doesn't miss too many cycles, then this is quite adequate for robotic control and servoing.

As evidence of this, let me present the PR2 and the Shadow Robot Hand:

PR2

This robot has about 20 degrees of freedom, all of which are servoed through ROS's main loop. Or how about the Shadow Robot Hand, which also has 20 DOFs, plus an array of tactile and other sensors, and is also servoed through ROS's main loop.

The ROS main loop suffers from a little jitter, sometimes as much as 100us, and even sometimes misses cycles altogether. But it doesn't matter if 99.9% of cycles are executed successfully.

The use of many cores within the host PCs means that one whole core is dedicated to running the main loop, and so it is very rarely delayed by other tasks.

The main reason for using a really real-time OS for a robotic system is for safety. If the robot is working in a safety critical situation, then it is your responsibility to guarantee 100% control up-time, and part of this is guaranteeing the real time nature of it.

Whether you use a real-time OS or not, your servos should do something safe in the event that the control loop dies entirely. This safety system would also be helpful on the rare occasion your non-real-time OS skips more than a cycle. For example, on the Shadow Hand, the motors are halted if the control loop misses more than 20 cycles (20ms). I have never seen this happen though.


Added

Another way to think about it is this: What update rate does your servo system actually need? If it's a largish arm, and doesn't need super high performance, high speed positioning, then 500Hz might be sufficient. For driving around, 200Hz is probably sufficient. In both of these cases, if your loop is actually running at 1000Hz, then a late or missing cycle really is no problem at all, as long as your control algorithm takes into account the actual elapsed time between loops.

$\endgroup$
  • $\begingroup$ So in short, you are saying that real-time is often not used, because non-real-time software works "good enough"? $\endgroup$ – Shahbaz Oct 25 '12 at 13:52
  • $\begingroup$ @Shahbaz - I can't comment on exactly how often it's actually used, but I can say that if it is used, then it may well be unnecessary. We used to use RTAI, then abandoned it because it was actually hindering more than helping. $\endgroup$ – Rocketmagnet Oct 25 '12 at 15:10
  • 1
    $\begingroup$ I would like to stress one point: you have so many processing power on the PR2 that you might get something "good enough". I worked on a robot with "only" a Core2 Duo. That's not an option there: the complete stack is taking each core 100% most of the time. Here, Rock (Orocos) and RT-Linux were necessary to hold the 1kHz control loop together. $\endgroup$ – sylvain.joyeux Oct 25 '12 at 20:34
  • $\begingroup$ @sylvain.joyeux - I agree. ROS performs pretty badly for control when you have only 2 cores. $\endgroup$ – Rocketmagnet Oct 25 '12 at 20:45
  • $\begingroup$ @Rocketmagnet I'm sorry to have to downvote this one, but the PR2 part is wrong. On the PR2 there is a single real-time loop running at 1000Hz parallel to ROS (on Linux + RT PREEMPT), which is communicating via Ethercat with the motor controller boards, doing the actual motor control of each DOF. You have to be careful when programming controllers (e.g. a joint trajectory controller) in order to not break real-time and they also have special tools to manage them (e.g. load/unload them). Look here for more details. $\endgroup$ – bit-pirate Nov 8 '12 at 0:23
4
$\begingroup$

The purpose of the software determines whether it needs to be strictly real-time.

Where the purpose is path planning or localization, often a low frequency is sufficent, for example, 10Hz. In these cases, a player/stage setup running on Linux is fine. We can see that there are few problems if one time step is a bit longer or shorter.

Strictly real-time behaviour is required if the robot dynamics are fast. For example, moving a robotic arm to track a position, or to handle/grip objects, and move them. If a time step is missed, the position may overshoot undesirably, and we may want more predictable behaviour. In this case, we may have a frequency up to 1kHz or more. If an operating system is used, we want a real-time operating system.

Real-time behaviour can be accomplished in embedded applications, by using timers and interrupts (compiled C code on a microcontroller). In this case, we must ensure that the processing load is not too high so that a regular sampling frequency can be maintained.

Robots using computers/microprocessors (because more processing power is required), will need to use a real-time operating system to guarantee high sampling rates.


Therefore, whether real-time behaviour is required depends on what the developer intends to use it for.

$\endgroup$
  • $\begingroup$ Thanks for the reply. Maybe my question needs better wording, but I didn't want to ask "when to use real-time", but "how often do people actually use real-time when real-time is needed". Nevertheless, your real-time behavior on microcontrollers, without the need for a real-time platform was a good point I hadn't considered. $\endgroup$ – Shahbaz Oct 25 '12 at 11:37
  • $\begingroup$ On a side-note, real-time and fast are two orthogonal concepts. If a path planner has to decide strictly within one minute, then it is a real-time application. Although I understand why you'd mention a fast robot. $\endgroup$ – Shahbaz Oct 25 '12 at 11:38
2
$\begingroup$

Our company builds robots using FreeRTOS running on PIC microcontrollers. For us, the main reasons to use FreeRTOS is the ease of rearranging priorities on tasks, handling multiple communication lines simultaneously, and easy communication between interrupt handlers and main tasks. Microcontrollers are far cheaper than putting a full linux machine into each robot we produce as well.

$\endgroup$
2
$\begingroup$

If you actually need real-time, then you use a real-time operating system. Safety monitoring, data acquisition, and constant sample rate control loops are common subsystems that use real-time scheduling.

The real-time portion of the programming is usually as small as possible, because it's more difficult to debug and less code is easier to check for correctness. Documentation on real-time OS's is usually pretty good (including RTAI/Xenomai).

I've used QNX and RTAI->Xenomai in different real robotics projects. I preferred QNX but Xenomai was just as effective.

$\endgroup$
2
$\begingroup$

Orocos is a mature real time robotics control software framework. I have seen it used to successfully control high speed robotic manipulators with hard real time requirements. It has many of the same framework level components as ROS, communications, configuration, serialization, and component based packaging.

$\endgroup$
2
$\begingroup$

Start thinking about your robot in terms of multiple CPUs and the real time question shifts. If you have an algorithm that needs high speed reliable feedback such as a two wheel balancer or a quad-copter stabilizer or a servo pulse out, real time is extremely important, but the task is also very constrained.

You can offload a control loop like this to a dedicated real time CPU such as the cheap 8 bit AVR or low end 32 bit ARMs found in the Arduino class of devices. There's nothing preventing you from adding many dozens of these small MCUs running dedicated control loops, USB device enumeration even makes this easy.

Now that you have the timing sensitive control loops handled by a dedicated CPU, you can relax the real time needs of the 'brain' of the robot which can run higher end logic using components such as ROS on Linux or Kinect for Windows.

$\endgroup$
0
$\begingroup$

In response to "when/in which case" real-time systems are used:

In my experience, motion control is the main application for real-time systems. For controlling motors a high frequency (100hz, 1000hz and more) and low jitter (time variations) are important. Safety is a big point here. Consider a robot among humans: For example, you want/need to ensure that the robot (arm) stops in a specific time frame/distance.

For other tasks such as path planning, vision processing and reasoning real-time system are not that important and often avoided due to the overhead in development time and hardware costs.

Nowadays, big robots such as the PR2 combine both worlds. In the real-time context of the RT enabled operating system (e.g. Linux + Xenomai) motion control is happening and in the non-real-time part (user land), vision processing and planning are embedded in systems like ROS. Both can run on the same computer.

I'm happy to edit this answer, once the question has been clarified. :-)

$\endgroup$
0
$\begingroup$

Yes, assuming the hardware resources can meet the timing requirements (enough processing power, low enough latency), when the scheduler can not sequence processes and threads appropriately, then one uses a real-time scheduler, usually attached to a kernel specifically optimized for the challenges of this. Hardware drivers can also be optimized for real-time conditions.

Yes, if ones software cannot be guaranteed to do it's job in the required time constraints, then one uses real-time approaches.

$\endgroup$
0
$\begingroup$

One good solution is to do BOTH. A design I use place "hard" real time functions on a small micro controller, tight servo control loops and such. Then there is another CPU that is larger, running Linux and ROS. I let ROS handle the higher level tasks and the uP handle things like controlling a stepper motor or running a PID loop.

As said above by others you CAN allow a non-real-time OS to run 1KHz control loops but in order for this to work you need a gross over-kill sized computer that spends most of it's time in an idle loop. If you run the Linux/ROS computer at near 100% CPU utilization the real-time performance is poor. Using a two tier design allows you to always get very good RT performance and also use a smaller, less power hungry computer (possibly a Pi2 higher level tasks.) My uP currently does not run any OS but I'm moving to FreeRTOS

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.