# When should FPGAs be used in Robotics?

FPGA has good points such as a lot of IO points but then again you need to think things on very low level with flip-flops and pioneer on areas where things are not yet mature -- for example see this question here about development-tools on FPGAs -- this is my understanding currently! Now FPGA has been used to create excellent dexterity in robotic hands like here. Now some people market FPGA for fast prototyping and "forward looking" designs like here, I don't fully understand them: if you don't need a lot of IO points for things such as sensors, why to choose FPGA for a robot? So

When should FPGA be chosen for a project in robotics?

• If you need parallelism then there is a sense to use FPGA as you could achieve that. Also, you can tailor your own "chip design" and "easily" upgrade some functionality without new hardware chip. I saw examples on multiple object detection from camera in realtime. For sure it is overhead for smaller projects. – Gossamer Apr 3 '13 at 5:05

I'm not sure what in the linked question about development tools makes you think that they are not mature. It is true that most of them are proprietary and not open source. But I thought that the tools were quite mature even back when that question was asked three years ago. Today Xilinx, Altera, and Lattice all have freely available development environments that run on Windows and Linux with 32 and 64-bit support. If you insist on open source Icarus Verilog can do quite a bit. But I'd personally stick with the vendor specific tools.

FPGA's are more than just a bunch of configurable flip flops. Consider the Altera NIOS II. A 32-bit soft-core processor capable of running Linux. Mix and match communications and other peripherals from OpenCores or from Xilinx, Altera, or Lattice and you've built a custom microcontroller with everything you need and nothing you don't. You can even write C code for the NIOS II processor and have certain functions implemented directly in the FPGA hardware if you need the speed. Is that high level enough thinking for you?

FPGA's have a high initial investment cost. But it's cheaper and easier to add functionality later in the design process. That's what's meant by "forward looking" designs. Consider a project where you want to add some additional motors to your robot but you've run out of PWM generators on your microcontroller. What do you do? Add another microcontroller? Buy separate PWM chips and connect them to a communications interface? What if all your communications interfaces are in use? With an FPGA, this is reduced almost entirely to a copy and paste operation. It's a lot easier, cheaper, and quicker to scale a design to include new functionality without buying as much additional hardware with an FPGA.

But probably the biggest reason to use an FPGA is speed. I'm not talking about pure clock speed. But when you need a lot of things to happen simultaneously. As Gossamer mentioned, parallelism is what FPGA's are really good at. Filter designs especially benefit from this kind of parallel architecture. If you need to filter and respond to a rapidly changing sensor, then you need an FPGA. They are also quite good at video processing as this too benefits from parallel processing.

Take for instance this high-speed hand robot. It needs to process the video from the high-speed (1000 frames per second) machine vision camera as well as the tactile sensors and respond by manipulating numerous actuators in only a few milliseconds. This type of project is when you need to choose FPGAs.

• Could you clarify this "FPGA's have a high initial investment cost."? Related here: prices look very low. Do you mean high training cost or some other figure? – hhh Apr 3 '13 at 16:27
• @hhh I wasn't talking about raw chip cost. I was talking about development cost. Consider a microcontroller development kit such at the PandaBoard. Contains an ARM Cortext A9 for $181. An FPGA development board that can hold an A9 such as the Cyclone V is$1,099. – embedded.kyle Apr 3 '13 at 17:14
• @hhh There are of course cheaper FPGA development kits. But I was trying to find some metric by which to compare two things that are so different. So a development kit that has (very) roughly the same processing power is about 10x more. – embedded.kyle Apr 3 '13 at 17:16
• @hhh The cheapest ARM on Mouser is $0.80. The cheapest FPGA is$2.80. – embedded.kyle Apr 3 '13 at 17:22
• @hhh That hand is cutting edge research with likely tens if not hundreds of thousands of dollars of custom equipment running it. Video processing in general tends to be on the higher end. See here and here. And neither could probably process 1kfps video. More like 30-60fps. The board you listed doesn't have any video inputs. Though it most certainly could control the actuators and process the tactile sensors. Defiantly could emulate the hand you linked. – embedded.kyle Apr 3 '13 at 18:39

I think there might be a misconception around what an FPGA actually is and when it is used in designs. So let me try to explain that part first.

FPGAs and when one might want to have one...

At risk of simplication, an FPGA is little more than a processor that happens to be 'reconfigurable'.

Why would someone want a 'reconfigurable' microprocessor? Well, because it costs far too much to fabricate a silicon processor to afford to do it for one, two, or even two hundred chips. You'd need to be running batches of thousands of chips to make it economically feasible.

Since no design is ever bug-free the first time round, going the silicon route effectively commits a company to at least two if not more test runs, all at huge fabrication costs.

Which essentially means that very few commercial companies are going to be interested in designing and bringing to fabrication anything other than chip with broad enough commercial appeal.

So, if you come up with a brilliant design that justifies a custom microprocessor, then you might try to bring your idea to life on a shoestring budget by implementing it in an FPGA, where the actual chip is simply a collection of gates, and a 'program' (typically VHDL or Verilog) arranges those gates into an actual microprocessor.

But that's quite a ways down an entirely different (microprocessor design!) road...

Which leads me to suggest that

... FPGAs are not essential to robotics -- at least at the outset

What you need for robotics is a processor. An FPGA is just a special kind of processor (reconfigurable).

You ask whether you 'should'? Well that depends on your knowledge as well as where your real interests lie -- microprocessor design first or robotics first?

If microprocessors are your passion, then by all means! And robotics is a great application area for custom microprocessors -- perhaps a vision cortex with many parallel decision pathways handled directly in hardware, or special tensor multiplication in hardware -- basically anything that generic processors can't handle well off-the-shelf.

But if you're going to microprocessor designs in FPGAs, well, then of course you'll need to have all of that low level knowledge you mention, and more -- because fundamentally you'll be designing your own processor. Which has nothing really to do with robotics, even though your target application might be robotics.

Most designers can probably find an existing processor that will do most of what they'll ever need.

So I think you probably don't need an FPGA.

A starter roadmap: from commercial capabilities to (maybe) an FPGA...

First goal in my opinion is to try to get all of your creative concepts built up into a working robot (quite an intensive challenge in itself).

If you find that you've got specific and significant processing bottlenecks in your design, then the next goal is optimising the selection / design of your processors, still commercially available chips. Maybe bigger, faster (trade-off with power draw, heat dissipation). Maybe smaller, dedicated chips handling specific tasks and interfacing with the main brain (trade-off with algorithm / logic complications).

Only if there's capabilities that you just won't be able to get commercial chips to handle, then you might consider implementing a specialised processor inside an FPGA because at that point there will be very clear advantages that you hope to gain by 'rolling your own', and you can focus on just implementing those capabilities, with the right interfaces to rest of your design.

Otherwise, you're likely to get side-tracked from your main goal (which I'm going to assume is to actually build a robot!)

Bottom line: FPGAs are a distraction from getting started in robotics --- until you're quite a ways down the robotics road -- and have gained some strong chops in digital electronics / microprocessor design somewhere along the way.

• I can think many examples in areas such as video-processing where parallel processing with FPGA is essential for speed and more accurate results. Seen failures where people try to implement noisy iris detection real-time with a single-cpu. No. FPGA has its own specific areas where it rocks such as robotic hands of high dexterity or precise/fast video-processing. I cannot undertand juxtaposing FPGA and microprocessor like that. They are very different beasts with their own good/bad sides. Mistaken? Any idea whether this kind of good areas for FPGA listed somewhere? – hhh Apr 6 '13 at 23:09
• You're certainly correct that there are plenty of areas where FPGAs are an advantage -- I have not excluded those in my answer. The issue is where you are on the experience curve, and how clear you are on the specific advantages you want to gain out of using an FPGA. If you don't know the answer to that, then you probably don't need one! (This echoes Jakob's answer below.) – Assad Ebrahim Apr 7 '13 at 9:43

Having worked with both FPGAs and Microcontrollers in robotics projects, I would actually say now: whatever the person implementing the task has the most experience with. If you know both equally well, you wouldn't be asking yourself the question. If you don't know either well, than these would be the points to go by:

• Is there complex timing on I/O ports required? 3-Phase motor control can have such requirements. FPGAs have a slight advantage here.
• Can you parallelize your super low latency algorithm? Here the FPGA doesn't really compete with microcontrollers, but more with embedded PCs. Most things I would go for the embedded PC, but you may have an advantage with an FPGA for some specific applications. Dense stereo processing for example is often done in FPGAs. Lots of cameras use FPGAs for processing the data stream.

What is also often used are hybrid solutions. Either with two chips, so one microcontroller for the program code, and an FPGA for IO or some other task. There also are a number of FPGAs that actually have a microcontroller embedded.

I wouldn't be so harsh on FPGAs like some other posts, but would also generally argue, that unless you know your VHDL well, you will be better off using microcontroller, or even better still embedded PCs.

• Can you show some example projects that would help to understand what kind of things are good to do with FPGAs and which with microcontrollers? Suppose for example some line-following robot: it works pretty well with a single microcontroller on flat clean surface until more blurry lines that requires image-sharpening/convolutions/etc. If a line in forest (or other noisy surface), it would be far harder challenge (I doubt whether even possible to do with fpga). Have you done any real-world robotic project where you had used fpga for example for image/video-processing realtime? +1 for VHDL. – hhh Apr 6 '13 at 22:53