Could you implement a simple neural network on a microprocessor such as the Arduino Uno to be used in machine learning?
It's certainly possible to implement this on an Arduino. Here are 3 such Arduino libraries that implement neural networks:
The complexity of the network that the Arduino can handle is a separate question, especially when it comes to training -- tens of thousands of iterations on training data. Training on a fast machine and then copying the neuron weights to the Arduino will be a smarter way to develop your implementation.
Yes. If you only run it in feed-forward mode and do your training off-line somewhere else:
I programmed a 3-layer (5-5-2) feedforward ANN on an Arduino UNO. It ran on a mobile robot. Whenever the robot would hit something, it would re-train the network. The feedforward portion of the net ran in real-time; while the back-propagation training took on the order of ~5 to 20 seconds. I suppose you could trim the size of the network as well as the play with the parameters to make it run a bit faster, but if you plan on doing backpropagation on an Arduino, I think it would be too slow.
Some thoughts to speed things up include:
- Use fixed vs floating point (for MCU's w/o an FPU)
- Use an MCU that has a FPU
- Use a simpler activation function (ie. $\tanh$) instead of Sigmoid
- Have the training phase occur offline on a PC
Here's a quick write-up I did of the network.
Yes indeed, it's possible to embed neural network in microcontrollers. There are many such examples of this in the scientific literature but I can cite a striking example of what can be done with a very simple MCU if you're smart enough. In Evolutionary Bits'n'Spikes, the authors describe the implementation of a real time spiking neural network AND a genetic algorithm to train it, in order to control a differential wheel robot. The whole code runs in a tiny PIC16F628 4MHz MCU embedded on the 1-cubic-inch Alice robot.