# How to detect writing on a chalkboard? [closed]

I've got an idea for a simple robot (with an Arduino or something similar) that will play noughts-and-crosses (aka tic-tac-toe) on a chalkboard, so that kids can play against it.

It will use a couple of servos to move an articulated arm holding a piece of chalk which will draw the O's and X's. Then the opponent will draw an O or X and press a button to tell the robot to make the next move.

The tricky bit is: how will the robot know where the opponent made their mark? I could use a camera and some sort of motion detection software, but that sounds complex.

I was wondering if there's a way to detect the touch of the chalk on the board, perhaps using something like a MTCH101 chip -- one per square on the board. The board will have a wooden wipe-clean surface, but chips could be embedded just under the surface.

Does anyone have any suggestions?

Edit 4 Jan 2018: Perhaps a force sensitive resistor would work -- such as http://www.trossenrobotics.com/store/p/6496-1-5-Inch-Force-Sensing-Resistor-FSR.aspx

• any limits on creativity? you can use a few microphones at different locations and then teach (machine learning) your program to sense where the opponent touched. Feb 1, 2018 at 18:50
• That sounds like a massive overkill with too little accuracy... Feb 1, 2018 at 19:36
• @GürkanÇetin, that is a very good idea. you could acoustically separate each square, so that marking a square would not be sensed on adjacent squares Feb 1, 2018 at 20:21
• How do you want to acoustically separate areas on a flat surface? Feb 2, 2018 at 7:47
• Welcome to Robotics Chris Dennis, but I'm afraid that Unbounded Design Questions are off-topic because there are many ways to solve any given design problem. We prefer practical, answerable questions based on actual problems that you face, so questions which ask for a list of approaches or a subjective recommendation on a method (for how to build something, how to accomplish something, what something is capable of, etc.) are off-topic. Please take a look at How to Ask & tour for more information on how stack exchange works.
– Chuck
Feb 5, 2018 at 15:32

Since you are going to have a mechanism that can draw within a square why not put a light sensor on it and use the same mechanism scan for the opponent's move?

1. At the start of the game, scan all the squares and record the brightness value for each square. If the board is clear, all the values should be low.
2. After your opponent is done and before each move, scan the "open" squares to confirm that a move has been made and where it was made (that cell should be brighter then the original scan).
3. Record the square where the opponent made a move as no longer "open" then make your move and record that square as no longer "open".
4. Back to 2. until someone looses.
5. Do something fun with the mechanism to show who won. This can add "personality" to the game.

This Color Sensor from adafruit works well and uses I2C so it is very easy to use with Arduino. It has a white LED light you can turn on to improve reflective readings.

Adafruit also has a lot of other options if this one doesn't work for you. There are also sensors like the TCS3200 that have more lights; but, some don't support I2C and may be be more difficult to use.

Note: Determining whether your opponent drew an 'X' or an 'O' is really not necessary. Just say 'X' always goes first. If you are first assume he drew an 'O' and if he is first, assume he drew an 'X'.

• That might work -- thanks. It would be a bit sensitive to changes in ambient lighting, but I suppose it could check each square each time and consider the relative brightnesses. As you suggest, I wasn't going to try to distinguish an O from an X. Feb 2, 2018 at 16:20
• Adding a light to the sensor will also help reduce the impact of ambient light changes. Feb 2, 2018 at 21:00

You can set a fixed camera points towards the chalkboard to check the content in every cell. This can be done by simply crop the image taken by the camera.

Then, by setting a color (brightness) threshold, you should be able to isolate the hand-written content from the background. In this way you can differentiate cells with marks from the vacant ones. Note that this threshold can be dynamic: you can get the color (brightness) of the background by check the part of chalkboard that is not being used.

A Handwritten Digit Recognition system (e.g. here) should be able to recognize the content of the processed image (which only contains pure white and black) of each cell. A "O" will be recognized as number 0. "X" will be recognized as other values.

• This would be really cool; but it may not fit well if your need to be mobile. * Portability and connectivity will likely be a challenge. * The camera has to setup where it can see the board and the mechanism has to setup where it can draw on the board. You would then need to connect them which with kids opens up the risk of the cables getting tripped over. * This approach would probably work best in a fixed setup environment. Feb 3, 2018 at 1:38
• It would definitely have the advantage of versatility. You could likely play different games with the same setup. Feb 3, 2018 at 1:40
• Thank you for those ideas, but it all seems to be getting a bit complex for what I had in mind. Feb 3, 2018 at 12:15
• Why not put the camera directly onto the arm? Feb 5, 2018 at 12:41
• @Malcolm I don’t know what kind of robot arm is used, so I’m not sure if this is easily applicable. But it is indeed a good idea. Feb 5, 2018 at 16:07

Ok I'm going to pitch another answer with a different take :).

Why not have the user tell the robot where they wrote their X or O?

Make an arm with the servo motors you have as you have already planned to do. For the player to make its move, make it grab the end effector of your arm holding the chalk and write down the X or O. Now, the robot can memorise the movement it has done through its encoders, while writing the user's answer on the board, with the user. Thus it can figure out where the user drew their symbol. Then, just memorise where the robots own answers are and you can play tic-tac-toe.

I believe it would be quite fun to grab the robot arm to directly write on the board and then see the robot answer by itself :).

For added fun, make it understand when the user is erasing the board by doing it with the user. For even more added fun, make it understand when it is drawing the double cross to play in (I don't the name of this) so you don't avec to play at a fixed location on the board.

• That's a good idea -- assuming that the servos can work in reverse like that. Would it require special servos? Feb 6, 2018 at 22:36
• As far as I remember no you don't need special servos for that if the servo are free. However, having compliant servos, which would be nicer, is not easy :) Feb 6, 2018 at 23:10
• What do you mean by "free" and "compliant"? Feb 7, 2018 at 8:54
• Yeah free wasn't a very good term. Free as in no input current, the servo isn't locked in a position and can be moved manually without forcing. Compliant: the servo still has a position but move according to the pressure given by the user. A free motor will always feel down where gravity takes it. A compliant won't move unless you push it and then will stay there. Checkout haptic feedback for example Feb 7, 2018 at 8:55
• Thanks -- I'll look for suitable servos and control circuits. Feb 7, 2018 at 10:49