How to move the robot using Raspberry Pi?

Question

I used a Raspberry Pi 3 model B and a camera to make a robot detect fire, after watching Train a custom object detection model using your data.

However I am not sure how to make this robot move after it detects fire. The L298N motor driver (DC motors are connected to this motor driver) and a Logitech camera of this robot are connected to the Raspberry Pi of this robot. I know that it maybe better to use fire sensors to detect fire, however for my project I decided to use a camera.

My last lines of code in the Raspberry Pi terminal are (these are same as the lines of code at about 10:41 in the YouTube link above except instead of using android.tflite, fire.tflite is used):

source tflite/bin/activate
cd examples/lite/examples/object_detection/raspberry_pi/
cp ~/Downloads/fire.tflite .
python detect.py --model fire.tflite

So is there a way to make the robot move using raspberry pi each time it detects fire?

---

The following is a more detailed description of the problem (Note if non of the links below work remove the full stop at the end of the link and try again):

Hardware setup

The pin configuration of the Raspberry Pi (RPi) 3 model B is at Raspberry Pi Pinout Diagram | Circuit Notes

The IN3, IN4 and ENB pins of the L298N motor driver is connected to GPIO 17, GPIO 27 and GPIO 22 respectively of the RPi model B. The OUT 3 and OUT 4 pins of the L298N motor driver is connected to a DC motor. A male to female jumper wire is connected to the ground pin of the L298N motor driver and to the ground at the GPIO header 6 at Raspberry Pi Pinout Diagram | Circuit Notes. A male to male jumper wire is also connected to the ground pin of the L298N motor driver and to the ground of a 11.1V battery. A male to male jumper wire is connected to the +12V pin of the L298N motor driver and to the other pin of the 11.1V battery. The display connector of the RPi is connected to a mini display.

Software setup

I took about 70 images of fire to be used as training data. Then I took about 10 images of fire to used as validation data. Then I annotated the fire in each of the images of the training data and validation data using the LabelImg (or labelimg) software. Then I went to the following Google Colab (GC) link on Raspberry Pi: Train a custom object detection model with TensorFlow Lite Model Maker This link was in the video description at Train a custom object detection model using your data.

In the GC link above I logged in, then selected “runtime”, then selected “change runtime type”, then selected “GPU” as the “Hardware accelerator” and then selected “Save”. Then I uploaded my previously annotated images of training data and validation data in a zip folder called “fire_detectororigpics5.zip” to GC by using the upload option on the left side of GC. Within this zip folder there is a folder named “fire_detectororigpics5” then within this “fire_detectororigpics5” folder there are two folders one folder named “train” which contains the training data and another folder named “validate” which contains the validation data. Then I selected connect in GC which is that same as about 5:22 in the youtube link (YL) above. Then in GC under “Preparation” I selected the play button which is the same as about 5:30 it the YL above. Then in GC under “Import the required packages” I selected the play button which is the same as about 5:38 it the YL above. Next under the “We start with downloading the dataset” in GC I remove the following code:

!wget https://storage.googleapis.com/download.tensorflow.org/data/android_figurine.zip
!unzip -q android_figurine.zip

Which is shown at about 5:56 in the YL above and replaced this code with

!unzip -q fire_detectororigpics5.zip

Then I selected the play button to the left of this !unzip -q fire_detectororigpics5.zip code

Then under the “Train the object detection model” at GC I replace

train_data = object_detector.DataLoader.from_pascal_voc(
    'android_figurine/train',
    'android_figurine/train',
    ['android', 'pig_android']
)

val_data = object_detector.DataLoader.from_pascal_voc(
    'android_figurine/validate',
    'android_figurine/validate',
    ['android', 'pig_android']
)

as shown at about 6:11 in the YL above, with

train_data = object_detector.DataLoader.from_pascal_voc(
    'fire_detectororigpics5/train',
    'fire_detectororigpics5/train',
    ['fire']
)

val_data = object_detector.DataLoader.from_pascal_voc(
    'fire_detectororigpics5/validate',
    'fire_detectororigpics5/validate',
    ['fire']
)

Then I selected the play button to the left of this

train_data = object_detector.DataLoader.from_pascal_voc(
    'fire_detectororigpics5/train',
    'fire_detectororigpics5/train',
    ['fire']
)

val_data = object_detector.DataLoader.from_pascal_voc(
    'fire_detectororigpics5/validate',
    'fire_detectororigpics5/validate',
    ['fire']
)

code. Then under the “Step 2: Select a model architecture” in GC, I pressed the play button to the left of spec = model_spec.get('efficientdet_lite0') code which is the same as about 6:35 in the YL above.

Next under “Step 3: Train the TensorFlow model with the training data” in GC, I pressed the play button to the left of the following code:

model = object_detector.create(train_data, model_spec=spec, batch_size=4, train_whole_model=True, epochs=20, validation_data=val_data)

which is the same as about 6:40 in the YL above. After this, under “Step 4” in GC, I selected the play button to the left of the following code

model.evaluate(val_data)

which is the same as about 7:23 in the YL above.

Next under “Step 5” in GC, I replaced the code that is

model.export(export_dir='.', tflite_filename='android.tflite')

which is the same code as shown about 8:04 in the YL above, with

 model.export(export_dir='.', tflite_filename='fire.tflite')

Then I selected the play button to the left of this

model.export(export_dir='.', tflite_filename='fire.tflite')

Then under “Step 6” in GC, I replaced

model.evaluate_tflite{‘android.tflite’,val_data}

which is the same as about 8:15 in the YL above, with

model.evaluate_tflite{‘fire.tflite’,val_data}

then I selected the play button to the left of

model.evaluate_tflite{‘fire.tflite’,val_data}

Next I selected the “Files” tab on the left of GC, then right clicked the document named “fire.tflite”, then selected “Download”.

Then I opened the Raspberry Pi terminal and entered the following code:

source tflite/bin/activate
cd examples/lite/examples/object_detection/raspberry_pi/
cp ~/Downloads/fire.tflite . 
python detect.py --model fire.tflite

The preceding four lines of code opens the Logitech webcam connected to the Raspberry Pi and detects fire whenever fire is put in front of this camera

So is there a way to make the robot move using Raspberry Pi each time it detects fire?

Answer 1

Right now you're running detect.py. You need to have a script that calls that instead and manages the communication between detect.py and a hypothetical move.py. Then, instead of

python detect.py --model fire.tflite

you would call

python detect_and_move.py --model fire.tflite

Then your detect_and_move.py script forwards the model value to the detect script, gets a callback from the move script, and passes that callback to the detect script.

Then, in some hypothetical OnFireDetected method in your detect script, you would call the Move() callback, which invokes the move method in the move.py script.

Answer 2

Your question is quite vague as we do not know what your setup is. But in general, if you want to make your robot move you will need a controller to decide where and how to move your robot. This controller will receive some input from the camera (giving for example the location of the fire) and decide on an action (e.g., move towards the fire). Then you will have a lower level controller that will send a command to your driver to input x volts to motor1 and y volts to motor2.

If you are able to detect fire, all you have to do is send this info to a controller script that will decide whether to move or not. For this, as suggested in @chuck's answer, I also suggest using ROS to make scripts communicate.

One script will detect the fire, and send a signal that it detected fire. Another script will receive this message and decide whether and where to move based on this info, and send a message to another script telling it where to move. This last script will control the position (for example) of your robot and enable IN3, IN4 or ENB depending on how you need to move.