0
$\begingroup$

I used a raspberry pi 3 model B and a camera to make a robot detect fire, using https://www.youtube.com/watch?v=-ZyFYniGUsw However I am not sure how to make this robot move after it detects fire. The L298N motor driver (DC motors are connected to this motor driver) and a logitech camera of this robot are connected to the raspberry pi of this robot. I know that it maybe better to use fire sensors to detect fire, however for my project I decided to use a camera.

My last lines of code in the raspberry pi terminal are (these are same as the lines of code at about 10:41 in the youtube link above except instead of using android.tflite, fire.tflite is used):

  • source tflite/bin/activate
  • cd examples/lite/examples/object_detection/raspberry_pi/
  • cp ~/Downloads/fire.tflite .
  • python detect.py --model fire.tflite

So is there a way to make the robot move using raspberry pi each time it detects fire?

Update: The following is a more detailed description of the problem (Note if non of the links below work remove the full stop at the end of the link and try again):

Hardware setup

The pin configuration of the raspberry pi (RPi) 3 model B is at https://www.jameco.com/Jameco/workshop/circuitnotes/raspberry-pi-circuit-note.html The IN3, IN4 and ENB pins of the L298N motor driver is connected to GPIO 17, GPIO 27 and GPIO 22 respectively of the RPi model B. The OUT 3 and OUT 4 pins of the L298N motor driver is connected to a DC motor. A male to female jumper wire is connected to the ground pin of the L298N motor driver and to the ground at the GPIO header 6 at https://www.jameco.com/Jameco/workshop/circuitnotes/raspberry-pi-circuit-note.html. A male to male jumper wire is also connected to the ground pin of the L298N motor driver and to the ground of a 11.1V battery. A male to male jumper wire is connected to the +12V pin of the L298N motor driver and to the other pin of the 11.1V battery. The display connector of the RPi is connected to a mini display.

Software setup

I took about 70 images of fire to be used as training data. Then I took about 10 images of fire to used as validation data. Then I annotated the fire in each of the images of the training data and validation data using the LabelImg (or labelimg) software. Then I went to the following google colab (GC) link on raspberry pi: https://colab.research.google.com/github/khanhlvg/tflite_raspberry_pi/blob/main/object_detection/Train_custom_model_tutorial.ipynb This link was in the video description at https://www.youtube.com/watch?v=-ZyFYniGUsw. In the GC link above I logged in, then selected “runtime”, then selected “change runtime type”, then selected “GPU” as the “Hardware accelerator” and then selected “Save”. Then I uploaded my previously annotated images of training data and validation data in a zip folder called “fire_detectororigpics5.zip” to GC by using the upload option on the left side of GC. Within this zip folder there is a folder named “fire_detectororigpics5” then within this “fire_detectororigpics5” folder there are two folders one folder named “train” which contains the training data and another folder named “validate” which contains the validation data. Then I selected connect in GC which is that same as about 5:22 in the youtube link (YL) above. Then in GC under “Preparation” I selected the play button which is the same as about 5:30 it the YL above. Then in GC under “Import the required packages” I selected the play button which is the same as about 5:38 it the YL above. Next under the “We start with downloading the dataset” in GC I remove the following code:

!wget https://storage.googleapis.com/download.tensorflow.org/data/android_figurine.zip !unzip -q android_figurine.zip

Which is shown at about 5:56 in the YL above and replaced this code with !unzip -q fire_detectororigpics5.zip Then I selected the play button to the left of this !unzip -q fire_detectororigpics5.zip code

Then under the “Train the object detection model” at GC I replace train_data = object_detector.DataLoader.from_pascal_voc( 'android_figurine/train', 'android_figurine/train', ['android', 'pig_android'] )

val_data = object_detector.DataLoader.from_pascal_voc( 'android_figurine/validate', 'android_figurine/validate', ['android', 'pig_android'] )

as shown at about 6:11 in the YL above, with train_data = object_detector.DataLoader.from_pascal_voc( 'fire_detectororigpics5/train', 'fire_detectororigpics5/train', ['fire'] )

val_data = object_detector.DataLoader.from_pascal_voc( 'fire_detectororigpics5/validate', 'fire_detectororigpics5/validate', ['fire'] )

Then I selected the play button to the left of this

train_data = object_detector.DataLoader.from_pascal_voc( 'fire_detectororigpics5/train', 'fire_detectororigpics5/train', ['fire'] )

val_data = object_detector.DataLoader.from_pascal_voc( 'fire_detectororigpics5/validate', 'fire_detectororigpics5/validate', ['fire'] )

code. Then under the “Step 2: Select a model architecture” in GC, I pressed the play button to the left of spec = model_spec.get('efficientdet_lite0') code which is the same as about 6:35 in the YL above. Next under “Step 3: Train the TensorFlow model with the training data” in GC, I pressed the play button to the left of the following code model = object_detector.create(train_data, model_spec=spec, batch_size=4, train_whole_model=True, epochs=20, validation_data=val_data) which is the same as about 6:40 in the YL above. After this, under “Step 4” in GC, I selected the play button to the left of the following code model.evaluate(val_data) which is the same as about 7:23 in the YL above. Next under “Step 5” in GC, I replaced the code that is model.export(export_dir='.', tflite_filename='android.tflite') which is the same code as shown about 8:04 in the YL above, with model.export(export_dir='.', tflite_filename='fire.tflite')

Then I selected the play button to the left of this model.export(export_dir='.', tflite_filename='fire.tflite'). Then under “Step 6” in GC, I replaced model.evaluate_tflite{‘android.tflite’,val_data} which is the same as about 8:15 in the YL above, with model.evaluate_tflite{‘fire.tflite’,val_data} then I selected the play button to the left of model.evaluate_tflite{‘fire.tflite’,val_data}

Next I selected the “Files” tab on the left of GC, then right clicked the document named “fire.tflite”, then selected “Download” Then I opened the raspberry pi terminal and entered the following code:

• source tflite/bin/activate

• cd examples/lite/examples/object_detection/raspberry_pi/

• cp ~/Downloads/fire.tflite .

• python detect.py --model fire.tflite

The preceding four lines of code opens the Logitech webcam connected to the raspberry pi and detects fire whenever fire is put in front of this camera

So is there a way to make the robot move using raspberry pi each time it detects fire?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

Your question is quite vague as we do not know what your setup is. But in general, if you want to make your robot move you will need a controller to decide where and how to move your robot. This controller will receive some input from the camera (giving for example the location of the fire) and decide on an action (e.g., move towards the fire). Then you will have a lower level controller that will send a command to your driver to input x volts to motor1 and y volts to motor2.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Please use the update above that describes the hardware and software setup in detail. Please use this to suggest how to solve this problem $\endgroup$
    – Viv
    Mar 21 at 10:05
  • $\begingroup$ If you are able to detect fire, all you have to do is send this info to a controller script that will decide whether to move or not. For this, as @chuck suggested below, I also suggest using ROS to make scripts communicate. One script will detect the fire, and send a signal that it detected fire. Another script will receive this message and decide whether and where to move based on this info, and send a message to another script telling it where to move. This last script will control the position (for example) of your robot and enable IN3, IN4 or ENB depending on how you need to move. $\endgroup$
    – JoElPoncho
    Mar 21 at 12:47
0
$\begingroup$

Right now you're running detect.py. You need to have a script that calls that instead and manages the communication between detect.py and a hypothetical move.py. Then, instead of

python detect.py --model fire.tflite

you would call

python detect_and_move.py --model fire.tflite

Then your detect_and_move.py script forwards the model value to the detect script, gets a callback from the move script, and passes that callback to the detect script.

Then, in some hypothetical OnFireDetected method in your detect script, you would call the Move() callback, which invokes the move method in the move.py script.

Improve this answer
$\endgroup$
3
  • $\begingroup$ If you don't want to write your own communication framework then I'd highly recommend checking out ROS. You could write your detect script as a ROS publisher, have move subscribe to that topic, or have move setup as a ROS service and have detect make service calls. For what you're doing you could probably manage everything from one parent script, as I've described, but that approach won't scale. $\endgroup$
    – Chuck
    Mar 17 at 15:08
  • $\begingroup$ Thank you for your answer. I only recently started using raspberry pi and python. So is there a way to learn how to create a python script from scratch that makes the robot detect fire? $\endgroup$
    – Viv
    Mar 21 at 6:36
  • $\begingroup$ @Viv - you mentioned having a detect.py script, so just study what that script is doing. A lot of people get into programming by taking example code and making edits until it does what they want. Eventually you'll get to a point where you realize what you're doing is really clunky, or you're writing the same code over and over, then you try to find out what you can do to eliminate that repetition and you learn about loops and functions, then you learn about classes, then design patterns, and then suddenly it's 15 years later and you're a senior programmer lol. $\endgroup$
    – Chuck
    Mar 21 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.