I have a WEBOTS Simulation with an automobile. Now while WEBOTS provides a camera overlay, I want to display the result of image processing along with sensor data in a comprehensive manner.

For this I have written a simple GUI application using tkinter and customtikinter libraries in a separate script inside the same folder as the WEBOTS Controller.

When I try to call this script from the WEBOTS Controller, the app launches perfectly but my WEBOTS Simulation is paused while the app is running.

Is there any way to pass, let's say an array from my WEBOTS Controller to the GUI Script without affecting the Simulation?

Controller Code:

import cv2
import dataviz #<-- I tried to import the GUI Script here. The script launches but pauses the Webots simulation
import numpy as np
from controller import Camera, GPS, Compass
from vehicle import Driver

driver = Driver()
timestep = int(driver.getBasicTimeStep())

camera = driver.getDevice('camera')
gps = driver.getDevice('gps')
compass = driver.getDevice('compass')

def image_processing():
    data = camera.getImage()
    img = np.frombuffer(data, dtype=np.uint8).reshape((camera.getHeight(), camera.getWidth(), 4))
    gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
    # dataviz.called(img) # I also tried to create a function in the GUI script to pass the img array but it had the same results.            
    cv2.imshow("Video Feed", gray)

def get_gps_data():
    gps_xyz = gps.getValues()
    print('Speed', gps.getSpeed())
    print('Speed Vector', gps.getSpeedVector())
# Main control loop
while driver.step() != -1:

GUI Code:

import tkinter as tk
import customtkinter as CTK
from PIL import Image, ImageTk

import numpy as np
import cv2

CTK.set_appearance_mode("System")  # Modes: "System" (standard), "Dark", "Light"
CTK.set_default_color_theme("blue")  # Themes: "blue" (standard), "green", "dark-blue"

class App(CTK.CTk):
    def __init__(self):
        gray = np.ones(1,1) #<-- This is a dummy array. I want the image captured by the camera to be passed here. 
        # gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #This was just to test if the code worked standalone.
        height, width = np.shape(gray)
        im = Image.fromarray(gray)
        ph = CTK.CTkImage(im, size=(width,height))

        # configure window
        self.title("CustomTkinter complex_example.py")

        self.grid_rowconfigure(0, weight=1)
        self.grid_columnconfigure(0, weight=1)

        self.bottombar_frame = CTK.CTkFrame(self, height= 10, corner_radius=0, fg_color="green")
        self.bottombar_frame.grid(row=1, column=0, rowspan=4, sticky="nsew")
        self.bottom_label = CTK.CTkLabel(self.bottombar_frame, text = "Data Visualiser")
        self.bottom_label.grid(row=0, column=0, columnspan=2, padx=5, pady=5, sticky="nsew")

        # create tabview
        self.tabview = CTK.CTkTabview(self)
        self.tabview.grid(row=0, column=0, padx=5, pady=5, sticky="nsew")
        self.tabview.add("Tab 1")
        self.tabview.add("Tab 2")
        self.tabview.add("Tab 3")

        self.label_tab_2 = CTK.CTkLabel(self.tabview.tab("Tab 2"), image=ph, text="")
        self.label_tab_2.grid(row=0, column=0, padx=20, pady=20)

app = App()

#def called(image):  # I thought keeping the GUI loop inside a callable function might solve the problem but it did not. 
    #app = App()
    #return 0

  • 1
    $\begingroup$ Often if the simulation is paused and doesn't progress is because the driver.step() function is not executed continuously. So maybe you should check if your main control loop is executed while your GUI app is running and make sure that the GUI app is executed in parallel to the main Webots controller and doesn't block it. $\endgroup$ Mar 28 at 7:30


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.