ESP32-CAM-Based Ball Detector

Hi folks! Welcome back. OpenCV is fun, isn’t it? It’s a very effective Python library to add ‘eyes’ to your project. Be it human recognition or object detection, OpenCV is by your side to ensure precise detection and tracking. Today, we are back with another interesting project using ESP32-CAM and OpenCV. It’s a ball detector. The ESP32-CAM will capture images and host an HTTP server. It will let us access individual JPEG images over a local network. For the ball following task, we will use a Python script. The Python application is based on OpenCV. It retrieves image frames from the ESP32-CAM. It looks for a ball in the camera. When it finds one, it marks that. Numpy plays a crucial supportive role in this project. It enables OpenCV to decode the image.

The applications of this ball detector can be

• Pick and place robot.
• Autonomous robots can use this system to follow objects, like a colored ball or a person wearing a specific color.
• You can track colored markers on a user's hand, enabling gesture control or interaction with digital interfaces (e.g., a "virtual mouse" or air drawing).
• Can be adapted for touchless control panels, especially in healthcare or cleanroom environments.

List of components

Circuit diagram

The schematic diagram for this project is shown below.

Fig: Circuit diagram

If you have never used an ESP32 board before, please perform the board installation first. Download and install the ESP32-CAM library. To make the camera functional, the CP210x USB driver and the FTDI driver must be properly installed on your computer. Here is a detailed tutorial that shows how to get started with the ESP32-CAM.

Once you are done with the code uploading, disconnect the IO0 pin of the camera from GND. Then press the RST pin. The following messages will appear.

ig: Code successfully uploaded to ESP32-CAM

You need to copy the IP address and insert it into the corresponding section of your Python script.

 Setting Up the Python Environment

To process the video stream from the ESP32-CAM and perform ball detection using computer vision, we need to set up a Python environment with the necessary libraries. This setup ensures that your system can capture images, process them in real-time, and display the tracking output.

 1. Install Python

Ensure you have Python 3.7 or later installed.

• #include <WebServer.h>: Helps the creation of a lightweight HTTP server on the ESP32.
• #include <WiFi.h>: WiFi.h lets  ESP32 connect to Wi-Fi networks.
• #include <esp32cam.h>: Contains functions for managing the ESP32-CAM module, including camera setup and image capture.
• esp32cam::capture: Captures a frame from the camera.
• Failure Handling: If the frame capture fails, an error is logged and a 503 Service Unavailable response is sent.
• Success Logging: Displays the resolution and file size of the successfully captured image.
• Serving the Image:
  • Specifies the content length and MIME type as image/jpeg, then transmits the image data to the client.
• handleJpgHi: Changes the camera resolution to high using esp32cam::Camera.changeResolution(hiRes) and then invokes serveJpg.
  Error Logging: If the resolution change is unsuccessful, an error message is printed to the Serial Monitor.

  Serial Initialization:

• Initializes the serial port for debugging.
• Sets baud rate to 115200.

  Camera Configuration:

• Defines the pin configuration for the AI Thinker ESP32-CAM module.
• Sets the default resolution, number of frame buffers, and JPEG quality to 80%.
• Tries to initialize the camera and logs whether the initialization was successful or not.

  Wi-Fi Setup:

• Connects to the Wi-Fi network in station mode.
• Waits until the connection is established and then logs the device’s IP address.

  Web Server Routes:

• Maps URL endpoint ( /cam-hi.jpg).
•   Server Start:
• Starts the web server.
• server.handleClient(): Continuously monitors for incoming HTTP requests and responds according to the configured endpoints.

Summary of Workflow

1. The ESP32-CAM connects to Wi-Fi and starts a web server.
2. The URL endpoint /cam-hi.jpg allows the user to request high-resolution images.
3. The camera captures the image and sends it to the client in JPEG format.
4. The system continuously handles new client requests.

Testing

1. Power up the ESP32-CAM and check if it is successfully connected to Wi-Fi.
2. Run the Python script. Correctly set the ESP32-CAM URL.
3. Show a ball in the ESP32-CAM.
4. The system detects the ball and marks it. 

                 Fig: Ball detected