I'm using ESP32-CAM for video streaming using ESP-IDF framework. I was able to create server on ESP32-CAM and stream video sucessfully. For streaming I'm using HTTP protocol with Content-Type: multipart/x-mixed-replace; boundary=" PART_BOUNDARY "\r\n headers and it's working great :)
Now, I want to add a servo motor to the camera to adjust pan remotely. I think of 2 ways to do it.
Create another endpoint for servo, but for that I need to disconnect video streaming untill servo request is completed. This gives significant lag in video.
send data in between video streaming connection and get minimum lag in video.
I was able to use 2nd option in Arduino IDE for WebServer, there we have an option of reading binary data from client in an ongoing request. Example below
// only showing the relevent code ...
// create server.
WebServer server(80);
// then register different endpoint handlers ...
// ...
// ...
void video_stream_handler(){
// initilize camera stuff, nothing to worry here
camera_fb_t * fb = NULL;
esp_err_t res = ESP_OK;
size_t _jpg_buf_len = 0;
uint8_t * _jpg_buf = NULL;
char buf[32];
sensor_t* sensor_settings = esp_camera_sensor_get();
sensor_settings->set_framesize(sensor_settings, FRAMESIZE_VGA);
sensor_settings->set_quality(sensor_settings, 20);
// get client handle
WiFiClient client = server.client();
// now we can write headers as well as data to client. this works in ESP-IDF as well :)
client.write(<some-headers>, <header-length>);
// now this is interesting.
// we can read from client as well
client.read(); // gives bytes read from client
My question is, is it possible with esp-idf to do something like this? Or if there is any other better alternative than all of it? I want minimum lag in video streaming while still performing servo action in between.
Hardware: ESP32-CAM(single core) with 4MB PSRAM
P.S. I'm using Python socket to read/process the video stream and to send binary data in ongoing connection.