Trying to write a UDP streaming webcam service on C

Question

folks.

I'm not entirely sure how should I send the encoded image over the UDP datagram. Once the UDP service is established, I'm running this loop:

Server side:

FrameCapture = cvCaptureFromCAM(-1);
frameORG = cvQueryFrame(FrameCapture);  
small = cvCreateImage(cvSize(frameORG->width / 2, frameORG->height / 2), frameORG->depth, 3);   

while((cvWaitKey(40) & 0xFF) != ESC_KEY)
{
    frameORG = cvQueryFrame(FrameCapture);

    VideoBuffer = cvEncodeImage(bufUDP, &small, 0);
    memcpy(bufUDP, VideoBuffer, UDPMAX);

    if (sendto(sockUDP, bufUDP, strlen(bufUDP), 0, (struct sockaddr *)&servUDP, length) < 0)
    {
        if (errno == EPIPE)
        {
            printf("Connection lost.);
        }
        else
        {
            perror("Transmission failed.");
        }
        exit(1);
    }
}

Client side:

bufUDP = (char*) malloc(UDPMAX);
VideoBuffer = (CvMat *) malloc(UDPMAX);
cvNamedWindow("UDP Streaming from Server", CV_WINDOW_AUTOSIZE);

while ((cvWaitKey(40) & 0xFF) != ESC_KEY) 
{
    if (recvfrom(sockUDP, bufUDP, strlen(bufUDP), 0, (struct sockaddr *)&servUDP, &length) < 0)
    {
        if (errno == EPIPE)
        {
            printf("Connection lost.\n");
        }
        else
        {
            perror("Problem recieving data.");
        }
        exit(1);
    }
    memcpy(VideoBuffer, bufUDP, UDPMAX);
    img = cvDecodeImage(VideoBuffer,CV_LOAD_IMAGE_COLOR);
    cvShowImage("UDP Video Receiver", &img);
}

The program is compiling without any warnings (using -Wall) but the output of this from the server side is an exception from the OpenCV function.

OpenCV Error: Bad argument (Unknown array type) in cvarrToMat, file /../../../../../../../matrix.cpp, line 698 terminate called after throwing an instance of 'cv::Exception' what(): /../../../../../../../matrix.cpp:698: error: (-5) Unknown array type in function cvarrToMat

I'm not entirely sure how should I handle the timing so I tried sending the packets every 40ms (25fps).

Thanks in advance

Answer 1

I finally got around the answer myself. Hope this helps for anybody who struggles with UDP+OpenCV+CapturefromCAM. The whole use of cvEncodeImage and cvDecodeImage is not really necessary. Since the struct IplImage already holds its data in image->imageData. Hence the following codes:

Server side:

cvNamedWindow("Webcam from Server", CV_WINDOW_AUTOSIZE);
FrameCapture = cvCaptureFromCAM(0); 
frameORG = cvQueryFrame(FrameCapture);  
small = cvCreateImage(cvSize(frameORG->width, frameORG->height), frameORG->depth, 3);

if(!FrameCapture) {
    printf("Could not iniciate webcam\n");
    return;
}

while((cvWaitKey(40) & 0xFF) != ESC_KEY)
{
        small = cvQueryFrame(FrameCapture);                                     
        for(times = 0; times < MAX_PAQS; times++)
        {
            memset(bufUDP, 0, UDPMAX);
            memcpy(bufUDP, small->imageData + UDPMAX*times, UDPMAX);
            if (sendto(sockUDP, bufUDP, UDPMAX, 0, (struct sockaddr *)&servUDP, length) < 0)
            {
                // check errno for the problem or conection lost
            }
        }
    }
    cvShowImage("Webcam from Server", small); // feedback on server webcam
}

Client side:

cvNamedWindow("UDP Streaming from Server", CV_WINDOW_AUTOSIZE);
img = cvCreateImage(cvSize(640, 480), IPL_DEPTH_8U, 3);

while((cvWaitKey(40) & 0xFF) != ESC_KEY)
{
    for(times = 0; times < MAX_PAQS; times++)
    {
        if (recvfrom(sockUDP, bufUDP, UDPMAX, 0, (struct sockaddr *)&servUDP, &length) < 0)
        {
                // check errno for the problem or conection lost
        }
        memcpy(img->imageData + UDPMAX*times, bufUDP, UDPMAX);
    }

    cvShowImage("UDP Streaming from Server", img);
}

Notice that there is no frame sync here (I didn't copy that part). UDPMAX has to hold an amount equal to widthheightchannels/MAX_PAQS. As you can see, I used a 640*480-3 channel frame.