More efficient way to read a video frame-by-frame with Qt and openCV

Question

I am currently using Qt and OpenCV to get a frame-by-frame video from a local file (1920*1280, 30 frames per second, uncompressed)

bool MainWindow::foo()

{

    const std::string name = loadFileName.toStdString();

    cv::VideoCapture cap(name);

    if(!cap.isOpened()) 

        return false;

    cap.set(cv::CAP_PROP_BUFFERSIZE, 3);

    cv::Mat frame;

    while (cap.isOpened())

    {

        CHiResTimer timer; // custom timer class

        timer.Start();

        cap >> frame;

        timer.Stop();

        QTest::qWait(1);

    }

    frame.release();

    cap.release(); 

    return true;

}

But only the cap >> frame line takes 10-12 ms and it is too slow for me because I want to do some processing and show back 30 fps video with minimal delay. I found that pipelines from gstreamer can help with faster reading from a file, but I'm absolutely not familiar with this framework, so I don't know if it's necessary to use it for just one pipeline. Is there any other way to speed up reading (even without OpenCV)?

Answer 1

If you want to have 30 fps output then you have 33 ms of time per frame.

Doing it all (read, process and show) in sequence results with you having 10-12 ms for reading and 21-23 ms for further processing and showing. If that time is not enough then it is unlikely that twice quicker input (5-6 ms so 27-28 ms for processing and showing) saves you. It is only 22% more time for processing and showing.

You may need to use other ways to speed it up. For example if you have 3 separate threads that process frames (and do nothing else) then each has 100 ms for processing a frame.