Handling and Displaying High-Frequency Flat Panel Frames in Qt Quick (Mercu0909F Integration)

Wertyus

Hello,

I am currently working on a user interface using Qt Creator with C++ and QML to display frames from a Mercu0909F flat panel detector. The integration involves connecting to the device and handling various functionalities such as receiving X-Ray images via UDP, using the SDK provided by Mercu0909F.

The detector sends frames at 30 FPS, with a resolution of 1024x1024, in Gray-8 format, resulting in 2 MB per frame. According to the device's documentation, the frames must be processed using a deep copy. While I can successfully save the frames as files on my computer, I am facing challenges when attempting to display these frames in real-time on a QML interface.

Issues I Am Facing:
Deep Copy Crashing the Application:
During the deep copy process, the application crashes with a segmentation fault. Managing the high-speed data stream effectively seems to be a challenge.

No Frames Displayed in QML:
I am unable to display even a single frame on the QML side. Although I am using QQuickImageProvider to send frames to QML, they do not appear in the interface.

My Implementation:
C++ Code:
I use a boost::lockfree::spsc_queue to store frames and signal the QML side for updates. Here is a simplified version of my FrameProvider class:

class FrameProvider : public QObject {
    Q_OBJECT
public:
    explicit FrameProvider(QObject* parent = nullptr) : QObject(parent) {}

    void onNewFrame(unsigned short* pInFrame, int width, int height) {
        if (!frameQueue.push(FrameData{pInFrame, width, height})) {
            qDebug() << "Frame queue is full!";
        } else {
            emit newFrame(QImage((uchar*)pInFrame, width, height, QImage::Format_Grayscale8));
        }
    }

signals:
    void newFrame(const QImage &frame); // Signal to send the frame to QML

private:
    struct FrameData {
        unsigned short* data;
        int width;
        int height;
    };

    boost::lockfree::spsc_queue<FrameData, boost::lockfree::capacity<1024>> frameQueue;
};

QML Code:
I use Connections to listen for new frames in QML and update the displayed image:

Connections {
    target: frameProvider
    onNewFrame: {
        displayedImage.source = frame; // Show the new frame in the image
    }
}

Image {
    id: displayedImage
    width: 1024
    height: 1024
    source: "image://frameProvider"
}

Tried Alternatives:
I used the Boost library (boost::lockfree::spsc_queue) to store frames in a lock-free manner, but I still encountered segmentation faults during the deep copy process.
Saving the frames as files works without any issues, but I cannot render the frames in real time in the Qt interface.

Questions:
Deep Copy Issues:
Is there an alternative to deep copy for processing frames, or how can I optimize this process to avoid segmentation faults?

QML Rendering:
What would be the best approach to ensure frames are reliably displayed in the QML interface?

Boost Alternative:
Can I use a different solution, such as QSharedPointer or any other Qt-based structure, instead of Boost for managing the frame queue?

Handling High-Speed Data Streams:
How can I efficiently handle and render high-frequency (30 FPS, 2 MB) frames in a Qt-based application?

Any advice or suggestions would be greatly appreciated. Thank you in advance for your help! 😊

JoeCFD

Check if Mercu0909F is able to compress the frames into specific formats with gstreamer or FFmpeg encoders and then send them out as RTSP streaming. Or it may be a RTSP streaming with raw frames. Your receiver can handle the frames in a QML sink of gstreamer in real time and no need to store these frames in files. A good GPU can help a lot to reduce latency. Cameras work this way.

SGaist

Hi,

In the code you provided, you are not doing any deep copy hence the crash you are getting. The QImage constructor that you are using explicitly state that you are responsible for the lifetime management of the underlying data.