Qt_Camera crash on Jetson Nano

Pranav_nc

Hi Qt Experts,

I am new to Qt framework and I am trying use read frame by frame as QImage using camera capture method for that I have written small code and same code I have attached here. When I run same application code on x86 machine its working fine but I see a warning message(CameraBinn warning: Resource busy or not available) ."

With same code when I tried to run on Jetson Nano platform I see segfault as well as the above warning message.

Please have a look the attached source code and help me to resource this issue.

Below is the source code
main.cpp

#include "mainwindow.h"
#include<iostream>
#include <QApplication>

int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();
return a.exec();

}

mainwindow.cpp

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include<qwindow.h>
#include<qscreen.h>
#include<iostream>
#include<unistd.h>

MainWindow::MainWindow(QWidget *parent)
: QMainWindow(parent)
, stop(false),ui(new Ui::MainWindow)
{
ui->setupUi(this);

m_camera = new QCamera;

m_camera->setCaptureMode(QCamera::CaptureStillImage);

QCameraViewfinder *viewfinder = new QCameraViewfinder;



m_imageCapture = new QCameraImageCapture(m_camera);
m_imageCapture->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);
m_camera->setViewfinder(viewfinder);

QObject::connect(m_imageCapture, &QCameraImageCapture::imageCaptured, [=] (int id, QImage img) {
    std::cout<<"img.width"<<img.width()<<"img.height"<<img.height()<<std::endl;
    processCapturedImage(id,img);
});

m_camera->start();

}

MainWindow::~MainWindow()
{
delete ui;
}
//void MainWindow::displayCapturedImage()
//{
// //ui->stackedWidget->setCurrentIndex(1);
//}

void MainWindow::processCapturedImage(int requestId, const QImage& img)
{
Q_UNUSED(requestId);

auto pe1 = std::chrono::high_resolution_clock::now();
QPixmap pixmap;
QImage qimg = img.convertToFormat(QImage::Format_RGBA8888);
pixmap = QPixmap::fromImage(qimg);
ui->lastImagePreviewLabel_2->setPixmap(pixmap);
ui->lastImagePreviewLabel_2->setScaledContents(true);
ui->lastImagePreviewLabel_2->show();
auto pe2 = std::chrono::high_resolution_clock::now();
auto pe_duration = std::chrono::duration_cast<std::chrono::microseconds>( pe2 -  pe1).count() / 1000;
std::cout<<"latency read "<<pe_duration<<std::endl;

}
void MainWindow::on_pushButton_clicked()
{
while(1){

    if(stop==false){
        m_imageCapture->capture();

    }else{
        stop=false;
        break;
    }
    qApp->processEvents();
}

}

void MainWindow::on_pushButton_2_clicked()
{
stop=true;
}
mainwindow.h

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <QScopedPointer>
#include <QCamera>
#include <QMediaService>
#include <QMediaRecorder>
#include <QCameraViewfinder>
#include <QCameraInfo>
#include <QMediaMetaData>
#include <QCameraImageCapture>
#include <QBuffer>
QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE

class MainWindow : public QMainWindow
{
Q_OBJECT

public:
MainWindow(QWidget *parent = nullptr);
~MainWindow();
QCamera *m_camera;
QCameraImageCapture *m_imageCapture;
private:
bool stop;
Ui::MainWindow *ui;
// QScopedPointer<QCamera> m_camera;
//QScopedPointer<QCameraImageCapture> m_imageCapture;
signals:
void processCapturedImage_signal();
void imageCaptured(int, const QImage);
private slots:
void processCapturedImage(int requestId, const QImage &img);
//void processCapturedImage(const QImage &img);

// void displayCapturedImage();
// void displayViewfinder();
// void on_pushButton_clicked();
void on_pushButton_clicked();
void on_pushButton_2_clicked();
};
#endif // MAINWINDOW_H

SGaist

Hi and welcome to devnet,

Did you check that the camera is not used by another process ?
Did you try to use it with GStreamer directly ?

Your while loop is a bad idea, use Qt's asynchronous nature rather than such loops.

Pranav_nc

Hi SGaist,

Thanks for your reply.

Did you check that the camera is not used by another process ?
Yes, no other process is using camera. Camera is used by this application only.
Did you try to use it with GStreamer directly ?
I will check it and let you know.

Your while loop is a bad idea, use Qt's asynchronous nature rather than such loops.
Ok, I will try to explore about Qt's asynchronous and how to use it here?
Is there any example by which I can understand about Qt's asynchronous usage ? that would be great help full for me.

Thanks in Advance,

Pranav_nc

Hi @SGaist ,

Did you try to use it with GStreamer directly ?
With Gstreamer plugin I have verified and I didn't see any issue and its playing
I have used below command to read and display my laptop camera
$gst-launch-1.0 v4l2src ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Thanks, in Advance

SGaist

Are you sur the camera object you are building is valid ?

You should check QCameraInfo::defaultCamera and also list what is available on your system.

Pranav_nc

@SGaist ,

Checked the available camera device and this application is using it.

I have enabled gstreamer debug message and it may help you to guide me further on this issue.

ory "vaapisink"!
80 0:00:00.141855231 15991 0x14e478d0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element fact ory "vaapisink"!
81 "/dev/video0"
82 mythread
83 0:00:00.560768762 15991 0x1531a850 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:preview-appsrc:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
84 mythread
85 onValueChanged
86 mythread
87 mythread
88 onValueChanged
89 0:00:07.014636435 15991 0x14e478d0 WARN base_camera_src gstbasecamerasrc.c:193:gst_base_camera_src_start_capture:<camera_sourc e> Capturing already ongoing
90 0:00:07.014785448 15991 0x14e478d0 WARN camerabin gstcamerabin2.c:1058:gst_camera_bin_handle_message:<camerabin> Capture failed, reason: Resource busy or not available. - gstbasecamerasrc.c(197): gst_base_camera_src_start_capture (): /GstCameraBin:camerabin/ GstWrapperCameraBinSrc:camera_source
91 CameraBin warning: "Resource busy or not available."
92 mythread

Thanks in Advance,

Pranav_nc

@SGaist ,

My main intention for this example , I just want to capture video frame as a QImage.
Once I have QImage then I want to do some processing on it and display.
I went through the default camera example as I am new to QT GUI framework and not able to understand it completely .

Also as you mentioned asynchronous I have added wait and signal and still I am facing same issue, please have a look at the below code:

mainwindow.cpp:

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include<qwindow.h>
#include<qscreen.h>
#include<iostream>
#include<unistd.h>

MainWindow::MainWindow(QWidget *parent)
: QMainWindow(parent)
, stop(false),ui(new Ui::MainWindow)
{
ui->setupUi(this);
mThread = new MyThread(this);
m_camera = new QCamera;

m_camera->setCaptureMode(QCamera::CaptureStillImage);

QCameraViewfinder *viewfinder = new QCameraViewfinder;

const QList<QCameraInfo> cameras = QCameraInfo::availableCameras();
for (const QCameraInfo &cameraInfo : cameras)
    qDebug() << cameraInfo.deviceName();


m_imageCapture = new QCameraImageCapture(m_camera);
m_imageCapture->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);
m_camera->setViewfinder(viewfinder);
connect(mThread, SIGNAL(valueChanged(int)),this, SLOT(onValueChanged(int)));
QObject::connect(m_imageCapture, &QCameraImageCapture::imageCaptured, [=] (int id, QImage img) {
    std::cout<<"img.width"<<img.width()<<"img.height"<<img.height()<<std::endl;
    mThread->condition.wakeOne();

    processCapturedImage(id,img);

});

m_camera->start();
mThread->start();

}
void MainWindow::onValueChanged(int count){

std::cout<<"onValueChanged" <<std::endl;
m_imageCapture->capture();
mThread->condition.wakeOne();
//    condition.wakeOne();
qApp->processEvents();

}
MainWindow::~MainWindow()
{
delete ui;
}
//void MainWindow::displayCapturedImage()
//{
// //ui->stackedWidget->setCurrentIndex(1);
//}

void MainWindow::processCapturedImage(int requestId, const QImage& img)
{
Q_UNUSED(requestId);
std::cout<<"after processCapturedImage"<<std::endl;
auto pe1 = std::chrono::high_resolution_clock::now();
QPixmap pixmap;
QImage qimg = img.convertToFormat(QImage::Format_RGBA8888);
pixmap = QPixmap::fromImage(qimg);
ui->lastImagePreviewLabel_2->setPixmap(pixmap);
ui->lastImagePreviewLabel_2->setScaledContents(true);
ui->lastImagePreviewLabel_2->show();
auto pe2 = std::chrono::high_resolution_clock::now();
auto pe_duration = std::chrono::duration_caststd::chrono::microseconds( pe2 - pe1).count() / 1000;
std::cout<<"latency read "<<pe_duration<<std::endl;

}
void MainWindow::on_pushButton_clicked()
{

mThread->Stop=false;
mThread->condition.wakeOne();

}

void MainWindow::on_pushButton_2_clicked()
{
mThread->Stop=true;
}

mythread.cpp:

#include "mythread.h"
#include <QDebug>
#include<QMutex>
#include<iostream>
MyThread::MyThread(QObject *parent, bool b) :
QThread(parent), Stop(b)
{
}

// run() will be called when a thread starts
void MyThread::run()
{
for(;;)
{
std::cout<<" mythread "<<std::endl;
mutex.lock();

    condition.wait(&mutex);
    std::cout<<" mythread "<<std::endl;
    if(this->Stop) {
        this->Stop=false;

        mutex.unlock();
        condition.wait(&mutex);
       // break;
    }
    emit valueChanged(0);

    mutex.unlock();

    //condition.wakeOne();
    // slowdown the count change, msec
    this->msleep(30);
}

}

main.cpp:

#include "mainwindow.h"
#include<iostream>
#include <QApplication>

int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();
return a.exec();

}

Pranav_nc

@SGaist ,

Further I have modified code a little bit and make sure the code has implemented in asynchronous , I didn't see the earlier warning message "CameraBin warning: "Resource busy or not available."", but still I am seeing the same segfault issue.

Please help me to resolve thiis issue

Here is the Gstreamer debug messages:

available for media type: application/x-gst-av-webp
77 0:00:00.067371505 19108 0x3da392d0 WARN default descriptions.c:1229:gst_pb_utils_get_codec_description: No description available for media type: application/x-gst-av-wtv
78 0:00:00.067440256 19108 0x3da392d0 WARN default descriptions.c:743:format_info_get_desc: Unexpected MPEG-1 layer in au dio/mpeg, mpegversion=(int)1
79 0:00:00.136766277 19108 0x3da392d0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element fact ory "vaapisink"!
80 0:00:00.142514305 19108 0x3da392d0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element fact ory "vaapisink"!
81 "/dev/video0"
82 mythread
83 0:00:00.528898667 19108 0x3df0cb20 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:preview-appsrc:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
84 mythread
85 mythread
86 onValueChanged
Segmentation fault (core dumped)

Thanks in Advance,

SGaist

Can you run your application through the debugger and show the back trace ?

In any case, your use of synchronisation primitive and threading is just adding complexity for nothing and adds another layer of issues.

Pranav_nc

Hi @SGaist ,

Somehow I was failed to run remote machine using qt-creator and I didn't get the back trace.
When I run application using gdb on target platform I got only above call stack which I mentioned below as well.

backtrace:

Thread 10 "queue5:src" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fa8cce130 (LWP 14841)]
0x0000007fb01f8cbc in gst_memory_unmap () from /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
(gdb) bt
#0 0x0000007fb01f8cbc in gst_memory_unmap () at /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#1 0x0000007fa98df0b8 in () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so
#2 0x0000000000a15550 in ()
(gdb)

Also I have notice another issue, when I configure the QCameraImageCapture as a CaptureToBuffer, why this capture evening saving the image(.jpg) file on to disk location(Pictures folder ).

To check further I put some prints and its showing expected output .

qDebug() << m_imageCapture->isCaptureDestinationSupported(QCameraImageCapture::CaptureToFile);
qDebug() << m_imageCapture->isCaptureDestinationSupported(QCameraImageCapture::CaptureToBuffer);
m_imageCapture->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);
qDebug() << m_imageCapture->captureDestination();

true
true
QFlags(0x2)

Kindly help me why jpg files are saving and how to stop it?

Thanks, in advance,

SGaist

Can you check the example shown here ?

Pranav_nc

Hi @SGaist ,

Thanks for you quick reply on this issue .

Here is my observations please kindly check.

Suggested example is working fine on x86 machine, but it comes to hardware platform I still see a segfault and first frame itself the application crash.

Thread 11 "queue5:src" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fa8d74130 (LWP 16081)]
0x0000007fb0170cbc in gst_memory_unmap () from /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
(gdb) bt
#0 0x0000007fb0170cbc in gst_memory_unmap () at /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#1 0x0000007fa98dc0b8 in () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so
#2 0x0000000000a13910 in ()

Example:

mainwindow.cpp:

#include "ui_mainwindow.h"
#include "mainwindow.h"
MainWindow::MainWindow(QWidget *parent)
: QMainWindow(parent)
, ui(new Ui::MainWindow)
{
ui->setupUi(this);

QCamera *cam = new QCamera;

cam->setCaptureMode(QCamera::CaptureStillImage);

QCameraViewfinder *viewfinder = new QCameraViewfinder;

viewfinder->show();

QCameraImageCapture *cap = new QCameraImageCapture(cam);
cap->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);

cam->setViewfinder(viewfinder);

QObject::connect(cap, &QCameraImageCapture::imageCaptured, [=] (int id, QImage img) {
    QByteArray buf;
    QBuffer buffer(&buf);
    buffer.open(QIODevice::WriteOnly);
    img.save(&buffer, "PNG");
});

QObject::connect(cap, &QCameraImageCapture::readyForCaptureChanged, [=] (bool state) {
   if(state == true) {
       cam->searchAndLock();
       cap->capture();
       cam->unlock();
   }
});

cam->start();

}

MainWindow::~MainWindow()
{
delete ui;
}

main.cpp:

#include "mainwindow.h"

#include <QApplication>

int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
//w.show();
return a.exec();
}

SGaist

Which Linux distribution are your running on your target ?

Pranav_nc

Hi @SGaist ,

I am not sure how to get that details, but from dmesg command I could see below message

Here is my Linux details ,

Linux version 4.9.140-tegra (buildbrain@mobile-u64-2713) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #1 SMP PREEMPT Mon Dec 9 22:47:42 PST 2019

Thanks in Advance,

SGaist

What does "uname -a" return ?

Pranav_nc

@SGaist said in Qt_Camera crash on Jetson Nano:

uname -a
Linux nanoboard-1 4.9.140-tegra #1 SMP PREEMPT Mon Dec 9 22:47:42 PST 2019 aarch64 aarch64 aarch64 GNU/Linux

Pranav_nc

@SGaist ,

One more quick update from end is, now I am able to run the above example without crash. But again I stuck with same core problem where I want to read continuous frames from camera and do process on it.

The crash is happening due to below issue:
https://forums.developer.nvidia.com/t/mjpeg-patch-makes-jpegdec-segfault/119810

The example which you pointed here, with that also image file has creating every capture call

Thanks in Advance,

Pranav_nc

Hi @SGaist ,

As per my understanding right, reading frame from camera is very basic operation any one can expect from QT camera API.

From one week I am working on this and I feel so difficult for reading frame using QCamera API.
I have been working with OpenCV its so straightforward
Looks like am I missing something ? Please kindly help on this.

Thanks in Advance,

SGaist

Qt uses GStreamer on Linux. If GStreamer has an issue, Qt can't do miracles.

Pranav_nc

@SGaist ,

Thanks for the information, is there any API by which I can access the Qimage other than capture API?
Because capture is doing both the things in my case, file saving and filling same data into Qimage object.
As we know right performance and memory consumption will affect due to file writes.

Thanks in advance,