Play audio source from camera? (Capture Card)

[BrianL]

Hello, I am currently creating an application that display camera view from a camera source, I have got that part working with QCamera and QCameraViewfinder. I would also like to play the audio from the camera device too (it is a game capture card Avermedia LGP Lite), however I do not seem to find any mention about audio/sounds in the QCamera library, I tried to check out QAudioDeviceInfo, but that only recognize microphones and speakers.

Is there a way to get the audio from cameras and play it (probably with QAudioOuput)?

P.S. when I start the camera in my application, it shows few warnings in the application output, but the camera still works just fine.

Failed to get the video control
failed to find the video proc amp
	 avc

However when I try to launch it with debugger it gives me a seg fault, though it does not crash or show those warnings when I use my built-in laptop webcam. Here is how a launch the camera:

for (QCameraInfo const& info : QCameraInfo::availableCameras())
{
    if (ui->CB_Cameras->currentText() == info.description())
    {
        m_camera = new QCamera(info, this);
        m_cameraCapture = new QCameraImageCapture(m_camera, this);
        m_camera->setCaptureMode(QCamera::CaptureStillImage);
        m_camera->setViewfinder(m_cameraView);
        m_camera->start();
        break;
    }
}

[Psnarf]

Audio is not a function of the camera, but the microphone in a laptop. What kind of camera do you have? If it has a built-in microphone, then you need to deal with the audio separately. Maybe use QMediaControl?

[BrianL]

The camera is a game capture card Avermedia LGP Lite, in OBS for example when you add the capture card to the scene, its audio is also added, I wanted to do something similar but also playback the audio on PC

[BrianL]

Anyone have ideas on this?

[SGaist]

Hi,

Is it seen as a microphone by the system ?

[BrianL]

@SGaist Do you mean QAudioDeviceInfo? No, it only show the microphone array and stereo mix of my laptop. The capture card audio is not in the list of Recording devices (including Disabled/Disconnected devices). One thing I do notice is that when my application occupies the camera, obviously OBS cannot get video footage, but it also cannot get audio either.

[SGaist]

Are you on Linux ?

[BrianL]

@SGaist on Windows

[SGaist]

QCamera does not do audio therefore on Linux you could have "cheated" with GStreamer but I do not think you can do something similar on Windows.

Maybe switch the capture part over to VLC ?

[BrianL]

@SGaist thanks for the suggestion, upon looking into the libraries, it looks like I can pass QUrl into it, but wouldn't that imply I need to stream the footage on the network first before I can pass to VLC?

[SGaist]

Sorry, I was suggesting to use libVLC and VLC-Qt in your application.

[BrianL]

@SGaist Yep, soo I was able to get VLC-Qt working somewhat in my application, however there are a few warnings a errors I am getting

1. When I initialize the vlc instance:

m_instance = new VlcInstance(VlcCommon::args(), this);

it gives me this warning:

QObject: Cannot create children for a parent that is in a different thread. (Parent is MainWindow(0x73fe00), parent's thread is QThread(0x1e41c7b8), current thread is QThread(0x1fbd3b08)

I suppose this was not the biggest problem since I can just not parent with my mainwindow and delete it manually.

2. I was able to get the video footage, however the audio does not work, I start the media with the following code:

QString url = "dshow://";
    m_media = new VlcMedia(url, m_instance);
    QStringList list;
    list << ":dshow-vdev=LGPLite Stream Engine";
    list << ":dshow-adev=LGPLite Stream Engine";
    list << ":dshow-aspect-ratio=16:9";
    m_media->setOptions(list);
    m_player->open(m_media);

I grabbed the option strings directly from when I was testing getting footage in VLC Player (32-bit), and it was working perfectly fine there, so I was not quite sure why it is not working here, perhaps missing settings/options? The error I am getting is this:

dshow demux error: can't use device: LGPLite Stream Engine, unsupported device type
libvlc: can't use device: LGPLite Stream Engine, unsupported device type
core demux error: Capture failed
libvlc: Capture failed
core demux error: The device you selected cannot be used, because its type is not supported.
libvlc: The device you selected cannot be used, because its type is not supported.
dshow demux error: can't open audio device
libvlc: can't open audio device

EDIT: I tried setting up OBS Virtual Camera with options:

list << ":dshow-vdev=OBS-Camera";
list << ":dshow-adev=OBS-Audio";

I was able to get the audio too, but obviously that way is not very intuitive and it relies on yet another application to get this to work...

3. When I tried to use VlcWidgetVideo (this should allow me to display video on my application, without this I can still get video but it creates a separate window for me), it gave me this error:

QWidget: Must construct a QApplication before a QWidget

I did this is two ways, one is to promote a QWidget to the VlcWidgetVideo with path VLCQtWidgets/WidgetVideo.h (I have already included the path in .pro). The other way is I initialize it dynamically in code:

m_video = new VlcWidgetVideo(this);
ui->verticalLayout->addWidget(m_video);

However both is giving me the same error message and crashed the application :(

[SGaist]

Are you able to have both in VLC itself ?

The error makes it look like you are not doing things in the right order.

Did you forgot QApplication ?

[BrianL]

@SGaist
When I test it in VLC Player I was able to get video and audio just fine.

The VlcWidgetVideo class is initialized in mainwindow.cpp, not in main.cpp, the main.cpp is just the default one with one line added for Qt-VLC:

int main(int argc, char *argv[])
{

    QApplication a(argc, argv);
    VlcCommon::setPluginPath(a.applicationDirPath() + "/plugins");

    MainWindow w;
    w.show();
    return a.exec();
}

I do have the plugins at the exe directory and it wasn't the cause of the problems

[SGaist]

Do you have the same issue if you use the VLC-Qt C++ example ?

[BrianL]

@SGaist Yep, it has the exact same problem

QWidget: Must construct a QApplication before a QWidget

Just a little more info my build is Qt 5.13.1 MinGW 32-bit, and I have the MinGW version of VLC-Qt downloaded, it compiles without any error

[BrianL]

So I tried this demo code instead of VLC-Qt:
https://wiki.videolan.org/LibVLC_SampleCode_Qt/

And first it couldn't get the audio yet again, but then I tried to replace the plugin and .dll from the latest VLC player as I was using the ones from VLC-Qt, turns out those are very outdated and now the audio is working! And this is also able to display within Qt widget by directly using libvlc code instead of custom widget from VLC-Qt.

[SGaist]

Nice !

I haven't had the time to check for the VLC-Qt error but it is quite strange as it usually has to do with some static widget creation when the QApplication object is not missing.