How to use QMediaCaptureSession and QAudioBufferInput

csab6597

I try to set up a media capture session, where I want to provide custom audio samples. So following the docs for https://doc.qt.io/qt-6/qmediacapturesession.html and https://doc.qt.io/qt-6/qaudiobufferinput.html and https://doc.qt.io/qt-6/qmediarecorder.html I have

QAudioFormat format;
format.setSampleFormat(QAudioFormat::Float);
format.setSampleRate(44100);
format.setChannelConfig(QAudioFormat::ChannelConfigStereo);
QMediaCaptureSession mSession = new QMediaCaptureSession();
QAudioBufferInput mAudioInputBuffer = new QAudioBufferInput(format);
mSession->setAudioBufferInput(mAudioInputBuffer);
QMediaRecorder recorder;
mSession->setRecorder(&recorder);
recorder.setOutputLocation(QUrl::fromLocalFile("c:/test"));
recorder.record();

But this never gives me the QAudioBufferInput::readyToSendAudioBuffer() signal and QAudioBufferInput::sendAudioBuffer(const QAudioBuffer &audioBuffer) always returns false, while recorder.recorderState() is QMediaRecorder::RecordingState and recorder.error() is QMediaRecorder::NoError

So how to start the capture process, what is needed to get the readyToSendAudioBuffer signal
??

SGaist

Hi,

Based on your code snippet, your recorder is stack based and likely in a function which means that it's going to be destroyed as soon as the function ends.

csab6597

Well ok I simplified. Here is some fully working code demonstrating the situation. I do not get the readyToSendAudioBuffer signal, while the recorder state change is indeed emitted.

WorkerThread.hpp

#pragma once

#include <QThread>
#include <QMediaCaptureSession>
#include <QAudioBufferInput>
#include <QMediaRecorder>

class WorkerThread : public QThread {
	Q_OBJECT //do not put Q_OBJECT macro into a source file -> linker error

private:
	QAudioFormat mFormat;
	QMediaCaptureSession* mSession;
	QAudioBufferInput* mAudioInputBuffer;
	QMediaRecorder mRecorder;

public:
	WorkerThread();
	void run() override;
};

main.cpp

#include <QCoreApplication>
#include <QThread>
#include <QAudioFormat>
#include <QUrl>

#include "WorkerThread.hpp"

WorkerThread::WorkerThread() {
	mFormat.setSampleFormat(QAudioFormat::Float);
	mFormat.setSampleRate(44100);
	mFormat.setChannelConfig(QAudioFormat::ChannelConfigStereo);
	mSession = new QMediaCaptureSession();

	auto bufferReady = [] { qDebug() << "buffer ready to send"; };
	mAudioInputBuffer = new QAudioBufferInput(mFormat);
	connect(mAudioInputBuffer, &QAudioBufferInput::readyToSendAudioBuffer, this, bufferReady);
	mSession->setAudioBufferInput(mAudioInputBuffer);

	auto recorderState = [] (QMediaRecorder::RecorderState state) { qDebug() << "new state:" << state; };
	connect(&mRecorder, &QMediaRecorder::recorderStateChanged, this, recorderState);
	mSession->setRecorder(&mRecorder);
	mRecorder.setOutputLocation(QUrl::fromLocalFile("f:/test"));
	mRecorder.record();
}

void WorkerThread::run() {
	QByteArray bytes(1024, 0);
	QAudioBuffer buf(bytes, mFormat, 0);
	bool result = mAudioInputBuffer->sendAudioBuffer(buf);
	qDebug() << "result:" << result << mRecorder.recorderState() << mRecorder.error();
	QThread::msleep(200);
}

int main(int argc, char** args) {
	QCoreApplication a(argc, args);
	std::shared_ptr<QThread> qthread = std::make_shared<WorkerThread>();
	qthread->start();
	a.exec();
	qthread->wait();
	return 0;
}

This will give me the output:

Available HW decoding frameworks:
     d3d11va
     dxva2
     d3d12va
Available HW encoding frameworks:
     d3d11va
     dxva2
     d3d12va
new state: QMediaRecorder::RecordingState
result: false QMediaRecorder::RecordingState QMediaRecorder::NoError

SGaist

Which version of Qt are you using ?
On Windows ?

csab6597

I am using Qt 6.8 on Windows 11 with Visual Studio 2022