@Bonnie Thank you for the reply.
Since my last post, I made some significant progress by debugging the Windows Qt multimedia source. I swapped QBuffer for a QRingBuffer. As it turns out, the performance issues were not related to the QBuffer in my previous implementation, however I prefer to use a lighter weight object as the Single Producer Single Consumer (SPSC) buffer between the microphone (the default input device) and the speaker (the default output device).
The remaing problem that I need help with is how to schedule a restart after I encounter a buffer underrun or Eof condition (where there are no bytes available in the SPSC buffer).
I swapped QBuffer for Qt's private QRingBuffer class which is used in other multimedia QIODevice derived objects. This class is not really well documented but it is relatively straight forward to understand - this link shows the impmentation). The ring buffer is basically made up from a list of RingChunks - each of which is effectively a wrapper around a QByteArray (with supporting head and tail offsets).
The problem now is that I once the pull mode AudioSink encounters an error in its timer callback method - in this case the pullSource method (see below for Qt's windows AudioSink implementation) (QWindowsAudioSink::pullSource), the audio output changes its state to QAudio::IdleState with QAudio::IOError or QAudio::UnderrunError, and stops the pull timer. The pull timer callback is resposible for requesting the next raw chunk of audio from the QRingBuffer via the m_pullSource->read(readLen)) and writing it to the speaker ourput device. Meanwhile the capture slot in my worker thread keeps appending microphone data to this shared mpSinkDevice - so the buffer keeps growing (which would prevent the buffer underrun/eof condition) but I have no idea how to restart the audio output.
void
RtpWorker::handleAudioAvailable(const QAudioBuffer& rAudioBuffer) const {
// append captured audio to mpSinkDevice's QRingBuffer
mpSinkDevice->write(rAudioBuffer.constData<
const char>(), rAudioBuffer.byteCount());
}
The m_pullSource field in the code below is a pointer to the QIODevice containing the QRingBuffer which was opened for Read/Write (write is required by the audio capture slot to copy the raw audio from the microphone to the QRingBuffer).
void QWindowsAudioSink::pullSource()
{
qCDebug(qLcAudioOutput) << "Pull source";
if (!m_pullSource)
return;
auto bytesAvailable = m_pullSource->isOpen() ? qsizetype(m_pullSource->bytesAvailable()) : 0;
auto readLen = qMin(bytesFree(), bytesAvailable);
if (readLen > 0) {
QByteArray samples = m_pullSource->read(readLen);
if (samples.size() == 0) {
deviceStateChange(QAudio::IdleState, QAudio::IOError);
return;
} else {
write(samples.data(), samples.size());
}
}
auto playTimeUs = remainingPlayTimeUs();
if (playTimeUs == 0) {
deviceStateChange(QAudio::IdleState, m_pullSource->atEnd() ? QAudio::NoError : QAudio::UnderrunError);
} else {
deviceStateChange(QAudio::ActiveState, QAudio::NoError);
m_timer->start(playTimeUs / 2000);
}
}
Here is my SinkDevice
//! Modeled after the Generator class example from QT 6.x audio output example.
class SinkDevice : public QIODevice {
Q_OBJECT
public:
/**
* Explicit constructor
*
* @param parent [in] parent.
*/
explicit SinkDevice(QObject* parent = nullptr)
: QIODevice(parent)
, mBuffer{}
{}
/**
* Explicit constructor
*
* @param rByteArray [in] array of multi-channel audio
* samples.
* @param parent [in] parent.
*/
explicit SinkDevice(const QByteArray& rByteArray, QObject* parent = nullptr)
: QIODevice(parent)
, mBuffer{}
{
mBuffer.append(rByteArray);
}
~SinkDevice() override = default;
/**
* Start the IO device - open in read/write mode
* so it can act like a ring buffer.
*/
void start();
/**
* Close IO device.
*/
void stop();
//! Audio device should give sequential access
[[nodiscard]] bool isSequential() const override {
return true;
}
[[nodiscard]] qint64 bytesAvailable() const override {
return mBuffer.size() + QIODevice::bytesAvailable();
}
// Our size
[[nodiscard]] qint64 size() const override {
return mBuffer.size();
}
protected:
[[nodiscard]] qint64 readData(char* data, qint64 maxlen) override;
[[nodiscard]] qint64 writeData(const char* data, qint64 maxlen) override;
private:
// disable copy & move semantics on QObject subclasses
// https://www.cleanqt.io/blog/why-qobject-subclasses-are-not-copyable
Q_DISABLE_COPY_MOVE(SinkDevice)
void generateData(const QAudioFormat& format, qint64 durationUs, int sampleRate);
QRingBuffer mBuffer;
};
