Skip to content
  • 0 Votes
    3 Posts
    216 Views
    J

    @Bonnie Thank you for the reply.

    Since my last post, I made some significant progress by debugging the Windows Qt multimedia source. I swapped QBuffer for a QRingBuffer. As it turns out, the performance issues were not related to the QBuffer in my previous implementation, however I prefer to use a lighter weight object as the Single Producer Single Consumer (SPSC) buffer between the microphone (the default input device) and the speaker (the default output device).

    The remaing problem that I need help with is how to schedule a restart after I encounter a buffer underrun or Eof condition (where there are no bytes available in the SPSC buffer).

    I swapped QBuffer for Qt's private QRingBuffer class which is used in other multimedia QIODevice derived objects. This class is not really well documented but it is relatively straight forward to understand - this link shows the impmentation). The ring buffer is basically made up from a list of RingChunks - each of which is effectively a wrapper around a QByteArray (with supporting head and tail offsets).

    The problem now is that I once the pull mode AudioSink encounters an error in its timer callback method - in this case the pullSource method (see below for Qt's windows AudioSink implementation) (QWindowsAudioSink::pullSource), the audio output changes its state to QAudio::IdleState with QAudio::IOError or QAudio::UnderrunError, and stops the pull timer. The pull timer callback is resposible for requesting the next raw chunk of audio from the QRingBuffer via the m_pullSource->read(readLen)) and writing it to the speaker ourput device. Meanwhile the capture slot in my worker thread keeps appending microphone data to this shared mpSinkDevice - so the buffer keeps growing (which would prevent the buffer underrun/eof condition) but I have no idea how to restart the audio output.

    void RtpWorker::handleAudioAvailable(const QAudioBuffer& rAudioBuffer) const { // append captured audio to mpSinkDevice's QRingBuffer mpSinkDevice->write(rAudioBuffer.constData< const char>(), rAudioBuffer.byteCount()); }

    The m_pullSource field in the code below is a pointer to the QIODevice containing the QRingBuffer which was opened for Read/Write (write is required by the audio capture slot to copy the raw audio from the microphone to the QRingBuffer).

    void QWindowsAudioSink::pullSource() { qCDebug(qLcAudioOutput) << "Pull source"; if (!m_pullSource) return; auto bytesAvailable = m_pullSource->isOpen() ? qsizetype(m_pullSource->bytesAvailable()) : 0; auto readLen = qMin(bytesFree(), bytesAvailable); if (readLen > 0) { QByteArray samples = m_pullSource->read(readLen); if (samples.size() == 0) { deviceStateChange(QAudio::IdleState, QAudio::IOError); return; } else { write(samples.data(), samples.size()); } } auto playTimeUs = remainingPlayTimeUs(); if (playTimeUs == 0) { deviceStateChange(QAudio::IdleState, m_pullSource->atEnd() ? QAudio::NoError : QAudio::UnderrunError); } else { deviceStateChange(QAudio::ActiveState, QAudio::NoError); m_timer->start(playTimeUs / 2000); } }

    Here is my SinkDevice

    //! Modeled after the Generator class example from QT 6.x audio output example. class SinkDevice : public QIODevice { Q_OBJECT public: /** * Explicit constructor * * @param parent [in] parent. */ explicit SinkDevice(QObject* parent = nullptr) : QIODevice(parent) , mBuffer{} {} /** * Explicit constructor * * @param rByteArray [in] array of multi-channel audio * samples. * @param parent [in] parent. */ explicit SinkDevice(const QByteArray& rByteArray, QObject* parent = nullptr) : QIODevice(parent) , mBuffer{} { mBuffer.append(rByteArray); } ~SinkDevice() override = default; /** * Start the IO device - open in read/write mode * so it can act like a ring buffer. */ void start(); /** * Close IO device. */ void stop(); //! Audio device should give sequential access [[nodiscard]] bool isSequential() const override { return true; } [[nodiscard]] qint64 bytesAvailable() const override { return mBuffer.size() + QIODevice::bytesAvailable(); } // Our size [[nodiscard]] qint64 size() const override { return mBuffer.size(); } protected: [[nodiscard]] qint64 readData(char* data, qint64 maxlen) override; [[nodiscard]] qint64 writeData(const char* data, qint64 maxlen) override; private: // disable copy & move semantics on QObject subclasses // https://www.cleanqt.io/blog/why-qobject-subclasses-are-not-copyable Q_DISABLE_COPY_MOVE(SinkDevice) void generateData(const QAudioFormat& format, qint64 durationUs, int sampleRate); QRingBuffer mBuffer; };
  • 0 Votes
    1 Posts
    93 Views
    No one has replied
  • 0 Votes
    4 Posts
    373 Views
    SGaistS

    I would check the internals of the other platform plugins to see how they are used by these classes.

  • 0 Votes
    1 Posts
    271 Views
    No one has replied
  • 0 Votes
    5 Posts
    493 Views
    R

    Thank you for your help, I've probably identified the problem - one of the input parameters was a char (8 bit). I've tried to fix it using the following code to reformat it so that it can be used properly for 16 bit but I keep getting the same problem

    /* Function with char as one of the parameters - qint64 XYSeriesIODevice::writeData(const char *data, qint64 maxSize) - same as in the Qt audio example**/ int bitsPerSample = 16; int bytesPerSample = (bitsPerSample / BITS_IN_BYTE); int signalLength = (strlen(data) ) / numChannels / bytesPerSample; short* signal = (short*)malloc(sizeof(short) * signalLength); for(int i = 0; i < signalLength; i++){ int index = i * numChannels * bytesPerSample; memcpy(&signal[i], &data[index], sizeof(short)); }

    Any suggestions?

  • 1 Votes
    6 Posts
    2k Views
    SGaistS

    Then you either have to play your video through QtMultimedia and use a QAudioProbe or you'll have to do it with some additional hardware.

  • 0 Votes
    1 Posts
    362 Views
    No one has replied