Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

No Audio-Output with QAudioOutput::start(QIODevice*)



  • I have this problem, that I need to generate audio output to the systems default audio device. The audio is generated speech from text (Text-to-Speech, TTS). The speech generation works fine, since I can write it to a file (which is played correctly) and play with QAudioOutput in Push-Mode, by calling QIODevice *QAudioOutput::start().

    • Saving to file is not what I need, but I'm using this method for an AutoTest, which works fine.
    • Push-Mode seems like a bad option, since waiting a hard coded amount of time can cause interferences in the output

    The problem I have is, that I don't fill the buffer completely. The TTS library I'm working with (Nuance Vocalizer Embedded) works with callbacks. I have a start the audio stream callback, which I use to call QAudioOutput::start(QIODevice*) with a buffer I implemented, based on the AudioOutput Example Generator-Class from Qt. After that the state is set to QAudio::ActiveState with QAudio::NoError

    When the TTS Library starts generating a new chunk of data there is a buffer request callback, which I answer by creating a C-String with a size of 4096 (const char *buffer = new char[4096]).

    After that I pass it back to the TTS, which generates the audio data and puts it into the pointer. After that I get a buffer is ready callback, with the previously generated C-String as void* with a buffer length, that can differ from what I set earlier (i.e. if there is no more audio). I use it to write to my buffer, while the QAudioOutput state is still QAudio::ActiveState

    I expect QAudioOutput to notice, that the buffer is being written and call the reimplemented qint64 QIODevice::readData(char*, qint64 maxlen) function, so the new data is written to the audio device and cleared from the buffer. However, readData is actually never called, although the buffer is being written and emits the bytesWritten-Signal.

    I would think, that my assumption is just wrong and QAudioOutput does not care if the buffer gets new data after QAudioOutput::start, but, I'm rewriting old code (new design, tests, etc.), which already works in production and uses the same principle. start ... request buffer -> generate buffer -> fill buffer -> write to device-buffer ... output audio on audio device ... request buffer -> ........
    My program also runs without errors. It just doesn't generate any audio output.

    Thanks in advance!

    Here is some of my code. It's work in progress and has debug code snippets and such, which don't belong into the end product:

    /* My Reimplemented Audiobuffer */
    #include <algorithm>
    #include <iostream>
    #include <string>
    #include <QIODevice>
    
    class AudioBuffer final: public QIODevice
    {
    public:
        explicit AudioBuffer();
        qint64 size() const override;
    protected:
        qint64 readData(char *data, qint64 maxlen) override;
        qint64 writeData(const char *data, qint64 len) override;
        qint64 bytesAvailable() const override;
    
    private:
        qint64 m_pos = 0;
        std::string m_buffer;
    };
    AudioBuffer::AudioBuffer()
    {}
    
    qint64 AudioBuffer::size() const
    {
        return static_cast<qint64>(m_buffer.size());
    }
    
    qint64 AudioBuffer::readData(char *data, qint64 maxlen)
    {
        std::cout << "calling readData()" << std::endl;
        qint64 total = 0;
        if (!m_buffer.empty()) {
            while (maxlen - total > 0) {
                /* choose 4096 or remaining data-size */
                size_t chunk = static_cast<size_t>(std::min((maxlen - total), static_cast<qint64>(4096)));
                memcpy(data + total, m_buffer.data() + m_pos, chunk);
                m_pos = (m_pos + static_cast<qint64>(chunk)) % static_cast<qint64>(m_buffer.size());
                total += chunk;
            }
        }
        std::cout << "buffer-size: " << m_buffer.size() << std::endl;
        return total;
    }
    
    qint64 AudioBuffer::writeData(const char *data, qint64 len)
    {
        auto oldSize = m_buffer.size();
        m_buffer.append(data, static_cast<size_t>(len));
        auto written = static_cast<qint64>(m_buffer.size() - oldSize);
        emit bytesWritten(written);
        return written;
    }
    
    qint64 AudioBuffer::bytesAvailable() const
    {
        return static_cast<qint64>(m_buffer.size()) + QIODevice::bytesAvailable();
    }
    
    /* My AudioOutput Class (Wraps QAudioOutput and reimplemented AudioBuffer) */
    #include "audiooutputinterface.h" /* pure virtual class, only used as interface */
    #include <memory>
    #include <QAudioOutput>
    #include "audiobuffer.h"
    
    class AudioOutput final : public AudioOutputInterface
    {
    public:
        AudioOutput(const int audioBufferSize = 4096);
    
        bool start();
        size_t write(const char *data, size_t length);
        void suspend();
        void resume();
        void stop();
    
    private:
        QAudioOutput m_audioOutput;
        std::unique_ptr<AudioBuffer> m_buffer;
        const int m_audioBufferSize;
    };
    
    QAudioFormat audioFormatFactoryFunction()
    {
        QAudioFormat result;
        result.setSampleRate(22000);
        result.setChannelCount(1);
        result.setSampleSize(16);
        result.setSampleType(QAudioFormat::SignedInt);
        result.setCodec(QStringLiteral("audio/pcm"));
        return result;
    }
    
    AudioOutput::AudioOutput(const int audioBufferSize)
        : m_audioOutput(audioFormatFactoryFunction())
        , m_buffer(new AudioBuffer)
        , m_audioBufferSize(audioBufferSize)
    {
        if (!m_buffer->open(QIODevice::ReadWrite))
            qFatal("Could not open AudioBuffer ... ");
        QObject::connect(&m_audioOutput, &QAudioOutput::stateChanged,
                         QCoreApplication::instance(), [] (QAudio::State state) -> void
        {
            std::cout << "[DBG] state changed to: " << state << std::endl;
        });
    }
    
    bool AudioOutput::start()
    {
    //    m_buffer.reset(m_audioOutput.start());
    //    m_audioOutput.start(m_buffer.get());
        return m_buffer != nullptr;
    }
    
    /* in push-mode I write, then wait a fixed amount of time (calculated). Works, but not as wanted. This is pull-mode */
    size_t AudioOutput::write(const char *data, size_t length)
    {
        if (m_buffer == nullptr)
            return static_cast<size_t>(-1);
        std::cout << "[DBG] buffer-size: " << m_buffer->size() << std::endl;
        return static_cast<size_t>(m_buffer->write(data, static_cast<long long>(length)));
    }
    
    /* TODO: implement suspend */
    void AudioOutput::suspend()
    {
    
    }
    
    /* TODO: implement resume */
    void AudioOutput::resume()
    {
    
    }
    
    void AudioOutput::stop()
    {
        m_buffer->close();
        m_audioOutput.stop();
    }
    
    /* The callback-function, to show how it works and calls the methods */
    /* Stuff like with VE_, NUAN and other strange names come directly from the TTS-Engine API */
    NUAN_ERROR TtsNuanceEngine::onOutNotify(VE_CALLBACKMSG *pcbMessage)
    {
        switch (pcbMessage->eMessage) { /* what kind of callback is this? */
            case VE_MSG_BEGINPROCESS: { /* begin the audio output */
                if (!m_audioOutput->start()) /* My AudioOutput::start */
                    return NUAN_E_TTS_AUDIOOUTOPEN;
                break;
            }
            case VE_MSG_ENDPROCESS: /* end the process */
                setState(State::Ready);
                break;
            case VE_MSG_PAUSE: /* pause the audio output */
                m_audioOutput->suspend();
                setState(State::Paused);
                break;
            case VE_MSG_RESUME: /* resume the audio output */
                m_audioOutput->resume();
                setState(State::Speaking);
                break;
            case VE_MSG_STOP: /* stop the audio output */
                m_audioOutput->stop();
                break;
            case VE_MSG_OUTBUFREQ: { /* request buffer for audio data */
                auto *outData = static_cast<VE_OUTDATA*>(pcbMessage->pParam); /* VE_OUTDATA is a struct */
                outData->pOutPcmBuf = new char[kAudioBufferSize]; /* pOutPcmBuf = Pointer to Output PCM-Buffer */
                memset(outData->pOutPcmBuf, 0, kAudioBufferSize); /* set everything to 0 */
                outData->cntPcmBufLen = kAudioBufferSize; /* cntPcmBufLen = PCM-Buffer Length. kAudioBufferSize = 4096 */
                outData->pMrkList = Q_NULLPTR; /* unneeded (special voice tonation commands etc. */
                outData->cntMrkListLen = 0; /* see line above */
                break;
            }
            case VE_MSG_OUTBUFDONE: {
                auto *outData = static_cast<VE_OUTDATA*>(pcbMessage->pParam);
                auto *bufferData = static_cast<const char*>(outData->pOutPcmBuf); /* buffer is passed as void*. Cast it back */
                if (0 < outData->cntPcmBufLen) { /* PCM-Buffer length is not 0 */
                    if (m_audioOutput->write(bufferData, outData->cntPcmBufLen) /* That's my AudioOutput::write */
                        == static_cast<size_t>(-1)) {
                        return NUAN_E_TTS_AUDIOOUTWRITE;
                    }
                } else if (pcbMessage->lValue != 0xFFFF) {
                    return NUAN_E_BUFFERTOOSMALL;
                }
                delete[] bufferData; /* delete this buffer, since it's not needed any longer */
                bufferData = Q_NULLPTR;
                break;
            }
            default:
                // LOG ERROR
        }
    
        return NUAN_OK;
    }
    

  • Lifetime Qt Champion

    Hi,

    From what I recall, I think the simple road would be to pass the QIODevice you get from QAudioOutput to the callback and write the chunk of data you get there directly to it.



  • This post is deleted!


  • @SGaist That's what I mean when talking about push-mode. It's not working as intended


  • Lifetime Qt Champion

    From the looks of it, seems that you could directly write to the QAudioOutput when your callback is called.

    However, do I understand correctly that you would rather get all the generated data before sending them to your audio output ?



  • @SGaist No, I would rather just write every 4096-Byte Block directly into the device when I get the callback with the data. I tried it with first loading the buffer and to call QAudioOutput::start(QIODevice*) after the VE_MSG_ENDPROCESS callback. Still didn't work, although it should, as far as I know. Nothing seems to work and I'm currently using the push-mode method, which lags sometimes.


Log in to reply