Repeatedly playing contents of QByteArray through QAudioOutput



  • The idea was to populate QByteArray with enough samples for 1 cycle of a sine wave, then start playing and once the buffer was exhausted, re-start from the beginning.
    The program prints that sound goes into the active state, but instead of playing sound it crashes before it even reaches the code that re-starts playing. I do not understand why.
    Is that due to QDataStream going out of scope? Does QDataStream need to be a class member?

    #ifndef _MAINWINDOW_H
    #define _MAINWINDOW_H
    
    #if defined(Q_OS_WIN) || defined(Q_OS_WIN32) || defined(Q_OS_WIN64)
    #include <QtMultimedia/QAudioDeviceInfo>
    #include <QtMultimedia/QAudioOutput>
    #include <QtMultimedia/QAudioInput>
    #else
    #include <QAudioDeviceInfo>
    #include <QAudioOutput>
    #include <QAudioInput>
    #endif
    #include <QDebug>
    #include <QtMath>
    #include <QBuffer>
    #include <QFile>
    #include "ui_MainWindow.h"
    
    class MainWindow : public QDialog {
    	Q_OBJECT
    private slots:
      void handleBtnStart();
      void handleBtnStop();
      void handleBtnPlayFile();
      void handleStateChanged(QAudio::State newState);
    public:
        MainWindow();
    	virtual ~MainWindow();
        void deviceCapabilities(QAudioDeviceInfo& di);
    private:
    	Ui::MainWindow widget;
        QAudioOutput* audio;
        QByteArray* buf;
        QBuffer* b;
        bool stop = false;
        QFile sourceFile;
    };
    
    #endif /* _MAINWINDOW_H */
    
    void MainWindow::handleBtnStart(){
        QAudioDeviceInfo di = QAudioDeviceInfo::defaultOutputDevice();
        QAudioFormat af = QAudioFormat();
        af.setCodec("audio/pcm");
        af.setSampleRate(44100);//192000);
        af.setSampleSize(16); // also tried 8 bit with char sample
        af.setByteOrder(QAudioFormat::LittleEndian);
        af.setSampleType(QAudioFormat::UnSignedInt);
        af.setChannelCount(1);
    
        if(!di.isFormatSupported(af)){
            af = di.nearestFormat(af);
        }
        qDebug() << "Supported!";
        audio = new QAudioOutput(af, this);
        audio->setNotifyInterval(50);
        audio->setBufferSize(32768);
    
        connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));
    
        buf = new QByteArray();
        QDataStream s(buf, QIODevice::ReadWrite);
    
        for(float ii=0.0f; ii<360.0f; ii+=(360.0f*1000.0f/af.sampleRate())){
            int sample = ((int)(qSin(qDegreesToRadians(ii)) * 65536));
            // char sample = (char)(qSin(qDegreesToRadians(ii)) * 256); // also tried 8 bit with char sample
            s << sample;
            qDebug() << sample;
        }
        qDebug() << "Len: " << buf->length();
        audio->start(s.device());
        qDebug() << "State: " << audio->state();
        qDebug() << "Error: " << audio->error();
    }
    
    void MainWindow::handleStateChanged(QAudio::State newState)
    {
        switch (newState) {
            case QAudio::IdleState: // Finished playing (no more data)
                if(stop){
                    audio->stop();
                    qDebug() << "Stopped audio" << newState;
                    delete audio;
                    delete b;
                    delete buf;
                }
                else{ // restart from scratch
                    QDataStream s(buf, QIODevice::ReadWrite);
                    audio->start(s.device());
                }
                break;
    
            case QAudio::StoppedState: // Stopped for other reasons
                if (audio->error() != QAudio::NoError) { // Error handling
                    qDebug() << "Audio error: " << newState;
                }
                break;
    
            default:
                // ... other cases as appropriate
                qDebug() << "Something else: " << newState;
                break;
        }
    }
    
    void MainWindow::handleBtnStop(){
        stop = true;
    }
    

    This prints all generated samples and length of the buffer, then this:

    Something else:  ActiveState
    State:  ActiveState
    Error:  NoError
    

    which is supposed to mean that it successfully started playing...

    At the same time this code successfully plays an audio file (with the necessary edits to the state handler), which tells me that sound plugin is working fine:

    void MainWindow::handleBtnPlayFile(){
        sourceFile.setFileName("C:/Program Files/Microsoft Office/Office14/MEDIA/CHIMES.WAV");
        sourceFile.open(QIODevice::ReadOnly);
    
        QAudioFormat format;
        // Set up the format, eg.
        format.setSampleRate(22050);
        format.setChannelCount(1);
        format.setSampleSize(8);
        format.setCodec("audio/pcm");
        format.setByteOrder(QAudioFormat::LittleEndian);
        format.setSampleType(QAudioFormat::UnSignedInt);
    
        QAudioDeviceInfo info(QAudioDeviceInfo::defaultOutputDevice());
        if (!info.isFormatSupported(format)) {
            qWarning() << "Raw audio format not supported by backend, cannot play audio.";
            return;
        }
    
        audio = new QAudioOutput(format, this);
        connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));
        audio->start(&sourceFile);
    }
    

  • Qt Champions 2016

    hi
    http://doc.qt.io/qt-5/qaudiooutput.html#start
    start(QIODevice *device)
    seems to want a pointer.
    and u have
    QDataStream s(buf, QIODevice::ReadWrite);
    audio->start(s.device());

    So I would try to make it a class member so it dont die at end of function.



  • But if I declared in the header:

    QDataStream* s;
    

    and used it like that, then there would be a compilation error:

    s = new QDataStream(buf, QIODevice::ReadWrite);
        char sample = (char)(qSin(qDegreesToRadians(ii)) * 256);
        s << sample; // mainwindow.cpp:286: error: C2296: '<<' : illegal, left operand has type 'QDataStream *'
    

    How can I use a pointer to QDataStream with << operator?


  • Qt Champions 2016

    hi
    no need to make it pointer.
    Just have it as before but as member
    QDataStream s;
    It might eat
    (*s) << sample;
    But just use non pointer to check if its
    out of scope issue.



  • I left the pointer there, but instead populated the buffer with buf.append(sample); using 8 bits and char sample
    It is no longer crashing, but the speaker only burps once and there is no sound emitted.
    I dumped the samples and they form a correct sine wave, the rate seems to be reasonable too, so I am at a loss to understand why there is no sound.

    Also tried your suggestion to use (*s) << and it worked as well, but still no sound.

    Basically, there are the samples I am shoving down the audio output: samples chart


  • Qt Champions 2016

    hmm, would it be possible to save the raw "sound" to file and try
    to play in other program just to verify that it is indeed what expected?



  • As that file would only have 1 sine wave, I do not expect that anything could possibly play it, short of me writing a program for that.

    At least the 8 bit file using char sample directly inserted into the buffer

            char sample = (char)(qSin(qDegreesToRadians(ii)) * 128);
            buf->append(sample);
    

    looks fine:

    0000000000: 00 12 23 35 45 53 60 6B │ 74 7A 7E 7F 7E 7A 74 6C   ↕#5ES`ktz~⌂~ztl
    0000000010: 61 54 45 35 24 13 00 EF │ DD CC BC AE A1 95 8D 86  aTE5$‼ ïÝ̼®¡•†
    0000000020: 82 81 82 85 8B 94 9F AB │ BA CA DB ED FF           ‚‚…‹”Ÿ«ºÊÛíÿ
    

    That is the one that emits a weak 'burp' when it starts.
    The 16 bit samples populated via stream look all right to me too, but wit this one I cannot hear anything at all:

    0000000000: 00 00 00 00 00 00 12 2C │ 00 00 23 FB 00 00 35 0F        ↕,  #û  5☼
    0000000010: 00 00 45 0F 00 00 53 AA │ 00 00 60 92 00 00 6B 85    E☼  Sª  `’  k…
    0000000020: 00 00 74 4B 00 00 7A B5 │ 00 00 7E A3 00 00 7F FF    tK  zµ  ~£  ⌂ÿ
    0000000030: 00 00 7E C4 00 00 7A F7 │ 00 00 74 AC 00 00 6C 03    ~Ä  z÷  t¬  l♥
    0000000040: 00 00 61 2B 00 00 54 5A │ 00 00 45 D4 00 00 35 E3    a+  TZ  EÔ  5ã
    0000000050: 00 00 24 DB 00 00 13 13 │ 00 00 00 E9 FF FF EE BB    $Û  ‼‼   éÿÿî»
    0000000060: FF FF DC E5 FF FF CB C6 │ FF FF BB B5 FF FF AD 07  ÿÿÜåÿÿËÆÿÿ»µÿÿ­•
    0000000070: FF FF A0 08 FF FF 94 FA │ FF FF 8C 17 FF FF 85 8E  ÿÿ ◘ÿÿ”úÿÿŒ↨ÿÿ…Ž
    0000000080: FF FF 81 80 FF FF 80 02 │ FF FF 81 1D FF FF 84 C9  ÿÿ€ÿÿ€☻ÿÿ↔ÿÿ„É
    0000000090: FF FF 8A F5 FF FF 93 80 │ FF FF 9E 3E FF FF AA F7  ÿÿŠõÿÿ“€ÿÿž>ÿÿª÷
    00000000A0: FF FF B9 69 FF FF C9 4A │ FF FF DA 46 FF FF EC 06  ÿÿ¹iÿÿÉJÿÿÚFÿÿì♠
    00000000B0: FF FF FE 2E             │                          ÿÿþ.
    

  • Qt Champions 2016

    Hi
    Well im not sure if it will work but
    http://manual.audacityteam.org/man/importing_audio.html
    can import raw.



  • Emits a beep in Audacity as one would expect (if pasted 1000 times).


  • Qt Champions 2016

    Ok so the beep would be somewhat like the burb you get with Qt?

    Im wondering about
    QAudioFormat format;
    and
    new QAudioOutput(format, this);
    If it will copy it. ( I assume yes) (+ wav file works)
    so dont matter it goes out of scope.

    also
    else{ // restart from scratch
    QDataStream s(buf, QIODevice::ReadWrite);
    audio->start(s.device());
    }
    That seems to be a new / another QDataStream and not the one you made a class member?



  • Since then I've changed the code to use the member stream:

            else{ // restart from scratch
                audio->start(s->device()); // s is a class member here
            }
    

    No, the 'burp' is a short burst, a short abrupt chirp, nothing like a proper sine wave.
    The audio output clearly thinks it is playing audio, as there are no events until I hit the Stop button and then it prints

    Stopped audio IdleState
    

    into the debug output. It's just that there is no sound coming from the speakers. The program also appears in the Windows mixer once the audio output starts.


  • Qt Champions 2016

    My best guess is that it dont like the format but
    I cannot see why not.
    Also
    http://stackoverflow.com/questions/32049950/realtime-streaming-with-qaudiooutput-qt
    which seems same code, it seems it plays for him.
    And you seem to check errors etc so Im out of suggestions for now :(



  • I am somewhat suspicious of the repeated starting of the output from the same stream.
    Should there be something to re-wind the stream to the beginning? Like this:

            else{ // restart from scratch
                s->resetStatus();
                audio->start(s->device());
            }
    

    which does not actually help, I am just posting this as a question.
    That was one of the reasons I wanted to create a local variable for the stream in both places: for initial start and re-start.

    Or perhaps do I need to create a completely separate stream for reading from the buffer? I am getting impression that the stream is either at the end after writing data into the buffer, or when I want to re-start.


  • Qt Champions 2016

    Hi
    I wondered the same.
    Like it plays once and then are at the end.
    You could maybe put sample generation in a function and create a new sample
    at restart. mostly for test.



  • No luck! I tried to add this to the state handler:

            else{ // restart from scratch
                delete buf;
                delete s;
    
                buf = new QByteArray();
                s = new QDataStream(buf, QIODevice::ReadWrite);
    
                for(float ii=0.0f; ii<360.0f; ii+=(360.0f*1000.0f/af.sampleRate())){
                    int sample = ((int)(qSin(qDegreesToRadians(ii)) * 32768));
                    (*s) << sample;
            //        char sample = (char)(qSin(qDegreesToRadians(ii)) * 128);
            //        buf->append(sample);
            //        qDebug() << (int)sample;
                }
    
                audio->start(s->device());
            }
            break;
    

    Still no sound.
    Also tried completely removing re-start and simply repeating the wave 1000 times to get 1 second of sound, but to no avail:

    for(int i=0; i<1000; i++){
        for(float ii=0.0f; ii<360.0f; ii+=(360.0f*1000.0f/af.sampleRate())){
            int sample = ((int)(qSin(qDegreesToRadians(ii)) * 32768));
            (*s) << sample;
    //        char sample = (char)(qSin(qDegreesToRadians(ii)) * 128);
    //        buf->append(sample);
            qDebug() << (int)sample;
        }
    }
    

    It saved a 180,000 byte file which plays a nearly perfect tone when imported into Audacity (as far as I can tell with the cheap speakers).


  • Qt Champions 2016

    Ok so data is good
    but it seems it dont like. I wonder if some of the format settings
    is wrong but I really cant spot it.

    Is it possible for me to have the project to play with ?



  • @mrjj
    You can grab it from here: qtcreator project
    Thank you for your time!


  • Lifetime Qt Champion

    Hi,

    I'd recommend taking a look at the spectrum example in the example of the QtMultimedia module. There's a small tone generator that should get you started.



  • This is it, that project set me onto the right track:

    s->device()->close();
    s->device()->open(QIODevice::ReadOnly);
    

    before starting playback fixed the issue! So that really was the stream at the end.
    Super, thanks a lot!

    Development is not over, as I suspect that closing and re-opening the device is time consuming and would cause jerky sound, but at least it is clear now why it was not working.

    For now when I try to play the buffer with 1 sine wave repeatedly, using s->device()->seek(0); before restarting audio, the speakers are just clicking fast instead of playing a sine wave. This approach may be entirely unsustainable due to overhead involved in restarting both IO device and Audio output.


  • Qt Champions 2016

    Super
    I tried with
    http://doc.qt.io/qt-5/qbuffer.html#details
    but had no luck.



  • Overhead of seeking to 0 and re-starting audio must be huge!
    When I am playing 1 second of 1000 waves and re-starting, there is about 1/10 second delay between the blocks.
    This is on i7-2600 3.4 GHz machine. I will need to change the approach completely, but at least you two got me onto the right track - appreciate your help!


  • Qt Champions 2016

    @nulluse
    You are most welcome :)
    Its pretty huge delay.
    Im surprised seek is that expensive.

    funny, i have i7-2600 also. Old now but still a fine CPU


  • Qt Champions 2016

    Note
    Adding just
    s->device()->seek(0);
    made it play for me.


  • Qt Champions 2016

    I was wondering if we subclass
    QIODevice and implement
    qint64 QIODevice::readData(char * data, qint64 maxSize)
    If we then would endless supply data and avoid the
    delay as we never restart then.
    I have never subclassed QIODevice so not sure what minimum
    override is.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.