Repeatedly playing contents of QByteArray through QAudioOutput

nulluse

The idea was to populate QByteArray with enough samples for 1 cycle of a sine wave, then start playing and once the buffer was exhausted, re-start from the beginning.
The program prints that sound goes into the active state, but instead of playing sound it crashes before it even reaches the code that re-starts playing. I do not understand why.
Is that due to QDataStream going out of scope? Does QDataStream need to be a class member?

#ifndef _MAINWINDOW_H
#define _MAINWINDOW_H

#if defined(Q_OS_WIN) || defined(Q_OS_WIN32) || defined(Q_OS_WIN64)
#include <QtMultimedia/QAudioDeviceInfo>
#include <QtMultimedia/QAudioOutput>
#include <QtMultimedia/QAudioInput>
#else
#include <QAudioDeviceInfo>
#include <QAudioOutput>
#include <QAudioInput>
#endif
#include <QDebug>
#include <QtMath>
#include <QBuffer>
#include <QFile>
#include "ui_MainWindow.h"

class MainWindow : public QDialog {
	Q_OBJECT
private slots:
  void handleBtnStart();
  void handleBtnStop();
  void handleBtnPlayFile();
  void handleStateChanged(QAudio::State newState);
public:
    MainWindow();
	virtual ~MainWindow();
    void deviceCapabilities(QAudioDeviceInfo& di);
private:
	Ui::MainWindow widget;
    QAudioOutput* audio;
    QByteArray* buf;
    QBuffer* b;
    bool stop = false;
    QFile sourceFile;
};

#endif /* _MAINWINDOW_H */

void MainWindow::handleBtnStart(){
    QAudioDeviceInfo di = QAudioDeviceInfo::defaultOutputDevice();
    QAudioFormat af = QAudioFormat();
    af.setCodec("audio/pcm");
    af.setSampleRate(44100);//192000);
    af.setSampleSize(16); // also tried 8 bit with char sample
    af.setByteOrder(QAudioFormat::LittleEndian);
    af.setSampleType(QAudioFormat::UnSignedInt);
    af.setChannelCount(1);

    if(!di.isFormatSupported(af)){
        af = di.nearestFormat(af);
    }
    qDebug() << "Supported!";
    audio = new QAudioOutput(af, this);
    audio->setNotifyInterval(50);
    audio->setBufferSize(32768);

    connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));

    buf = new QByteArray();
    QDataStream s(buf, QIODevice::ReadWrite);

    for(float ii=0.0f; ii<360.0f; ii+=(360.0f*1000.0f/af.sampleRate())){
        int sample = ((int)(qSin(qDegreesToRadians(ii)) * 65536));
        // char sample = (char)(qSin(qDegreesToRadians(ii)) * 256); // also tried 8 bit with char sample
        s << sample;
        qDebug() << sample;
    }
    qDebug() << "Len: " << buf->length();
    audio->start(s.device());
    qDebug() << "State: " << audio->state();
    qDebug() << "Error: " << audio->error();
}

void MainWindow::handleStateChanged(QAudio::State newState)
{
    switch (newState) {
        case QAudio::IdleState: // Finished playing (no more data)
            if(stop){
                audio->stop();
                qDebug() << "Stopped audio" << newState;
                delete audio;
                delete b;
                delete buf;
            }
            else{ // restart from scratch
                QDataStream s(buf, QIODevice::ReadWrite);
                audio->start(s.device());
            }
            break;

        case QAudio::StoppedState: // Stopped for other reasons
            if (audio->error() != QAudio::NoError) { // Error handling
                qDebug() << "Audio error: " << newState;
            }
            break;

        default:
            // ... other cases as appropriate
            qDebug() << "Something else: " << newState;
            break;
    }
}

void MainWindow::handleBtnStop(){
    stop = true;
}

This prints all generated samples and length of the buffer, then this:

Something else:  ActiveState
State:  ActiveState
Error:  NoError

which is supposed to mean that it successfully started playing...

At the same time this code successfully plays an audio file (with the necessary edits to the state handler), which tells me that sound plugin is working fine:

void MainWindow::handleBtnPlayFile(){
    sourceFile.setFileName("C:/Program Files/Microsoft Office/Office14/MEDIA/CHIMES.WAV");
    sourceFile.open(QIODevice::ReadOnly);

    QAudioFormat format;
    // Set up the format, eg.
    format.setSampleRate(22050);
    format.setChannelCount(1);
    format.setSampleSize(8);
    format.setCodec("audio/pcm");
    format.setByteOrder(QAudioFormat::LittleEndian);
    format.setSampleType(QAudioFormat::UnSignedInt);

    QAudioDeviceInfo info(QAudioDeviceInfo::defaultOutputDevice());
    if (!info.isFormatSupported(format)) {
        qWarning() << "Raw audio format not supported by backend, cannot play audio.";
        return;
    }

    audio = new QAudioOutput(format, this);
    connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));
    audio->start(&sourceFile);
}

mrjj

hi
http://doc.qt.io/qt-5/qaudiooutput.html#start
start(QIODevice *device)
seems to want a pointer.
and u have
QDataStream s(buf, QIODevice::ReadWrite);
audio->start(s.device());

So I would try to make it a class member so it dont die at end of function.

nulluse

But if I declared in the header:

QDataStream* s;

and used it like that, then there would be a compilation error:

s = new QDataStream(buf, QIODevice::ReadWrite);
    char sample = (char)(qSin(qDegreesToRadians(ii)) * 256);
    s << sample; // mainwindow.cpp:286: error: C2296: '<<' : illegal, left operand has type 'QDataStream *'

How can I use a pointer to QDataStream with << operator?

mrjj

hi
no need to make it pointer.
Just have it as before but as member
QDataStream s;
It might eat
(*s) << sample;
But just use non pointer to check if its
out of scope issue.

nulluse

I left the pointer there, but instead populated the buffer with buf.append(sample); using 8 bits and char sample
It is no longer crashing, but the speaker only burps once and there is no sound emitted.
I dumped the samples and they form a correct sine wave, the rate seems to be reasonable too, so I am at a loss to understand why there is no sound.

Also tried your suggestion to use (*s) << and it worked as well, but still no sound.

Basically, there are the samples I am shoving down the audio output: samples chart

mrjj

hmm, would it be possible to save the raw "sound" to file and try
to play in other program just to verify that it is indeed what expected?

nulluse

As that file would only have 1 sine wave, I do not expect that anything could possibly play it, short of me writing a program for that.

At least the 8 bit file using char sample directly inserted into the buffer

        char sample = (char)(qSin(qDegreesToRadians(ii)) * 128);
        buf->append(sample);

looks fine:

0000000000: 00 12 23 35 45 53 60 6B │ 74 7A 7E 7F 7E 7A 74 6C   ↕#5ES`ktz~⌂~ztl
0000000010: 61 54 45 35 24 13 00 EF │ DD CC BC AE A1 95 8D 86  aTE5$‼ ïÝÌ¼®¡•†
0000000020: 82 81 82 85 8B 94 9F AB │ BA CA DB ED FF           ‚‚…‹”Ÿ«ºÊÛíÿ

That is the one that emits a weak 'burp' when it starts.
The 16 bit samples populated via stream look all right to me too, but wit this one I cannot hear anything at all:

0000000000: 00 00 00 00 00 00 12 2C │ 00 00 23 FB 00 00 35 0F        ↕,  #û  5☼
0000000010: 00 00 45 0F 00 00 53 AA │ 00 00 60 92 00 00 6B 85    E☼  Sª  `’  k…
0000000020: 00 00 74 4B 00 00 7A B5 │ 00 00 7E A3 00 00 7F FF    tK  zµ  ~£  ⌂ÿ
0000000030: 00 00 7E C4 00 00 7A F7 │ 00 00 74 AC 00 00 6C 03    ~Ä  z÷  t¬  l♥
0000000040: 00 00 61 2B 00 00 54 5A │ 00 00 45 D4 00 00 35 E3    a+  TZ  EÔ  5ã
0000000050: 00 00 24 DB 00 00 13 13 │ 00 00 00 E9 FF FF EE BB    $Û  ‼‼   éÿÿî»
0000000060: FF FF DC E5 FF FF CB C6 │ FF FF BB B5 FF FF AD 07  ÿÿÜåÿÿËÆÿÿ»µÿÿ•
0000000070: FF FF A0 08 FF FF 94 FA │ FF FF 8C 17 FF FF 85 8E  ÿÿ ◘ÿÿ”úÿÿŒ↨ÿÿ…Ž
0000000080: FF FF 81 80 FF FF 80 02 │ FF FF 81 1D FF FF 84 C9  ÿÿ€ÿÿ€☻ÿÿ↔ÿÿ„É
0000000090: FF FF 8A F5 FF FF 93 80 │ FF FF 9E 3E FF FF AA F7  ÿÿŠõÿÿ“€ÿÿž>ÿÿª÷
00000000A0: FF FF B9 69 FF FF C9 4A │ FF FF DA 46 FF FF EC 06  ÿÿ¹iÿÿÉJÿÿÚFÿÿì♠
00000000B0: FF FF FE 2E             │                          ÿÿþ.

mrjj

Hi
Well im not sure if it will work but
http://manual.audacityteam.org/man/importing_audio.html
can import raw.

nulluse

Emits a beep in Audacity as one would expect (if pasted 1000 times).

mrjj

Ok so the beep would be somewhat like the burb you get with Qt?

Im wondering about
QAudioFormat format;
and
new QAudioOutput(format, this);
If it will copy it. ( I assume yes) (+ wav file works)
so dont matter it goes out of scope.

also
else{ // restart from scratch
QDataStream s(buf, QIODevice::ReadWrite);
audio->start(s.device());
}
That seems to be a new / another QDataStream and not the one you made a class member?

nulluse

Since then I've changed the code to use the member stream:

        else{ // restart from scratch
            audio->start(s->device()); // s is a class member here
        }

No, the 'burp' is a short burst, a short abrupt chirp, nothing like a proper sine wave.
The audio output clearly thinks it is playing audio, as there are no events until I hit the Stop button and then it prints

Stopped audio IdleState

into the debug output. It's just that there is no sound coming from the speakers. The program also appears in the Windows mixer once the audio output starts.

mrjj

My best guess is that it dont like the format but
I cannot see why not.
Also
http://stackoverflow.com/questions/32049950/realtime-streaming-with-qaudiooutput-qt
which seems same code, it seems it plays for him.
And you seem to check errors etc so Im out of suggestions for now :(

nulluse

I am somewhat suspicious of the repeated starting of the output from the same stream.
Should there be something to re-wind the stream to the beginning? Like this:

        else{ // restart from scratch
            s->resetStatus();
            audio->start(s->device());
        }

which does not actually help, I am just posting this as a question.
That was one of the reasons I wanted to create a local variable for the stream in both places: for initial start and re-start.

Or perhaps do I need to create a completely separate stream for reading from the buffer? I am getting impression that the stream is either at the end after writing data into the buffer, or when I want to re-start.

mrjj

Hi
I wondered the same.
Like it plays once and then are at the end.
You could maybe put sample generation in a function and create a new sample
at restart. mostly for test.

nulluse

No luck! I tried to add this to the state handler:

        else{ // restart from scratch
            delete buf;
            delete s;

            buf = new QByteArray();
            s = new QDataStream(buf, QIODevice::ReadWrite);

            for(float ii=0.0f; ii<360.0f; ii+=(360.0f*1000.0f/af.sampleRate())){
                int sample = ((int)(qSin(qDegreesToRadians(ii)) * 32768));
                (*s) << sample;
        //        char sample = (char)(qSin(qDegreesToRadians(ii)) * 128);
        //        buf->append(sample);
        //        qDebug() << (int)sample;
            }

            audio->start(s->device());
        }
        break;

Still no sound.
Also tried completely removing re-start and simply repeating the wave 1000 times to get 1 second of sound, but to no avail:

for(int i=0; i<1000; i++){
    for(float ii=0.0f; ii<360.0f; ii+=(360.0f*1000.0f/af.sampleRate())){
        int sample = ((int)(qSin(qDegreesToRadians(ii)) * 32768));
        (*s) << sample;
//        char sample = (char)(qSin(qDegreesToRadians(ii)) * 128);
//        buf->append(sample);
        qDebug() << (int)sample;
    }
}

It saved a 180,000 byte file which plays a nearly perfect tone when imported into Audacity (as far as I can tell with the cheap speakers).

mrjj

Ok so data is good
but it seems it dont like. I wonder if some of the format settings
is wrong but I really cant spot it.

Is it possible for me to have the project to play with ?

nulluse

@mrjj
You can grab it from here: qtcreator project
Thank you for your time!

SGaist

Hi,

I'd recommend taking a look at the spectrum example in the example of the QtMultimedia module. There's a small tone generator that should get you started.

nulluse

This is it, that project set me onto the right track:

s->device()->close();
s->device()->open(QIODevice::ReadOnly);

before starting playback fixed the issue! So that really was the stream at the end.
Super, thanks a lot!

Development is not over, as I suspect that closing and re-opening the device is time consuming and would cause jerky sound, but at least it is clear now why it was not working.

For now when I try to play the buffer with 1 sine wave repeatedly, using s->device()->seek(0); before restarting audio, the speakers are just clicking fast instead of playing a sine wave. This approach may be entirely unsustainable due to overhead involved in restarting both IO device and Audio output.

mrjj

Super
I tried with
http://doc.qt.io/qt-5/qbuffer.html#details
but had no luck.