Audio buffer with raw sounds - mixing

joaogatao

Hi and happy new year!

I am working on a drum machine / sequencer app for mobiles. Of course, precise timing and sound quality are important. I started out using QSoundEffect and a QTimer (max. precision) to trigger the sounds rhythmically. But the timing is really not precise enough, especially when several sounds are to be triggered simultaneously. It's just an audible mess.
On the other hand, sounds do not have to be triggered instantaneously. The drum beat is repeating and so it is known in advance which sounds are heard when, as set by the user.

Thinking of alternatives with precise timing:

Is it possible to build up the audio buffer for a whole bar by adding up and offsetting (in time) the different sounds in raw format (or wav)?
This buffer would then be repeating; can that be achieved without lag?
Even better would be if sounds can be added to an audio stream while playing (so the user can edit the drum beat while it is playing).

joaogatao

I was playing around with a raw sound file to play and add sounds in a buffer. But not really getting there.
Some issues I experienced:

QByteArray can be used to concatenate the raw data, but not to add / subtract in a way to mix sounds together.
I don't really understand why/how to delete the audio output and buffer (as here). It seems wrong in my case and leads to the app crashing with memory error.

In general, is there any good guide to audio programming with Qt somewhere?

The following trial with a clap sound raw file plays the sound twice with a 1sec delay in between when 'playBytes' is called.
But how to really mix sounds?
And how to make the buffer repeat?

#include "beep.h"
#include <QDebug>
#include <QBuffer>
#include <QtMath>
#include <QtMultimedia/QAudioOutput>

Beep::Beep(QObject *parent) : QObject(parent)
{
    // QFile m_file; //declared in header
    m_file.setFileName(":/LM1_5_clap.raw");
    m_file.open(QIODevice::ReadOnly);
    QByteArray m_data; //declared in header
    m_data = m_file.readAll();    // first clap sound
    m_data.append(48000*2,0);    // 1sec delay (48000 Hz * 2 bytes)
    m_data.append(m_data);    // second clap sound

    // create and setup a QAudioFormat object
    // QAudioFormat m_audioFormat; //declared in header
    QAudioDeviceInfo deviceInfo(QAudioDeviceInfo::defaultOutputDevice());
    m_audioFormat.setSampleRate(48000);
    m_audioFormat.setChannelCount(1);
    m_audioFormat.setSampleSize(16);
    m_audioFormat.setCodec("audio/pcm");
    m_audioFormat.setByteOrder(QAudioFormat::LittleEndian);
    m_audioFormat.setSampleType(QAudioFormat::SignedInt);

    if(!deviceInfo.isFormatSupported(m_audioFormat))
    {
        qWarning() << "Raw audio format not supported by backend, cannot play audio.";
        qWarning() << "Specified raw audio format is not supported by backend, cannot play audio. Supported options:";
        qWarning() << deviceInfo.supportedByteOrders();
        qWarning() << deviceInfo.supportedChannelCounts();
        qWarning() << deviceInfo.supportedCodecs();
        qWarning() << deviceInfo.supportedSampleRates();
        qWarning() << deviceInfo.supportedSampleSizes();
        qWarning() << deviceInfo.supportedSampleTypes();
        return;
    }
}

void Beep::playBytes()
{
    // Create audio buffer with the raw audio data array
    QBuffer *input = new QBuffer(&m_data);
    input->open(QIODevice::ReadOnly);   // set the QIODevice to read-only

    // Create an audio output with our QAudioFormat
    QAudioOutput *audio = new QAudioOutput(m_audioFormat, this);

    // connect up signal stateChanged to a lambda to get feedback
    connect(audio, &QAudioOutput::stateChanged, [audio, input](QAudio::State newState)
    {
        if (newState == QAudio::IdleState)   // finished playing (i.e., no more data)
        {
            qWarning() << "finished playing sound";
            delete audio;
            delete input;
        }
    });

    // start the audio (i.e., play sound from the QAudioOutput object that we just created)
    audio->start(input);

}

SGaist

Hi,

For more advanced audio processing like you want to do, Qt is not the right tool. You should rather look at something like PortAudio for the playback.

This stack overflow answer has some interesting suggestions.

joaogatao

Ok, thanks, I will look into it.
But can PortAudio be used within the same Qt app? I've already set up the GUI with all its functions.
Cheers!

SGaist

No problem with using PortAudio and Qt together. I did it a long time ago and no issue at all.

joaogatao

@SGaist Hi, I stalled this hobby project for a while cause of numerous reasons. But mainly cause I got lost at this point.
I found portaudio and would like to try it. But I simply don't know how to include the external library in my project. And how to compile it for an android app.
Would you know of any kind of simple example / tutorial that shows the way? For portaudio or for another external library where the process is similar?
Thanks :)

SGaist

For Android, I have seen this project to add support for the Androïd backend however I do not know how current and valid it is.

joaogatao

Thank you! I will take a closer look at that project. But then still need to figure out how to use it with Qt.
Anyway, thanks a lot for your help.

SGaist

If memory serves well I wrote a custom QIODevice on top of PortAudio.

Call for Presentations - Qt World Summit

Unsolved Audio buffer with raw sounds - mixing