QAudioOutput and QSoundEffect outputs through ALSA directly without PulseAudio

Anthony Abboud

Hi Qt Community,

I've been working on making QAudioOutput play sounds through ALSA directly since my Embedded Linux Image does not contain a PulseAudio server (which is required for primitive functions such as QSound and QMediaPlayer). My embedded image contains Docker containers, where my Qt app resides.

I tried two different things to make this work.

Attempt 1
I used QAudioOutput and QAudioDeviceInfo to read the .WAV file, put it in a buffer and send it out to the correct device, as seen below:

QFile* m_audioFile;
QAudioOutput* audio;
QBuffer audio_buffer;
QByteArray audio_data;
void SoundManager::playAudio()
{
    m_audioFile = new QFile("/eclipse_qml/startup-sound.wav");
    if(m_audioFile->open(QIODevice::ReadOnly)) {
        // Build QByteArray
        m_audioFile->seek(44); // skip wav header
        audio_data = m_audioFile->readAll();
        m_audioFile->close();

        // Set QBuffer with data
        audio_buffer.setBuffer(&audio_data);
        audio_buffer.open(QIODevice::ReadOnly);
        qDebug() << "Audio Buffer Size: " << audio_buffer.size();

        QAudioFormat format;
        // Set up the format, eg.
        format.setSampleRate(44100);
        format.setChannelCount(2);
        format.setSampleSize(16);
        format.setCodec("audio/pcm");
        format.setByteOrder(QAudioFormat::LittleEndian);
        format.setSampleType(QAudioFormat::SignedInt);

        // Fetch Audio Device List
        QList<QAudioDeviceInfo> devices = QAudioDeviceInfo::availableDevices(QAudio::AudioOutput);
        foreach (QAudioDeviceInfo i, devices){
            // Find desired Audio device
            if(i.deviceName() == "sysdefault:CARD=imx6qapalissgtl")
            {

                qDebug() << "Attempting to play on device: " << i.deviceName() << Qt::endl;

                // Format support check
                if (!i.isFormatSupported(format)) {
                    qDebug() << "Raw audio format not supported by backend, cannot play audio." << Qt::endl;
                    return;
                }
                else{
                    qDebug() << "Format supported" << Qt::endl;
                    // Load source file and play audio
                    
                    audio = new QAudioOutput(i, format, this);
                    connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));

                    audio->start(&audio_buffer);
                    qDebug() << "Sound Player Started" << Qt::endl;
                    break;
                }
            }
        }
    }
}

void SoundManager::handleStateChanged(QAudio::State newState)
{
    switch (newState) {
        case QAudio::IdleState:
            // Finished playing (no more data)
            qDebug() << "Finished playing, no more data" << Qt::endl;
            audio->stop();
            qDebug() << "Sound Player Stopped" << Qt::endl;
            delete m_audioFile;
            delete audio;
            qDebug() << "Freed data" << Qt::endl;
            break;

        case QAudio::StoppedState:
            if(audio->error() != QAudio::NoError){
                // Error handling
                qDebug() << "Reached Stopped state: " << QString(audio->error()) << Qt::endl;
            }
            break;

        default:       
            break;
    }
}

The sound outputs and I can hear the music, but it's choppy (maybe ~20 cracks and pops), and it outputs a bunch (~20-30) of

ALSA lib pcm.c:8545:(snd_pcm_recover) underrun occurred

Attempt 2

I see that the function QSoundEffect allows to select an output device in the constructor, so I drafted up small code like so:

QList<QAudioDeviceInfo> devices = QAudioDeviceInfo::availableDevices(QAudio::AudioOutput);
    foreach (QAudioDeviceInfo i, devices){
        // Find desired Audio device
        if(i.deviceName() == "sysdefault:CARD=imx6qapalissgtl")
        {
            QSoundEffect audioPlayer(i, this);
            audioPlayer.setSource(QUrl("qrc:/sounds/FAR001-A44-01 Startup.wav"));
            audioPlayer.setVolume(1.0f);
            audioPlayer.play();
        }
    }

Where the hardcoded device name is where I'm trying to output to. Note that I am able to same the same sound to that device using the aplay command when in terminal.

However, this piece of code does not output any sound, even though I selected the correct audio device.

Question
For either of these alternatives, would anyone be familiar enough to identify what could be wrong?

Thanks in advance, any help is appreciated.

KH-219Design

I wish I could help in a more direct and specific way. I have used Qt on imx8, but I did not do anything with audio, nor with docker. (The docker part of your situation is "probably" irrelevant but it was wise of you to mention it. Maybe someone else will point out that docker is indeed relevant.)

All I can do now is make some generic observations. Things I would probably experiment with if I were in this situation.

(stating the possibly-obvious) the "underrun" message would seem to indicate that something in your code or the QAudio code is not filling the pcm buffer as quickly as it should.
It is very encouraging that aplay produces the sound correctly.

First thought:

Because of those 2 observations, and also because you already made (what I would call) "special accommodations" by forgoing QSound and QMediaPlayer due to platform limitations... (in other words, you are making code that is in some ways "tied to" or "hamstrung by" your platform already)

perhaps you would entertain just using libasound (<alsa/asoundlib.h>) directly? It could be an instructive exercise even if you ultimately are able to then switch back to using QAudio.

You could look at the source code of aplay to see what it does (since it successfully plays your sound clips): https://github.com/bear24rw/alsa-utils/blob/master/aplay/aplay.c

Second thought:

Look for ways to configure the buffer size. If your sound clips are small enough, and if the PCM and/or sound layers and hardware support it, maybe you can preload the audio into one big buffer before playing it. This might involve buffer flags in QAudio and/or buffer settings in the linux system somewhere.

Third thought:

Is there anything you can do to improve the overall performance of your application so it could "keep up" with its buffer-filling duties? Are there too many competing threads when audio is trying to play? Is an important audio-related thread blocked that shouldn't be?

It could also be as simple as compiling a fully optimized build, if you are not yet doing that. If your app binary is currently unoptimized (i.e. compiled with -O0), then simply adding optimization flags at compile time might help.

cordlandwehr

@Anthony-Abboud for the crackling sound issue, I could fix my issues with https://codereview.qt-project.org/c/qt/qtmultimedia/+/503818 and this is such a generic error that I expect that you hit the same problem.
For the missing sound output, the following changed helped me https://codereview.qt-project.org/c/qt/qtmultimedia/+/503819 . Yet for that I am not sure yet if it is a proper solution or just a workaround.