Can't understand the reasons behind complaints; "QAudioInput: failed to set input volume" and "Got a buffer underflow!"?

faisal.tum

I'm trying to develop a audio streaming server that can serve audio to the client as well as receive audio from the client side when the client decides to send audio. I have four classes for this job. The Server does the job of serving, AudioInput does the job of producing the audio data from the microphone and the data is sent via the server to the client, AudioOutput does the job of producing the sound that can be heard in the server side PC once the audio data is received by the server from the client. The AudioStreamer class does the job of coordinating all the classes and user interactions together.

The UI looks like as shown in the link

The problem is that when ever I click Serve Audio button so that the server starts listening to the microphone and send the listened audio to the client, QAudioInput complains saying "QAudioInput: failed to set input volume ". Then when from the client side any audio is sent and the audio is supposed to be transformed into sound in the server side PC by AudioInput class, there is again the complain that says "Got a buffer underflow!".

I haven't found this problem in Windows, it seems it only happens in linux, but I need to make it work in linux. I can't understand what the problem is. I haven't any solution yet. I'm confident that these problems are only because of something wrong with the server side, not with the client side. Please see the project code and help me. Actually the complain "Got a buffer underflow!" is also happening in the client side as it receives the audio data and tries to produce the sound from that.

Here I will post the whole code of the server side so that you clearly get an idea of what I am doing.

The code in Server:

Server::Server(quint16 port, QObject *parent) :
    QObject(parent)
{
    socket = 0;
    server = new QTcpServer(this);
    connect(server, SIGNAL(newConnection()), this, SLOT(registerNewConnection()));
    server->listen(QHostAddress::Any, port);
}

void Server::setAudioHeader(QByteArray audioData)
{
    audioHeader = audioData;
}

void Server::registerNewConnection()
{
    socket = server->nextPendingConnection();
    connect(socket, SIGNAL(readyRead()), this, SLOT(readyRead()));
    socket->write(audioHeader);
}

QAudioFormat Server::currentAudioFormat()
{
    return format;
}

void Server::readyRead()
{
    if (!format.isValid())
    {
        if (socket->bytesAvailable() < 8)
            return;

        QByteArray data = socket->read(8);
        readHeader(data);

        output = new AudioOutput(currentAudioFormat(), this);
    }

    QByteArray data;

    while (socket->bytesAvailable() > 0)
        data.append(socket->readAll());

    emit audioReady(data);
    output->writeData(data);
}

void Server::readHeader(QByteArray data)
{
    quint8 channelCount;
    quint32 sampleRate;
    quint8 sampleSize;
    quint8 byteOrder;
    quint8 sampleType;

    QDataStream stream(&data, QIODevice::ReadWrite);
    stream >> channelCount;
    stream >> sampleRate;
    stream >> sampleSize;
    stream >> byteOrder;
    stream >> sampleType;

    format.setCodec("audio/pcm");
    format.setChannelCount(channelCount);
    format.setSampleRate(sampleRate);
    format.setSampleSize(sampleSize);
    format.setByteOrder((QAudioFormat::Endian)byteOrder);
    format.setSampleType((QAudioFormat::SampleType)sampleType);
}

void Server::writeAudioData(QByteArray data)
{
    if(socket)
    {
        socket->write(data);
    }
}

Server::~Server()
{
    delete server;
}

The code in AudioInput class which produces the audio via the device that is sent to the client by the server:

AudioInput::AudioInput(QAudioDeviceInfo devinfo, QAudioFormat format, QObject *parent) :
    QObject(parent), audioFormat(format)
{
    audio = new QAudioInput(devinfo, audioFormat, this);
    audio->setBufferSize(16384);
    device = audio->start();
    connect(device, SIGNAL(readyRead()), this, SLOT(readData()));
}

void AudioInput::readData()
{

    QByteArray audioData;
    //Check the number of samples in input buffer
    int len = audio->bytesReady();
    //Read sound samples from input device to buffer
    if(len)
    {
        audioData.resize(len);
        device->read(audioData.data(), len);
    }

    emit record(audioData);
    emit dataReadyToTransmit(audioData);
}

QByteArray AudioInput::header()
{
    QByteArray headerData;
    QDataStream stream(&headerData, QIODevice::ReadWrite);
    stream<<(quint8)audioFormat.channelCount();
    stream<<(quint32)audioFormat.sampleRate();
    stream<<(quint8)audioFormat.sampleSize();
    stream<<(quint8)audioFormat.byteOrder();
    stream<<(quint8)audioFormat.sampleType();
    return headerData;
}

void AudioInput::closeAudio()
{
    device->close();
    audio->stop();
}

AudioInput::~AudioInput()
{
    delete audio;
}

AudioOutput class that which gets the audio data that is sent from the client and that produces the sound that can be heard in the server side:


AudioOutput::AudioOutput(QAudioFormat format, QObject *parent) : QObject(parent)
{
    QAudioDeviceInfo::availableDevices(QAudio::AudioOutput);
    audio = new QAudioOutput(format, this);
    audio->setBufferSize(16384);

    device = audio->start();
}

void AudioOutput::writeData(QByteArray data)
{
    int readlen = audio->periodSize();

    int chunks = audio->bytesFree() / readlen;

    while (chunks)
    {
        QByteArray middle = data.mid(0, readlen);
        int len = middle.size();
        data.remove(0, len);

        if (len)
            device->write(middle);

        chunks--;
    }
}

Finally AudioStreamer, the class that coordinates all the other classes and the user interactions, has the following code:


AudioStreamer::AudioStreamer(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::AudioStreamer)
{
    ui->setupUi(this);
    ui->audioLabel->setVisible(false);
    getDeviceInfo();
    connect(ui->deviceBox, SIGNAL(currentIndexChanged(int)), this, SLOT(setBoxes(int)));
}

AudioStreamer::~AudioStreamer()
{
    delete ui;
}

void AudioStreamer::on_listen_clicked()
{
    QAudioDeviceInfo devinfo = ui->deviceBox->itemData(ui->deviceBox->currentIndex()).value<QAudioDeviceInfo>();
    QAudioFormat audioFormat;
    audioFormat.setChannelCount(ui->channelBox->itemData(ui->channelBox->currentIndex()).toInt());
    audioFormat.setCodec("audio/pcm");
    audioFormat.setSampleRate(ui->sampleRateBox->itemData(ui->sampleRateBox->currentIndex()).toInt());
    audioFormat.setSampleSize(16);
    audioFormat.setByteOrder(QAudioFormat::LittleEndian);
    audioFormat.setSampleType(QAudioFormat::SignedInt);
    input = new AudioInput(devinfo, audioFormat, this);
    bool check;
    qint16 port = (quint16)ui->port->text().toInt(&check, 10);
    server = new Server(port, this);
    server->setAudioHeader(input->header());
    connect(input, SIGNAL(dataReadyToTransmit(QByteArray)), server, SLOT(writeAudioData(QByteArray)));
    ui->deviceBox->setEnabled(false);
    ui->sampleRateBox->setEnabled(false);
    ui->channelBox->setEnabled(false);
    ui->port->setEnabled(false);
    ui->listen->setEnabled(false);

}

void AudioStreamer::getDeviceInfo()
{
    QList<QAudioDeviceInfo> deviceList = QAudioDeviceInfo::availableDevices(QAudio::AudioInput);
    if(deviceList.isEmpty())
    {
        QMessageBox::warning(this, "Error", "No input device availabe!");
        ui->deviceBox->setVisible(false);
        ui->channelBox->setVisible(false);
        ui->sampleRateBox->setVisible(false);
    }
    else
    {
        for(int i = 0; i<deviceList.size(); i++)
        {
            ui->deviceBox->addItem(deviceList.at(i).deviceName(), qVariantFromValue(deviceList.at(i)));
        }
        setBoxes(ui->deviceBox->currentIndex());
    }
}

void AudioStreamer::setBoxes(int index)
{
    QAudioDeviceInfo devinfo = ui->deviceBox->itemData(index).value<QAudioDeviceInfo>();
    QList<int> supportedChannels = devinfo.supportedChannelCounts();
    ui->channelBox->clear();
    for(int i = 0; i<supportedChannels.size(); i++)
    {
        ui->channelBox->addItem(QString::number(supportedChannels.at(i)), QVariant(supportedChannels.at(i)));
    }

    QList<int> supportedSampleRates = devinfo.supportedSampleRates();
    ui->sampleRateBox->clear();
    for(int i = 0; i<supportedSampleRates.size(); i++)
    {
        ui->sampleRateBox->addItem(QString::number(supportedSampleRates.at(i)), QVariant(supportedSampleRates.at(i)));
    }
}

void AudioStreamer::on_stopListening_clicked()
{
    server->~Server();
    input->closeAudio();
    input->~AudioInput();
    ui->deviceBox->setEnabled(true);
    ui->sampleRateBox->setEnabled(true);
    ui->channelBox->setEnabled(true);
    ui->port->setEnabled(true);
    ui->listen->setEnabled(true);
}

If you want to download the project files, the server application source is provided here:
https://www.dropbox.com/s/d1miuwyr3xkyzhw/PushToTalkServer.zip?dl=0

and the client application source is provided here:
https://www.dropbox.com/s/wp16lw6uov7f148/PushToTalkClient.zip?dl=0

faisal.tum

Is it the right branch to post this question,? In which branch should I post this so that I have higher probability of getting an answer? Is the way I have formulated the question is a hindrance to getting any answer at all?

SGaist

Hi and welcome to devnet,

Please allow 24 to 48 hours before bumping your own thread. This forum is community driven and not all people lives in the same timezone as you.

That said, both problems comes from Pulse. Is it working properly ?

faisal.tum

@SGaist

Hi,
thanks for the reply. Well I'm not sure if pulse is working correctly all the time. It seems unpredictable sometimes. Although I can't see these complaints in windows, but they are persistent in linux.

Anyway, I have changed my application as I don't need any more server client connection. Currently all I'm doing is a simple redirecting of QAudioInput to QAudioOutput. The main methods for that are as below:

void MainWindow::on_startButton_clicked()
{
    QAudioDeviceInfo inputDevinfo = ui->inputDevice->itemData(ui->inputDevice->currentIndex()).value<QAudioDeviceInfo>();
    QAudioFormat audioFormat;
    audioFormat.setChannelCount(2);
    audioFormat.setCodec("audio/pcm");
    audioFormat.setSampleRate(44100);
    audioFormat.setSampleSize(16);
    audioFormat.setByteOrder(QAudioFormat::LittleEndian);
    audioFormat.setSampleType(QAudioFormat::SignedInt);

    if (!inputDevinfo.isFormatSupported(audioFormat))
    {
        //Default format not supported - trying to use nearest
        audioFormat = inputDevinfo.nearestFormat(audioFormat);
    }
    m_audioInput = new QAudioInput(inputDevinfo, audioFormat, this);

    QAudioDeviceInfo outputDevinfo = ui->outputDevice->itemData(ui->outputDevice->currentIndex()).value<QAudioDeviceInfo>();
    if (!outputDevinfo.isFormatSupported(audioFormat))
    {
        //Default format not supported - trying to use nearest
        audioFormat = outputDevinfo.nearestFormat(audioFormat);
    }
    m_audioOutput = new QAudioOutput(outputDevinfo, audioFormat, this);
    m_audioOutput->start(m_audioInput->start());
}

void MainWindow::on_stopButton_clicked()
{
    m_audioOutput->stop();
    m_audioInput->stop();
    m_audioOutput->~QAudioOutput();
    m_audioInput->~QAudioInput();
}

Despite such simple approach of redirecting, where all the memory is supposedly dealt natively by Qt, there are at times complaints like "QAudioInput: failed to set input volume" and "Got a buffer underflow!". The application seems to work smoothly, but not quite dependable enough as the behaviour differs with different input and output devices. I have no idea what could be the problem. Is it the problem with my code or is there a bug in Qt's Audio module? The issues seem persistent in linux, in Windows I haven't faced many issues.

I didn't want to flood my response with codes, so if you want to fully test the code you can get it from the following Dropbox link.
https://www.dropbox.com/s/qwckg9midiqgp4q/duplexaudio.zip?dl=0

Well, just checked this code in my windows system. I cannot hear any sound there at all. So, in linux it complains "QAudioInput: failed to set input volume" and "Got a buffer underflow!" and in windows, no sound at all, but no complaints. Do you have any idea why such is the case?

Thanks.

SGaist

Calling a destructor like that is not the right way to delete an object. Either use the delete operator or, since they are QObject derived classes, deleteLater.

Did you found a pattern when these messages can be seen ?

faisal.tum

@SGaist

Thank you for an important rectification of my code.

As for the pattern, well, the code runs in Linux, it complains "QAudioInput: failed to set input volume" exactly when QAudioInput object is started by calling the start() function, while "Got a buffer underflow!" is rather an unpredictable complaint, I couldn't find a definitive stage when it would make that complaint.

As I have told you in the previous response, this approach of redirecting QAudioInput object to QAudioOutput object produces mere quietness in Windows, but the approach shown in this link:
http://www.codeproject.com/Articles/421287/Cross-Platform-Microphone-Audio-Processing-Utility
works perfectly in Windows, yet produces rampant "Got a buffer underflow!" complaints and eventual crash in Linux. The project from that link itself, when run in Linux leads to an eventual crash along with the aforementioned complaints, while running smoothly in Windows.

Why is such different behaviour for two different operating systems?

SGaist

Because both platform have rather radically different multimedia frameworks. Windows has DirectX and the Windows Media Foundation and linux has e.g. Pulse, GStreamer.

faisal.tum

@SGaist

So, this sort of difference should be expected then? Should I be worried that the code that actually manages to do the job is different for two different OS or this is normal in this case?

I'm sorry, I'm rather new and inexperienced, so I've these questions.

Thanks!

SGaist

Yes, it is to be expected, every OS is different so there will be differences. Qt abstracts them away for you quite nicely however there will some points where what the OS supports will show some differences. e.g. QMediaPlayer will play video on all OSes however Windows might require some additional Codecs to be installed where Linux would already read them because the default installed packages has supports for them already.

faisal.tum

@SGaist

Thanks!

So, is there any way I can get rid of the complaint "QAudioInput: failed to set input volume" in the Linux version?

SGaist

I'd say probably, but right now I don't know what might be interfering here to trigger that message.