How to get audio data from microphone via QAudioInput and QByteArray?

wzf999999

I bind QBuffer with QAudioInput, bind QByteArray with QBuffer.
After I start QAudioInput, the QByteArray will grow.
I need to process the audiodata per 50ms, but the QByteArray will grow without pausing.

It make me think the QByteArray is a bull shit.

Do any friends have ideas?
it make me crazy!

jsulm

@wzf999999 said in How to get audio data from microphone via QAudioInput and QByteArray?:

It make me think the QByteArray is a bull shit.

?
It's not QByteArrays fault if data processing is done in a wrong way. Don't blame others for own mistakes :-)

Please show your code, so others can actually tell you what is wrong in your code.

wzf999999

@jsulm ok thanks

wzf999999

audioutils.h

#include "audioutils.h"
#include <QAudioEncoderSettings>

#include <QDebug>

AudioUtils::AudioUtils()
{
    format.setSampleRate(16000);
    format.setChannelCount(1);
    format.setSampleSize(16);
    format.setCodec("audio/pcm");
    format.setByteOrder(QAudioFormat::LittleEndian);
    format.setSampleType(QAudioFormat::SignedInt);
    notifyTime = 10000;
    buffSize = 32*notifyTime;
}

AudioUtils::~AudioUtils()
{
    qDebug() << "~AudioUtils";
    if (timer.isActive()){
        timer.stop();
        timer.disconnect();
    }
    if ((audio != nullptr) && (audio->state() != QAudio::StoppedState)) {
        audio->stop();
        audio->disconnect();
        delete audio;
    }
}

void AudioUtils::stopRecording()
{
    if (audio->state() != QAudio::StoppedState) {
        audio->stop();
    }
    audio->disconnect();
    delete audio;
    audio = nullptr;
}

void AudioUtils::handleStateChanged(QAudio::State newState)
{
    qDebug() << "handleStateChanged" ;
    switch (newState) {
    case QAudio::StoppedState:
        if (audio->error() != QAudio::NoError) {
            // Error handling
        } else {
            // Finished recording
        }
        break;

    case QAudio::ActiveState:
        // Started recording - read from IO device
        break;

    default:
        // ... other cases as appropriate
        break;
    }
}

QIODevice * AudioUtils::startAudio()
{
    if (audio != nullptr)
        stopRecording();
    audio = new QAudioInput(format, this);
    connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));
//    connect(audio, SIGNAL(notify()), this, SLOT(handleNotify()));
    connect(audio, SIGNAL(notify()), this, SIGNAL(notify()));
//    audio->setBufferSize(buffSize);
    audio->setVolume(1.0);
    audio->setNotifyInterval(notifyTime);
    QIODevice * d = audio->start();
    return d;
}

void AudioUtils::startAudio(QIODevice *voicedevice)
{
    if (audio != nullptr)
        stopRecording();
    audio = new QAudioInput(format, this);
    connect(audio, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleStateChanged(QAudio::State)));
    connect(audio, SIGNAL(notify()), this, SLOT(handleNotify()));
    connect(audio, SIGNAL(notify()), this, SIGNAL(notify()));
    audio->setBufferSize(buffSize);
    qDebug() <<"buffer size" <<audio->bufferSize();
    audio->setVolume(1.0);
    qDebug() <<"volume" << audio->volume();
    audio->setNotifyInterval(notifyTime);
    qDebug() <<"notify interval" << audio->notifyInterval();
    audio->start(voicedevice);
}

void AudioUtils::handleNotify()
{
    qDebug() << "bytes ready" <<audio->bytesReady();
}

void AudioUtils::testtimer()
{
    qDebug() << "testtimer";
}

bool AudioUtils::setAudioParam(int sampleRate, int channel, int sampleSize)
{
    format.setSampleRate(sampleRate);
    format.setChannelCount(channel);
    format.setSampleSize(sampleSize);
    format.setCodec("audio/pcm");
    format.setByteOrder(QAudioFormat::LittleEndian);
    format.setSampleType(QAudioFormat::SignedInt);

    QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
    if (!info.isFormatSupported(format)) {
        qCritical() << "Default format not supported, trying to use the nearest.";
        format = info.nearestFormat(format);
        return false;
    }
    return true;
}

void AudioUtils::resetAudioBuffer()
{
        audio->reset();
}

audioutils.h

#ifndef AUDIOUTILS_H
#define AUDIOUTILS_H

#include <QObject>
#include <QAudioInput>
#include <QFile>
#include <QTimer>
#include <QDir>
#include <QDate>
#include <QTime>
#include <QString>

class AudioUtils: public QObject
{
    Q_OBJECT
public:
    AudioUtils();
    ~AudioUtils();

    void testtimer();
    bool setAudioParam(int sampleRate,int channel, int sampleSize);
    void resetAudioBuffer();
signals:
    void notify();

public slots:
    void stopRecording();
    void handleStateChanged(QAudio::State newState);
    void handleNotify();

    QIODevice *  startAudio();
    void startAudio(QIODevice *voicedevice );


private:
    QTimer timer;
    QAudioFormat format;
    QAudioInput *audio = nullptr;
    int buffSize = 0;
    int notifyTime = 0;

};

#endif // AUDIOUTILS_H

here i start and stop the audio

void AsrClient::on_start_clicked(bool checked)
{
    qDebug() << checked;
    if (checked){
        ui->start->setText("Stop");
      qDebug() << byteArray.length();
      connect(&buff,SIGNAL(readyRead()), this, SLOT(handleReadData()));
        buff.open(QIODevice::ReadWrite|QIODevice::Truncate);
        //iodevice =
                audio.startAudio(&buff);
    }
    else{
        QFile file(config.getPcmName());
        file.open(QIODevice::WriteOnly|QIODevice::Truncate);
//        file.write(buff.readAll());
        QDataStream ds(&file);
        ds << byteArray;
        ui->start->setText("Start");
        qDebug() << byteArray.length();

        audio.stopRecording();
//        iodevice = nullptr;
//        qDebug() << iodevice;
        file.close();
        buff.reset();
        buff.close();
    }
}

at first i connect the caller and audioinput

    connect(&audio, SIGNAL(notify()), this, SLOT(replyNotify()));

and in below, I found the qbytearray will grow

void AsrClient::replyNotify()
{
    qDebug() << "notify";
//    qDebug() << byteArray.length();
//    qDebug() << iodevice->readAll().length();
//    QByteArray ba = buff.data();
//    if (!buff.reset()){
//        qDebug() << "reset failed";
//    }
//    qDebug() << ba.length();
//    buff.seek(0);
//    ui->asrText->append(QString("len: %1   %2").arg(byteArray.length()).arg(file.size()));
//    buff.reset();
//    byteArray.resize(0);
//    buff.seek(0);
//    if (!byteArray.isEmpty() || !byteArray.isNull()){
//        byteArray.clear();
//        byteArray.resize(0);
//        byteArray.squeeze();
    //    }
}

jsulm

@wzf999999 said in How to get audio data from microphone via QAudioInput and QByteArray?:

if (audio != nullptr)
stopRecording();

You leek memory as you do not delete previous audio.

"and in below, I found the qbytearray will grow" - which byte array do you mean?

Please show your handleReadData()

wzf999999

@jsulm
Sorry,but I delete it in fact.
in function stopRecording(), I delete it.

void AudioUtils::stopRecording()
{
    if (audio->state() != QAudio::StoppedState) {
        audio->stop();
    }
    audio->disconnect();
    delete audio;
    audio = nullptr;
}

wzf999999

@jsulm
@jsulm

in function replyNotify(), I comment it.
the bytearray is bind to buff, buff is bind to audioinput

in slot replayNotify()
I qDebug the bytearray.length(), it will grow every time when the slot was trigger.

handleReadData() is below

void AsrClient::handleReadData()
{
    qDebug() << "handleReadData";
    qDebug() << buff.data().length();
//    buff.readAll();
//    qDebug( "%x",buff.buffer().data()[0]);
}

jsulm

@wzf999999 Well, at some point you need to remove data from the byte array. What do you do with the data?

wzf999999

@jsulm

I need to send audio data every 50ms with 0x05,0x00,0x00,0x00 prefix to server.

how to remove the data?
yesterday, the exec the function bytearray.remove(0, bytearray.length()) in handleReadData, but it doesn't work.

jsulm

@wzf999999 said in How to get audio data from microphone via QAudioInput and QByteArray?:

but it doesn't work

In what way it did not work? And why bytearray and not buff which is the one growing if I understood you correctly?

wzf999999

@jsulm said in How to get audio data from microphone via QAudioInput and QByteArray?:

@wzf999999 said in How to get audio data from microphone via QAudioInput and QByteArray?:

but it doesn't work

In what way it did not work? And why bytearray and not buff which is the one growing if I understood you correctly?

but the buff doesn't have the function remove() i don't know how to remove the data from buff. :-(

jsulm

@wzf999999 It's buff.data()
What type is buff?

wzf999999

@jsulm
The buff is QBuffer.
And the buff.data() is const QByteArray.

Am I use the wrong type of buff?

jsulm

@wzf999999 Do you read at some point from buff? See https://doc.qt.io/qt-5/qbuffer.html
Your audio data is constantly written into buff, but if you do not read from it (or not read fast enough) it will grow.

wzf999999

@jsulm
I have seen it before.
I need to confirm the buffer size.

Now I know if I don't read it will grow.
I will try it now.

wzf999999

@jsulm

There is no suitable function in QBuffer.
What i see is to read from QByteArray which is bind to QBuffer. But it doesn't work.

Below is code read from QBuffer.

void AsrClient::handleReadData()
{
    qDebug() << "handleReadData Before Read" << buff.data().length();
    buff.read(buff.data().length());
    QByteArray ba = buff.readAll();
    qDebug() << "handleReadData After Read" << buff.data().length();


//    buff.buffer().remove(0,buff.buffer().length());
//    buff.readData(audioBuff,65536);
//    buff.data().data();
//    buff.seek(0);
//    buff.reset();
//    buff.readAll();
//    qDebug( "%x",buff.buffer().data()[0]);
}

and the result is below:

handleReadData Before Read 31284
handleReadData After Read 31284
handleReadData Before Read 61412
handleReadData After Read 61412
handleReadData Before Read 93856
handleReadData After Read 93856
handleReadData Before Read 125122
handleReadData After Read 125122
handleReadData Before Read 157496
handleReadData After Read 157496
handleReadData Before Read 189876
handleReadData After Read 189876
handleReadData Before Read 222134
handleReadData After Read 222134
handleReadData Before Read 253308
handleReadData After Read 253308

below is read from QByteArray.

void AsrClient::handleReadData()
{
    qDebug() << "handleReadData Before Read" << buff.data().length();
    buff.read(buff.data().length());
//    QByteArray ba = buff.readAll();
    QByteArray ba = buff.data();
    ba.clear();
    qDebug() << "handleReadData After Read" << buff.data().length();


//    buff.buffer().remove(0,buff.buffer().length());
//    buff.readData(audioBuff,65536);
//    buff.data().data();
//    buff.seek(0);
//    buff.reset();
//    buff.readAll();
//    qDebug( "%x",buff.buffer().data()[0]);
}

the result is :

handleReadData Before Read 34436
handleReadData After Read 34436
handleReadData Before Read 67734
handleReadData After Read 67734
handleReadData Before Read 101112
handleReadData After Read 101112
handleReadData Before Read 134566
handleReadData After Read 134566

the same.

I have read some webpages and the qt creator help manual. I can not found the right way.

I almost sure that the bytearray will not be freed when the audioinput is running.

At least when I run on my Pc.
The QAudioInput QBuffer QByteArray combines a bull shit.
It makes me tired.
sign~

wzf999999

After I clear the buff, the length() of buff.data() will be 0 at soon.

But next get data from QAudioInput, the size of buff.data() will grow more . it looks like the clear operation is doesn't work.

it seems that QAudioInput has own buffer. And it will not clean.

So where I should read the data? The QBuffer? QByteArray? or QAudioInput? And How?

I need help!

JKSH

@wzf999999 said in How to get audio data from microphone via QAudioInput and QByteArray?:

it seems that QAudioInput has own buffer. And it will not clean.

So where I should read the data? The QBuffer? QByteArray? or QAudioInput? And How?

I need help!

There is an overload of QAudioInput::start() which returns an internal QIODevice as the buffer. You can read from that buffer directly. You don't need your own QBuffer.

I suggest you start by studying an example: https://doc.qt.io/qt-5/qtmultimedia-multimedia-audioinput-example.html (Search for "Audio Input Example" in Qt Creator)

wzf999999

@JKSH
OK, Thanks!
I noticed that the example was override the function writeData() in the class who derived from QIODevices.

Here I guess I need to create my own QIODevice and override the function writeData();

And maybe when I use QAudioOutput, I need to override the function readdate().

If it works I will close the topic.

wzf999999

This time I know that the QByteArray doesn't suitable to storage the big fast audio data with QBuffer.
If we want get the audio from QAudioInput, we need to create our own QIODevice class and override the function

qint64 writeData(const char * data, qint64 len);