Streaming audio using custom IODevice (com-port) - problem with delays

[nexx124]

Good afternoon. I ask for your help. Before I start - please excuse me for my English, I'm not good at it, but I'll try to write correctly. The task is to read the data (sound) through the device from the com-port and transfer it to the computer speakers.
Development is carried out using Qt. In order to output audio, I inherited from QIODevice, redefined the methods readData and writeData. Then I used the QAudioOutput class to output data to the computer speakers.
The Generator class acts as an intermediary between the com port and the audio output class. In the AudioClass class, there is a getData method that is called in the external program when the required amount of data arrives.
Actually the problem itself - the sound goes with a periodic delay. By long clarification, I realized that when executing the code in the function Generator :: readData (char * data, qint64 len) it happens that the condition !m_buffer.isEmpty () is not executed, and the data for the output to the speakers is fulled with zeros - that's why I have the pause in streaming sound. I think this problem can be corrected if I call readData only after enough data has been accumulated in the buffer, but this function is called by itself (I obviously do not call it in the code by myself), and I do not know how to disable calling this function. I ask for help.
It will be very good if someone can give an example of how to correctly read the sound from the device in com port and transmit it to the computer speakers without delays. Below is the code
audiooutput.h

#ifndef AUDIOOUTPUT_H
#define AUDIOOUTPUT_H
 
#include <math.h>
 
#include <QAudioOutput>
#include <QByteArray>
#include <QComboBox>
#include <QIODevice>
#include <QLabel>
#include <QMainWindow>
#include <QObject>
#include <QPushButton>
#include <QSlider>
#include <QTimer>
 
class Generator : public QIODevice
{
    Q_OBJECT
 
public:
    Generator(const QAudioFormat &format, qint64 durationUs, int sampleRate, QObject *parent);
    ~Generator();
 
    void start();
    void stop();
 
    qint64 readData(char *data, qint64 maxlen);
    qint64 writeData(const char *data, qint64 len);
    qint64 bytesAvailable() const;
 
//private:
    void generateData(const QAudioFormat &format, qint64 durationUs, int sampleRate);
 
private:
    qint64 m_pos;
    QByteArray m_buffer;
    //QByteArray test_buffer;
};
 
class AudioTest : QObject
{
    Q_OBJECT
 
public:
    AudioTest();
    void getData(QByteArray * data);
    void stop();
    ~AudioTest();
 
private:
    void initializeAudio();
    void createAudioOutput();
 
private:
    QTimer *m_pushTimer;
 
    QAudioDeviceInfo m_device;
    Generator *m_generator;
    QAudioOutput *m_audioOutput;
    QIODevice *m_output; // not owned
    QAudioFormat m_format;
 
    bool m_pullMode;
    QByteArray audio_buffer;
    //QByteArray test_buffer;
 
private slots:
    void pushTimerExpired();
    void toggleSuspendResume();
};
 
#endif // AUDIOOUTPUT_H

audiooutput.cpp

#include <QAudioDeviceInfo>
#include <QAudioOutput>
#include <QAudioDecoder>
#include <QDebug>
#include <QVBoxLayout>
#include <QMediaPlayer>
#include <qmath.h>
#include <qendian.h>
 
#include "audiooutput.h"
 
#define SUSPEND_LABEL   "Suspend playback"
#define RESUME_LABEL    "Resume playback"
 
const int DurationSeconds = 2;
const int ToneSampleRateHz = 600;
const int DataSampleRateHz = 8000;
const int BufferSize      = 16384;
 
QAudioFormat m_format;
qint64 m_durationUs;
int m_sampleRate;
 
Generator::Generator(const QAudioFormat &format,
                     qint64 durationUs,
                     int sampleRate,
                     QObject *parent)
    :   QIODevice(parent)
    ,   m_pos(0)
    ,   m_buffer(BufferSize + 100, 0)
{
 
    m_format = format;
    //m_durationUs = durationUs;
    //m_sampleRate = sampleRate;
   /* if (format.isValid())
        generateData(format, durationUs, sampleRate);*/
}
 
Generator::~Generator()
{
 
}
 
void Generator::start()
{
    open(QIODevice::ReadWrite);
}
 
void Generator::stop()
{
    m_pos = 0;
    //close();
}
 
qint64 Generator::readData(char *data, qint64 len)
{
    if (!m_buffer.isEmpty()) {
        memset(data, 0, len);
        if (m_buffer.size() < len) {
            len = m_buffer.size();
        }
        if (len > 0) {
            memcpy(data, m_buffer.left(len).data(), len);
            m_buffer.remove(0, len);
        }
    } else {
        memset(data, 0, len);
    }
    return len;
}
 
qint64 Generator::writeData(const char *data, qint64 len)
{
    qDebug() << "writeData" << m_buffer.length();
    m_buffer.append(data, len);
    qDebug() << "writeData finish" << m_buffer.length();
    return len;
}
 
qint64 Generator::bytesAvailable() const
{
    return m_buffer.size() + QIODevice::bytesAvailable();
}
 
AudioTest::AudioTest()
    :   m_pushTimer(new QTimer(this))
    ,   m_device(QAudioDeviceInfo::defaultOutputDevice())
    ,   m_generator(0)
    ,   m_audioOutput(0)
    ,   m_output(0)
    ,   m_pullMode(true)
    //,   audio_buffer(/*BufferSize, */0)
{
    initializeAudio();
}
 
void AudioTest::initializeAudio()
{
    m_format.setSampleRate(DataSampleRateHz);
    m_format.setChannelCount(1);
    m_format.setSampleSize(16);
    m_format.setCodec("audio/pcm");
    m_format.setByteOrder(QAudioFormat::LittleEndian);
    m_format.setSampleType(QAudioFormat::SignedInt);
 
    QAudioDeviceInfo info(m_device);
    if (!info.isFormatSupported(m_format)) {
        qWarning() << "Default format not supported - trying to use nearest";
        m_format = info.nearestFormat(m_format);
    }
 
    if (m_generator)
        delete m_generator;
    m_generator = new Generator(m_format, DurationSeconds*1000000, ToneSampleRateHz, this);
 
    createAudioOutput();
}
 
void AudioTest::getData(QByteArray * data)
{
    m_generator->writeData(data->data(), data->length());
    m_audioOutput->resume();
}
 
void AudioTest::createAudioOutput()
{
    delete m_audioOutput;
    m_audioOutput = 0;
    m_audioOutput = new QAudioOutput(m_device, m_format, this);
    m_audioOutput->setVolume(qreal(0.5f));
    m_audioOutput->setBufferSize(BufferSize);
    m_audioOutput->suspend();
    m_generator->start();
    m_audioOutput->start(m_generator);
}
 
AudioTest::~AudioTest()
{
 
}
 
void AudioTest::stop()
{
    m_generator->close();
    m_audioOutput->suspend();
}

[SGaist]

Hi and welcome to devnet,

Why not use QSerialPort for your com-port communication ?

[nexx124]

@SGaist thanks for your reply.
I use QSeralPort for reading and transferring data, but before the data gets to the computer speakers it is processed some functions (decryption, parsing, etc.) therefore it is necessary to use AudioClass. I can not output data directly to the speakers after I get them from the device:(

[SGaist]

Still, why do you need AudioClass Generator to be a QIODevice if its goal is to do some processing on the audio data ?

Shouldn't you have something like:

1. Controller with:
2. QSerialPort
3. AudioProcessor
4. QAudioOutput

With your Controller that stores some amount of received data from the QSerialPort, when enough has arrived make it pass through your AudioProcessor that can emit a new buffer of audio data that you will send to QAudioOutput ?

[nexx124]

@SGaist AudioClass does not inherit the QIODevice. Class Generator do it. There is also a class that inherits QSeralPort, which receives data from the port, processes it and sends it to AudioClass. If I write something like AudioClass-> start(QSeralPort), then the data will directly go to the computer's speakers, and processing will not be performed. Therefore, such a complex scheme was built: QSerialPort receives data from the device, processes it, then sends to the class Generator, which is being listened by AudioClass.

[nexx124]

@SGaist But do u know on what system or Qt event the readData function called? This is overridden function inherited from QIODevice in class Generator

[SGaist]

One case where readData is called is for example when you call readAll.

In any case, I still think you are a bit over-engineering here. Connect the readyRead signal from your QSerialPort to a slot of AudioProcessor, once you're done with processing emit a new signal with your processed audio data and that signal should be connected to the write slot of your QAudioOutput object.