How to transfer Audio data via UDP?

mikeh64

Dear all,
I want to design a LAN voice talk application, but it can't work properly, there is a lot of noise. I guess the problem is happening on onAudioInReadyRead() and onRxReadyRead(). I don't know how to fix them.

AudioUdp::AudioUdp(QObject *parent)
    : QObject{parent}
{
    initUdp();
    initAudioIn();
    initAudioOut();
    connect(m_audioInputIO,SIGNAL(readyRead()),this,SLOT(onAudioInReadyRead()));
    connect(m_socket,SIGNAL(readyRead()),this,SLOT(onRxReadyRead()));
}

void AudioUdp::initAudioIn()
{
    QAudioDevice m_audioInputDevice = QMediaDevices::defaultAudioInput();
    QAudioFormat audioFormat = m_audioInputDevice.preferredFormat();
    m_audioInput = new QAudioSource(m_audioInputDevice,audioFormat,this);
    m_audioInputIO = m_audioInput->start();
}

void AudioUdp::initAudioOut()
    {
        QAudioDevice m_audioOutputDevice = QMediaDevices::defaultAudioOutput();
        QAudioFormat audioFormat = m_audioOutputDevice.preferredFormat();
        m_audioOutput = new QAudioSink(m_audioOutputDevice,audioFormat,this);
        m_audioOutputIO = m_audioOutput->start();
    }

void AudioUdp::initUdp()
{
    m_address.setAddress("127.0.0.1");
    m_socket = new QUdpSocket(this);
    m_socket->bind(QHostAddress::AnyIPv4, 9000);
}

void AudioUdp::onAudioInReadyRead()
{
    QByteArray datagram = m_audioInputIO->readAll();
    m_socket->writeDatagram(datagram, m_address, 8000);
}

void AudioUdp::onRxReadyRead()
{
    QNetworkDatagram datagram = m_socket->receiveDatagram();
    m_audioOutputIO->write(datagram.data());
}

SGaist

Hi and welcome to devnet,

Are you sure your output is configured to match the input data ?
UDP has no data integrity guaranty so are you sure you are getting everything ?

mikeh64

@SGaist Hi. Thanks for your reply.
Yes, I am sure there is nothing wrong with the output and input device. I match the format via QAudioFormat audioFormat = m_audioInputDevice.preferredFormat();
I don't think it's related to the characteristics of UDP, I use the loopback address 127.0.0.1 to test. It might be the problem with how to configure the buffer of QIODevice. And I want to know how to handle network and audio data properly in the code.
Here is the part of header files of AudioUdp.cpp

    QAudioSource *m_audioInput;
    QIODevice *m_audioInputIO;
    QAudioSink *m_audioOutput;
    QIODevice *m_audioOutputIO;

mikeh64

@SGaist There are a lot of excellent Qt examples. I have reviewed the code of Audio Input Example, Audio Output Example, Multicast Receiver and Multicast Sender. Yes, multicast is also need to be used in my application. Unfortunately, I couldn't find any examples related to both Audio and Network.

SamiV123

You'd better served from the get go by using a better protocol and an existing implementation. Have you looked at WebRTC. This sort of use case is essentially what it was made for.

In fact unless you have a particular reason, you might want to consider doing this application is a web app in the browser?

SGaist

@mikeh64 Qt examples can't cover all the possible use cases people may find for creating applications.

As for the issue at hand, @SamiV123 raises a good point. There are already protocols and tools existing which might be preferable to use. For example GStreamer allows you to create a stream server with a minimal amount of code.

mikeh64

@SamiV123 Hi. Thanks for your suggestion.
I have heard of WebRTC, but it's not suitable for LAN scenario. Have you heard of ZeroTier? It provides virtual LAN services for people who own public address around the world. Actually, it uses UDP as the transport protocol.
I have set up a virtual LAN with many frineds, we play games that support LAN connection via it. But we lack a voice chat application. So I want to make a desktop or mobile app rather than a web app.

mikeh64

@SGaist Thanks for your recommendation.
GStreamer might be a good framework, but I think Qt's bulit-in framework is sufficient to implement the function, so I don't want to learn other tools temporarily.

SamiV123

@mikeh64 said in How to transfer Audio data via UDP?:

@SamiV123 Hi. Thanks for your suggestion.
I have heard of WebRTC, but it's not suitable for LAN scenario. Have you heard of ZeroTier? It provides virtual LAN services for people who own public address around the world. Actually, it uses UDP as the transport protocol.
I have set up a virtual LAN with many frineds, we play games that support LAN connection via it. But we lack a voice chat application. So I want to make a desktop or mobile app rather than a web app.

No, I haven't heard of it. But I don't see why that would be a blocker for using WebRTC. If you can send raw UDP packets yourself then you can also send UDP packets via WebRTC.

mikeh64

@SamiV123
To be honest, I don't know how to write WebRTC code. LOL.

I want to design a cross platform applications. Unfortunately, web application have its limitations, for example, it's inconvenient to run on Xbox or PS5. I need to keep it connected and running.

I spent these days researching how to fix the code. And, it's working! I plan to put the code on Github later. Here is my answer:

private:
    void speaking();
    QByteArray m_audioOutputBuffer;
private slots:
    void onAudioInReadyRead();
    void onRxReadyRead();

void AudioUdp::onAudioInReadyRead()
{
    static const qint64 bufferSize = 4096;
    const qint64 len = qMin(m_audioInput->bytesAvailable(), bufferSize);
    QByteArray buffer(len, 0);
    qint64 l = m_audioInputIO->read(buffer.data(), len);
    //qDebug() << "data size" << datagram.size();
    if (l > 0){
        m_socket->writeDatagram(buffer, h_address, 8000);
    }
}

void AudioUdp::onRxReadyRead()
{
    QNetworkDatagram datagram = m_socket->receiveDatagram();
    m_audioOutputBuffer += datagram.data();
    if (!m_audioOutputBuffer.isEmpty()){
        speaking();
    }
}

void AudioUdp::speaking(){
    qint64 len = m_audioOutput->bytesFree();
    if (len){
        m_audioOutputIO->write(m_audioOutputBuffer, len);
        m_audioOutputBuffer.remove(0, len);
    }
}

SamiV123

And you expect to be able to run a Qt based app on Xbox or PS 5 ?

mikeh64

@SamiV123
I gave a inappropriate example. Actually, they all have official comprehensive voice chat tools, unless you are concerned about the information security. By the way, Qt supports UWP development.
A suitable example would be iOS platform, its tombstoning freezes the background process. So web application is not a choice if I need to keep it connected and running.