how to forward captured audio over UDP/RTP

johnco3

I have a QT 6 multimedia application that captures audio from the microphone and forwards the audio over UDP (fixed length datagrams defined by the MTU).

I need some help with how to design an intermediary circular buffer or some equivalent approach that I can use to forward these captured datagrams (in chunks of MTU) as they arrive. The main thread will be filling this array while the timer thread is draining it. The crude array approach I am thinking about is pretty crude and I don't know if it will work. Also I would like to convert the PCM 16 to PCM32 before sending out over UDP - this latter part would require a pretty simple conversion - not sure if QT has some low latency converters out of the box. I searched but could not find any.

I always see fixed 7056 byte blocks of audio data containing pcm16 - I need to forward these samples in 1500 byte UDP chunks to an RTP server application on the local network. The commented out block of code attempts to fill an array of fixed length MTU sized datagrams contained in mTXBufferList and these are transmitted using the timer callback (also shown below). I would appreciate any advice on how to do this better.

void
RtpWorker::handleAudioCaptured(const QAudioBuffer& rAudioBuffer)
{
    // log the capture event every 50 packets
    static int counter = 0;
    if (++counter % 3 == 0) {
        Q_EMIT audioSampleCaptured(rAudioBuffer);
        Q_EMIT addLogEntry(level::info, std::format(
            "captured {} bytes", rAudioBuffer.byteCount()).c_str());
    }
    // append captured audio data to the capture buffer
    mpCaptureBuffer->append(
        rAudioBuffer.constData<const char>(),
        rAudioBuffer.byteCount());

    mTXCounter = 0;
    mTXBufferList.clear();
    const auto headerLen = gHeaderLenLambda(*mpHeader);
    if (headerLen >= sizeof(rtp_hdr_t)) {
        // copy construct the existing header
        const auto rawHeader = std::make_unique<QByteArray>(*mpHeader);
        const auto rtpHeader = reinterpret_cast<rtp_hdr_t*>(rawHeader->data());
        auto seqNum = rtpHeader->getSeqNum();
        auto timeStamp = rtpHeader->getTimestamp();
        //for (const auto& next : rAudioData) {
        //    // set the marker indicating a significant boundary condition
        //    if (mTXBufferList.empty()) {
        //        rtpHeader->m = 1u;
        //    }
        //    else {
        //        rtpHeader->m = 0u;
        //    }
        //    rtpHeader->setSeqNum(seqNum);
        //    rtpHeader->setTimestamp(timeStamp);
        //    // each datagram consists of an rtp_hdr_t (12 bytes +
        //    // optional extensions enclosed via headerLen) followed by
        //    // the raw audio data.
        //    QByteArrayList temp = {
        //        *rawHeader,
        //        next
        //    };
        //    mTXBufferList.emplace_back(temp.join());
        //    seqNum++;
        //    timeStamp += mTSIncrement;
        //}
    }
}

Here is the timer callback that transmits the data over UDP.

/**
 * Timer callback event handler.
 *
 * @param event [in] timer event.
 */
void
RtpWorker::timerEvent(QTimerEvent* event)
{
    // send all mTXData datagrams
    if (event->timerId() == mpTimer->timerId()) {
        // blast out the entire block queued up datagrams at once
        for (auto& next : mTXBufferList) {
            mpSocket->writeDatagram(
                next, mTargetIP, mTargetPort);
        }
        mTXCounter += mTXBufferList.size();
        // TODO change to use a sent packet stats counter
        const auto lastTxHeader = reinterpret_cast<rtp_hdr_t*>(
            mTXBufferList.back().data());
        const auto lastSentSeq = lastTxHeader->getSeqNum();
        const auto lastSentTS = lastTxHeader->getTimestamp();
        // update all rtp headers in the mTXBufferList
        for (auto i = 0u; i < mTXBufferList.size(); ++i) {
            // update each header in the mTXBufferList
            // making sure to increment the timestamp, sequence number and
            // reset the marker bit as it has just been transmitted
            const auto rtpHeader = reinterpret_cast<
                rtp_hdr_t*>(mTXBufferList[i].data());
            // increment the sequence numbers
            rtpHeader->setSeqNum(lastSentSeq + i + 1);
            // increment timestamp
            rtpHeader->setTimestamp(lastSentTS + (i + 1) * mTSIncrement);
            // reset the marker bit so it doesn't keep getting transmitted
            rtpHeader->m = 0u;
        }
    } else { // pass unhandled timer event up the chain
        QObject::timerEvent(event);
    }
}