Unsolved VoIP QTcpSoket(audio streaming)
-
-
@raven-worx Actually nothing is crashing, just output audio device do not produce any sound. But in a log I see that Client trying to send some data just after start streaming call.
-
does the output device ever produce any sound?
I'm not sure QAudioOutput is as smart as you want it to be. When you call
audio->start(audioclientConnection );
you would like the audio to respond toreadyRead()
signals and stream what it can read but I think everythingstart
does is read everything that is already available and stream it out (Think as if the input was a file) -
@VRonin I rewrite server and client sides. With readyRead approach using QBuffers and QByte Arrays The server looks as follows:
MainDialog.h class MainDialog: public QWidget { ... private: ... QByteArray byte_buffer; QBuffer m_audioOutputIODevice; } MainDialog.cpp MainDialog::MainDialog(QWidget *parent) : QDialog(parent), ui(new Ui::MainDialog), host_index(0), port(50306) { ... audio->setBufferSize(512000); audio->setNotifyInterval(10); qDebug()<<"Real buffer size "<<audio->bufferSize(); qDebug()<<"NotifyInterval "<<audio->notifyInterval(); byte_buffer.clear(); m_audioOutputIODevice.close(); m_audioOutputIODevice.setBuffer(&byte_buffer); m_audioOutputIODevice.open(QIODevice::ReadOnly); } void MainDialog::AudioStream() { qDebug()<<"In audio stream"; audioclientConnection = audioserver->nextPendingConnection(); connect(audioclientConnection,SIGNAL(readyRead()), this,SLOT(ReadStream())); audio -> start(&m_audioOutputIODevice); } void MainDialog::ReadStream() { qDebug()<<"ReadAudio"; byte_buffer.append(audioclientConnection->readAll()); }
Client side
AudioClient.h class AudioClient : public QObject { ... private: ... QBuffer m_audioInputIODevice; QByteArray byte_buffer; } AudioClient.cpp AudioClient::AudioClient (): hostName("127.0.0.1"), port(50306) { ... qDebug()<<"Created audio input"; audio = new QAudioInput(format, this); audio->setBufferSize(512000); byte_buffer.clear(); m_audioInputIODevice.close(); m_audioInputIODevice.setBuffer(&byte_buffer); m_audioInputIODevice.open(QIODevice::WriteOnly); connect(audio,SIGNAL(stateChanged(QAudio::State)), SLOT(handleStateChanged(QAudio::State))); } void AudioClient::StartStream() { qDebug()<<"Connecting to the host Audio"; socket->connectToHost(hostName, port); if (!socket->waitForConnected(Timeout)) { emit error(socket->error(), socket->errorString()); return; } qDebug()<<" "; qDebug()<<"Audio stream from thread "<<QThread::currentThreadId(); qDebug()<<" "; audio -> start(&m_audioInputIODevice); } void AudioClient::handleStateChanged(QAudio::State newState) { int bytes_ava = 0; switch (newState) { case QAudio::StoppedState: ... break; case QAudio::ActiveState: qDebug()<<"Start recording "; bytes_ava = audio->bytesReady(); qDebug()<<"Data ready"<<bytes_ava ; if(bytes_ava == 0) { qDebug()<<"Nothing to send"; } else { qDebug()<<"Writing to socket"; socket->write(byte_buffer); } break; case QAudio::SuspendedState : ... break; case QAudio::IdleState: qDebug()<<"IdleState streamin"; .... break; } }
I doubt that I understood it correctly. But now the problem is that on client side program goes to handleStateChanged, check that nothing to send and never sends anything to server.
-
handleStateChanged is called when the audio input device becomes available and then never more.
this part:bytes_ava = audio->bytesReady(); qDebug()<<"Data ready"<<bytes_ava ; if(bytes_ava == 0) { qDebug()<<"Nothing to send"; } else { qDebug()<<"Writing to socket"; socket->write(byte_buffer); }
should be put in a slot connected to
QAudioInput::notify()
-
@VRonin Thank you for help. From qDebug I see that client write data to soket and send it. But on the server side still no sound, also buffer size is differ from the client side.
void MainDialog::ReadStream() { qDebug()<<"Read Audio"; byte_buffer.append(audioclientConnection->readAll()); qDebug()<<"Size of Byte_buffer "<<byte_buffer.size(); qDebug()<<"Size of Buffer "<<m_audioOutputIODevice.size(); }
I connect function to notify of AudioOutput and it seems that server never starts to process audio.
void MainDialog::ReadNotify() { qDebug()<<"Here in Notify Output"; qDebug()<<"Notify Byte_buffer "<<byte_buffer.size(); }
-
The problem is in the client, not the server, the
case QAudio::ActiveState:
section should respond to theaudio,&QAudioInput::notify
signalalso if you do not clean
byte_buffer
but keep appending you will send over all the audio from the start of the recording every time -
I rewrote the client side as you said. And now it looks:
void AudioClient::handleStateChanged(QAudio::State newState) { switch (newState) { ... case QAudio::ActiveState: // Started recording - read from IO device qDebug()<<"Device active "; connect(audio,SIGNAL(notify()), SLOT(SendData())); break; case QAudio::SuspendedState : qDebug()<<"SuspendedState"; // ... other cases as appropriate; break; ... } } void AudioClient::SendData() { qDebug()<<"Send Data "; int bytes_ava = byte_buffer.size(); qDebug()<<"Data ready in notify "<<bytes_ava ; if(bytes_ava == 0) { qDebug()<<"Nothing to send"; //audio->resume(); } else { qDebug()<<"Writing to socket"; socket -> write(byte_buffer); byte_buffer.clear(); } }
And on server side
MainDialog.cpp MainDialog::MainDialog(QWidget *parent) : QDialog(parent), ui(new Ui::MainDialog), host_index(0), port(50306) { ... connect(audio,SIGNAL(notify()), SLOT(ReadNotify())); } void MainDialog::AudioStream() { qDebug()<<"In audio stream"; audioclientConnection = audioserver->nextPendingConnection(); audio -> start(&m_audioOutputIODevice); } void MainDialog::ReadStream() { qDebug()<<"Read Audio"; byte_buffer = audioclientConnection->readAll(); qDebug()<<"Size of Byte_buffer "<<byte_buffer.size(); qDebug()<<"Size of Buffer "<<m_audioOutputIODevice.size(); }
On the server it is never visit ReadNotify.
-
as i know, if you using TCP, there will have delay on network, so, your output buffer of audio will be drain to empty, so on ReadNotify(), you need check buffer of output audio, if it empty, you should generate some default data for it
i think it better to using UDP, and using Opus codec to encode, decode & generate default data on case UDP lost package.
i find an example about it, can you check
https://github.com/antonypro/AudioStreaming -
@kd_wala Thank you for example. I build it and run on my laptop, and audio start to repeat sounds after some time. Like playing from the begining on the background. Also on the first look example looks rather complicated. Is there way of doing it through AudioInput and AudioOutput through the start method?
-
"Is there way of doing it through AudioInput and AudioOutput through the start method?"
I thinks , should not, because, when you have transfer over internet, you will have some delay, package lost(on udp), encode, decode ... so you must handle this before you put data to output ...
I think this example not complicated,