[QAudioOutput] Internal working system ?
-
Hi !
First, sorry if my english is not that good, i accept any remark to improve it !![Optional to read] My situation : Im developping a centralized vocal chat. It doesnt communicate with UDP sockets, to prevent clients to be able to get other client's IP addresses. I'd like to distribute it at first in video games communities, and many people in competitive games try to find peers' IP to DoS them in future games. All audio data pass through a server with TCP sockets, then are redistributed to all clients (IP addresses are only available on the server, so players are safe).
The first problem was this. Each client had only one QAudioOutput for all clients. When more than 2 clients were communicating, sounds received were not mixed but just "concatenated". So the final sound was jerky. To solve this problem, I set up an ID system for each client. For each one, there is a QAudioOutput, so the received sounds dont have to mix together. Thats my "audio channels" solution.
But, even after doing this, the final sound (coming from multiple QAudioOutput) was still jerky. By decreasing the sound quality, 3 clients communicating is ok, but 4 is super jerky.
So i need to know something before I begin again from scratch : can we play multiple QAudioOuput simultaneously without threading them ? How the QAudioOutput works inside ? Should i set up a socket for each peer because of the slots' queued connection ? (Maybe the sounds received are accumulating in the socket and they are not analyzed fast enough ? Kinda strange, my PC is beast and it supposed to work on bad laptops...)
Sorry if Im not clear on some points, tell me if you need me to explain again some things !! I dont post code because its pretty dirty at the moment, so i just ask if someone knows how the QAudioOutput properly works. Couldnt find much informations in the documentation, I feel like all sound related doc is pretty empty..
Thank's so much for reading !
-
Hi
I'm a big fan of Qt but in the end I gave up with QAudioOutput. My app wasn't as complicated as yours sounds, I'm picking up audio data from a remote Android TCP/IP Client.
The problems I had were various and I'm afraid this post may cause more confusion rather than helping. My code ran ok. for about five minutes then the sound gradually degraded (buffer underruns) until eventually it was just noise. Although the audio output stuff was disconnected from the main GUI (it was a separate thread) there was a definite dependency on the GUI thread (moving the window caused the output thread to stop). The QAudioInput thread I had was an analogue of the output thread, it worked fine and was not affected by moving the window.
I followed all the advice I could find and tried it in both Windows and Linux, no luck.
I concluded there were bugs, gave up and re-wrote it in Java, the remote Android end stayed the same throughout. Everything's fine now.
I should be more helpful and give a proper bug report on this but my time is limited.
John
-
Hi
Well... Cant believe your story. I understand now why there is so few results to all my research about QAudioOutput, mixing, multiple channels, etc... Thank you for this. This is not confusing, I've already heard that the low level audio framework in Qt was tricky, some people just said "it's bad". I guess i have a concrete example now.
Im gonna try to set up a unique socket for each client. Maybe i can try this before restarting from zero with an other framework, since I dont know jack about Java.
Do you have some C++ references for a good audio output framework with channel mixing ? Which is implemented in C++ and not in C (like PortAudio is, for example..).
Thank's again. If someone already lived this audio channel problem, don't hesitate to talk about it and how you resolved the problem, even if it's by using an other framework / langage !
Thank's for reading.
NdFeB
-
This is first time I've done any digital audio work so I can't help any further I'm afraid. Both the Qt and the Java implementations amounted to nothing more than using the example code for setting up and writing to the audio output instance. Qt didn't work, Java did. Sorry and good luck.
If you've got C++ experience then Java isn't that different. It's conceptually a bit weird, particularly leaving stuff for the garbage collector to sort out. But it's a very quick way of throwing complicated GUIs together and of course it's instantly cross-platform except for Android ironically. Am I off topic?
John
-
Alright thank's. No you are not off topic, you just sold me out Java ahah ! I will check it out. Maybe i can insert a Java plugin made with Qt Jambi in a C++ Qt application ? And if i understand you first post, i can make a full Java app communicating with a C++ made server ? (as long as the packets have exactly the same architecture).
I wanted to learn xhtml next... Well, it will wait a bit more !
I dont have the time now, but if my attempt to communicate with unique sockets for each different client turns to work out, i will post it here ! Pretty sure someone can find this thread by searching on google, even if I feel like nobody uses QAudioOutput (surely for the same reason as you).
Thank's again John !
NdFeB
-
Hi,
The QtMultimedia module is currently not about advanced audio processing (e.g. multiplexing/channel mixing), it's more a high-level module. If you want to stay in the C++ world. gstreamer might be more suited to your needs.
-
@NdFeB
A language is a means, nothing more. I don't see what Java will give you that C++ doesn't. That said, I'm pretty sure you can have only oneQAudioOutputactive at any one time because of the notes given here. Bear in mind that IO operations are inherently serial, so I'd somewhat expect such a behavior. Also this would explain the jerkiness of your sound, when it comes from different sources, as multipleQAudioOutputinstances would be "fighting" for the same device, and writes would be serialized. Additionally, if I'm right, this would defeat the purpose of putting multipleQAudioOutputinstances in different threads. I would suggest implementing your own mixing (you could probably use QAudioDecoder for that purpose) and writing to the device from a single place. Also you might want to consider some thread brokering in your server application, since the throughput you'd need to handle may get quite large. As a note, the mixing part can be done in the server part, to reduce the amount of data transferred between the server and the clients. -
Hi there !
Thank you both for your answers !
@SGaist Thank you for the reference, i will check it out for sure after trying my multiple sockets solution !
@kshegunov Yes you are right, i should just keep trying to solve this in C++. I also got a look at the Audio QML type, which offer a pretty simple way to play sound.
I also tried to play simultaneous sounds from differents .wav files with multiple QAudioOutput, without threading anything. I was able to play up to 6 sounds, there were no jerkiness at all. I didn't try more because the final sound was just turning horrible, but playing them didn't seem to be a problem.
About the "fighting" issue for the same device : at the moment, my application (the jerky one) has one socket which receive all sounds. But the multiple QAudioOutput don't read sounds from it. The socket is read, get the sender ID, and gives the sound as a QByteArray to the associated peer audio system. At this point, the QByteArray is written (with append()) in my internal QByteArray, which is used as the buffer of a QBuffer. This QBuffer is the source device for the QAudioOutput. So every single QAudioOutput has its own buffer.
I think my problem is here : maybe the socket's readyRead() slot queue is overloaded, leading to latencies. I asked some people if QTcpSocket was internaly threaded. They told me it was, but the processing of the connected slot was not because processed in the main thread.
My next move consist to make a new QTcpSocket for each connected peer, with my slot connected to it processing in a different thread, so maybe i can get a low latency, resulting in a fluid sound.
Also, the server is made like this : a QTcpServer is listening to a port. When it has a new connection, it immediately sends a list of "sub servers" to the client. Those sub servers represent chat rooms. Each room listens to a different port, and is limited to 5 or 6 (i don't know yet) simultaneous connections. So each "vocal server" shouldn't be overloaded.
To finish, i don't want to make the mixing part on the server. I want to receive the sounds separately on the client, then move it on different channels. So the user can control every channel independantly as he want to, for the volume control for example. This way, if Mister X doesn't know how to reduce his microphone volume, then Mister Y can reduce the sound coming from him :-)
Thanks again for your answers, helps me a lot in my progress !! So good to be able to talk about it and to get external ideas about the problem's source :-)
-
Hello,
TheQAudioOutputclass is probably not reentrant (it's not mentioned in the docs), so if that's the case you won't be able to use its instances from different threads. My guess is that the "jerkiness" comes from appending the buffer(s) from the different channels when they're sent to the sound device. You really should, in that case, implement your own channel mixing. You could still do it on the server, assuming that you'll be willing to write some code for session management, still it's up to you (the program designer) to decide how the application should work. -
@kshegunov indeed, unless stated otherwise the class are not reentrant.
@NdFeB If you would like to do it in the client side, may be DSPFilters might be of some help
-
NdFeB is right the language is irrelevant. I choose Java because it was the line of least resistance; it has a high level audio interface, I'm familiar with it, and it's cross-platform, but I could have used anything. I had hoped to use Qt so that I could eventually implement this on a tablet however if I am to do this I will have to rewrite it again using Android Java.
Point is, I couldn't get a single-threaded, single-channel implementation of QAudioOutput to work. I appreciate that this might be ignorance but i used all the examples I could find in both Windows and Linux over several days and still it behaved in exactly the way described above. The code was less than a hundred lines and wasn't exactly a conceptual challenge. I used a broadly similar implementation in Java and it just worked. I dare say if I'd chosen gStreamer or whatever Visual Studio offers it would have worked too.
