Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Is Qt capable of small-buffer low-latency audio applications (e.g. soft synth)?

Is Qt capable of small-buffer low-latency audio applications (e.g. soft synth)?

Scheduled Pinned Locked Moved Unsolved General and Desktop
qtmultimediaaudio
15 Posts 4 Posters 8.0k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P Offline
    P Offline
    paulmasri
    wrote on 27 Nov 2015, 15:03 last edited by paulmasri
    #1

    I have been playing with / hacking the Spectrum example to create my own soft-synth. I have hit an issue that the audio glitches if I put the buffer size below about 5000 samples (125ms, at 44.1kHz sample rate). This is about 20x the length of buffer I want to achieve.

    I posted a question on StackOverflow and 2 respondents basically said that Qt is the wrong tool for low latency audio, even if I were to make the app multithreaded. I'd be grateful for some genuine insight from Qt developers and the Qt team.

    (NB: I'm not interested in 'my platform is better than yours' responses - I want to genuinely know if Qt is the right tool for the job here, from people who know Qt and understand non-blocking, high priority, low latency programming. I already love Qt for its QML + JavaScript GUI building side.)

    1 Reply Last reply
    0
    • S Offline
      S Offline
      SGaist
      Lifetime Qt Champion
      wrote on 27 Nov 2015, 22:20 last edited by SGaist
      #2

      Hi,

      If you want to reach Qt developers you should rather post this question on the interest mailing list. You'll find there Qt's developers/maintainers (this forum is more user oriented)

      As for your original question, low latency is currently not in available with Qt but that doesn't mean you can't write your application with Qt. You can use a framework like PortAudio to handle the audio part of your application and write the rest with Qt (they play well together)

      Hope it helps

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      P 1 Reply Last reply 30 Nov 2015, 18:32
      2
      • S SGaist
        27 Nov 2015, 22:20

        Hi,

        If you want to reach Qt developers you should rather post this question on the interest mailing list. You'll find there Qt's developers/maintainers (this forum is more user oriented)

        As for your original question, low latency is currently not in available with Qt but that doesn't mean you can't write your application with Qt. You can use a framework like PortAudio to handle the audio part of your application and write the rest with Qt (they play well together)

        Hope it helps

        P Offline
        P Offline
        paulmasri
        wrote on 30 Nov 2015, 18:32 last edited by
        #3

        @SGaist Thanks for the heads up about the mailing list.

        Your comments about wrapping PortAudio echo the response on StackOverflow (suggesting the same thing but with RTAudio). It's useful to hear this confirmed.

        1 Reply Last reply
        0
        • S Offline
          S Offline
          SeeLook
          wrote on 30 Nov 2015, 20:21 last edited by
          #4

          Hi
          I also may confirm well cooperation Qt and RtAudio.
          I give up from QtMultimedia and with RtAudio I was able to get 64 frames buffer and callback in RtAudio thread where Qt signal was emitted.
          All together works flawlessly.
          Good luck!

          1 Reply Last reply
          2
          • P Offline
            P Offline
            paulmasri
            wrote on 1 Dec 2015, 10:06 last edited by
            #5

            Hi. Thank you both for your responses. I know that the audio will need to be kept in a separate high priority thread but I can't see how to integrate RTAudio/PortAudio with Qt and with this threading.

            Could you give some advice? Or is there sample code I could look at?

            Many thanks,
            Paul.

            1 Reply Last reply
            0
            • S Offline
              S Offline
              SeeLook
              wrote on 1 Dec 2015, 11:13 last edited by SeeLook 12 Jan 2015, 11:45
              #6

              I will try, feel free to ask if I'm not clear enough.

              I suggest to use RtAudio - it works over PulseAudio/JACK/ALSA and Windows/Mac audio as well

              First of all try some simple RtAudio example from their site: https://www.music.mcgill.ca/~gary/rtaudio/

              Input audio data You will get through implementing call back function.
              This is static method, so to be able do something in your Qt class from there, create static instance member in your class:

              class MyAudioInput public QObject
              {
              Q_OBJECT
                MyAudioInput()
                {
                  m_instance = this;
                  m_buffer1 = new float[64];
                  m_buffer2 = new float[64];
                  m_currentBuffer = m_buffer1;
                  m_readyBuffer = 0;
                  m_posInBuffer = 0;
                  m_thread = new QThread();
                  moveToThread(m_thread);
                  connect(m_thread, &QThread::started, this, &MyAudioInput::process);
                }
              ~MyAudioInput() 
              {
                delete m_buffer1;
                delete m_buffer2;
                m_instance = 0;
                delete m_thread;
              }
              
              static MyAudioInput* instance() { return m_instance;}
              
              static int inputCallBack(void*, void *inBuffer, unsigned int nBufferFrames, double, RtAudioStreamStatus status, void*) {
                for(int i = 0; i < nBufferFrames; i++;) {
                  // copy audio data to current buffer
                  *(instance()->m_currentBuffer + instance()->m_posInBuffer) = *(inBuffer + i);
                  instance()->m_posInBuffer++;
                  if (instance()->m_posInBuffer == 64 { // switch buffers when full
                    instance()->m_posInBuffer = 0;
                    if (instance()->m_currentBuffer == instance()->m_buffer1) {
                      instance()->m_currentBuffer = instance()->m_buffer2;
                      instance()->m_readyBuffer = instance()->m_buffer1;
                    } else {
                      instance()->m_currentBuffer = instance()->m_buffer1;
                      instance()->m_readyBuffer = instance()->m_buffer2;
                    }
                  // start processing in separate thread
                  instance()->m_thread->start();
                  }
                }
              }
              
              void process() {
                // do something with m_readyBuffer here
                // or/and emit dataReady() to inform the rest of Your app
              }
              
              
              signals:
                void dataReady();
              
              private:
                static MyAudioInput* m_instance;
                float *m_buffer1, *m_buffer2;
                float *m_currentBuffer, *m_bufferReady;
                int m_posInBuffer;
              
                QThread *m_thread; // processing thread;
              }
              

              Then You may open RtAudio stream like this:

                RtAudio adc;
                RtAudio::StreamParameters parameters;
                parameters.deviceId = adc.getDefaultInputDevice();
                parameters.nChannels = 2;
                parameters.firstChannel = 0;
                unsigned int sampleRate = 44100;
                unsigned int bufferFrames = 64; // 64 sample frames
                try {
                  adc.openStream( NULL, &parameters, RTAUDIO_FLOAT32,
                                  sampleRate, &bufferFrames, &MyAudioInput::inputCallBack );
                  adc.startStream();
                }
              

              When signal dataReady() will be handled, m_readyBuffer points to data for processing and code on it has to be performed in 2ms, so probably it is better to have bigger buffer size or more buffers. I'm using 512 samples.
              But to get this working You have to add to this example some "real" things.

              1 Reply Last reply
              2
              • P Offline
                P Offline
                paulmasri
                wrote on 2 Dec 2015, 08:25 last edited by
                #7

                Thank you! That's an awesome help. I shall take a look at the RTAudio examples and play with the sample code you've posted.

                1 Reply Last reply
                0
                • S Offline
                  S Offline
                  SGaist
                  Lifetime Qt Champion
                  wrote on 2 Dec 2015, 09:03 last edited by
                  #8

                  Depending on your application design, you might want to consider implementing a custom QIODevice for easier integration with Qt's API (see QAudioInput/QAudioOutput)

                  Interested in AI ? www.idiap.ch
                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                  1 Reply Last reply
                  0
                  • S Offline
                    S Offline
                    SeeLook
                    wrote on 2 Dec 2015, 10:01 last edited by
                    #9

                    In my experience with QtMultimedia I found it a bit unpredictable.
                    If one set audio buffer QAudioInput::setBufferSize(someValue) it will never be set to desired value. And value forced by QtMultimedia is quite big as mentioned in the first post.

                    When one get this value after device opening QAudioInput::bufferSize() it is good for nothing - when one expecting to obtain the 'buffer size' of data in readyRead() signal.
                    Usually every emit of readyRead() gives different amount of audio data.

                    Still, it is possible to manage all of above, but with RtAudio one gets every call with data amount of exactly declared buffer size value. Rt = Real Time and it is.
                    Simply - it is much easier to work with it.

                    Anyway, I don't mean to deny QtMutlimedia. I came back to it under Android and it is bearable...
                    I made a blog-post how to manage QIODevice with audio output:
                    http://nootka-app.blogspot.com/2015/08/android-qaudiooutput-and-callback-in.html
                    I hope it may be usable for someone.

                    1 Reply Last reply
                    1
                    • S Offline
                      S Offline
                      SGaist
                      Lifetime Qt Champion
                      wrote on 2 Dec 2015, 10:07 last edited by
                      #10

                      I just meant QAudioInput as an example of API not necessarily to use it. i.e. You get a QIODevice or give a QIODevice when you want to get the audio data and then you can use the usual Qt APIs to get notified and read the data.

                      Interested in AI ? www.idiap.ch
                      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                      1 Reply Last reply
                      0
                      • S Offline
                        S Offline
                        SeeLook
                        wrote on 2 Dec 2015, 10:43 last edited by
                        #11

                        @SGaist After I posted, I thought the same:
                        It would be cool thing to have RtAudio wrapper by QIODevice
                        ...but I'm too small yet to do such a thing :-)

                        1 Reply Last reply
                        0
                        • S Offline
                          S Offline
                          SGaist
                          Lifetime Qt Champion
                          wrote on 2 Dec 2015, 10:46 last edited by
                          #12

                          Why too small ?

                          Interested in AI ? www.idiap.ch
                          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                          S 1 Reply Last reply 2 Dec 2015, 11:10
                          0
                          • S SGaist
                            2 Dec 2015, 10:46

                            Why too small ?

                            S Offline
                            S Offline
                            SeeLook
                            wrote on 2 Dec 2015, 11:10 last edited by SeeLook 12 Feb 2015, 11:11
                            #13

                            @SGaist Well...
                            I wrote about my solutions to combine Qt and RtAudio and it just works but it is a bit of head breaking for me how to manage this with QIODevice.
                            Maybe after some time

                            1 Reply Last reply
                            0
                            • V Offline
                              V Offline
                              VaL Doroshchuk
                              wrote on 12 Nov 2018, 13:31 last edited by
                              #14

                              Hi @paulmasri, sorry if it is not relevant any more but curious, why you wanted to decrease the buffer size?
                              As I understood, if the buffer size is small enough, the backend will play faster than it is filled by new data, which produces glitches/pops/noise.

                              Also I read the post from SO: "In practice I find I get glitches if the buffer is less than around 100ms. That's way too long for good responsiveness."

                              Could you please explain why it is too long? Thanks

                              S 1 Reply Last reply 12 Nov 2018, 18:30
                              0
                              • V VaL Doroshchuk
                                12 Nov 2018, 13:31

                                Hi @paulmasri, sorry if it is not relevant any more but curious, why you wanted to decrease the buffer size?
                                As I understood, if the buffer size is small enough, the backend will play faster than it is filled by new data, which produces glitches/pops/noise.

                                Also I read the post from SO: "In practice I find I get glitches if the buffer is less than around 100ms. That's way too long for good responsiveness."

                                Could you please explain why it is too long? Thanks

                                S Offline
                                S Offline
                                SeeLook
                                wrote on 12 Nov 2018, 18:30 last edited by SeeLook 11 Dec 2018, 21:53
                                #15

                                @VaL-Doroshchuk
                                Hi!
                                "big buffer" is quite relative and that "big" depends on needs.
                                In my case I'm using audio data to pitch recognition and depending on pitch range to recognize, I'm using 512, 1024 or 2048 samples buffer, so it is apparently 11, 23 and 45ms (more less). Those power-of-two buffer sizes are required by FFT routines.
                                But my app is flexible enough, and when underlaying OS is not capable to keep that buffer size it will portion any audio data size as needed for pitch detection algorithm.
                                (on Android Qt Audio is used and with low level device a delay between data ready call could sometimes be about 100ms)
                                However smaller buffer gives faster app response (displaying pitch in score).
                                And no any glitches was notices even if buffer is set to 64 frames (2ms)

                                And if we take simpler example, which could be passing input mike data to an output device - more quicker we send incoming data to the out - less delay will be in speakers - it means less buffer - faster response.

                                1 Reply Last reply
                                1

                                • Login

                                • Login or register to search.
                                • First post
                                  Last post
                                0
                                • Categories
                                • Recent
                                • Tags
                                • Popular
                                • Users
                                • Groups
                                • Search
                                • Get Qt Extensions
                                • Unsolved