Is Qt capable of small-buffer low-latency audio applications (e.g. soft synth)?

paulmasri

@SGaist Thanks for the heads up about the mailing list.

Your comments about wrapping PortAudio echo the response on StackOverflow (suggesting the same thing but with RTAudio). It's useful to hear this confirmed.

SeeLook

Hi
I also may confirm well cooperation Qt and RtAudio.
I give up from QtMultimedia and with RtAudio I was able to get 64 frames buffer and callback in RtAudio thread where Qt signal was emitted.
All together works flawlessly.
Good luck!

paulmasri

Hi. Thank you both for your responses. I know that the audio will need to be kept in a separate high priority thread but I can't see how to integrate RTAudio/PortAudio with Qt and with this threading.

Could you give some advice? Or is there sample code I could look at?

Many thanks,
Paul.

SeeLook

I will try, feel free to ask if I'm not clear enough.

I suggest to use RtAudio - it works over PulseAudio/JACK/ALSA and Windows/Mac audio as well

First of all try some simple RtAudio example from their site: https://www.music.mcgill.ca/~gary/rtaudio/

Input audio data You will get through implementing call back function.
This is static method, so to be able do something in your Qt class from there, create static instance member in your class:

class MyAudioInput public QObject
{
Q_OBJECT
  MyAudioInput()
  {
    m_instance = this;
    m_buffer1 = new float[64];
    m_buffer2 = new float[64];
    m_currentBuffer = m_buffer1;
    m_readyBuffer = 0;
    m_posInBuffer = 0;
    m_thread = new QThread();
    moveToThread(m_thread);
    connect(m_thread, &QThread::started, this, &MyAudioInput::process);
  }
~MyAudioInput() 
{
  delete m_buffer1;
  delete m_buffer2;
  m_instance = 0;
  delete m_thread;
}

static MyAudioInput* instance() { return m_instance;}

static int inputCallBack(void*, void *inBuffer, unsigned int nBufferFrames, double, RtAudioStreamStatus status, void*) {
  for(int i = 0; i < nBufferFrames; i++;) {
    // copy audio data to current buffer
    *(instance()->m_currentBuffer + instance()->m_posInBuffer) = *(inBuffer + i);
    instance()->m_posInBuffer++;
    if (instance()->m_posInBuffer == 64 { // switch buffers when full
      instance()->m_posInBuffer = 0;
      if (instance()->m_currentBuffer == instance()->m_buffer1) {
        instance()->m_currentBuffer = instance()->m_buffer2;
        instance()->m_readyBuffer = instance()->m_buffer1;
      } else {
        instance()->m_currentBuffer = instance()->m_buffer1;
        instance()->m_readyBuffer = instance()->m_buffer2;
      }
    // start processing in separate thread
    instance()->m_thread->start();
    }
  }
}

void process() {
  // do something with m_readyBuffer here
  // or/and emit dataReady() to inform the rest of Your app
}


signals:
  void dataReady();

private:
  static MyAudioInput* m_instance;
  float *m_buffer1, *m_buffer2;
  float *m_currentBuffer, *m_bufferReady;
  int m_posInBuffer;

  QThread *m_thread; // processing thread;
}

Then You may open RtAudio stream like this:

  RtAudio adc;
  RtAudio::StreamParameters parameters;
  parameters.deviceId = adc.getDefaultInputDevice();
  parameters.nChannels = 2;
  parameters.firstChannel = 0;
  unsigned int sampleRate = 44100;
  unsigned int bufferFrames = 64; // 64 sample frames
  try {
    adc.openStream( NULL, &parameters, RTAUDIO_FLOAT32,
                    sampleRate, &bufferFrames, &MyAudioInput::inputCallBack );
    adc.startStream();
  }

When signal dataReady() will be handled, m_readyBuffer points to data for processing and code on it has to be performed in 2ms, so probably it is better to have bigger buffer size or more buffers. I'm using 512 samples.
But to get this working You have to add to this example some "real" things.

paulmasri

Thank you! That's an awesome help. I shall take a look at the RTAudio examples and play with the sample code you've posted.

SGaist

Depending on your application design, you might want to consider implementing a custom QIODevice for easier integration with Qt's API (see QAudioInput/QAudioOutput)

SeeLook

In my experience with QtMultimedia I found it a bit unpredictable.
If one set audio buffer QAudioInput::setBufferSize(someValue) it will never be set to desired value. And value forced by QtMultimedia is quite big as mentioned in the first post.

When one get this value after device opening QAudioInput::bufferSize() it is good for nothing - when one expecting to obtain the 'buffer size' of data in readyRead() signal.
Usually every emit of readyRead() gives different amount of audio data.

Still, it is possible to manage all of above, but with RtAudio one gets every call with data amount of exactly declared buffer size value. Rt = Real Time and it is.
Simply - it is much easier to work with it.

Anyway, I don't mean to deny QtMutlimedia. I came back to it under Android and it is bearable...
I made a blog-post how to manage QIODevice with audio output:
http://nootka-app.blogspot.com/2015/08/android-qaudiooutput-and-callback-in.html
I hope it may be usable for someone.

SGaist

I just meant QAudioInput as an example of API not necessarily to use it. i.e. You get a QIODevice or give a QIODevice when you want to get the audio data and then you can use the usual Qt APIs to get notified and read the data.

SeeLook

@SGaist After I posted, I thought the same:
It would be cool thing to have RtAudio wrapper by QIODevice
...but I'm too small yet to do such a thing :-)

SGaist

Why too small ?

SeeLook

@SGaist Well...
I wrote about my solutions to combine Qt and RtAudio and it just works but it is a bit of head breaking for me how to manage this with QIODevice.
Maybe after some time

VaL Doroshchuk

Hi @paulmasri, sorry if it is not relevant any more but curious, why you wanted to decrease the buffer size?
As I understood, if the buffer size is small enough, the backend will play faster than it is filled by new data, which produces glitches/pops/noise.

Also I read the post from SO: "In practice I find I get glitches if the buffer is less than around 100ms. That's way too long for good responsiveness."

Could you please explain why it is too long? Thanks

SeeLook

@VaL-Doroshchuk
Hi!
"big buffer" is quite relative and that "big" depends on needs.
In my case I'm using audio data to pitch recognition and depending on pitch range to recognize, I'm using 512, 1024 or 2048 samples buffer, so it is apparently 11, 23 and 45ms (more less). Those power-of-two buffer sizes are required by FFT routines.
But my app is flexible enough, and when underlaying OS is not capable to keep that buffer size it will portion any audio data size as needed for pitch detection algorithm.
(on Android Qt Audio is used and with low level device a delay between data ready call could sometimes be about 100ms)
However smaller buffer gives faster app response (displaying pitch in score).
And no any glitches was notices even if buffer is set to 64 frames (2ms)

And if we take simpler example, which could be passing input mike data to an output device - more quicker we send incoming data to the out - less delay will be in speakers - it means less buffer - faster response.

Call for Presentations - Qt World Summit

Unsolved Is Qt capable of small-buffer low-latency audio applications (e.g. soft synth)?