Which sample type (not size) to choose in QtMultimedia's QAudioFormat for 24, 32 and 64 bit audio?

[maestro22]

Hello everyone, I am writing a media player with Qt, but I'm now facing some unknown situation. Actually, I'm trying to use QAudioOutput and QAudioDecoder to play high res music (24, 32 or even 64 bit audio). But QAudioFormat (the glue between all audio classes) specify a sampleType as in the following table:

Constant	Value	Description
QAudioFormat::Unknown	0	Not Set
QAudioFormat::SignedInt	1	Samples are signed integers
QAudioFormat::UnSignedInt	2	Samples are unsigned intergers
QAudioFormat::Float	3	Samples are floats

Now, the problem arise when I also set the sample size to something greater than 16bits. I now have one hypothesis that I need confirmation :

• assuming ints are 32bits in size, if I want to support up to 32bit sample sizes I have to use QAudioFormat::SignedInt with pcm audio for 24 and 32 bit audio (filling with 0 for 24bit audio).

But what if there is a higher sample size (eg: 64bit audio for dsd converted to pcm). Should I assume that I still set the sample type to QAudioFormat::SignedInt but that each "sample" of 64bits is stored in two ints ? Or is it simply not supported by QtMultimedia ?

I'm open to any enlightenment 😙!

[maestro22]

I also posted the same question on stackoverflow and concerning the size of the "type" in audioformat some awesome guy responded, so I'll take it as a valid answer before I try it :

From the documentation for QAudioFormat::setSampleSize():

void QAudioFormat::setSampleSize(int sampleSize)

Sets the sample size to the sampleSize specified, in bits.

This is typically 8 or 16, but some systems may support higher sample
sizes.

Therefore, to use 64-bit samples, you'd need to call setSampleSize(64). That could be called in combination with a call to setSampleType() to specify whether the samples will be fixed-point-signed vs fixed-point-unsigned vs floating-point -- note that the values in setSampleType() do not imply any particular sample size.

For 64-bit audio, each sample will be stored as 64 bits of data; you could access each sample as a long long int, or alternatively as an int64_t (or unsigned long long int or uint64_t for unsigned samples, or as a double for floating-point samples).

(Of course none of this guarantees that your Qt library's QtMultimedia actually supports 64-bit samples; it may or may not, but at least the API supports telling Qt what you want :) )

EDIT: here is a link to the stack overflow thread : https://stackoverflow.com/questions/57636058/which-sample-type-not-size-to-choose-in-qtmultimedias-qaudioformat-for-24-32

[SGaist]

Hi and welcome to devnet,

QtMultimedia provides support through the OS official APIs, therefore, does your platform natively support such high quality audio formats ?

Note that the module can also be improved to support a wider range of audio formats.

You might want to check with the interest mailing list. You'll find there Qt developers/maintainers. This forum is more user oriented.

[maestro22]

Hi thank you for the link to the mailing list, I did'nt know it existed 🙃. Concerning Qt Multimedia I already know that it is a highly platform specific part of Qt and thus plan to have alternatives to both QAudioDecoder and QAudioOutput through libavcodec and libsoundio respectively. I only want to implement Qt Multimedia because it may provide a more efficient (hardware accelerated) way of doing things on some platforms.

On the matter of platform support, most desktop platforms indeed support up to 64bit audio. You can quickly check it by running the audio devices exemple. I ran it both on mac and windows and it does support 64 bit and up to 384Khz (both being what my hi-res external DAC supports). Thus, I think that if any limitation there is, it's more on the side of Qt's private API glueing everything. I'll try to contact the maintainer of QtMultimedia directly because of this issue and also with another problem of a not unique output device id on windows with some compilers (I think it was with MinGW).

By the way do you know if Qt Multimedia will come to WebAssembly down the road ? I would be super interested in having a highly efficient player on the web 😋.

[maestro22]

I also posted the same question on stackoverflow and concerning the size of the "type" in audioformat some awesome guy responded, so I'll take it as a valid answer before I try it :

From the documentation for QAudioFormat::setSampleSize():

void QAudioFormat::setSampleSize(int sampleSize)

Sets the sample size to the sampleSize specified, in bits.

This is typically 8 or 16, but some systems may support higher sample
sizes.

Therefore, to use 64-bit samples, you'd need to call setSampleSize(64). That could be called in combination with a call to setSampleType() to specify whether the samples will be fixed-point-signed vs fixed-point-unsigned vs floating-point -- note that the values in setSampleType() do not imply any particular sample size.

For 64-bit audio, each sample will be stored as 64 bits of data; you could access each sample as a long long int, or alternatively as an int64_t (or unsigned long long int or uint64_t for unsigned samples, or as a double for floating-point samples).

(Of course none of this guarantees that your Qt library's QtMultimedia actually supports 64-bit samples; it may or may not, but at least the API supports telling Qt what you want :) )

EDIT: here is a link to the stack overflow thread : https://stackoverflow.com/questions/57636058/which-sample-type-not-size-to-choose-in-qtmultimedias-qaudioformat-for-24-32

[SGaist]

Nice !

Thanks for the feedback !

One library that you might want to consider is VLC, they also provide a Qt integration module that could be of interest.

I don't know what is planned for web assembly concerning QtMultimedia, sorry.

By the way, can you post a link to the Stack Overflow thread ? It might interest other people :-)

[maestro22]

@sgaist Yep, I added a link to stack overflow. Concerning vlc I already looked at it but sadly there is some major drawbacks :

-the documentation is poor or inexistant.
-it does not allow fine tuning what you want at compile time as much as ffmpeg allows it (to reduce shared libs size).
-it is partially based on ffmpeg anyway and I try to reduce as much as possible the amont of dependencies (for future compatibility and maintaining on my free time).

Again thank you so much for the quick replies and the mailing list 🤩.