QT 6.x error message "qt.multimedia.audiooutput: Failed to setup resampler"

johnco3

I am creating a windows based audio tester application to capture raw PCM audio from the microphone and forward it to both a UDP socket and the pc speaker (earphone plugged to avoiod echo).

The microphone QAudioDevice corresponds to QMediaDevices::defaultAudioInput() .

Upon querying this device via the supportedSampleFormats function, it indicates that it supports 3 formats: Uint8, int16 and Float. It also indicates that the preferred QAudioFormat is Int16, 2 channels (stereo), with a sample rate of 44,100. The microphone's QWindowsAudioDeviceInfo indicate that is supports sampling rates from 11025 to maximum 96000.

On the output side, the audio output (headset) has identical properties to those mentioned above queried via the QMediaDevices::defaultAudioOutput().

When I eventually start the audio processing by calling start on the QAudioSink (QWindowsAudioSink) in the following sequence of calls:

    const auto& defaultOutput = QMediaDevices::defaultAudioOutput();
    const auto formats = defaultOutput.supportedSampleFormats();
    mpAudioSink = std::make_unique<QAudioSink>(
        defaultOutput, defaultOutput.preferredFormat());
    // copy synthesized data to IODevice.  speaker will pull data from it on demand.
    mpIODeviceSink->setData(rAudioData.join());
    // opens the device for read only
    mpIODeviceSink->start();
    // starts streaming using the default pull service
    mpAudioSink->start(mpIODeviceSink.get());

I get qt.multimedia.audiooutput: Failed to setup resampler as a console error.

I tracked the source of the error to somewhere deep within the windows implementation of QWindowsAudioSink . It appears that the first call to initialize the resampler has 2 slightly different sample rates for the input vs the output 44100 vs 48000 and also int16 vs Float so it will not be a pass through. The float and 48000 sampling rates for the speaker were determined via a couple of calls:
m_audioClient->GetMixFormat(&pwfx);

Using the returned pwfx we convert that into a QAudioFormat via *QWindowsAudioUtils::waveFormatExToFormat(pwfx)

   bool QWindowsAudioSink::open() 
   {
      if (!m_resampler.setup(m_format, QWindowsAudioUtils::waveFormatExToFormat(*pwfx))) {
        qCWarning(qLcAudioOutput) << "Failed to setup resampler";
        CoTaskMemFree(pwfx);
        return false;    
   }

This is the problematic setup call.

   bool QWindowsResampler::setup(const QAudioFormat &fin, const QAudioFormat &fout)
   {
    qCDebug(qLcAudioResampler) << "Setup audio resampler" << fin << "->" << fout;

    m_totalInputBytes = 0;
    m_totalOutputBytes = 0;

    if (fin == fout) {
        qCDebug(qLcAudioResampler) << "Pass through mode";
        m_inputFormat = fin;
        m_outputFormat = fout;
        return true;
    }

    if (!m_resampler)
        return false;

    QWindowsIUPointer<IMFMediaType> min = QWindowsAudioUtils::formatToMediaType(fin);
    QWindowsIUPointer<IMFMediaType> mout = QWindowsAudioUtils::formatToMediaType(fout);

    HRESULT hr = m_resampler->SetInputType(m_inputStreamID, min.get(), 0);
    if (FAILED(hr)) {
        qCWarning(qLcAudioResampler) << "Failed to set input type" << hr;
        return false;
    }

    hr = m_resampler->SetOutputType(0, mout.get(), 0);
    if (FAILED(hr)) {
        qCWarning(qLcAudioResampler) << "Failed to set output type" << hr;
        return false;
    }

    MFT_OUTPUT_STREAM_INFO streamInfo;
    hr = m_resampler->GetOutputStreamInfo(0, &streamInfo);
    if (FAILED(hr)) {
        qCWarning(qLcAudioResampler) << "Could not obtain stream info" << hr;
        return false;
    }

    m_resamplerNeedsSampleBuffer = (streamInfo.dwFlags
             & (MFT_OUTPUT_STREAM_PROVIDES_SAMPLES | MFT_OUTPUT_STREAM_CAN_PROVIDE_SAMPLES)) == 0;

    m_inputFormat = fin;
    m_outputFormat = fout;

    return true;
   }

Eventually, I found the source of the problem but I do not know how to work around it or why it happens. The call to CoCreateInstance fails to return an m_sampler if I include the following calls.

#if defined (RESAMPLER_BUG_FIXED)
    // opens the device for write by mpAudioSource
    mpIODeviceSource->start();
    mpAudioSource->start(mpIODeviceSource.get());
#endif

   QWindowsResampler::QWindowsResampler()
   {
    CoCreateInstance(CLSID_CResamplerMediaObject, nullptr, CLSCTX_INPROC_SERVER,
                     IID_IMFTransform, (LPVOID*)(m_resampler.address()));
    if (m_resampler)
        m_resampler->AddInputStreams(1, &m_inputStreamID);
   }

vlada

I just tried the Audio Output example and I got exactly the same error.

The reported format is:

QAudioFormat( 44100 Hz,  2 Channels,  Int16 Format )

As soon as the example calls

m_audioOutput->start(m_generator.data());

I get this error.

I do believe there might be a bug in QtMultimedia causing this.

srce

I was seeing the same problem in my application. Looks like the OP created a bug here:

https://bugreports.qt.io/browse/QTBUG-108383

For me, seems to be a problem with Qt 6.4.1, but 6.4.0 is OK.

johnco3

@vlada I found the bug yesterday - the windows resampler requires a dll called "mfplat.dll" and the load library code call searches for "mfplat.dll.dll".

I cannot believe that the Qt developers would release such a bug ridden set of updates without testing their own examples on each of the platforms. Amazing. Anyway the bug I encountered had to do with initializing the windows resampler. The workaround is to copy "mfplat.dll" from your system library folder to some temporary path (in my case c:\tools\qtworaround) .

Then rename this to "mfplat.dll.dll". and make sure that the path is appended to you PATH environment variable. I opened a bug against this which so far has not been reviewed.

Bonnie

Oh, that should be the reason I got segmentation fault when testing the new ffmpeg plugin built by Qt6.4.
But when I test with Qt6.5 snapshot it seems to be OK.

johnco3

@Bonnie I am tempted to try the snapshot - although I have a workaround for the time being

mnbv

@johnco3 Good catch on that DLL. What a silly error.

I filed a bug report too (which now I see is a duplicate) before I found this forum post; https://bugreports.qt.io/browse/QTBUG-108669.

Also good to know it works in 6.4.0, I might go back to that version then. I jumped directly from Qt 5.15 to 6.4.1 yesterday and am migrating some code, I skipped right over 6.4.0.

This is all it takes to illustrate the error:

int main (int argc, char *argv[]) {

    QCoreApplication a(argc, argv);

    QAudioDevice device = QMediaDevices::defaultAudioOutput();
    QAudioFormat format = device.preferredFormat();
    QAudioSink *output = new QAudioSink(device, format);
    output->start();

    return a.exec();

}

Btw, for the workaround, I also got it working by just copying mfplat.dll to mfplat.dll.dll in the application exe's directory. An alternative if you don't feel letting setting up PATH.

mnbv

Also, maybe I'm crazy, but I'm even more bothered by the fact that they used "setup" as a verb in that error message than I am by the bug itself, 😂.

QtWS: Super Early Bird Tickets Available!

QT 6.x error message "qt.multimedia.audiooutput: Failed to setup resampler"