@JonB : Yes, it's a mixed app as it relays onto some 3rd party libs that are only available in C#. Not nice, but without alternative for now...
@hskoglund : I've been convinced that you were on to something until I learned, that Dispose-Pattern is very useful to free unmanaged resources immediately instead of wating for the garbage collector. But there is no way in C# for an object to destroy itself and free the used memory leaving nothing but a dangling pointer behind...
I dug a little deeper into the topic and it seems, that this isn't Qt related. I tried the same, using only standard C++ and the ComBaseApi (combaseapi.h), and got pretty much to the same result. The COM objects reference count actually drops to zero as it is supposed.
Here I learned tha when creating a COM object a Runtime Callable Wrapper (RCW) is created to own the object and hold a strong reference to it which could possibly prevent tha GC from doing it's job. I guess I will have a closer look on that...