For anyone coming across this thread, the answer is...
tl;dr: You need to add the runFullTrust capability.
With Qt 5, there was support for UWP apps. This made it possible for Windows apps to be delivered through the Microsoft Store. (This is how my app was built and delivered.)
Then Microsoft opened up the Microsoft Store for Win32 apps too; i.e. standard desktop apps. As a developer you have a choice of either packaging your own installer and delivering that through the Microsoft Store, or just your app, letting Microsoft Store handle installation. The latter option behaves more like the handling of a UWP app and Microsoft handles the code-signing certificate for you.
As a consequence, Qt decided to drop support for UWP as of Qt 6. This was done with no public notification to the Qt community 🤬, with just a message on the Qt development mailing list. I eventually found this out via a convoluted bug report.
What Qt themselves did not realise is that if you submit a Win32 app via the Microsoft Store, because it is a desktop app and does not sit nicely in a sandbox like a UWP app, you need to add the runFullTrust capability. If you don't do this, you get the weird behaviour of the application window opening in a separate window described above.
However be aware that when you add runFullTrust Microsoft will require you to justify why your app needs access to the whole Windows system. I was able to pass certification with a message along the lines of "My app is built with the Qt framework that only generates Win32 apps. My app does not operate outside the sandbox and behaves nicely."
One more gotcha if you're going this route. The path QStandardPaths::AppDataLocation was different for UWP apps than Win32 apps. The following only applies if you had a UWP app in Qt 5 and stored data there as the path will be wrong for your Win32 app in Qt 6. To fix this, I use the following: https://gist.github.com/paulmasri/75dcd3386a4e8eaf705058240ef547d1