So I got it to deploy the dlls (Qt5Core.dll ...) to a subfolder with --libdir $(TargetDir)\qt\libs
And added that path to the qt.conf file.
It works fine in visual studio, but when I run the exe itself it still misses those dlls
This is the not working layout.
Moving the contents of libs back to the appdir and removing the Libraries entry from the .conf and windeployqt works fine.