It turns out the solution to this is simple, though I had to read a whole bunch of confusing descriptions of scalability. It really should be easier to find. And the Qt support pages on the topic could be less confusing.
https://stackoverflow.com/questions/41331201/pyqt-5-and-4k-screen
Put this in your setup code before creating the QGuiApplication, to support High DPI scaling on Windows:
os.environ["QT_AUTO_SCREEN_SCALE_FACTOR"] = "1"
sys.argv += ['--style', 'fusion']
app = QGuiApplication(sys.argv)
The font scaling ends up a little bit different, but that is acceptable for my use case.
For C++ users, look here:
https://doc.qt.io/qt-5/qtquickcontrols2-highdpi.html
Whew! I was getting nervous about my decision to create this cross platform UI.