Drawing error when using Qt6.6.1 using windows native event WM_NCCALCSIZE

[AlexZhZZ]

I try to draw FramelessWindow using Qt6.6.1 with this method:

• set FramelessWindowHint
• using windows api SetWindowLong set WS_THICKFRAME style
• using WM_NCCALCSIZE message to cover title bar
  it is completely correct when window is not maximized.
  However, when maximizing, the right and bottom sides will exceed the drawable range.

But in old Qt version like 6.3.2 has no problem, it seems to have occurred after 6.4.2, and also can be reproduced in PyQt.
This is the effect in Qt6.3.2

and the result in Qt6.6.1

It can be observed that the third button exceeds the boundary.
Is there any way to get same result in Qt6.6.1
The full test code like:

#include "mainwindow.h"
#include <QGuiApplication>
#include <QWindow>
#include "ui_mainwindow.h"
#include <Windows.h>
#include <dwmapi.h>

#pragma comment(lib, "user32.lib")
#pragma comment(lib, "dwmapi.lib")

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    this->setWindowFlags(this->windowFlags() | Qt::FramelessWindowHint);
    auto style = GetWindowLong((HWND) this->winId(), GWL_STYLE);
    SetWindowLong((HWND) this->winId(),
                  GWL_STYLE,
                  style | WS_MINIMIZEBOX | WS_MAXIMIZEBOX | WS_CAPTION | CS_DBLCLKS | WS_THICKFRAME
                      | WS_OVERLAPPED);
    SetWindowPos((HWND) this->winId(),
                 HWND_TOP,
                 0,
                 0,
                 0,
                 0,
                 SWP_NOMOVE | SWP_NOSIZE | SWP_NOZORDER | SWP_FRAMECHANGED);
    this->showMaximized();
}

MainWindow::~MainWindow()
{
    delete ui;
}

bool isMaximized(HWND hWnd)
{
    WINDOWPLACEMENT windowPlacement;
    windowPlacement.length = sizeof(WINDOWPLACEMENT);
    bool ret = GetWindowPlacement(hWnd, &windowPlacement);
    if (!ret) {
        return false;
    }

    return windowPlacement.showCmd == SW_MAXIMIZE;
}

QWindow *findWindow(HWND hWnd)
{
    if (!hWnd)
        return nullptr;
    auto windows = QGuiApplication::topLevelWindows();
    if (windows.size() == 0)
        return nullptr;
    int id = int(hWnd);
    foreach (auto w, windows) {
        if (int(w->winId()) == id) {
            return w;
        }
    }
    return nullptr;
}

int getResizeBorderThickness(HWND hWnd, bool hor = true)
{
    auto window = findWindow(hWnd);
    if (!window) {
        return 0;
    }
    auto frame = hor ? SM_CXSIZEFRAME : SM_CYSIZEFRAME;
    auto size = GetSystemMetrics(frame) + GetSystemMetrics(92);
    if (size > 0)
        return size;
    int res = 0;
    DwmIsCompositionEnabled(&res);
    auto thickness = res ? 8 : 4;
    return int(thickness * window->devicePixelRatio());
}

bool MainWindow::nativeEvent(const QByteArray &eventType, void *message, qintptr *result)
{
    if (eventType != "windows_generic_MSG") {
        return false;
    }
    MSG *msg = (MSG *) message;
    if (msg->hwnd == nullptr)
        return false;
    switch (msg->message) {
    case WM_NCCALCSIZE: {
        auto isMax = ::isMaximized(msg->hwnd);
        auto setRectMax = [](HWND hWnd, LPRECT rect) {
            int tx = getResizeBorderThickness(hWnd);
            int ty = getResizeBorderThickness(hWnd, false);
            rect->top += ty;
            rect->bottom -= ty;
            rect->left += tx;
            rect->right -= tx;
        };
        if (msg->wParam) {
            LPNCCALCSIZE_PARAMS rect = (LPNCCALCSIZE_PARAMS) (msg->lParam);
            if (isMax) {
                setRectMax(msg->hwnd, rect->rgrc);
            }
        } else {
            LPRECT rect = (LPRECT) (msg->lParam);
            if (isMax) {
                setRectMax(msg->hwnd, rect);
            }
        }
        *result = msg->wParam ? 0 : WVR_REDRAW;
        return true;
    }
    }
    return false;
}

[AlexZhZZ]

@AlexZhZZ
Also, if the implementation of WM_NCCALCSIZE is removed, and the title bar is no longer overridden, it can be observed that the window drawing is also incorrect in the non-maximized state.
This seems to happen when both FramelessWindowHint and SetWindowLong are used simultaneously, even if only the style obtained through GetWindowLong is set.

[AlexZhZZ]

@AlexZhZZ
See QTBUG-120196