QDockWidget provides a nice framework for this. If you need extra functionality outside of what the QDockWidget default title bar offers, you can roll your own using setTitleBarWidget().
No idea what you want to achieve. All top-level widgets get a title bar automatically. What do you mean by a 'custom button', and where would you like to have it? In the title bar itself, or somewhere else?