Strange choice. What I do in my project is to give buttons their own MouseAreas, and place the buttons above (using 'z' value) the background (which also has a MouseArea). This way, when user clicks on a button, I can handle the action for this button, while the main MA remains untouched. If user does not click on the button, though, main MA kicks in.
Anyway, obviously you know your project better, maybe your solution is better in your case. The two ways to achieve this that I provided should work, after some adjustments.