Solved it: by checking is the cursor in an "interactive" region of the graphics item in QGraphicsItem::mousePressEvent I can determine whether the item will become a mouse grabber (by either accepting or ignoring the event). When it comes to the graphics scene I simply prioritise the event propagation to the selection area resizing and I ignore accepted events.
I checked if a new image was returned by saving each image data to a file before it gets returned and the image seems to be correct.
I will try to replace the graphicsview/scene by a QLabel, perhaps this will work.
It might be better to have this handled in the parent class. When a graphics item is selected (clicked) store this information and move the item from the parent when the right key presses are seen. You might have a problem with losing focus otherwise.