Your code/algorithm is incorrect. This is basic coding, please study what your code does. You "actively" encourage reading across file boundaries, i.e. what you call "too much").
You (seem to) start by setting size to the total number of bytes expected for the file. Then you call read(buffer, size);. But when that returns with fewer bytes than size you still call it again next time round asking it to read up to size further bytes. This can/will exceed the number of bytes in the first file. You obviously need to reduce what size you pass to read() accordingly on subsequent calls. And then be careful what you compare in your if (... || numReadTotal == size). Adjust your code however to make it right.
I previously said that the whole synchronous approach of repeatedly calling waitForReadyRead() in a loop is not the event-driven way to do things. Allow bytes to arrive in repeated readyRead() signals asynchronously instead.