As long as you do not save the image to its initial source file you may do several scalings. E.g. you start your application with different sizes of your graphic scene and you always read it from original file, you should not face problems. The quality of the displayed may change with the actual size, but a new start from original file will have different scaling and a different result. See also the input of Jota for scaling quality. Consequence, you should not destroy your original file.
If you load your image once and you do several scalings with different sizes, you will certainly have deterioration of the image.