I'm not aware of any particular tangible, scientific measure for overall image quality. There are certain types of artifacts that are to be expected and if you know what you are looking for you can identify distortions that are caused by them, but ultimately, it's still a somewhat subjective measure at some level.

The general idea is to measure how closely the image captured corresponds to what was present in the scene. Sharpness can certainly be a big part of image quality measures, as can the dynamic range of the sensor (which can be measured more scientifically) and the color accuracy of the sensor (which can be compared to a dedicated colorometer or spectrometer. Other things like various types of lens distortion have known looks that can be examined based on how they look. How small of an area is impacted, how much are lines and gradients distorted, etc.

Really, anything that impacts the level of detail and accuracy of an image contributes towards the image quality and the amount we can objectively vs subjectively measure depends on the characteristic that is being examined. Also, how various factors should be balanced is really subjective and can differ from one type of image to another.