What you are going wrong is not giving the camera enough latitude. You fixed the aperture and ISO, so all the camera can do is set the shutter-speed and flash power. It must be not as low-light as you think because most often you would get an under-exposed image doing what you are doing.

The camera has a shutter-speed range it can use with the flash. The maximum speed is known as the flash sync-speed and usually varies between 1/160 and 1/250, depending on the model. This is the fastest speed at which the shutter-speed is fully open at some time during the exposure.

Flash power can be varied too. Most on-camera flashes work at full power, half-power, quarter-power and so on. If the flash is automatic the camera will chose power itself but you can also have the flash on manual power on some cameras. Depending on the model, if you set the flash to Forced (On), it may always fire at maximum power though.

Shutter-priority also has some interaction with flash. If you select a speed faster than the sync-speed, then you will get a partially lit image. If you select anything less, than flash illumination does not change because a flash burst is very fast, in the order of 1/20000s to 1/80000s, so choosing a longer shutter-speed only gets more light from ambient lighting.

On more thing, flash illumination is inversely proportional to the square of the distance, so if your subject is very close, even low-power flash may be too bright.