This is highly likely due to the charging time of the flash. Depending on the camera, one of the following happens in manual (M) mode:

1. The camera always fires the flash at full-intensity.
2. The flash power is automatically set by the camera.
3. Flash power is manually set between 1, 1/2, 1/4, 1/8, etc of full power.

For #1, you will get the full recharge cycle of your flash every time. If you were in any Automatic mode, then the camera may not need to fully charge the flash and you would get a shorter lag.

For #2, the flash recharge cycle will be depending on the aperture and ISO you set. Try the highest ISO and widest aperture you have and then the opposite, to see if this is the case.

For #3, see the Custom Setting Menu on the D40, the lag will be depend on the manually chosen power setting. Try different power settings and you should see the lag change, particularly if you choose 1/32.

On some cameras, there is a setting in the setup menu which lets the camera take a shot even if the flash is not sufficiently charged. From memory, I do not recall the D40 having this.