This strongly suggests a loose connection at the flash foot/hot shoe interface, or an internal fault related to that connector. The key clue is that nudging the flash slightly changes whether E-TTL works. That usually means the electrical contacts are only making intermittent contact, so the camera and flash can’t reliably communicate.

When TTL communication drops out, exposure can become erratic, and intermittent contact can also cause misfires or dark frames.

Since cleaning the external contacts didn’t fix it, the problem is likely mechanical or internal rather than simple dirt. The practical solution is to have the flash repaired or replace it.

Your reduced reliability is unlikely to be normal behavior for the 580EX II if it previously performed better under the same conditions.