PNG compression quality varies greatly from compressor to compressor. The standard PNG compression in Photoshop for example can sometimes commonly be beat by large percentage points. This is primarily due to more intelligent switching algorithms when it picks the kind of prediction to do for a certain set of pixels. Most of the "additional" compression is achieved in that way, but there is also a certain set of specialty tools out there which will also just do a plain old better entropy encoding of the data (through reverse compression, sounds funny but works very well). That is probably more technical of an answer than you were looking for though.

JPG can be compressed better by better compressors as well while achieving the same or nearly the same quality. This usually works by finding optimal tuning numbers for internal quality settings and doing automatic comparisons with the original photo. In this way you can reduce quality without it being entirely noticeable. AFAIK, this is how google's tools work for web image optimization for example.

Its a very large topic. Feel free to ask a question and I'll do my best to answer.