This is the nature of lossy JPEG compression. The compression process works by splitting an image into 8x8 pixel patches, and compressing each of them individually, with no concern for the boundary conditions where each patch joins with its neighbors. The theory is that very small details can be modified and/or discarded without our visual system taking too much notice. As a result, discontinuities show up on the edges of these patches, and become more noticeable at higher compression (or, alternatively, lower quality) settings. In the extreme, the compression algorithm could theoretically convert each patch into a single color, which would be pretty similar to scaling a picture down by a factor of 8, and then scaling it back up by simply replicating pixels.

PNG compression, on the other hand, does nothing to modify the actual pixel colors. It applies a lossless compression algorithm to the image contents, which, depending on the contents of the image, can vary widely in its effectiveness - highly detailed images compress less, while a more uniform image would compress better. This is also why PNG images, even with their maximum compression settings, will still almost always be larger than a JPEG image with even moderate settings - it doesn't discard image details like JPEG does.

The CAPTCHA images you reference, are compressed at a relatively low-quality/high-compression setting, in order to make them as small as possible to preserve your network bandwidth (and storage on the server end). These settings would not generally be used for images intended for print/display/archival/etc.