There does not appear to be a well-known, dedicated service for comparing multiple JPEG compression engines side by side. A practical starting point is to build your own tests with tools such as ImageMagick, which can batch-process files and expose JPEG-related options.

That said, one answer notes an important limitation: JPEG is a standardized compression method, so implementations are often more similar than different. In many cases, the biggest variation is not a completely different algorithm, but how each program maps its “quality” setting to actual encoder parameters. So comparing “different compressors” may yield less distinction than expected.

For visual inspection of artifact differences, you can compare an original and recompressed file using a difference blend/layer method in Photoshop or similar editors: non-black areas show changes.

One responder also reports that they could not find any decent facilities specifically for comparing JPEG libraries, and later found their original hypothesis was not supported by further testing.

So: use command-line tools like ImageMagick for controlled experiments, but be aware that JPEG encoder differences may be subtler than the question assumes.