When you correct the distortion in an image from a fisheye lens, you get undesirable side-effects.

• You lose a lot of LOT of diagonal angle of view from cropping, to get a rectangular image out of it. See the below example of a rectilinear conversion (yellow indicates the largest usable rectangular area after fisheye to rectilinear conversion). So after correction, you have lost some image information.

  

• You lose a LOT of resolution on the corners of the frame. For an example, look at the corners of the first example image (above). They are really blurry the further you get to the corners.

• Rectilinear doesn't mean no distortion. Simply due to the huge angle of view, you'll still get things looking very stretched at the edges, even though it's technically what you'd get from a rectilinear lens that had the same angle of view. See this second example of a rectilinear conversion (below), which has already been cropped to the largest usable rectangle. The people on the far left and right look stretched, even though that's how they actually look, if you were able to photograph them with a really wide rectilinear lens. You can verify that it's a rectilinear projection because the straight lines are all straight, not curved.

  

  In reality the photographer would have been very close to this group of people and using a fish-eye lens. At least the people on the ends wouldn't have looked so wide in the original fisheye!

  Also, even in the previous image I showed you of the buildings, notice how exaggerated the divergence of vertical lines is, since the camera is slightly angled upwards.