From Wikipedia's Pinhole camera article,

The best pinhole is perfectly round (since irregularities cause higher-order diffraction effects), and in an extremely thin piece of material. Industrially produced pinholes benefit from laser etching, but a hobbyist can still produce pinholes of sufficiently high quality for photographic work.

There is a theoretical optimal size to a pinhole aperture. From a geometric standpoint, the smaller the pinhole aperture, the sharper the produced image will be — but this holds only to a point. Smaller than this size, the image sharpness becomes diffraction limited. The wave-like behavior of the light interacting with the aperture walls creates its own interference pattern, and at (or beyond) the diffraction-limited aperture size, this dominates the image clarity.

A circle is the only shape with constant curvature. Any other shape will have either tighter curves or closer edges than a circle. If you strictly maintain minimum curvature and edge distance, but neither are constant, then by definition you have part of the shape that is larger than it needs to be. Thus, a circle is the optimal shape to balance the desire for a smaller aperture (for image clarity purposes) with a lower limit to aperture size due to diffraction limitation.