Sensors are rectangular by tradition, based on the historically traditional shape of image media.

But there is a technology/business decision that drives them to be rectangular, also. Sensors are rectangular because they are made using semiconductor fabrication techniques. These techniques call for “printing” multiple sensor circuits onto a silicon wafer. Today these wafers can be 300 mm in diameter and manufacturers are moving toward 450 mm diameter (see here). A lot of sensors can be printed on wafers that large.

Sensors are tiled onto the wafer to efficiently use the space available and in a way that makes them easy to cut apart into “dies” (or the individual sensors, in this case). The process is called dicing. The most cost effective shape for dies is rectangular. Usually a saw or scribe is used to cut the wafers in straight lines. Imagine if the dies (sensors in this case) were supposed to be round (a wasteful and costly use of the material) or hexagonal (efficient use of the material but the cuts are not straight across the whole wafer). (See here for more info.)

B) Lenses made of high quality glass are generally ground using lathes. (This can be seen in this video. Watch around the 7:00 minute mark in particular. Sorry, it's in Japanese, but the video is very fascinating and revealing.) It is easier to spin, grind, and polish a round lens in these machines because there are no edges to catch on the tooling as the lens spins around. It also is consistent with the optical symmetry they are trying to achieve in the finished lens.

Lenses that are not round would generally be cut from round lenses, a step that adds cost to the production of the lens assembly. Lenses don’t need to be round. For heaven’s sake, most eyeglasses are not round! When your eyeglasses are made, you must be aware that the lens maker isn’t stocking a lens for every shape of eyeglass frame. He’s cutting or grinding round lenses to fit the frame.

Once the lens manufacturer has his round lenses, what would motivate him to cut it into a different shape? As many people have pointed out in various forums, the lens shape does not determine the image shape or quality (apart from diffraction caused by edges, which can be mitigated, and some second order aberration effects, maybe), and for the most part, every point on the lens can gather light from every point on the object and focus each point on the image plane. I’ve already pointed out that changing the shape of the lens adds cost. There really isn’t any practical reason (generally) for changing the shape.