Making large semiconductor devices with no, or only a very small number, of defects is very hard. Smaller ones are much less demanding to make.

In particular the yield – the proportion of the ones you make which are usable – for semiconductors drops as you try and make them larger. If the yield is low, then you have to make a lot of devices for each good one, and this means that the cost per device becomes very high: possibly higher than the market will bear. Smaller sensors, with the resulting higher yields, are then strongly preferred.

Here's a way of understanding the yield curve. Let's say that the chance of a defect per unit area in a process is c, and that such a defect will kill any device which is made out of that bit of semiconductor. There are other models for defects in devices but this is quite a good one.

If we want to make a device which has an area A then the chance of it not having a defect is (1 - c)A. So if A is 1 then the chance is (1 - c) and it gets smaller (since (1 - c) is less than one) as A gets larger.

The chance of a device of area A not having a defect is the yield: it's the proportion of good devices of area A we get. (In fact the yield may be lower, because there may be other things that can go wrong).

If we know the yield yA for decives of some some area A, then we can work out c: c = 1 - yA1/A (you get this by taking logs of both sides and rearranging). Equivalently we can compute the yield for any other area a as y = yAa/A.

So now, let's say that when we make 24x36mm (full-frame) sensors we get a yield of 10%: 90% of the devices we make are no good. Manufacturers are shy about saying what their yields are, but this is not implausibly low. This is equivalent to saying that c, the chance of a defect per mm2 is approximately 0.0027.

And now we can compute the yields for other areas: in fact we can just plot the yield curve against area:

In this plot I've marked the expected yields for sensors of various less-than-full-frame sizes if the full-frame yield is 10% (these may be approximate, as APS-C can mean various things, for instance). As you can see smaller sensors get much higher yields.

Over time, as manufacturing processes improve, this yield curve flattens, and the yields for big sensors improve. As this happens, larger sensors drop in price to the point where the market will bear their cost.