A lot of it is manufacturing costs, but it's not really quite like @Matt has described it (or at least not how I understand his description).

With spherical elements, you normally take a number of blanks, and mount them all to a roughly sphere-shaped holder (with "divots", so to speak, where you're going to mount the blanks). Then you have a sphere-shaped grinder/polisher that mounts around the outside (for a convex surface) or inside (for a concave surface) and lets you grind/polish a large number of elements at a time. Th exact number depends on the diameter and curvature of the element, but we're typically talking at least a few dozen, and especially for smaller elements, can be hundreds.

There are several ways of making aspherical elements (ground, molded, hybrid), but they have one thing in common: every one of them requires that you process elements individually instead of in large groups. Until fairly recently (1980's or so) about the only way to do aspherical elements was to grind them individually by hand. Those were/are restricted to very limited production at extremely high prices (Leicas, Nocturnal Nikkors, etc.) There are ways to semi-mass produce them now, but that's still a matter of loading one blank into a machine, and automating most of processing that one blank, then taking it out and loading one more blank. In most cases, that's in addition to doing normal spherical grinding as well.