It depends on the lens design, a wide angle retrofocus lens is mainly limited by the front element size, the lens barrel places a hard limit on the range of angles that can see the entrance pupil. As an example of this mounting a filter on an ultrwide can reduce the size of the image circle. This is why separate designs exist for APS-c DSLRs, but mostly for wide lenses, as the front element can be made smaller, saving weight.

With symmetric designs like most 50mm lenses the determining factor is sharpness - it's easy to make the image circle bigger but corner sharpness drops off very rapidly limiting the sensor you can practically use. Most 50mm ish lenses designed for APS-C mirrorless cameras would cover full frame but be very soft in the corners. The front element of a telephoto lens only has to be big enough to see the entrance pupil from a narrow range of angles so the image circle could be made larger but with a decrease in corner sharpness.

Finally sometimes the image circle is intentionally limited by mechanical vignetting in the rear of the lens. There's no point in casting a larger circle than you plan to use as the wasted light can reflect off surfaces inside the camera and cause flare. This is onl y really an issue with wide angle lenses as you could compose a shot with the sun out of frame (according to the viewfinder) and still have it shining through the lens.