No, they're not. That doesn't mean that cheap plastics are used; in fact, the non-glass elements are usually considerably more expensive (and more difficult to produce) than the optical glass elements.

Optical glasses are made in a number of different formulations (such as crown glass and flint glass) that have different optical properties, including different indices of refraction (which has the effect of bending light more or less with the same shape of lens) and different dispersion characteristics (the amount that the light spectrum is spread out). No single lens is perfect, so multiple elements of different shapes and optical characteristics are used to correct one another.

The corrective elements are often made of exotic non-glass crystals, like fluorite. Less often, aspherical molded elements are cast from an optical resin (plastic, if you prefer) bonded onto a more conventional glass element. (These days, the cast aspherical elements are more likely to be glass, not so much because resin is a bad thing, but because of the consumer acceptance factor. The main problem with resin lenses is that they are easily scratched or pitted, which is not really a problem when the element is buried deep within the lens body.) These non-glass elements are usually found in better, more expensive (and longer) lenses, usually to reduce chromatic aberration and approach true apochromaticity.