Like many things regarding lens design, most everything is a compromise of one sort or another.

Examples: A zoom lens can't be optimized to correct all aberrations equally at all focal lengths - correcting barrel distortion at the wide end usually increases pincushion distortion on the long end and vice-versa. Even with prime lenses correcting one type of aberration often introduces another - spherical and coma aberration correction often introduces astigmatism.

Using IS can help solve problems related to handholding a camera at slower shutter speeds, but using IS can also introduce additional problems. This is true even when the IS system is operating as designed. Additionally, as with just about any mechanical or technological device, increasing the complexity of the overall design increases the chances that something may not work as intended under certain conditions or that a piece of the design will fail.

When operating as designed an IS system corrects for slight movements by moving the IS lens group inside the lens to compensate. Depending on the speed and severity, in terms of angular change, of the movement the position of the IS group moves to match the movement in the opposite direction. This movement of the IS group does not come without a price. Tilting or decentering the optical elements in the IS group has the same consequence as tilting or decentering any other group in the lens: the lens is no longer in the same optical alignment. Assuming the lens is best lined up when the IS group is parked at a "centered" position, the further the IS group moves from that centered position the more the effect of the difference in optical performance will be. Since the IS group continues to move in an attempt to stabilize the lens while the shutter is open, the optical performance of the lens will vary over the duration of the exposure. Note that these differences should be very slight, but they are present and can have the effect of "...smooth(ing) everything out." The greater the degree of the corrections needed to counteract movement, the greater the degree of the slight differences in the optical characteristics of the lens over the course of these movements. All movements of any mechanical device can also introduce vibrations. Normally, though, the vibrations introduced by the IS unit are insignificant compared to the external vibrations they are intended to counteract.

When operating outside of the system design an IS system can degrade image quality to a much more significant degree. The optical elements of the IS unit may be continuously out of alignment, even when parked at the "centered" position. One of the two servos that move the IS elements in the lens in question may become inoperative or operate erratically. Since both servos are designed to be coordinated in their movements, if one servo responds properly to a command from the camera and the other does not, it can throw the total movement/alignment of the IS unit well out of the intended range of movement. A defective IS unit may also vibrate excessively, introducing blur.

From this blog written by Roger Cicala, founder and lens guru at lensrentals.com, you can see that IS repairs made up about 15% of the repairs needed by their collection of EF 70-200mm f/2.8 L IS II lenses for their July 2012- June 2013 period. For the first six months of 2012 IS unit replacements were 29% of total repairs to the same lens. As Roger points out in the article (bold emphasis added):

I think the other trend that’s very obvious is that large, f/2.8 zoom lenses are likely to have issues. Sony, Canon, and Nikon 70-200 f/2.8 lenses all made the list. The Canon 100-400 is there, too, while the Nikon 80-400 and Canon 28-300 barely missed (and usually make the list). The Sigma 50-500 OS and 120-300 OS are on it, too, with the Sigma 70-200 f/2.8 OS barely missing (in a small sample size of 20 copies).

Each seems to have its own issues. The Canon 70-200 IS II has problems with the nylon reverse gears in the zoom ring and its IS unit. The 100-400 with its outer focusing barrel and notorious smooth-tighten ring. The Sigmas have HSM motor problems, OS issues, and barrel/cam issues. The plate for Nikon’s tripod foot is too thin and bends frequently, and the AF system is a bit finicky. The takeaway message is these are among the most complex lenses made today and the complexity shows in higher repair rates.

Without visual examples that show the degree of severity with which you are experiencing the problem you have identified it is difficult to say if what you have noticed is within the expected normal range of using IS or if it is outside that range and indicates the IS unit may be in need of attention from a Canon service center.

---

Regarding using the EF 70-200mm f/2.8 L IS mounted on a tripod: The EF 70-200mm f/2.8 L IS auto senses when it is solidly mounted, such as on a stable tripod, and adjusts the IS in such a use case. From his review of the lens at The-Digital-Picture Bryan Carnathan says:

This IS version is tripod-sensing - The Canon EF 70-200mm f/2.8L IS USM Lens knows that a tripod is being used when vibrations go below a certain level. Keep IS turned on when mounting on a tripod to take advantage of this secondary IS mode - reducing mirror slap, shutter and tripod vibrations. The downside is the slightly additional battery drain.

Concerns about IS introducing blur due to being on a tripod apply to lenses with IS that do not have such an auto sensing feature.

The EF 75-300mm f/4-5.6 IS (1995), EF 300mm f/4 L (1997), and EF 28-135mm f/3.5-5.6 IS (1998), Canon's first three IS lenses, did not have auto sensing. The EF 100-400mm f/4.5-5.6 L IS (1998) was the first Canon IS lens to include the auto switching feature. The Super Telephoto IS series of lenses introduced in 1999 went a step further and not only detect when they are solidly mounted but switch to a "tripod" IS mode that is tuned to correct for internal camera vibrations such as mirror slap and shutter vibrations as well as very minor tripod movements.