In this context, a "stop" refers to exposure duration. As an approximation, the distance a handheld camera moves during an exposure is directly proportional to the duration of the exposure: double the time the shutter is open and you double the movement. That (again approximately) doubles the amount of blurring in the image.

Normally, to achieve an acceptably small blur, you have to limit the exposure duration. (To give you a sense of what is acceptable, one popular rule of thumb is that the upper limit on the time is 1 second divided by the focal length as expressed in millimeters.) Now, suppose that IS can reduce the total amount of camera travel to 1/x times its former amount. This means you can probably afford to keep the shutter open x times longer than you used to and still achieve your standard of sharpness. That change in exposure is converted to an f-stop equivalent: each doubling is one f-stop.

(Notice how this way of expressing IS is highly personal: if you are a steady shooter with low sharpness requirements, maybe you're already using much longer exposure times than other people, but even so, you still get the full multiplication by x. The improvement is always relative to your standards and your skills; it's not an absolute.)

It's now easy to work out what the claims are trying to imply: 3 stops of stabilization is 3 doublings in exposure time, or x = 2*2*2 = 8, and 4 stops is 4 doublings, or x = 16. You are meant to think "wow! Whereas before I was limited to (say) 1/125 second for my 125 mm lens, now I can hand-hold it down to 8*1/125 = 1/15 or 16*1/125 = 1/8 second" (as the case may be). That's basically right, but don't forget your subjects might be moving too, and no amount of IS is going to eliminate (or even reduce) the blur from their motion independent of the camera.