There can be differences. For some subjects, a longer exposure just works better.

One of the classic examples is taking a picture of moving water in a river or stream. With a short exposure, the water looks rather strange and frozen. With a longer exposure, you get a smooth, flowing look that many people find more pleasing. Depending on how fast the stream is moving, it often takes an exposure around 10-15 seconds (or so) to achieve this look, so in bright light you often just about need an ND filter to get a long enough exposure.

Another place that a really long exposure can be helpful is taking pictures of popular tourist attractions. If you're patient (and have a tripod) with a heavy ND filter, you can take something like a five minute exposure. The people (who rarely sit still very long) mostly disappear. Take a half dozen or so, and combine them properly in Photoshop, and most of the tourists simply disappear.

Of course, there are also cases where you really want a short exposure. The most obvious is almost anything (like sports) that involves fast action. With too long of an exposure, most of this would turn into an incoherent blur. On the other hand, I think many people to a bit overboard with this, trying to completely "freeze" everything in the shot. I generally prefer to use a long enough exposure that a few of the fastest moving parts are still at least slightly blurred, to retain a sense of motion.

Just for example, consider this shot:

Here I worked pretty hard to find what I considered the "right" shutter speed. The obvious strategy would have been to shoot with the aperture wide open, to get the shortest shutter speed, so everything would be frozen and sharp looking.

I chose, instead, to use a longer exposure and pan with the bike, so the background and spokes of the wheels are blurred, so you get clarity but get a clear sense of motion. In this case, I only used the aperture to get a slower shutter speed, but if I'd had the right ND filter with me, I'd have used it--shallower depth of field to further blur the background would be an improvement.

This can get a little tricky though. For example, another shot from the same night:

Part of the time I think this would be better if I'd used a faster shutter speed. Other times, I think the kind of...frenetic feel of it is just about right. Kind of depends on what sort of result you want though--at least to me, the way it is right now is strong on "feel", but would probably be a problem if you wanted a really factual presentation.