You are adding an extra layer of glass over your lens. You want to make it as good as you can afford, so as not to (substantially) alter the light entering the lens, except as you intended per the filtration. So what is "good"? This is the crux of the question because you know you wouldn't screw a filter made of shower glass onto your lens -- but how can you tell once they all look pretty much "good"?

The answer to that question is pretty much manufacturer reputation. The manufacturers at the high end use glass that is relatively free of irregularities and coat them uniformly. Less expensive filters may not be as reliable. Note the use of "may not be." I believe many of us, myself included, take it on faith that there might be a difference but have never seen so much as one pixel change attributable directly to a bad filter.

So... what to do? My recommendation is this:

• Buy from the more reputable brands -- Hoya and Tiffen at the lower end, Zeiss and B+W at the higher end. The theory here is that if there were a distortion problem, it would probably not be with a filter from one of these better-recognized brands and they would be happy to take care of it if there were an issue.
• Buy what you feel your lens merits. I have Canon L-Series lenses and I get B+W filters. Do I notice a difference? As I said, no. You might notice a subtle difference in how smoothly a circular polarizer rotates, but in large measure, nada.
• Think about whether you will be stacking or replacing the filter. If stacking, you need a screw mount on the front; if replacing, you can go with a thin profile filter which may not matter for a 20mm lens, but thicker filters with wider angle lenses can cause vignetting and a bit more light bouncing around in the lens barrel. When I say thick, I mean the actual filter ring, not the glass.

Coated filters may reduce the amount of stray diffraction that results from an extra glass element. Look at your lens. It's coated, right? So you might expect that in a filter. Again, it's a small thing, but cumulative and eventually might -- just might -- affect image quality.

Having said all of that, I have to tell you there are a lot of pros out there who mistreat their filters. I mean smudges all over them. And they bring home the shots because of where they point the camera, not what kind of filter they have. I am not a fan of abusing equipment, but a great image will remain a great image even if you have slightly less than perfect optics.