Yes, it's worth worrying about. Generally these fix software bugs or even add new features. It's very expensive and time consuming to get a software product near-bug free at launch — a luxury camera makers don't have in a fast-moving market. Firmware is the "operating system" of your camera, and keeping up with the latest version will make sure your camera is operating at its best.

Cameras companies tend to be somewhat secretive, often fixing things beyond what's listed, so even if the changelog doesn't seem interesting, it's usually still a good idea. However, unless I have an urgent probem, I usually wait a few weeks after a firmware is released to apply it to my camera. I've never heard of a catastrophic firmware release, buy I figure I'll let someone else be the guinea pig.

The procedure varies in details from manufacturer to manufacturer, but in general, you download a file, extract that blob to a memory card (sometimes using a custom program), and power on the camera with some button held. There will be instructions on the support site for your camera. The main risk is loss of power during the upgrade — if that happens, the camera may be bricked — the technical term for "turned into a doorstop" — and need to be sent in for repair (rarely covered by warranty). So make sure you have a fresh, fully-charged battery (or use an AC adaptor if you have one).