A lot depends on which 24-70mm f/2.8 lens you are comparing to the EF 24-105mm f/4 L IS.

There are three very good 24-70mm f/2.8 lenses you could consider:

• Canon EF 24-70mm f/2.8 L II This lens is the most expensive at a little over $2,000 U.S., but is the sharpest from 24mm all the way to 70mm at f/2.8. At the wide end where you want to use it there isn't a lot of difference between it and the other two 24-70 lenses listed below.
• Tamron SP 24-70mm f/2.8 Di VC This lens is a great value that hangs with the Canon "II" until between about 35mm and 50mm. From there to 70mm when both are wide open at f/2.8 the Canon is sharper, but not by a lot. But this lens also includes Vibration Control (VC), Tamron's version of IS. It sells for around $1,300 US and is an excellent value. I have seen it for as low as $1,000 U.S, during recent special promotions and is very attractive at that price.
• Canon EF 24-70mm f/2.8 L This is the older version of the Canon. While very good, it is a step behind both the Canon "II" and the new Tamron at longer focal lengths when used wide open at f/2.8. Used prices vary, but in general they are higher than the Tamron (and the going price when this lens was in production), but lower than the Canon "II".

At 24mm and f/2.8 all of theses lenses perform very well and close to each other. The older Canon is not quite as sharp in the center, but is just as sharp at the edges. When used at f/2.8, the Canon II starts to pull away from the Tamron and older Canon somewhere between 35mm and 50mm. The most difference between these lenses is at 70mm and f/2.8 where the "II" is clearly sharper, especially on the edges. At f/4 there is very little difference between the Canon "II" and the Tamron until between 50mm and 70mm. By f/5.6 there is no real difference between any of these three lenses at or below 50mm, but the older Canon is a little softer on the edges above 50mm. There is no real difference between the Canon "II" and the Tamron at f/5.6 or narrower at any focal length. Side-by-side comparison of theses three lenses at DxO Mark. Ignore the overall scores, they are weighted more for things you (and I) probably aren't as concerned about. Click on "Measurements-->Sharpness-->Profiles" and play around with the focal lengths and apertures for each lens to see how they compare. The left of the chart is center sharpness, the right is edge sharpness. You can also compare vignetting, chromatic aberration, distortion, and T-stop values this way.

The Canon EF 24-105mm f/4 L IS is the other lens you mentioned. It is my favorite "walk around" lens. At 24mm and f/4 it is just as sharp as the Canon 24-70 II or the Tamron 24-70 from the center out to about 40% of the frame (almost the entire frame on an APS-C body) and then slightly softer from there to the edges. At 35mm there is no real difference at f/4. But from 50mm and above the 24-105 looses ground to both the Canon II and the Tamron 24-70. It is still very good, just not quite at the same level as the 24-70 lenses. The good news is that from 70m to 105mm the sharpness of this lens stays fairly constant and gives you a little more reach than the 24-70mm lenses do. Side-by-side comparison of the Canon 24-70 II, the Tamron 24-70, and the Canon 24-105 at DxO Mark. Please see the instructions above regarding the other comparison.

For your stated purpose of shooting at focal lengths below 50mm, the best value would either be the Tamron SP 24-70mm f/2.8 Di VC or the Canon EF 24-105mm f/4 L IS. The Canon II doesn't give you that much more performance in terms of IQ until about 50mm and above, it lacks IS, and costs twice as much. Both The Tamron 24-70 and the Canon 24-105 have very good image stabilization. If you shop around and can wait for promotions that have been fairly frequent for both lenses lately the Canon is a little cheaper at around $800, the Tamron gives you f/2.8 for about $200 more. Since you say you tend to shoot in low light I'd go for the Tamron if I were you.

Note: This answer was written based on the assumption you are using a Canon Full Frame body. If you are shooting APS-C, the ideal lens for your stated usage would be the Canon EF-S 17-55mm f/2.8 IS. It is sharper than the EF 24-105mm f/4 L IS and the list price for it has been recently lowered by Canon. It can be had for around $830 U.S. It is every bit the optical match of its "L" series cousins, but Canon does not designate any of their EF-S lenses as "L" glass.

Shot with the EF 24-105mm f/4 L IS at 24mm and f/4, 1/100 second, ISO 1000. Canon 5D Mark II. Cropped slightly and then downsized for web viewing. The detail of the Sousaphone holds up very well in the full resolution version.