There is a HUGE difference. The 17-40 is good for northern lights, moon-illuminated landscape and such where but it will struggle a bit with noise in other "darker" situations. The Samyang 24mm is very, very fast and quite sharp even wide open if you get a good copy...(BUT NOT VERY WIDE, which is why many night photog specialists usually also have an ultra-wide ~ f2.8 as well)

Here are my personal experiences:

I owned the Canon 17-40mm L and now have the new Canon 16-35 mm L. Both capable lenses and usable at night and I also use the 6D. I recently purchased the Samyang 24mm f1/4 however and I was floored how much light it can suck in. I actually overexposed a number of northern lights shots because they were simply too bright and so I had to shorten exposure and lower ISO significantly which I guess is a good thing but there is a learning curve. The secret is; because of the shorter exposure on the 24mm (stars travel less across the sky), I've ended up shooting a number of panoramas both horizontally and vertically and it seems to work quite well so I am beginning to use this lens at night almost exclusively. Definitely a special purpose lens but there are other uses even if you already have an ultra wide. I found I love the ways this lens draws and it's colour rendering which produces images that have a completely different feel from the Canon lens. The bokeh is also quite amazing and silky smooth which opens up some artistic possibilities close up and for portraits. This was a surprise to me because I didn't expect it to be that good in these situations. This however this brings up a problem because the Canon version is not chipped and therefore there is no focus confirm which for hand-held shooting, up close and portraits if pretty essential for me. I am considering adding on an aftermarket chip. If versatility is important to you you could consider the new, albeit more expensive, Sigma 24 mm f1/4 which does have autofocus.

A vertical panorama taken with the Samyang 24mm f1/4 @ f1.4, 22 seconds