Yes it is and there are several ways to do that.

The easiest which works with any camera (still or video) is to split the video file into a sequence of stills using software like ffmpeg. Then pass the images to an Exposure Fusion software. While it was not it primary intention, Exposure Fusion works really well to blend images which results in lower noise, increased dynamic-range and greater depth-of-field (depending on the fusion weights).

Should you want more direct controls, it won't be hard to load all these images as layers in a software which supports this concept like Photoshop. Then all you need is to set the correct blending mode. I am guessing something like Average would do it.

As @Russell said, this can also be done from images using Multi-Frame noise-reduction. Sony is big on it and so is Fuji in their CMOS based cameras (F and HS series). They call it Pro Low-Light mode. @mattdm said Pentax does it too but from memory I do not remember which one.

I suspect but didn't try it that you would get very similar results using the Multi-Exposure mode present in most Pentax, Nikon, Olympus ILCs (plus the Canon 1D X and 5D Mark III). This is usually limited to 2-9 images. Note that you must enable Auto Gain on Nikon & Olympus and Auto EV Adjustment on Pentax for this to blend rather than add exposures.

Should you want to do your own Multi-Frame noise-reduction you want to use a proper exposure, not under-exposed images. This is because noise is much higher in dark areas of images. Obviously, if the scene is too dark to be exposed properly then you have to work with what you can get. Adding multiple images to create a brighter one is called Image Stacking and is used for Astrophotography. Given the context in your question, this may be what you were looking for.