Camera sensors (see this article for an overview) consist of a very large number of individual sensor elements, each of which can be regarded as a bucket that collects photons. These buckets have a maximum number of photons they can capture before they become full, which is called being saturated (this is when the highlights clip). This maximum capacity is not affected by the ISO setting of the camera.

As stated in this question the lowest or "base" ISO usually corresponds to the unamplified sensitivity of the sensor, increasing the ISO results in the readings from the sensor being multiplied, meaning that the maximum value that can be stored in a recorded image can be reached before the sensor wells become saturated.

Decreasing the ISO would therefore require a 'deamplification' of the sensor readings, which can only be done after reading the sensor levels and so cannot actually decrease the sensitivity of the sensor to incoming light. This will cause any artificial decrease in sensitivity to also result in a corresponding decrease in dynamic range as you are compressing the total range of sensor levels into a less-than-total part of the range of levels that can be recorded in an image.

All this is a long way of saying that camera sensors have a fixed minimum sensitivity to light, regardless of what ISO it is referred to as. Any artificial decrease of ISO below this sensor minimum will both not actually make the sensor less sensitive to light and will also decrease the range of light levels that can be recorded in an image.

Neutral density filters solve the problem of wanting a longer exposure (without clipping) than the camera's minimum sensitivity allows by reducing the amount of light that hits the sensor, which causes the sensor buckets to fill up at a slower rate than they would without the filter being used.

Decreasing the ISO below the minimum can only be done after the recorded values are read from the sensor and therefore cannot actually darken the image hitting the sensor which is what is required to get a 'genuinely' extended exposure time. What such a decrease would result in is an increase in perceived exposure time when looking at the final image that is actually just an artefact of squashing the light levels recorded on the sensor into a smaller range when writing them into the image. Neutral density filters do make the image hitting the sensor darker which does result in an extended 'genuine' exposure time.

The numerical value of the lowest ISO is calculated by reference to standards as described in this question.