You seem to be confusing the time it takes to expose the sensor that is the time each pixel well on the sensor collects light information and the time it takes to readout the sensor that is the time it takes to collect the information that each pixel well has recorded.

If I have a shutter speed of s-seconds, then does it mean that the camera takes s-seconds to capture each row of an image?

No, it means that each row is exposed to light for s-seconds. But it does not mean that each row must be exposed one at a time. It also does not mean that the sensor must be read out in the same direction it was exposed. In fact, Canon DLSRs expose from the top to bottom of the sensor when using mechanical shutter curtains but then after the second curtain has closed the sensor is read out from the bottom to the top. Since the image projected by the lens is inverted on the sensor, this means the image as it is viewed is exposed from the bottom to the top but the sensor reads out from the top to the bottom of the image as it is viewed.

I believe there must be a delay between capturing one row after another. Can this delay period be estimated via shutter lag?

No. Again, more than one row can be collecting light at the same time. At shutter times longer than a camera's flash sync speed all of the pixels are collecting light simultaneously for a portion of the exposure time. The CMOS sensor must be read out sequentially, but that doesn't mean it must be exposed sequentially. The time needed to readout the sensor will be the same regardless of how long or how short the exposure time was.

What about the delay that occurs after capturing one image and then capturing the next image during video recording using rolling shutter?

When recording video the mechanical shutter curtains are not used. Each line of the sensor is exposed for the designated interval. But the recording times for each line can overlap so that more than one line is collecting light at any given time. Only the readout, when the information each line has recorded is collected and sent to the camera's analog-to-digital convertor, must be done sequentially one line at a time. While one line is being read out the other lines can be collecting light.