The photo looks like I did not focus properly. However how can I focus if the stars are indefinitely far away from us?

Indeed.

For stars, you need to manually focus.

How to do it with a modern DSLR / MILC:

• First, find some very bright star
• Select the live view mode
• Aim the camera at the very bright star
• Zoom in the live view (not the lens!) as much as you can so that you can accurately manually focus. By this I mean use the camera's digital magnification button, not turning the zoom ring of the lens.
• Select the manual focus mode on the lens
• Turn the focusing ring of the lens very carefully to make the bright start as point-like as you can
• Recompose the shot as you wish without touching the focusing of the lens anymore
• Take the picture with proper exposure

Note that some focus-by-wire lenses may respond differently to slow and rapid movements of the focusing ring, i.e. turning it very little abruptly may be different from turning it the same amount extremely slowly. At least for canon RF lenses, you can change the behaviour of the focusing ring from the camera menu. Not all lenses support this, though.

Edit: some additional notes:

• Lenses are usually not parfocal. So set the zoom ring position first and only then focus, not vice versa.
• If the camera has exposure simulation, you need to have correct exposure set before you see the very bright star in the live view mode.