Kelvin is used because photographic color temperature is defined by comparison to an ideal heated object: a black-body radiator.

A black body emits light purely because of its temperature. As its temperature changes, the spectrum of light it emits changes in a predictable way: lower temperatures appear warmer/redder, and higher temperatures shift toward white and then blue-white. The color of these ideal emitters traces the Planckian locus in color space.

So a light source described as 6500K means its visible hue is similar to that of an ideal black body heated to 6500 kelvin. It does not mean the light itself is physically at that temperature, nor that the illuminated object is that hot.

This is why kelvin is not arbitrary here: it comes directly from the physics of thermal radiation. In photography, the term is used because many common light sources can be usefully compared to that black-body reference when setting white balance.