As long as your subject is at a large distance from the camera (non-macro), small translations in the position of the camera don't really have an impact on the image. Instead, the impact of your unsteadiness is in the pointing of the camera—that is, the rotation of the camera about its axes.

So the traditional 1/FL rule is saying that in 1/FL seconds, the angular pointing error of your hand-holding becomes significant. Significant compared to what? It has to be another angular quantity—namely the angular field of view of the camera. Thus, in the original 1/FL rule, FL is being used as a proxy for angular field of view. It's just convenient happenstance that it tends to work out well numerically for most people, when working on 35mm.

Since the FL is being used as a proxy for the quantity we really care about—field of view—it's necessary to adjust it to other sensor formats, and that means using the "35mm equivalent focal length" in the rule of thumb, not the true physical focal length.

Of course, there are lots of other caveats, not the least of which is image stabilization. These are discussed at length in another question here.