You're right: it's less of an issue, and your reasoning is also right.

To see this consider the absurd case: a really huge sensor. Let's imagine a sensor so large that its pixels are a centimetre on a side. To obscure a pixel you would need a bit of 'dust' a centimetre on a side, and as you say the physics of scale is such that an object that large simply won't stick to a vertical surface, not least because that's the kind of scale where gravity matters.

In fact the pixel size doesn't really matter. Imagine an even more absurd sensor: one the same size (say 10m on a side, which would be one megapixel sensor with square-centimetre-sized pixels), but with normal-sized pixels, a sensor which would have of the order of a trillion pixels (4 trillion if they are 5 micron pixels). Now obviously dust would obscure individual pixels on such a sensor, so if you pixel-peeped on it you would see dust. But, assuming you looked at such an image in a reasonable way then a bit of dust would have to obscure a huge numbers of pixels: at least tens of thousands of them. So dust is not a problem for a sensor like this eiter.

Obviously such sensors are absurd, but the scaling question is what matters: as sensors get bigger, the kind of dust which will stick to a sensor doesn't get bigger, so it becomes less important. More dust does get on the sensor (the amount of dust presumably goes as sensor area), but it becomes less visible.

And indeed there is very good evidence for this for another kind of sensor: photographic film. Dust is a problem for film not, usually, when it is exposed but when it is printed: it is a significant problem to keep negatives really clean when you are printing them, and you quite often have the frustrating experience of making a really nice print (on paper which has a significant per-sheet cost!) only to discover obtrusive dust. For 35mm film, this is a real problem, and people have blowers and soft cloths &c &c to try to keep things clean. But one of the delights of printing from large-format film (say 4x5in, which I print from) is that dust is almost a non-problem, because, although there is dust, it is so much less visible with such a large neg.