A better tool than Smudge for this is usually the Spot Healing Brush. For a small object against a simple background like sky/horizon, it can often remove it cleanly in one pass.

Tips:

• Use a brush slightly larger than the building.
• Let the brush cover the object and a little of the surrounding background so Photoshop has enough context to replace it.
• If the first pass isn’t perfect, use it as a starting point and refine with small additional healing or cloning.

The Smudge tool tends to smear pixels, which is why the result can look obviously retouched. Spot Healing is generally more natural for this kind of cleanup.

If needed, you can combine tools: start with Spot Healing, then do minor touch-up with cloning or another cleanup tool. In your example, the community result showed Spot Healing worked well and gave a much cleaner outcome than Smudge.