My bet is on the same thing you guess — light reflected from the diffuser bouncing around badly. This is especially likely if you've tipped the flash to point the diffuser dome forward, as I've seen some people do. The EF 85mm f/1.2L is not a physically long lens and even with a hood you might have the light source protruding way too far forward.

This is easily tested by making some images with and without the flash on. I'd suggest a neutral target (the infamous brick wall shot), and run through a range of apertures and flash power.

Assuming that's it — or actually, even if it's not — I'd suggest replacing the dome diffuser with an off-camera flash and possibly (depending on what you're doing) a larger-yet-portable softbox. That sounds like it might be inconvenient outdoors, but the thing is, it's exactly because you're shooting outdoors. This style of diffuser does increase the effective flash area a little bit, softening light some, but its main effect is to scatter light in every direction. Indoors, that light then bounces from ceilings and walls, but outdoors (or in a large room), the primary effect is simply to waste light. (See When and how to use a push-on flash diffuser? for more.)

You can get that same effect for free and without risk of weird stray light by leaving off the diffusion dome and simply cutting the flash power. So maybe you don't even need to go the off-camera route ... but that's an option that will give you a lot more flexibility.