I have also experienced that changing the lens is cumbersome and sometimes you tend to miss certain events while you are anxiously changing lenses

That's something that you can improve by training it. A good large bag can also help, because it allows you to carry the lenses without caps1. This makes changing lenses less cumbersome:

1. take old lens off
2. put old lens in bag
3. get new lens out of the bag
4. attach new lens

I want to know whether its a safe bet to just travel with one lens.

Depends on what you consider "safe". From your question it sounds like missing a shot is what you are most afraid of. So here's a little thought on what images you will miss when only bringing one lens:

1. 18-55mm:
   • you will miss all the shots with focal lengths from 55 to 300mm
   • you will miss all the shots with aperture values below 4 (or whatever your kit lens has) down to f1.8 at 50mm focal length
2. 55-300mm:
   • you will miss all the shots with focal lengths from 18 to 55mm
   • you will miss all the shots with aperture values below 4 (or whatever your kit lens has) down to f1.8 at 50mm focal length
3. 50mm:
   • you will miss all the shots with focal lengths from 18 to 49mm and 51 to 300mm,

Will missing all these images counterbalance the shots you missed while changing lenses? That's a decision you have to make on your own. I doubt it.

I highly advice against leaving any of your lenses at home, because you could miss a shot while changing lenses. If you encounter a precious moment, ... click did you hear that? That was the shutter button of your camera. There wasn't even time to finish that sentence that I started, because your instincts took the shortcut to take the image first and ask questions later, because if the moment's gone, it's gone. Would that other lens in your bag be technically better suited for the situation? Maybe yes, but not being mounted to your camera diminishes that advantage to 0.

Some rules of thumb:

• Changing lenses is optional. Not bringing additional lenses makes changing lenses impossible.
• The right camera is the one in your hand. Maybe that other lens would improve the image. Guess what? So would a better camera. And spreading some flash equipment across the scene for some nifty lighting. Everything looks better at sunrise/sunset, so while changing lenses you better also turn back time.
• You missed a shot? Get over it. It happens to all of us. The field of view of a human is roughly 135°, so you are missing almost 2/3 of everything around you for your entire life. So the next time you miss a shot due to a lens change, think about why you made that lens change? Right, to better adopt to the change in environment, which should yield some improvement of the shots you are going to take in the near future.

---

1 in general, the conditions you face will allow you to do this. A good bag should provide enough protection. I don't think you are going through a sand storm in the desert. And yes, this can damage your gear, just like using it. I have seen people doing this with gear that had the same price tag as yours, just with an additional 0 at the end. You have to figure out for yourself where you draw the line between protecting your gear and the protection getting in the way of your photography.