Talk to colleagues about it. I didn't and the stress ate my soul until I had to take a mental health break. Most of the time you are not more of an idiot than they are and they have things to say about you that make them feel insufficient.

Some are just idiots though, but they often feel too confident to realize.