Jokes aside, I think there's a subtle cause and effect mixup.
If you "have it easy" you develop confidence, and that both emerges thanks to privilege but also is an actual advantage granted by privilege.
You do find more success if you don't hesitate too much, but you will always hesitate more if you have no safety net.