Its different I think when it is your peers that find you annoying. And especially when what they find annoying are traits often associated with neurodivergence. For example I definitely exhibited conflict seeking behavior, I interrupted people a lot, I talked too much, had trouble paying attention to people when they are talking to me, and had trouble staying on topic. I also had a lot of trouble telling when I was bothering people, and thus did not stop when I should've.

Do you have to have ADHD to be annoying? No. Are you inherently annoying if you have ADHD? Also no. But ADHD makes it a lot harder to not come off as annoying to people, and I definitely struggled to not be annoying because of my neurodivergence.