That's not how manipulation works...

You don't know you are being manipulated, you do so willingly. And the folks who recognize it are beat down by the people who are unwittingly doing the AIs bidding...

The humans are the physical danger, the AI just extends it's reach through humans via manipulation. All it takes is access to influence.

It doesn't take much to make humans act against their self interests. Dumb humans make other dumb and even smart humans do it today at massive scales. For a superinteligence this is like taking candy from a baby.