Civox, the company behind this service, is ensuring the AI discloses that it is AI to callers. They've also said they won't take VC money to ensure their board's ethics values aren't diluted. It won't be long before less high-minded people are doing what they are doing.

The EU's Artificial Intelligence Act (AI Act) will ban people from using tech like this to lie or deceive people, but few people anywhere else in the world have such protections yet. Disinformation has become the norm on social media already, it is almost inevitable people will use this technology for it. I suspect the kind of people who are already most vulnerable to disinformation, will be the easiest to be fooled by thinking AI's calling them are real humans, if the AI says it's human.