There are two dangers in the current race to get to AGI and in developing the inevitable ANI products along the way. One is that advancement and profit are the goals while the concern for AI safety and alignment in case of success has taken a back seat (if it's even considered anymore). Then there is number two - we don't even have to succeed in AGI for there to be disastrous consequences. Look at the damage early LLM usage has already done, and it's still not good enough to fool anyone who looks closely. Imagine a non-reasoning LLM able to manipulate any media well enough to be believable even with other AI testing tools. We're just getting to that point - the latest AI Explained video discussed Gemini and Sora and one of them (I think Sora) fooled some text generation testers into thinking its stories were 100% human created. In short, we don't need full general AI to end up with catastrophe, we'll easily use the "lesser" ones ourselves. Which will really fuel things if AGI comes along and sees what we've done.