Alright, I see what you're saying now. We're on the same page.

As an additional thing regarding AGI, I think it should be noted that 'human-level' and 'human-like' are importantly distinct when talking about this topic.

In reality, if an AGI is ever created, it will most likely not be human-like at all. Humans think the way we do out of an evolutionary conditioning for survival, a history an AGI will not be coming from. One example given by Robert Miles is a staple making machine becoming an ASI, where it essentially would exist solely to make as many staples as it could with its hyperintelligence.

We mean to say that this AGI is a 'human-level' intelligence in that it can learn to utilize abstractions and tools, be able to function in a large variety of environments without intervention or training, and be able to learn in a realtime fashion.

Obviously, these criteria for any AI shows just how far away we are from achieving anything right now.these concepts are very vague and the arguments for each one's impossibility or inevitability are equally vague and philosophical. It's still mostly just stuffy academics arguing with each other.

One statement I agree with, though, comes from the AI safety collective: We don't know what we're doing, and we should really sort that out. If any of this is actually possible and we accidentally make an AGI/ASI before having any failsafes or contingencies, it could be very bad.