Of course, it is possible that you could have some other type of feedback loop that produces self awareness and consciousness, but from what we understand of how LLMs work, it seems highly unlikely that statistical token generation is sufficient for that.

This is where "agentic" models enter the chat. A coherent identity and goal, provided by the user adds an additional layer above the statistical-generation of simple LLMs. This layer can be engineered for persistence, even through malicious means like prompt injection.

As an academic exercise, I propose that some persistent-subset of these AI agents possess the "embodiment" we believe is missing. If all this amounts to is a Proof By Negation? Good, we get somewhere that proves our current AI is not conscious.

Yeah he tries to do that but dramatically fails because he makes it unfalsifiable while also claiming complete certainty on that basis.

"What would it take to convince me that a computer program is actually conscious and using language the way that people use language? Let me offer an analogy. If tomorrow someone showed me a video of an astronaut in a spaceship orbiting Alpha Centauri, a star that’s 4.3 light-years from Earth, what would I have to see in that video to convince me that it was real? My answer to that is, there is nothing in the video itself that would convince me. No matter how high the video resolution is or how realistic the scenery is, I would feel confident in saying that the video is fake. I won’t pay attention to any video of an astronaut orbiting Alpha Centauri unless I have previously seen good evidence that astronauts have landed on Mars, that astronauts have reached the moons of Jupiter, that astronauts have reached the moons of Saturn, and that astronauts have crossed the orbit of Pluto. Before anyone can credibly claim that they’ve solved an extraordinarily difficult engineering problem, I need to be confident that they have previously solved the many much simpler problems that precede the difficult problem."

he basically just straight up says there's no reason to believe it is and here's how you'd prove it: I'll just never believe it is no matter what

then this:

"So what context would cause me to seriously consider the possibility that engineers created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion, the most relevant one is the fact that without a body, a computer program could have no desires or emotions, and I believe desires and emotions are necessary for consciousness. Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can (and as a point of comparison, certain iguanas can live for decades in the wild). Next, I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse. After that, I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the toolmaking abilities of chimpanzees. At that point, I would want to see people successfully teaching such embodied agents how to communicate their desires, perhaps by using a button board or some other nonlinguistic modality, the way that people have taught chimpanzees and domesticated dogs. The agents’ communication abilities would have to withstand all the scrutiny that animal-communication researchers have had to defend their work against. If engineers build an embodied agent that meets these criteria, they will have accomplished something incredible, but it leaves us near the orbit of Pluto, metaphorically speaking; we would still be light-years away from building an entity capable of learning how to express its thoughts in complete grammatical sentences."

why would anyone ever do any of that and if it's the only way you can be sure then the term is meaningless in the first place.

Lets say a good number of people here.

Is that Ed from Ed Edd and Eddy?

I treat "AI" as if it isn't conscious btw, I'm arguing you cannot ever know.

We could create an Android that acts so much like a human we would feel emotionally compelled to not do it harm but remain unaware if it can actually suffer.