Well put! I think I kinda misunderstood what you were saying, I guess we sort of reached the same conclusion from different directions. And yeah, it does seem like we're hitting the limits of what can be achieved from the current underlying word-prediction mechanisms alone, with how diminishing the returns are from dumping more data in. Maybe something big will happen soon, but it looks to me like LLMs will stagnate for a while until they're taken in a fundamentally new direction.

Either way, what they can do now is pretty incredible, and equally interesting to me is how it's making us reevaluate our ideas of consciousness and intelligence on a large scale; it's one thing to theorize about what could happen with an 'intelligent' AI, but the reality of these philosophical questions being so thoroughly challenged and dissected in mundane legal and practical matters is wild.