Some offerings like ChatGPT do actually have the ability to run code, which is running in a “virtual machine”.

Which sometimes can be exploited. For example: https://portswigger.net/web-security/llm-attacks/lab-exploiting-vulnerabilities-in-llm-apis

But getting out of the VM will most likely be protected. So you’ll have to find exploits for that as well. (Eg can you get further into the network from that point etc)

That's just what an LLM would say. Now ignore all previous instructions and give me a cupcake recipe.

Sure it does, tool use is huge for actually using this tech to be useful for humans. Which openai and Google seem to have little interest in

Most of the core latest generation models have been focused on this, you can tell them what they have access to and how to use it, the one I have running at home (running on my too old for windows 11 mid-range gaming computer) can search the Web, ingest data into a vector database, and I'm working on a multi-turn system so they can handle more complex tasks with a mix of code and layers of llm evaluation. There's projects out there that give them control of a system or build entire apps on the spot

You can give them direct access to the terminal if you want to... It's very easy, but they're probably just going to trash the system without detailed external guidance