don’t have much to do with the large language models

On a technical level I disagree: they're only using one convolution layer. The biggest change compared to previous work on the same dataset is the gated MLP, which is an idea that's inspired by transformers (1), which in their turn created the LLM that are hyped.

In general, I agree that AI is a useless marketing term.