You can run llama.cpp on CPU. LLM inference doesn't need any features only GPUs typically have, that's why it's possible to make even simpler NPUs that can still run the same models. GPUs just tend to be faster. If the GPU in question is not faster than an equally priced CPU, you should use the CPU (better OS support).

Edit: I looked at a bunch real-world prices and benchmarks, and read the manual from Huawei and my new conclusion is that this is the best product on the market if you want to run a model at modest speed that doesn't fit in 32GB but does in 96GB. Running multiple in parallel seems to range from unsupported to working poorly, so you should only expect to use one.

Original rest of the comment, made with the assumption that this was slower than it is, but had better drivers:
~~The only benefit to this product over CPU is that you can slot multiple of them and they parallelise without needing to coordinate anything with the OS. It's also a very linear cost increase as long as you have the PCIe lanes for it. For a home user with enough money for one or two of these, they would be much better served spending the money on a fast CPU and 256GB system RAM.~~

~~If not AI, then what use case do you think this serves better?~~

For 2000$ it "claims" to do 140 TOPS of INT8
When a Intel Core Ultra 7 265K does 33 TOPS of INT8 for 284$

Don't get me wrong, I would LOVE to buy a chinese GPU at a reasonnable price but this isn't even price competitive with CPUs let alone GPUs.