Yeah, I use ChatGPT fairly regularly for work. For a reminder of the syntax of a method I used a while ago, and for things like converting JSON into a class (which is trivial to do, but using chatGPT for this saves me a lot of typing) it works pretty good.

But I'm not using it for precise and authoritative information, I'm going to a search engine to find a trustworthy site for that.

Putting the fuzzy, usually close enough (but sometimes not!) answers at the top when I'm looking for a site that'll give me a concrete answer is just mixing two different use cases for no good reason. If google wants to get into the AI game they should have a separate page from the search page for that.

They give zero fucks about their customers, they just want to pump that stock price so their RSUs vest.

This stuff could give you incurable highly viral brain cancer that would eliminate the human race and they'd spend millions killing the evidence.

True, and it's excellent at generating basic lists of things. But you need a human to actually direct it.

Having Google just generate whatever text is like just mashing the keys on a typewriter. You have tons of perfectly formed letters that mean nothing. They make no sense because a human isn't guiding them.

It's like the difference between being given a grocery list from your mum and trying to remember what your mum usually sends you to the store for.

I mean, it does learn, it just lacks reasoning, common sense or rationality.
What it learns is what words should come next, with a very complex a nuanced way if deciding that can very plausibly mimic the things that it lacks, since the best sequence of next-words is very often coincidentally reasoned, rational or demonstrating common sense. Sometimes it's just lies that fit with the form of a good answer though.

I've seen some people work on using it the right way, and it actually makes sense. It's good at understanding what people are saying, and what type of response would fit best. So you let it decide that, and give it the ability to direct people to the information they're looking for, without actually trying to reason about anything. It doesn't know what your monthly sales average is, but it does know that a chart of data from the sales system filtered to your user, specific product and time range is a good response in this situation.

The only issue for Google insisting on jamming it into the search results is that their entire product was already just providing pointers to the "right" data.

What they should have done was left the "information summary" stuff to their role as "quick fact" lookup and only let it look at Wikipedia and curated lists of trusted sources (mayo clinic, CDC, national Park service, etc), and then given it the ability to ask clarifying questions about searches, like "are you looking for product recalls, or recall as a product feature?" which would then disambiguate the query.

It really depends on the type of information that you are looking for. Anyone who understands how LLMs work, will understand when they'll get a good overview.

I usually see the results as quick summaries from an untrusted source. Even if they aren't exact, they can help me get perspective. Then I know what information to verify if something relevant was pointed out in the summary.

Today I searched something like "Are owls endangered?". I knew I was about to get a great overview because it's a simple question. After getting the summary, I just went into some pages and confirmed what the summary said. The summary helped me know what to look for even if I didn't trust it.

It has improved my search experience... But I do understand that people would prefer if it was 100% accurate because it is a search engine. If you refuse to tolerate innacurate results or you feel your search experience is worse, you can just disable it. Nobody is forcing you to keep it.