1592
Google AI making up recalls that didn’t happen
(lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 01 Jun 2024
1592 points (98.6% liked)
Technology
59648 readers
1543 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
In Canada there was a company using an LLM chatbot who had to uphold a claim the bot had made to one of their customers. So there's precedence for forcing companies to take responsibility for what their LLMs says (at least if they're presenting it as trustworthy and representative)
This was with regards to Air Canada and its LLM that hallucinated a refund policy, which the company argued they did not have to honour because it wasn't their actual policy and the bot had invented it out of nothing.
An important side note is that one of the cited reasons that the Court ruled in favour of the customer is because the company did not disclose that the LLM wasn't the final say in its policy, and that a customer should confirm with a representative before acting upon the information. This meaning that the the legal argument wasn't "the LLM is responsible" but rather "the customer should be informed that the information may not be accurate".
I point this out because I'm not so sure CVS would have a clear cut case based on the Air Canada ruling, because I'd be surprised if Google didn't have some legalese somewhere stating that they aren't liable for what the LLM says.
But those end up being the same in practice. If you have to put up a disclaimer that the info might be wrong, then who would use it? I can get the wrong answer or unverified heresay anywhere. The whole point of contacting the company is to get the right answer; or at least one the company is forced to stick to.
This isn’t just minor AI growing pains, this is a fundamental problem with the technology that causes it to essentially be useless for the use case of “answering questions”.
They can slap as many disclaimers as they want on this shit; but if it just hallucinates policies and incorrect answers it will just end up being one more thing people hammer 0 to skip past or scroll past to talk to a human or find the right answer.
But it has to be clearly presented. Consumer law and defamation law has different requirements on disclaimers
Yeah the legalise happens to be in the back pocket of sundar pichai. ???