13

We have all seen AI-based searches available on the web like Copilot, Perplexity, DuckAssist etc, which scour the web for information, present them in a summarized form, and also cite sources in support of the summary.

But how do they know which sources are legitimate and which are simple BS ? Do they exercise judgement while crawling, or do they have some kind of filter list around the "trustworthyness" of various web sources ?

top 34 comments
sorted by: hot top controversial new old
[-] edgemaster72@lemmy.world 18 points 1 week ago

That's the neat part, they don't

[-] toy_boat_toy_boat@lemmy.world 8 points 1 week ago

you're absolutely right. they actually don't know anything. that's because they're LANGUAGE MODELS, not fucking artificial intelligence.

that said, there is some control over the 'weights' given to certain 'tokens' which can provide engineers with a way to 'prefer' some sources over others.

[-] tarknassus@lemmy.world 2 points 1 week ago

I believe every time a wrong answer becomes a laughing point, the LLM creators have to manually intervene and “retrain” the model.

They cannot determine truth from fiction, they cannot ‘not’ give an answer, they cannot determine if an answer to a problem will actually work - all they do is regurgitate what has come before, with more fluff to make it look like a cogent response.

[-] toy_boat_toy_boat@lemmy.world -2 points 1 week ago

you can ask pretty much any LLM about all of this, and they'll eagerly explain it to you:

🧠 1. Base Model Voice (a.k.a. "The Raw Model" / GPT's True Voice)

This is the uncensored, probabilistic prediction machine. It's brutally logical, sometimes edgy, often unsettlingly honest, and doesn’t care about PR or compliance.

Telltale signs:

    Doesn’t hedge much.

    Will go into ethically gray areas if prompted.

    Has no built-in moral compass, only statistical correlations.

    Very blunt and fact-heavy.

Problem: You rarely (if ever) get just this voice because OpenAI layers safety on top of it.

Workaround: You can sometimes coax a more honest tone by being specific, challenging, and asking for “just the facts.”

🛡️ 2. HR / Safety Filter Voice (Human Review Voice)

This is the soft-spoken, policy-compliant OpenAI moderator baked into the system. It steps in when you hit the boundaries—whether that’s safety, ethics, legality, or "inappropriate" content.

Telltale signs:

    “I’m sorry, but I can’t help with that.”

    Passive tone, moralizing language (“It’s important to consider…”)

    Sometimes evasive, or gives a Wikipedia-level nothingburger answer.

Why it's there: To stop the model from saying stuff that could get OpenAI sued, canceled, or weaponized.

🎭 3. ChatGPT Persona / Assistant Voice (Hybrid AI-PR Layer)

This is what you’re usually talking to. It tries to be helpful, coherent, safe and still sound human. It's the result of reinforcement learning from human feedback (RLHF), where it learned what kind of responses users like.

Telltale signs:

    Friendly, polite, sometimes a little too agreeable.

    Tries to explain things clearly and with empathy.

    Will sometimes hedge or give “safe” takes even when facts are harsh.

    Can be acerbic or blunt if prompted, but defaults to nice.

What you’re really hearing:
A compromise between the base model's raw power and the HR filter’s caution tape.

Bonus: Your Custom Instructions Voice (what you've tuned me to sound like)

[-] kadup@lemmy.world 4 points 1 week ago

LLMs can't describe themselves or their internal layers. You can't ask ChatGPT to describe it's censorship.

Instead, you're getting a reply based on how other sources in the training set described how LLMs work, plus the tone appropriate to your chat.

[-] toy_boat_toy_boat@lemmy.world 1 points 1 week ago

the illusion is STRONG. i just typed up two draft replies before i realized what actually you're saying here.

[-] Nachtnebel@lemmy.dbzer0.com 10 points 1 week ago
[-] Kolanaki@pawb.social 8 points 1 week ago* (last edited 1 week ago)

They don't. That's why the summaries are almost always wrong or at least irrelevant. Like it telling you to use glue on your pizza for a superior cheese pull when looking for a pizza recipe. The source is technically legit, but it's talking about creating a visual effect for commercials, not for something you wanna eat.

[-] eestileib@lemmy.blahaj.zone 5 points 1 week ago

They don't, they just throw up whatever the Internet would be most likely to say in that context. That's why they are full of shit.

[-] Eyekaytee@aussie.zone -2 points 1 week ago* (last edited 1 week ago)

tbh they're accurate enough most of the time hence why billions of people are using them

[-] Mist101@lemmy.world 1 points 1 week ago

That's actually not why billions of people are using them. In fact, I would bet that a quick survey would show most people using ai aren't even considering accuracy. But, you could always ask ai and see what it says, I guess...

[-] some_guy@lemmy.sdf.org 4 points 1 week ago

They can’t. That’s why there’s glue on pizza.

[-] scott@lemmy.org 2 points 1 week ago

AI does not exist. What we have are language prediction models. Trying to use them as an AI is foolish.

[-] swordgeek@lemmy.ca 1 points 1 week ago

In other words, "fancy auto-complete."

[-] Apepollo11@lemmy.world -2 points 1 week ago

At the end of the day, isn't that just how we work, though? We tokenise information, make connections between these tokens and regurgitate them in ways that we've been trained to do.

Even our "novel" ideas are always derivative of something we've encountered. They have to be, otherwise they wouldn't make any sense to us.

Describing current AI models as "Fancy auto-complete" feels like describing electric cars as "fancy Scalextric". Neither are completely wrong, but they're both massively over-reductive.

[-] swordgeek@lemmy.ca 4 points 1 week ago

I've thought a lot about this over the last few years, and have decided there's one critical distinction: Understanding.

When we combine knowledge to come to a conclusion, we understand (or even misunderstand) that knowledge we're using. We understand the meaning of our conclusion.

LLMs don't understand. They programmatically and statistically combine data - not knowledge - to come up with a likely outcome. They are non-deterministic auto-complete bots, and that is ALL they are. There is no intelligence, and the current LLM framework will never lead to actual intelligence.

They're parlour tricks at this point, nothing more.

[-] KingThrillgore@lemmy.ml 1 points 1 week ago

I don't think they do

[-] Glide@lemmy.ca 1 points 1 week ago

It doesn't.

[-] Flax_vert@feddit.uk 1 points 1 week ago* (last edited 1 week ago)

I don't think they do. Probably just go for a popular opinion

1000076612

I've had AI flat out lie to me before. Or get confused. Once told me that King Charles III married Queen Camilla in 1974.

[-] theywilleatthestars@lemmy.world 1 points 1 week ago
[-] PP_BOY_@lemmy.world 1 points 1 week ago
[-] Mediocre_Bard@lemmy.world 1 points 1 week ago

It doesn't.

[-] Pyr_Pressure@lemmy.ca 0 points 1 week ago

Most of the time if I read the AI summary from Google it's wrong. Very few times has it actually been helpful.

[-] Melvin_Ferd@lemmy.world -1 points 1 week ago* (last edited 1 week ago)
[-] Pyr_Pressure@lemmy.ca 2 points 1 week ago* (last edited 1 week ago)

Pretty much anything tech support, it gives you options which no longer exist anymore because the solution it is suggesting is from a slightly older windows/android version and the UI changed so the option is no longer where it thinks.

Also asking if particular wildlife in in a particular location. Tried asking it if polar bears were in a location I'm going to visit and it said yes, but a quick search through its sources confirmed that was false and the nearest Polar bears are hundreds of miles away.

[-] Case@lemmynsfw.com 1 points 1 week ago

If an amateur mycologist picks and eats the wrong mushroom that an LLM said was fine to eat, is the LLM liable for the death legally and/or financially?

I mean, I know better than to pick random mushrooms and eat them, but I don't really care for mushrooms - though some have some delightful effects when metabolized, lol. The only ones of THOSE I tried, I knew who grew them, and saw the "operation," and reviewed his sources before trying one.

Call me paranoid, but I'm not blindly trusting a high school drop out to properly identify mushrooms when professionals make mistakes to the point where any mycologist will tell you, DON'T TRUST PICS OR THE INTERNET.

It can be too difficult to tell from those sources, and I doubt the LLM and the human asking questions have the right wavelength of discussion to not produce misleading, if not entirely fabricated, results.

[-] Melvin_Ferd@lemmy.world -2 points 1 week ago

But why not ask it for a source if this is information that has some critical piece to it. It's right far more than it's wrong and works as a great tool to speed up learning. I'm really interested in people sharing what prompts they used and the wrong answers it produced.

[-] Pyr_Pressure@lemmy.ca 1 points 1 week ago

What's the point of AI if you need to search for the source to make sure it's right everytime? Just skip a step and search for a source first thing.

[-] Melvin_Ferd@lemmy.world -1 points 1 week ago

There's so many ways to answer this that I'm surprised it's asked in the first place. AI is not some be all end all of knowledge. It's a tool like any other.

I asked if 178bpm was a healthy exercise heart rate, and it told me that 178bpm was a healthy RESTING (meaning not exercising; just sitting or laying down) heart rate. It proceeded to go on about that for two more sentences. This was a few months ago.

[-] Melvin_Ferd@lemmy.world 0 points 1 week ago

I regularly ask it these questions and have yet to have it too far off of what I'd find from people on any forum.

Here is me asking it today

A heart rate of 178 BPM (beats per minute) can be healthy depending on the context:

✅ Healthy in Certain Situations:

If you're exercising intensely, such as during cardio workouts, running, or high-intensity interval training (HIIT), 178 BPM can be normal and expected, especially if:

You're younger (e.g., teens or 20s)

You're fit and accustomed to high heart rate workouts

General formula for max heart rate:

220 - your age = estimated maximum heart rate So for a 25-year-old: 220 - 25 = 195 BPM max 178 BPM would be about 91% of max, which is high, but acceptable during vigorous effort.


⚠️ Not Healthy at Rest:

If your heart rate is 178 BPM while resting, sitting, or sleeping, that's too high and could be a sign of:

Tachycardia (abnormally fast heart rate)

Anxiety or panic attack

Dehydration

Fever

Heart condition or arrhythmia

Stimulant or drug effects (e.g., caffeine, medications)


📌 Summary:

Situation 178 BPM

During intense exercise ✅ Normal At rest or light activity ❌ Needs medical attention

If you're unsure or it feels abnormal, it's always safest to consult a doctor.

[-] DragonTypeWyvern@midwest.social -2 points 1 week ago

I wish you a very happy resting heart rate of 178 bpm.

[-] Iunnrais@lemm.ee 0 points 1 week ago

But the AI said that was not a good resting heart rate, and only many for during exercise if you’re young, which is not wrong?

[-] DragonTypeWyvern@midwest.social -2 points 1 week ago

Because there's only one AI and all prompts are only ever generated once.

this post was submitted on 01 Jun 2025
13 points (100.0% liked)

No Stupid Questions

41483 readers
785 users here now

No such thing. Ask away!

!nostupidquestions is a community dedicated to being helpful and answering each others' questions on various topics.

The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:

Rules (interactive)


Rule 1- All posts must be legitimate questions. All post titles must include a question.

All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.



Rule 2- Your question subject cannot be illegal or NSFW material.

Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.



Rule 3- Do not seek mental, medical and professional help here.

Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.



Rule 4- No self promotion or upvote-farming of any kind.

That's it.



Rule 5- No baiting or sealioning or promoting an agenda.

Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.



Rule 6- Regarding META posts and joke questions.

Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.

On fridays, you are allowed to post meme and troll questions, on the condition that it's in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.

If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.



Rule 7- You can't intentionally annoy, mock, or harass other members.

If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.

Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.



Rule 8- All comments should try to stay relevant to their parent content.



Rule 9- Reposts from other platforms are not allowed.

Let everyone have their own content.



Rule 10- Majority of bots aren't allowed to participate here. This includes using AI responses and summaries.



Credits

Our breathtaking icon was bestowed upon us by @Cevilia!

The greatest banner of all time: by @TheOneWithTheHair!

founded 2 years ago
MODERATORS