45
A computer can never be held accountable
(simonwillison.net)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 03 Feb 2025
45 points (97.9% liked)
Hacker News
2200 readers
601 users here now
Posts from the RSS Feed of HackerNews.
The feed sometimes contains ads and posts that have been removed by the mod team at HN.
founded 10 months ago
MODERATORS
This is 100% true and moral.
Some people think that a hypothetical AGI being intelligent is enough to put it on charge of decisions; it is not. It needs to be a moral agent, and you can't really have a moral agent that is not punishable or reward-able.
Most HN comments ITT are trash, so addressing the one with some merit:
Emphasis in the original. True, a corporation cannot be held accountable for its actions; thus it should not be in charge of management decisions.
Holding someone accountable doesn't undo their mistakes, once a decision is made, there is often nothing you can do about it. Humans make bad decisions too, whether unknowingly or intentionally. It's clear that accountability isn't some magic catch-all.
I find the idea that punishment and reward are prerequisites of morality rather pessimistic, do you believe people are entirely incompetent of acting morally in the absence of external motivation?
Whichever way, AI does essentially function on the principle punishment and reward, you could even say it has been pre-punished and rewarded in millions of iterations during its training.
AI simply has clear advantages in decision making. Without self-interest it can make truly selfless decisions, it far less prone to biases and takes much more information into account in its decisions.
Try throwing some moral, even political questions at LLMs, you will find they do surprisingly well, and these are models that aren't optimized for decision making.
It is not some magic catch-all on a pragmatic level, but it is still regardless a necessary condition to avoid an otherwise rational agent fucking everything up.
This "but what about humans..." does not contradict what I said. (Some humans cannot be held accountable for their decisions either.)
I don't care if this is "pessimistic" or "optimistic" or skibidi. I'm more concerned about it being contextually accurate or inaccurate, true or false, consistent or inconsistent, etc.
And at the end of the day, all morality boils down to
If you have other grounds for morality feel free to share them.
It is not a matter of incompetence. Incompetence, in this case, would be failure to generate moral rules based on the consequences of one's actions.
This is a sleight of hand, as the word "training" is being sloppily used to refer to both "tweaking an automated system" and "human skill development", and human skill development often requiring rewards vs. punishment.
Emphasis mine. You're being self-contradictory - first claiming AI can be rewarded/punished, then claiming that it lacks self-interest.
The main issue with this idea of punishment and reward, in the sense that you mean them, is that their results depend entirely on the criteria by which you are punished or rewarded. Say, the law says being gay is illegal and the punishment is execution, does that mean it's immoral?
Being moral boils down to making certain decisions, the method by which they are achieved is irrelevant if the decisions are "correct". Most moral philosophies agree that moral decisions can be made by applying rational reasoning to some basic principles (e.g. the categorical imperative). We reason through language, and these models capture and simulate that. The question is not whether AI can make moral decisions, it's whether it can be better than humans at it, and I believe it can.
I watched the video, honestly I don't find anything too surprising. ChatGPT acknowledges that there are multiple moral traditions (as it should) and that which decision is right for you depends on which tradition you subscribe to. It avoids making clear choices because it is designed that way for legal reasons. When there exists a consensus in moral philosophy about the morality of a decision, it doesn't hesitate to express that. The conclusions it comes to aren't inconsistent, because it always clearly expresses that they pertain to a certain path of moral reasoning. Morality isn't objective, taking a conclusive stance on an issue based on one moral framework (which humans like to do) isn't superior to taking an inconclusive one based on many. Really this is one of our greatest weaknesses, not being able to admit we aren't always entirely sure about things. If ChatGPT was designed to make conclusive moral decisions, it would likely take the majority stance on any issue, which is basically as universally moral as you can get.
The idea that AI could be immoral because it holds the stances of its developers is invalid, because it doesn't. It is trained on a vast corpus of text, which captures popular views and not the views of the developers.
On morality:
Shifting the goalposts from "it's pessimistic" to "it depends on criteria".
I was being simplistic to keep it less verbose. I didn't talk about consistency, for example, even if it also matters.
By "reward/punishment" I mean something closer to behaviourism than to law. I could've called them "negative stimulus" and "positive stimulus" too, it ends the same.
For the bigots implementing and supporting such idiotic laws? Yes, they consider being gay immoral. That's the point for them.
For gay people? It throws them into a catch-22, because following such a law is also punishing: they're forced into celibate and prevented from expressing their sexuality and sexual identity.
So even considering your example, rooting morality into punishment and reward still works. And you can even retrieve a few conclusions out of it:
Back to AI. Without ability to be rewarded/punished, not even a hypothetical AGI would be a moral agent. At most it would be able to talk about the topic, but not generate a consistent set of moral rules for itself. And no, model feeding, tweaking parameters, etc. are clearly not reward/punishment.
That's circular reasoning, given that the "correct decisions" will be dictated by moral.
That does not address the request.
I'll rephrase it: since you disagree that moral values ultimately come from reward and punishment, I asked where you think that they come from. For example, plenty of the moral philosophies that you mentioned root their moral values into superstitions, like "God" or similar crap.
Language and reasoning:
That's likely false, and also bullshit (i.e. a claim made up with no regards to its truth value).
While language and reasoning do interact with each other, "there is a substantial and growing body of evidence from across experimental fields indicating autonomy between language and reasoning" (paywall-free link.
That already dismantles your argument on its central point. But I'll still dig further into it.
They don't even capture language as a whole, let alone a different system like reasoning.
They output decent grammar and vocab. But they handle notoriously poorly meaning (semantics) and utterance purpose (pragmatics). They show blatant signs of not knowing what they are outputting.
You can test this by yourself by asking any LLM-powered chatbot of your choice about some topic that you know by heart. Then looking at the incorrect answers, and asking yourself why the bot is outputting that wrong piece of info ("hallucination").
Example here.
Except the fact that it directly contradicts your claim.
Emphasis mine.
This is the cherry of the cake because it shows that you're wasting my time with a subject that you're completely clueless about. I'm saying that those systems are amoral, not immoral.
Sorry to be blunt but I'm not wasting my time further with you.