[-] kromem@lemmy.world 1 points 1 day ago

Actually, they are hiding the full CoT sequence outside of the demos.

What you are seeing there is a summary, but because the actual process is hidden it's not possible to see what actually transpired.

People are very not happy about this aspect of the situation.

It also means that model context (which in research has been shown to be much more influential than previously thought) is now in part hidden with exclusive access and control by OAI.

There's a lot of things to be focused on in that image, and "hur dur the stochastic model can't count letters in this cherry picked example" is the least among them.

[-] kromem@lemmy.world 20 points 3 days ago

I was thinking the same thing!!

It's like at this point Trump is watching the show to take notes and stage direction.

[-] kromem@lemmy.world 7 points 3 days ago* (last edited 3 days ago)

Yep:

https://openai.com/index/learning-to-reason-with-llms/

First interactive section. Make sure to click "show chain of thought."

The cipher one is particularly interesting, as it's intentionally difficult for the model.

The tokenizer is famously bad at two letter counts, which is why previous models can't count the number of rs in strawberry.

So the cipher depends on two letter pairs, and you can see how it screws up the tokenization around the xx at the end of the last word, and gradually corrects course.

Will help clarify how it's going about solving something like the example I posted earlier behind the scenes.

[-] kromem@lemmy.world 5 points 3 days ago

You should really look at the full CoT traces on the demos.

I think you think you know more than you actually know.

[-] kromem@lemmy.world -3 points 3 days ago* (last edited 3 days ago)

I'd recommend everyone saying "it can't understand anything and can't think" to look at this example:

https://x.com/flowersslop/status/1834349905692824017

Try to solve it after seeing only the first image before you open the second and see o1's response.

Let me know if you got it before seeing the actual answer.

[-] kromem@lemmy.world 69 points 4 days ago* (last edited 4 days ago)

I fondly remember reading a comment in /r/conspiracy on a post claiming a geologic seismic weapon brought down the towers.

It just tore into the claims, citing all the reasons this was preposterous bordering on batshit crazy.

And then it said "and your theory doesn't address the thermite residue" going on to reiterate their wild theory.

Was very much a "don't name your gods" moment that summed up the sub - a lot of people in agreement that the truth was out there, but bitterly divided as to what it might actually be.

As long as they only focused on generic memes of "do your own research" and "you aren't being told the truth" they were all on the same page. But as soon as they started naming their own truths, it was every theorist for themselves.

[-] kromem@lemmy.world 12 points 5 days ago* (last edited 5 days ago)

The pause was long enough she was able to say all the things in it mentally.

[-] kromem@lemmy.world 3 points 5 days ago

They got off to a great start with the PS5, but as their lead grew over their only real direct competitor, they became a good example of the problems with monopolies all over again.

This is straight up back to PS3 launch all over again, as if they learned nothing.

Right on the tail end of a horribly mismanaged PSVR 2 launch.

We still barely have any current gen only games, and a $700 price point is insane for such a small library to actually make use of it.

128
submitted 3 months ago by kromem@lemmy.world to c/technology@lemmy.world

I often see a lot of people with outdated understanding of modern LLMs.

This is probably the best interpretability research to date, by the leading interpretability research team.

It's worth a read if you want a peek behind the curtain on modern models.

9
submitted 5 months ago by kromem@lemmy.world to c/technology@lemmy.world
79
submitted 5 months ago by kromem@lemmy.world to c/technology@lemmy.world
[-] kromem@lemmy.world 157 points 7 months ago

More like we know a lot more people that would have zombie bite parties because they "trust their immune system" and simultaneously don't believe in the zombie hoax.

8
submitted 7 months ago* (last edited 7 months ago) by kromem@lemmy.world to c/technology@lemmy.world

I've been saying this for about a year since seeing the Othello GPT research, but it's nice to see more minds changing as the research builds up.

Edit: Because people aren't actually reading and just commenting based on the headline, a relevant part of the article:

New research may have intimations of an answer. A theory developed by Sanjeev Arora of Princeton University and Anirudh Goyal, a research scientist at Google DeepMind, suggests that the largest of today’s LLMs are not stochastic parrots. The authors argue that as these models get bigger and are trained on more data, they improve on individual language-related abilities and also develop new ones by combining skills in a manner that hints at understanding — combinations that were unlikely to exist in the training data.

This theoretical approach, which provides a mathematically provable argument for how and why an LLM can develop so many abilities, has convinced experts like Hinton, and others. And when Arora and his team tested some of its predictions, they found that these models behaved almost exactly as expected. From all accounts, they’ve made a strong case that the largest LLMs are not just parroting what they’ve seen before.

“[They] cannot be just mimicking what has been seen in the training data,” said Sébastien Bubeck, a mathematician and computer scientist at Microsoft Research who was not part of the work. “That’s the basic insight.”

[-] kromem@lemmy.world 154 points 9 months ago

Just wait until they find out public schools are giving their children dihydrogen monoxide without asking for parental approval.

[-] kromem@lemmy.world 194 points 10 months ago* (last edited 10 months ago)

I've seen a number of misinformed comments here complaining about a profit oriented board.

It's worth keeping in mind that this board was the original non-profit board, that none of the members have equity, and literally part of the announcement is the board saying that they want to be more aligned as a company with the original charter of helping bring about AI for everyone.

There may be an argument around Altman's oust being related to his being too closed source and profit oriented, but the idea that the reasoning was the other way around is pretty ludicrous.

Again - this isn't an investor board of people who put money into the company and have equity they are trying to protect.

205
submitted 10 months ago by kromem@lemmy.world to c/world@lemmy.world
[-] kromem@lemmy.world 269 points 1 year ago

The bio of the victim from her store's website:

Lauri Carleton's career in fashion began early in her teens, working in the family business at Fred Segal Feet in Los Angeles while attending Art Center School of Design. From there she ran “the” top fashion shoe floor in the US at Joseph Magnin Century City. Eventually she joined Kenneth Cole almost from its inception and remained there for over fifteen years as an executive, building highly successful businesses, working with factories and design teams in Italy and Spain, and traveling 200 plus days a year.

With a penchant for longevity, she has been married to the same man for 28 years and is the mother of a blended family of nine children, the youngest being identical twin girls. She and her husband have traveled the greater part of the US, Europe and South America. From these travels they have nourished a passion for architecture, design, fine art, food, fashion, and have consequently learned to drink in and appreciate the beauty, style and brilliance of life. Their home of thirty years in Studio City is a reflection of this passion, as well as their getaway- a restored 1920's Fisherman's Cabin in Lake Arrowhead. Coveting the simpler lifestyle with family, friends and animals at the lake is enhanced greatly by their 1946 all mahogany Chris-Craft; the ultimate in cultivating a well appreciated and honed lifestyle.

Mag.Pi for Lauri is all about tackling everyday life with grace and ease and continuing to dream…

What a waste. A tragedy for that whole family for literally nothing. No reason at all other than small minded assholes.

view more: next ›

kromem

joined 1 year ago