[-] brucethemoose@lemmy.world 1 points 2 hours ago* (last edited 2 hours ago)

Ah. You can still run them in exllamav2, but you're probably better off with ik_llama.cpp then:

https://github.com/ikawrakow/ik_llama.cpp

It supports special "KT" quantizations, aka trellis quants similar to exllamav3, and will work with vulkan (or rocm?) on your 6800.

Quantizing yourself is not too bad, but if you want, just ping me, and I can make some 16GB KT quants, or point you to how to do it yourself.

It's also a good candidate for Qwen3 30B with a little CPU offloading. ik_llama.cpp is specifically optimized for MoE offloading.

[-] brucethemoose@lemmy.world 4 points 6 hours ago* (last edited 6 hours ago)

To expand on what others said, there has been tons of corporate dysfunction at Intel.

Some hardware efforts have gone pretty well, and had decent long-term trajectories but were sabotaged by direction changes or early cuts from up high. There seems to be some corporate "game of thrones" going on between branches too, and a lot of redundancy on the software dev side.

AMD has a similar issue actually, and it's starting to bite, especially on the GPU side.

[-] brucethemoose@lemmy.world 3 points 6 hours ago* (last edited 6 hours ago)

The ideal state is some kind of bastardized hybrid, federally funded and guided but still private.

This is how TSMC rose.

It's also kinda how Samsung Foundry exists, with how Samsung is so giant and tied to SK politics that's it blurs the line with the govt.

Trump is going in the opposite direction unfortunately, regulating the shit out of them in terms of policy (tariffs, politics), yet yanking grants and "freeing" the government of them.

[-] brucethemoose@lemmy.world 3 points 6 hours ago* (last edited 6 hours ago)

their new datacenter hardware is hyper tuned for LLMs at the expense of general compute, unlike AMD

This is not true. The AMD MI300X/MI325X are, if anything, even more tuned for AI. They're missing ROPs when Nvidia's datacenter GPUs (last I checked) still have them.

...And honestly the demand for datacenter GPUs outside of AI is pretty small, anyway.

Also, CUDA has always been and will be the dominant compute API.

I'm not trying to shill Nvidia here. Screw them. The MI cards are better hardware anyway, just with a worse and (ironically) more AI specialized software stack that has utterly sabotaged them.

[-] brucethemoose@lemmy.world 1 points 6 hours ago

GPUs are also not really taking off.

Arc Battlemage was great, albeit late. Not cutting the big die and sticking with it one more generation, they would have been golden.

Falcon shores seems like a management disaster, yeah...

On the CPU side, they still sell a ton of laptop CPUs, and some efforts like the small-core server CPUs are competitive.

Intel was not toast, they just need to stop the corporate dysfunction and stick with some efforts. But it seems they can't even manage that.

[-] brucethemoose@lemmy.world 3 points 18 hours ago

Oh, heavens, I can only imagine what crapware OEMs would cook up with full access to the OS…

How would you like 11 gigabytes of junkware in your kernel? That only works on that version? Oh, and your computer won’t work without it.

[-] brucethemoose@lemmy.world 5 points 19 hours ago* (last edited 19 hours ago)

Meta pays for PyTorch development as well!

Llama.cpp will be fine of course, it technically has nothing to do with Meta.

But yeah, it’s mostly disappointing IMO…

And kinda stupid. These are literally experimental models; they release one experiment with mixed results, and admittedly catastrophically marketing for it, and Zuck pulls the rug?

[-] brucethemoose@lemmy.world 16 points 22 hours ago

My impression was that Meta's backing out of Llama LLMs anyway, to focus on “products”

[-] brucethemoose@lemmy.world 4 points 1 day ago

Mmmmmmmm...

What about continuing the Dominion thread? I am wide open to ideas there, but shapeshifters would both be awesome in modern CG and awesome as officers.

[-] brucethemoose@lemmy.world 5 points 1 day ago* (last edited 1 day ago)

Pop sci-fi seems to be fond of intermediate-mass black holes (EG Interstellar, Star Trek StrangeNew Worlds), and for something kinda the size of a star, they are "scary."

In other instances (like in TV Foundation), a close orbit to the accretion disk is a source of suspense.

And then there's the "stealth" aspect. Stellar-mass ones and below are very small and (potentially) quiet for something with the mass of a star, eg easy to stumble upon.

And in some very advanced universes (eg the online Orion's Arm), even with "hard" sci fi, swimming through a star's nuclear plasma is totally doable. But a black hole is an impossible boundry of physics, and an particularly extreme object useful for astroengineering.

[-] brucethemoose@lemmy.world 2 points 2 days ago* (last edited 2 days ago)

Ah, right!

I'll also add that Enterprise is pretty 'standalone' too. It's not like how Voyager and DS9 directly continue TNG's politics/characters; you can go back and watch it later. You can watch other series without missing much from Enterprise.

If I were you, I'd watch Picard as "scenery change" to modern, high res, serial Star Trek, while all those characters and plots are still fresh. Call it a breather after your Voyager marathon, and a farewell VOY, DS9, and TNG, as it's basically built on Voyager and TNG lore.

Then I'd go back to classic, more episodic ST and watch Enterprise.

I dunno about movies, as I am not an expert on those (and which ones are good or not).

[-] brucethemoose@lemmy.world 8 points 2 days ago* (last edited 2 days ago)

Obviously make sure you have watched DS9, the best Star Trek (fight me!).

Star Trek: Enterprise is not bad like the internet would lead you to believe. I say watch it, and skip around if you find you don't like it. But if you watched all of Voyager, I think you will like all of Enterprise.

...But if you skip it... hmm. Chronologically and character-wise (other than movies) ST Picard seems like the next step. It kinda 'picks up' from where TNG and Voyager left off. Discovery and SNW are more standalone.

13

"We're seeing a unifying moment. The band is back together," MAGA podcaster Jack Posobiec told Axios.

"He gets attacked just relentlessly by the Wall Street Journal in such an uncalled for way, and we have his back 100% against this smearing and this slandering," Charlie Kirk added on his show.

59
submitted 1 month ago* (last edited 1 month ago) by brucethemoose@lemmy.world to c/leopardsatemyface@lemmy.world

As to why it (IMO) qualifies:

"My children are 22, 25, and 27. I will literally fight ANYONE for their future," Greene wrote. "And their future and their entire generation's future MUST be free of America LAST foreign wars that provoke terrorists attacks on our homeland, military drafts, and NUCLEAR WAR."

Hence, she feels her support is threatening her kids.

"MTG getting her face eaten" was not on my 2025 bingo card, though she is in the early stage of face eating.

32

"It's not politically correct to use the term, 'Regime Change' but if the current Iranian Regime is unable to MAKE IRAN GREAT AGAIN, why wouldn't there be a Regime change??? MIGA!!

47
submitted 1 month ago* (last edited 1 month ago) by brucethemoose@lemmy.world to c/world@lemmy.world

Video is linked. SFW, but keep your volume down.

1
submitted 2 months ago* (last edited 2 months ago) by brucethemoose@lemmy.world to c/usa@lemmy.ml

In a nutshell, he’s allegedly frustrated by too few policies favorable to him.

63
submitted 3 months ago* (last edited 3 months ago) by brucethemoose@lemmy.world to c/world@lemmy.world
  • The IDF is planning to displace close to 2 million Palestinians to the Rafah area, where compounds for the delivery of humanitarian aid are being built.
  • The compounds are to be managed by a new international foundation and private U.S. companies, though it's unclear how the plan will function after the UN and all aid organizations announced they won't take part
17
Qwen3 "Leaked" (huggingface.co)

Qwen3 was apparently posted early, then quickly pulled from HuggingFace and Modelscope. The large ones are MoEs, per screenshots from Reddit:

screenshots

Including a 235B/22B active and a 30B/3B active.

Context appears to 'only' be 32K unfortunately: https://huggingface.co/qingy2024/Qwen3-0.6B/blob/main/config_4b.json

But its possible they're still training them to 256K:

from reddit

Take it all with a grain of salt, configs could change with the official release, but it appears it is happening today.

23
submitted 3 months ago* (last edited 3 months ago) by brucethemoose@lemmy.world to c/localllama@sh.itjust.works

This is one of the "smartest" models you can fit on a 24GB GPU now, with no offloading and very little quantization loss. It feels big and insightful, like a better (albeit dry) Llama 3.3 70B with thinking, and with more STEM world knowledge than QwQ 32B, but comfortably fits thanks the new exl3 quantization!

Quantization Loss

You need to use a backend that support exl3, like (at the moment) text-gen-web-ui or (soon) TabbyAPI.

13
submitted 3 months ago* (last edited 3 months ago) by brucethemoose@lemmy.world to c/localllama@sh.itjust.works

Seems there's not a lot of talk about relatively unknown finetunes these days, so I'll start posting more!

Openbuddy's been on my radar, but this one is very interesting: QwQ 32B, post-trained on openbuddy's dataset, apparently with QAT applied (though it's kinda unclear) and context-extended. Observations:

  • Quantized with exllamav2, it seems to show lower distortion levels than nomal QwQ. Its works conspicuously well at 4.0bpw and 3.5bpw.

  • Seems good at long context. Have not tested 200K, but it's quite excellent in the 64K range.

  • Works fine in English.

  • The chat template is funky. It seems to mix up the and <|think|> tags in particular (why don't they just use ChatML?), and needs some wrangling with your own template.

  • Seems smart, can't say if it's better or worse than QwQ yet, other than it doesn't seem to "suffer" below 3.75bpw like QwQ does.

Also, I reposted this from /r/locallama, as I feel the community generally should going forward. With its spirit, it seems like we should be on Lemmy instead?

31
submitted 3 months ago* (last edited 3 months ago) by brucethemoose@lemmy.world to c/asklemmy@lemmy.world

So I had a clip I wanted to upload to a lemmy comment:

  • Tried it as an (avc) mp4... Failed.
  • OK, too big? I shrink it to 2MB, then 1MB. Failed.
  • VP9 Webm maybe? 2MB, 1MB, failed. AV1? Failed.
  • OK, fine, no video. Lets try an animated AVIF. Failed. It seems lemmy doesn't even take static AVIF images
  • WebP animation then... Failed. Animated PNG, failed.

End result, I have to burden the server with a massive, crappy looking GIF after trying a dozen formats. With all due respect, this is worse than some aging service like Reddit that doesn't support new media formats.

For reference, I'm using the web interface. Is this just a format restriction of lemmy.world, or an underlying software support issue?

-2

53% of Americans approve of Trump so far, according to a newly released CBS News/YouGov poll conducted Feb. 5 to 7, while 47% disapproved.

A large majority, 70%, said he was doing what he promised in the campaign, per the poll that was released on Sunday.

Yes, but: 66% said he was not focusing enough on lowering prices, a key campaign trail promise that propelled Trump to the White House.

44% of Republicans said Musk and DOGE should have "some" influence, while just 13% of Democrats agreed.

1
submitted 7 months ago* (last edited 7 months ago) by brucethemoose@lemmy.world to c/politics@lemmy.world

Here's the Meta formula:

  • Put a Trump friend on your board (Ultimate Fighting Championship CEO Dana White).
  • Promote a prominent Republican as your chief global affairs officer (Joel Kaplan, succeeding liberal-friendly Nick Clegg, president of global affairs).
  • Align your philosophy with Trump's on a big-ticket public issue (free speech over fact-checking).
  • Announce your philosophical change on Fox News, hoping Trump is watching. In this case, he was. "Meta, Facebook, I think they've come a long way," Trump said at a Mar-a-Lago news conference, adding of Kaplan's appearance on the "Fox and Friends" curvy couch: "The man was very impressive."
  • Take a big public stand on a favorite issue for Trump and MAGA (rolling back DEI programs).
  • Amplify that stand in an interview with Fox News Digital. (Kaplan again!)
  • Go on Joe Rogan's podcast and blast President Biden for censorship.
view more: next ›

brucethemoose

joined 1 year ago