The sad thing: Throwing hardware at a problem was actually cheaper for a long time. You could buy that $1500 CPU and put it in your dedicated server, or spend 40 developer hours at $100 a pop. Obviously I'm talking about after the easy software side optimizations have already been put in (no amount of hardware will save you if you use the wrong data structures).

Nowadays you pay $500 a month for 4 measly CPU cores in Azure. Or "less than 1 core" for an SQL Server.

Obviously you have a lot more scalability and reliability in the cloud. But for $500 a month each we had a 16 core, 512 GB RAM machine in the datacenter (4 of them). That kind of hardware on AWS or Azure would bankrupt most companies in a year.

Well, having been on the other side, sometimes the Dev is also trying to fight the good fight whilst having to use some crap 3rd party system/library that's imposed from above because somebody at the C-suite level after suitably dinned and wined (and who knows what more, including implied or even explicit promises for the future of their career) signed a massive agreement with one of the big corporate software providers so now those of us at the coalface have to justify to money spent on that contract by using every POS from said big corporate software provider.

I mean, I might be exagerating the overtly corrupt nature of the deal (in my experience its more a mix of CTO incompetence - or being pretty much powerless at the C-Suite level because his is not the core business, hence overriden - and the high-level management trading favours using company money and more for personal rather than corporate reasons) but even competent devs that know their thing can't really do much when they have to use a bug-riddled POS massive framework from some vendor that doesn't even have proper support, for "corporate reasons".

You and me both

We worked with a business unit to predict how many people they would migrate on to their new system week 1-2 … they controlled the migration through some complicated salesforce code they had written.

We were told “half a million first week”. We reserved capacity to be ready to handle the onslaught.

8000 appeared week 1.

That this is deemed brilliant is the sad part.

That reminded me about trying to compile a rust application (Pika Backup) on a laptop with 4 GB of RAM (because AUR).

That was a fun couple of attempts. Eventually I just gave up and installed a flatpak.

Depends on which crappy software vendor I'm dealing with in any given week. lol

Said app is Chrome

*cough* Oracle *cough*

Then we'd probably have to drop each and every vendor...😩

Hmm is unexpected factorial a sub here yet?

This is why Java rocks with ETL, the language is built to access files via input/output streams.

It means you don't need to download a local copy of a file, you can drop it into a data lake (S3, HDFS, etc..) and pass around a URI reference.

Considering the size of Large Language Models I really am surprised at how poor streaming is handled within Python.

I phrased that panel poorly. App in question always had 16 GB and still never used above 4.

Does the OS really eat up 7GB RAM?

Here is an alternative Piped link(s): https://piped.video/pFlcqWQVVuU?si=E7XxLW64539jGqL3

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I'm open-source, check me out at GitHub.