A hot DB should not run on HDDs. Slap some nvme storage into that server if you can. If you can’t, consider getting a new server and migrating to it.

Did this because of the convincing replies in this thread. Migrating to modern hardware and switch SQL server with PostgreSQL (because its used by the other system we work with already, and there is know-how available in this domain).

You should avoid scanning an entire table with a huge number of rows when possible, at least during requests.

But how can we then ensure that I am not adding/processing products which are already in the "final" table, when I have no knowledge about ALL the products which are in this final table?

Create an index and a table constraint on the relevant columns. ... just so that the DB can do the work for you. The DB is better at enforcing constraints than you are (when it can do so).

This is helpful and also what I experienced. In the peak of the period where the server was overloaded the CPU load was pretty much zero - all processing happened related to disk read/write. Which was because we implemented poor query design/architecture.

For read-heavy workflows, consider whether caches or read replicas will benefit you.

May you elaborate what you mean with read replicas? Storage in memory?

And finally back to my first point: read. Learn. There are no shortcuts. You cannot get better at something if you don’t take the time to educate yourself on it.

Yes, I will swallow the pill. but thanks to the replies here I have many starting points on where to start.

RTFM is nice - but starting with page 0 is overwhelming.

Omg this exists already?

Hey buddy. I feel you. Wanna talk about it?

Came for the thumbnail. But I was a little bit disappointed because I thought those were real pics.

Tl;dr; they arent.

Well you havent seen my corporation then...

Guys please upvote we all need an eli5

That sounds more like a punishment than a sleeping occasion

What else did your parents had in their parentory reportoire?

Probably about the rate, its a free service and money matters.

Im looking foward to play games with such hyperrealistic graphics with my grandkids.

If one looks what nvidia achieves in rendering with the help of AI maybe it will be my kids already.

yes actually this is what I mean. although android is linux, any distro wont run on the device

would gentoo be an option as everything is compilled?

right. makes sense. thanks for digging into this monster thread tho.

may your day be blessed.

cool thing. I assume it works by piping the linked comments to the API, with the private API key of the bot maker?