This is essentially how the Klingons originally worked. They would enslave weaker and/or less advanced planets and make those people produce raw materials for the Empire. It was very imperial/feudal/mercantilist.

The problem is that the Prime Directive as presented in Star Trek is shown as this fully formed dictate without anyone having a good idea as to why it is important. It is made worse with Star Trek's episodic form that doesn't go back to previous worlds to check up on them; Lower Decks makes it a point to show that the Federation is really bad at this.

In contrast, Stargate has a few episodes which show how such a policy would be good for the program.

On screen it suffers from the audience's availability heuristic. We see the times when it's hard to follow or should be broken because those are the encounters worth making an episode about. The mundane "Here's a planet in its Bronze Age. We take only holographs and leave only footprints."

The directive is simple though. Do not interfere.
The rest is just captains disregarding it.

Indeed.

A bit lacking in genocide, conquests, and annexation to be manifest destiny, eh?