I mean, its a lot of things, starting all the way back to the ARPAnet days. The fact that encryption and privacy were not core to the specifications being built at the time means for much of the Internet's history, information passed as clear text. The project started in academia, which is probably why privacy was never considered because they imagined that only working professionals and academics would use it.

It was also a military project with the stated of goal of decentralizing our telecommunications network so that in the event of a nuclear attack, the military could still maintain comms, even if one of the major nodes went down.

Another reason why so much of it was in clear text is that for the majority of the lifespan of encryption via computers, that technology was listed as a munition. Being on the munitions list and meant that it could not be exported internationally or utilized by regular citizens.

However, as more and more people came online and the commercialization of the internet began, it was pretty clear that this could be used as a method of surveillance, although I have no material evidence of that. There was likely a tipping point in the past where national security concerns outweighed the desire for domestic surveillance and encryption was removed from the munitions list.

With the corporatization of the internet however it meant that the capitalist mode of production could swing into full effect in this unregulated and untested market. This is what ultimately led to the .com bubble, but what came out of that bubble was the knowledge of how to monetize the internet as well as vast consolidation of technological advancement over the course of the bubble's life.

The true revolution in monetization came with the onset of interpersonal networks becoming centralized and privatized. Things like XMPP/Jabber loosing to privatized systems like gchat and facebook messanger. The like-button, googles search dominance, the social feed of facebook, and amazon becoming the internet marketplace meant a full circle of information for users: What are you looking for? Who do you know? Where do you live? What sites do you visit? What are you buying?

All these questions could be answered thanks to google, facebook and amazon. Its no surprise then that each of them operated their own ad service. This leads to the inevitable outcome we have today.