There's a lot of startups trying to make better search engines. Brave for example is one of them. There's even one Lemmy user, but I forget what the name of theirs is.

But it's borderline impossible. In the old days, Google used webscrapers and key word search. When people started uploading the whole dictionary in white text on their pages, Google added some antispam and context logic. When that got beat, they handled web credibility by the number of "inlinks" from other websites. Then SEO came out to beat link farmers, and you know the rest from there.

An indexable version of Archive.org is feasible, borderline trivial with ElasticSearch, but the problem is who wants that? Sure you want I may, but no one else cares. Also, let's say you want to search up something specific - each page could be indexed, with slight differences, thousands of times. Which one will you pick? Maybe you'll want to set your "search date" to a specific year? Well guess what, Google has that feature as well.