I suspect, with almost 0 evidence, that the reason clickbait seems so effective is that there's a large contingent of viewers that it works on. So when you try playing the game, you'll see a surge in popularity. But such easily-won viewers are also easily lost if you don't play the game well enough, or someone else does it better. So if you want to keep your numbers up, you have to play the game hard, but doing that can drive away the loyal viewers that would seek out your videos even if the algorithm stopped promoting them. You can't measure your success through metrics alone.

In a way, if you play the game how YouTube wants you to play it (which is of course what makes them the most money,) you stop being an artist using YouTube as your platform and start being a worker for YouTube instead.