I think it’s fair to say that AI yields a modest productivity boost in many cases when used appropriately

I think this is a mistake. The example in this post is some empirical evidence for that.

It's not clear that we can know in advance whether it's appropriate for any given usecase; rather it seems more likely that we are just pigeons pecking at the disconnected button and receiving random intermittent reinforcement.

This sounds awful to me. Passing the tests is the starting point. It’s also important to make sure the code makes sense and is documented so whoever reads it 2 years from now (be that you, someone else, or I guess another llm) will understand what they are looking at. And that’s not even getting into if the code is efficient, or has other bugs or opportunities for bugs not captured by the test cases.

This lines up with my experience as well and what you've described is very close to how I work with LLM agents. The people bragging about 10x are either blowing smoke or producing garbage. I mean, I guess in some limited contexts I might get 10x out of taking a few seconds to write a prompt vs a couple of minutes of manual hunting and typing. But on the whole, software engineering is about so much more than just coding and those things have become no less important these days.

But the people acting like the tech is a useless glorified Markov generator are also out of their mind. There are some real gains to be had by properly using the tech. Especially once you've laid the groundwork by properly documenting things like your architecture and dependencies for LLM consumption. I'm not saying this to try to sell anybody on it but I really, truly, can't imagine that we're ever going back to the before times. Maybe there's a bubble burst like the dotcom bubble but, like the internet, agentic coding is here to stay.