This is just a statement of faith in your ability to judge these things accurately. Nowhere in here do I see any evidence that you’ve even considered that the reason you’ve changed your attitude towards the tech is that it’s just gotten so good at fooling people that it’s finally got you.
You don’t gain much from trying to convince me, but you could gain a lot from being more sceptical. People invented science to address the fact that our intuitive understanding doesn’t always reflect reality.
Science and the collection of objective data stops us from doing this:
There are a bunch of things that our brains just don’t understand intuitively, so we need to check our intuition against measurement. There’s no shame in that, but when it’s pointed out, then you have a chance to check yourself.
But you don’t seem to understand that. When you say:
Anyway, you’ll see all this eventually, when some data gets published.
you are demonstrating that you are the perfect mark for this stuff, because you are not reflecting on your own thought process to see where it might be failing you.
This is just a statement of faith in your ability to judge these things accurately. Nowhere in here do I see any evidence that you’ve even considered that the reason you’ve changed your attitude towards the tech is that it’s just gotten so good at fooling people that it’s finally got you.
Yet in all of your replies, you seem to have assumed early on that I’ve been fooled, based on outdated data. Do you just assume that newer data just doesn’t exist anywhere, and I’m lying about it? (To be clear: I wouldn’t blame you. There’s an old proverb: “Believe nothing you hear, and only half of what you see,” or something like that.)
you could gain a lot from being more sceptical
Another assumption that I wasn’t skeptical.
Anyway, the rest of your reply continues with the assumption that there was no data or objectivity on my part, so I won’t keep beating a dead horse. Just wait for newer data. It might be old by the time you see it, but still useful.
Edit: I suppose the number of recent layoffs might be useful (or at least interesting) data. Suddenly many different, unrelated companies had too many engineers – quite a contrast to the engineer shortage just a few years ago. Correlation ≠ causation and all, but interesting nonetheless.
And even for complex coding projects like the ones studied, the researchers are also optimistic that further refinement of AI tools could lead to future efficiency gains for programmers. Systems that have better reliability, lower latency, or more relevant outputs (via techniques such as prompt scaffolding or fine-tuning) “could speed up developers in our setting,” the researchers write. Already, they say there is “preliminary evidence” that the recent release of Claude 3.7 “can often correctly implement the core functionality of issues on several repositories that are included in our study.”
Claude 3.7 was released in February 2025. Also, I highly doubt 3.7 was good enough to make engineers more productive, overall (though I don’t have data on those old models). Relative to the speed of evolution of LLMs, harnesses, and people’s skills in using them, the data behind this article is ancient.
Edit 3:
In that article you shared, they link to the study in the second paragraph. Follow that link, and you’ll see this at the top:
Update: In February 2026, we published new data on the productivity impact of late-2025 AI tools.
There were selection effects in the follow-up study, but seemed worth mentioning anyway.
This is just a statement of faith in your ability to judge these things accurately. Nowhere in here do I see any evidence that you’ve even considered that the reason you’ve changed your attitude towards the tech is that it’s just gotten so good at fooling people that it’s finally got you.
You don’t gain much from trying to convince me, but you could gain a lot from being more sceptical. People invented science to address the fact that our intuitive understanding doesn’t always reflect reality.
Science and the collection of objective data stops us from doing this:
There are a bunch of things that our brains just don’t understand intuitively, so we need to check our intuition against measurement. There’s no shame in that, but when it’s pointed out, then you have a chance to check yourself.
But you don’t seem to understand that. When you say:
you are demonstrating that you are the perfect mark for this stuff, because you are not reflecting on your own thought process to see where it might be failing you.
Yet in all of your replies, you seem to have assumed early on that I’ve been fooled, based on outdated data. Do you just assume that newer data just doesn’t exist anywhere, and I’m lying about it? (To be clear: I wouldn’t blame you. There’s an old proverb: “Believe nothing you hear, and only half of what you see,” or something like that.)
Another assumption that I wasn’t skeptical.
Anyway, the rest of your reply continues with the assumption that there was no data or objectivity on my part, so I won’t keep beating a dead horse. Just wait for newer data. It might be old by the time you see it, but still useful.
Edit: I suppose the number of recent layoffs might be useful (or at least interesting) data. Suddenly many different, unrelated companies had too many engineers – quite a contrast to the engineer shortage just a few years ago. Correlation ≠ causation and all, but interesting nonetheless.
Edit 2: I just noticed this paragraph in that link you shared:
Claude 3.7 was released in February 2025. Also, I highly doubt 3.7 was good enough to make engineers more productive, overall (though I don’t have data on those old models). Relative to the speed of evolution of LLMs, harnesses, and people’s skills in using them, the data behind this article is ancient.
Edit 3:
In that article you shared, they link to the study in the second paragraph. Follow that link, and you’ll see this at the top:
There were selection effects in the follow-up study, but seemed worth mentioning anyway.