So far, AI is changing software engineering rather than replacing software engineers. Controlled studies find large speedups on small, well-defined tasks and no reliable gain yet in large, mature codebases; US projections still show developer jobs growing; and the work is moving toward specifying, reviewing, testing and securing code. The pressure shows first in entry-level hiring.
This article goes through the evidence one study at a time, with its date and design, because the headline numbers disagree and the designs explain why. Then it covers what the job now involves, which skills are worth building, and how a team can adopt AI coding tools without losing review, security or the ability to measure. For the field-level view, with benchmarks, research directions and computing education, see the future of AI in computer science.
Will AI replace software engineers? The evidence so far
The answer depends on which evidence you read, so here it is in one place. Each row is a primary source; the sections below explain the design behind each number.
| Evidence | When | Design | Main result |
|---|---|---|---|
| Copilot experiment | Run in 2022, published February 2023 | 95 freelance developers, one task: an HTTP server in JavaScript | 55.8% faster with the assistant |
| Google trial | Published October 2024 | 96 Google engineers, one complex enterprise task | About 21% less time, with a wide confidence interval |
| METR trial | Data from early 2025, published July 2025 | 16 experienced open-source developers, 246 real issues in their own projects | 19% longer with AI allowed |
| METR follow-up | Published February 2026 | 57 developers and 800+ tasks, from August 2025 | 18% and 4% less time, estimates METR calls unreliable |
| DORA 2025 report | September 2025 | Survey of nearly 5,000 technology professionals | AI adoption goes with higher throughput and lower stability |
| Stack Overflow survey | 2025 edition | 49,000+ responses from 177 countries | 84% use or plan to use AI tools; 46% distrust their accuracy |
| BLS outlook | Updated August 27, 2026 | Official US employment projections, 2025 to 2035 | Software developers up 10%, computer programmers down 7% |
| Stanford payroll study | August 2025, revised August 2026 | Payroll records for millions of US workers | Workers aged 22 to 25 in AI-exposed jobs 19% below trend |
Read together: the tools help most on bounded tasks; the gains measured in real, complex codebases are small or uncertain; US employment of software developers is still projected to grow; and the clearest labour market signal so far is fewer young hires in AI-exposed occupations. None of these sources measures software engineering disappearing, and several show it changing. That is the evidence as of September 2026, not a forecast.
What controlled experiments measured about AI and developer speed
A randomized controlled trial (RCT) is the strongest design available here. Developers or tasks are assigned at random to work with or without AI, so a difference in time can be attributed to the tool. The trials below all used it, and they still disagree, because they measured different kinds of work.
Small, well-defined tasks: large speedups
In the GitHub Copilot experiment (Peng and colleagues, published February 2023), 95 professional developers recruited on Upwork in May and June 2022 were split at random into two groups and asked to write an HTTP server in JavaScript as fast as possible. Those with the AI coding assistant finished 55.8% faster: about 71 minutes on average against 161 for the control group, counting those who finished, with a 95% confidence interval of 21% to 89%. Developers with less programming experience benefited most.
Google's enterprise trial, published in October 2024, gave 96 of its full-time engineers one complex, enterprise-grade task, with or without three internal AI features. Its best estimate is about 21% less time on task, with a large confidence interval. The authors caution that a result from Google's internal tools in the summer of 2024 may not carry over to other tools or later dates.
Experienced developers in mature codebases: a slowdown in 2025
METR's randomized controlled trial, published on July 10, 2025, tested a different setting: 16 experienced developers working on 246 real issues, about two hours each, in large open-source repositories they had contributed to for years. Each issue was randomly assigned to allow or forbid AI. When it was allowed, developers mostly used Cursor Pro with Claude 3.5 and 3.7 Sonnet, frontier tools at the time. With AI allowed, issues took 19% longer.
The perception gap was as striking as the result. Before starting, the developers expected AI to cut their time by 24%; afterwards they believed it had cut it by 20%. The paper adds that experts in economics and machine learning had predicted savings of 39% and 38%.
Screen recordings show where the time went. With AI allowed, developers spent a smaller share of their time writing code and searching for information, and more of it prompting, waiting for the model, reviewing its output and sitting idle. They accepted fewer than 44% of the AI's generations, 75% said they read every line of generated code, and reviewing and cleaning AI output took about 9% of their time.

Why METR now calls that result out of date
METR has since labelled its 2025 results out of date. In February 2026 it reported a second experiment, started in August 2025 with 57 developers across 143 repositories and more than 800 tasks. The raw estimates pointed to speedups: 18% less time for developers from the original study and 4% less for new recruits, with confidence intervals that include no effect. METR judged the data unreliable because developers increasingly declined to work without AI: some would not join, and 30% to 50% said they held back tasks they did not want to do without it. A lower pay rate than in the first study also played a part. METR believes developers were likely sped up more in early 2026 than in early 2025, but calls its own data only very weak evidence of how much.
Why the results differ
- The size and shape of the task. A self-contained exercise with a clear specification suits an assistant. A real issue in a mature repository (METR's averaged 10 years old and more than 1.1 million lines of code), with conventions, tests and review standards, leaves less room.
- What the developer already knows. METR's developers slowed down more on issues they knew well, and reported that the AI did not use the implicit context of their repositories that they carried in their heads.
- Experience. Within the Copilot experiment, developers with less programming experience gained the most; METR's participants had years of experience in the very repositories they worked on.
- Self-reports. Developers in METR's trial believed they were faster while they were slower. Surveys of perceived productivity, including those below, measure belief, not time.
Note
Treat every headline speedup as a property of its task, its tools and its date. A number measured on a JavaScript exercise in 2022 says little about a ten-year-old codebase in 2026, in either direction.
What DORA's research found at team level
Individual speed is only part of the question. A business needs working software to reach users reliably, and that is what DORA, Google Cloud's long-running research program on software delivery, studies. Its 2025 report, State of AI-assisted Software Development, published on September 24, 2025 and still its latest annual report as of September 2026, draws on survey responses from nearly 5,000 technology professionals and more than 100 hours of qualitative data.
- Adoption is near universal. 90% of respondents use AI at work, and more than 80% believe it has increased their productivity.
- Trust lags. 30% report little or no trust in the code AI generates.
- Throughput up, stability down. Unlike in 2024, AI adoption now goes with higher software delivery throughput and product performance. It still goes with lower delivery stability.
DORA's reading is that AI amplifies what a team already has. Faster code generation exposes weaknesses downstream: without strong automated testing, mature version control and fast feedback loops, more changes mean more instability. Teams with loosely coupled architectures and fast feedback see gains; teams held back by tightly coupled systems and slow processes see little or no benefit.

The report identifies seven capabilities that amplify AI's positive impact, and DORA's capability catalog describes each one: a clear and communicated AI stance, healthy data ecosystems, AI-accessible internal data, version control, working in small batches, a user-centric focus and platform engineering. Only two mention AI by name; the other five are established engineering and data practices.
How developers use and trust AI tools
The 2025 Stack Overflow Developer Survey, the latest edition with published results as of September 2026, shows the same split between use and trust:
- Use keeps rising. 84% of respondents use or plan to use AI tools in their development process, up from 76% the year before, and 51% of professional developers use them daily.
- Sentiment is falling. Favourable views of AI tools dropped to 60%, from more than 70% in 2023 and 2024.
- Trust is low. More developers distrust the accuracy of AI output (46%) than trust it (33%), and 3% trust it highly. Experienced developers are the most cautious.
- "Almost right" is the main cost. 66% name "AI solutions that are almost right, but not quite" as their biggest frustration, and 45% say debugging AI-generated code takes more time.
- Agents are not yet the norm. 14% use AI agents at work daily, and 38% have no plans to adopt them. 72% say vibe coding, generating software from prompts, is not part of their professional work.
- High-responsibility work stays human. 76% do not plan to use AI for deployment and monitoring, and 69% not for project planning. 75% would still ask a person when they do not trust an AI's answer.
The pattern matches the trials. Almost-right code is the expensive kind, because it passes a glance and fails later, in review, in tests or in production.
Software engineer jobs and AI: projections, pay and early-career hiring
The US Bureau of Labor Statistics' Occupational Outlook Handbook, last updated on August 27, 2026, separates two occupations that AI affects differently:
| Occupation | Jobs in 2025 | Projected change, 2025 to 2035 | Median pay, May 2025 |
|---|---|---|---|
| Software developers | 1,717,800 | Up 10% (174,700 jobs) | $135,980 |
| Software quality assurance analysts and testers | 187,600 | Up 6% | $104,300 |
| Computer programmers | 110,800 | Down 7% (8,100 jobs) | $100,390 |
| All occupations | Up 3% |
BLS gives its reasons. It expects programming work to keep being automated, with companies using AI for repetitive programming tasks and some higher-skilled programming shifting to software developers. For developers it projects strong demand from software for AI, the Internet of Things, robotics and automation, and from security software, with about 106,100 openings a year across developers, QA analysts and testers. These are projections, not measurements, and they cover the US only; other countries publish their own outlooks.
Payroll data shows where the change is visible now. The Stanford Digital Economy Lab paper by Erik Brynjolfsson, Bharat Chandar and Ruyu Chen, first released in August 2025 and revised on August 12, 2026, uses ADP payroll records for millions of US workers through June 2026. It finds no evidence of widespread job displacement. But employment of workers aged 22 to 25 in occupations with high AI exposure, software development among them, stands 19% below where it would be had it kept pace with less exposed peers, while experienced workers show no comparable gap. The gap comes mainly from fewer young hires, not more layoffs, and it is concentrated where AI use substitutes for tasks rather than complementing the worker. The same pattern across office jobs is covered in the impact of AI on white-collar jobs.
The authors are careful about what that means. They present the findings as early, descriptive indicators, not causal estimates. The pattern holds when technology firms and computer occupations are excluded, but it weakens when they control for education, some of the divergence predates generative AI, and it is more pronounced in the ADP sample than in national survey data.
Their proposed mechanism fits software closely. AI substitutes more readily for codified knowledge, the kind taught in courses and written in documentation, and complements tacit knowledge gained through practice. Occupations that rely more on codified knowledge show slower entry-level employment growth, and those that rely on tacit knowledge show faster growth for mid-career and senior workers. That, too, is a correlation, not proof.
How the software engineer's job is changing
Put the studies side by side and the change is in the role of software engineers: where their time and judgment go, not whether they are needed.
From writing code to specifying and reviewing it
Coding agents shift the work toward delegation. In Anthropic's analysis of 500,000 coding conversations from April 2025, 79% of conversations with Claude Code, its coding agent, were automation, where the AI performs the task, against 49% on its Claude.ai chat interface. Of all agent interactions, 43.8% were near-complete delegation and 35.8% were feedback loops, where the developer checks results and passes errors back. Anthropic notes that its data comes from early adopters of its own products and does not measure the quality of the code that results.
When a model writes the first draft, the engineer's contribution moves to both ends of the work: a specification precise enough to generate against (acceptance criteria, tests, interfaces, constraints) and a review strict enough to own the result. METR's developers show what the second end costs: three in four read every line of generated code, and more than half said they often had to make major changes to clean it up. The first end pays off directly: the French and German cybersecurity agencies, ANSSI and BSI, note in their October 2024 guidance on AI coding assistants that the quality of generated code depends heavily on how clear the prompt is.
Testing and security of generated code
Generated code needs every check human code gets, and some new ones:
- Less secure code, more confidence. In a user study published at ACM CCS 2023, participants with an AI assistant wrote significantly less secure code than those without one, and were more likely to believe their code was secure. Those who trusted the AI less and worked on their prompts produced fewer vulnerabilities.
- Invented dependencies. A USENIX Security 2025 study of 576,000 code samples from 16 models found package hallucination on a large scale: on average, at least 5.2% of the packages recommended by commercial models, and 21.7% of those recommended by open-source models, did not exist. An attacker who registers a hallucinated name turns a model's mistake into a supply chain attack.
- Instructions hidden in content. ANSSI and BSI list indirect prompt injection among the new attack vectors: instructions an attacker writes into, for example, a package's documentation can steer an assistant that reads it.
- Automation bias. The same guidance warns that fluent, confident output invites acceptance without checking, and that developers who lean on AI too heavily may lose the skills to review code thoroughly.
Our secure coding checklist covers how to review AI-generated changes, and our overview of automation testing in 2026 covers the test gates they should pass.
System design and domain knowledge
What models do least well is work that depends on context they do not have. METR's developers reported that AI did not use the tacit knowledge of their repositories. DORA found gains in loosely coupled architectures and little or no benefit in tightly coupled ones, which makes architecture a condition for getting value from AI, not a separate topic. And DORA's user-centric focus capability exists because a team that AI makes faster can also move faster in the wrong direction. Knowing the users, the business rules and why the system is built the way it is, and making the trade-offs of system design, remain the engineer's work. Where those decisions sit in a project, from requirements to maintenance, is covered in our guide to the software development process.
Skills to invest in as a software engineer
| Skill | Why it matters now | Evidence |
|---|---|---|
| Reading and debugging code | You verify code you did not write | Debugging showed the largest gap in Anthropic's 2026 trial |
| Specifying and testing | Tests are what you generate against and verify with | DORA: without automated testing, more change means instability |
| Security review | Generated code repeats known flaws and invents dependencies | ACM CCS 2023 and USENIX Security 2025 studies |
| System design | Architecture decides whether AI helps at all | DORA: loosely versus tightly coupled systems |
| Domain and codebase knowledge | Models lack your users' and your repository's context | METR's implicit context finding; Stanford's tacit knowledge |
| Measuring outcomes | Perceived speed is unreliable | METR's perception gap; DORA's delivery metrics |
Anthropic's randomized trial on skill formation, published in January 2026, is a caution for anyone still learning. In it, 52 mostly junior developers learned an unfamiliar Python library, Trio, with or without an AI assistant. The AI group finished about two minutes faster, a difference that was not statistically significant, and averaged 50% on a quiz about the concepts they had just used, against 67% for those who coded by hand. The largest gap was in debugging. How people used the assistant mattered: those who asked it for explanations or conceptual answers scored well, while those who leaned on it to write or debug the code scored lowest. The sample was small and the quiz immediate, so the study says nothing yet about long-term skill.
For early-career engineers, that points to a working rule: use AI to explain, then write and debug the code yourself until you could have written it without help. Our tips for new coders cover how to learn with AI as a tutor. And since young workers in exposed occupations are where hiring has slowed, build evidence of the parts of the job this evidence points toward: projects with tests, code review, deployments and security fixes, not only features.
How engineering teams should adopt AI coding tools
- Decide and publish your AI stance. Say which tools are approved, for which code and which data. DORA counts a clear and communicated AI stance among its seven AI capabilities.
- Keep secrets out of prompts. Keys, tokens and detailed infrastructure configuration never go into a prompt, and only company-approved accounts are used (more on this below).
- Keep human review, and scale it. ANSSI and BSI say generated code should be checked and understood by developers, and that security and quality assurance capacity must grow with the volume of code. Review AI-written pull requests like anyone else's.
- Keep the safety nets tight. Version control, small batches, automated tests and fast feedback are what DORA found turns AI's extra output into throughput instead of instability.
- Measure before and after. Baseline DORA's five delivery metrics for each application (change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate), then compare. DORA warns against turning them into targets, which invites gaming. ANSSI and BSI recommend an evaluation phase that tracks figures such as new code checked in and security team workload. And plan for a slow start: DORA's report on the return on AI-assisted development, updated in April 2026, is written partly to help teams manage the initial productivity dip of a rollout.
- Give agents the least access they need. A coding agent that runs commands can be steered by what it reads, so run it in a sandbox with scoped credentials, and require a person's approval before anything is merged, deployed or sent.
- Protect learning and resilience. Pair junior developers' AI use with review and explanation, and make sure the team can still work without AI during an incident, as ANSSI and BSI recommend.
The second step needs the most care, because a leaked key cannot be recalled. ANSSI and BSI point out that, depending on the provider's contract terms, what developers type into an assistant can flow into its training data. They recommend company accounts with clear contract terms instead of employees' private accounts, which create shadow IT; checking whether the provider uses your data for training; working out, before rollout, how API keys and other secrets are managed and how they will be kept from leaking; and keeping keys and detailed infrastructure configuration, such as infrastructure as code, out of prompts. In practice, secrets live in a vault or the runtime environment where the running application reads them, and the assistant only ever sees code that does not contain them.

If you are turning an AI-built prototype into production software, our custom software development team starts with a fixed-scope codebase review: we reproduce the build and deploy, run the tests and security scans, and list what blocks production. For AI inside your business workflows rather than your codebase, AI and automation builds systems where every model, prompt or source change must answer the same set of real questions before it ships, and agents act only through tools that are listed and scoped.
What the evidence cannot tell you yet
Every source above has limits, and they matter more than any single number:
| Limit | Why it matters |
|---|---|
| Short windows, fast tools | The trials measured single tasks or a few months of use, with tools that have since changed |
| Selection | METR's 2026 experiment lost the developers and tasks where AI was expected to help most |
| Vendor data | Some evidence, such as Anthropic's usage analysis, comes from AI companies studying their own products |
| One country | The BLS projections and the ADP payroll data describe the US labour market |
| Projections and correlations | BLS projects rather than measures, and the Stanford authors describe rather than prove a cause |
What would change the picture is repeated evidence: new trials with fixed tasks, DORA's next annual report, the next Stack Overflow survey and the Stanford team's continued payroll tracking. Until then, read the evidence narrowly: the work is changing, US developer employment is still projected to grow, entry-level hiring is where the strain shows, and someone still has to specify, review and verify what the tools produce.


