Is AI Generating a World of Code Slop?

AI can increase code-production throughput. Whether a team can verify that output at the same rate is the question this article investigates.

That does not prove the code is bad. It creates a more uncomfortable possibility: when producing plausible changes gets cheaper without a proportional increase in verification capacity, teams can accumulate code faster than they can prove it is necessary, correct, secure, understandable, and worth maintaining.

That gap is where code slop grows.

The phrase is provocative, but the question behind it is measurable. Are AI-assisted repositories accumulating more unnecessary edits, warnings, complexity, churn, shallow tests, review work, vulnerabilities, and maintenance cost? Or are we mistaking unfamiliar authorship for poor quality?

The evidence through August 12, 2026 does not support the simple claim that AI-generated code is always worse than human code. Controlled studies sometimes find equal or slightly better local quality. At repository and production scale, however, newer studies report concerning complexity, review, verification, and resource signals, with important methodological caveats.

My conclusion is narrower than “AI is ruining software,” but still serious:

AI is not automatically generating a world of code slop. It can make low-evidence change cheap to create and submit while leaving verification expensive for the receiver.

TL;DR

• AI-written does not mean low quality. One preregistered controlled study found no systematic downstream maintainability penalty within its Java tasks, and GitHub reported modest quality improvements in one bounded Copilot exercise.
• The strongest warning appears at repository scale. Two related 2026 quasi-experimental studies from an overlapping research team reported concerning complexity estimates after AI-tool adoption. In the Cursor study, alternative estimators were directionally similar but less statistically conclusive; treat the results as evidence of risk rather than a universal causal verdict.
• Some slop is directly measurable. The July 2026 TRIM paper defines CodeSlop as modifications removable while preserving task behavior. Across successful repair trajectories, it removed 17.9% to 32.9% of patch lines in evaluated Live-kBench settings; on SWE-Bench, 327 of 330 minimized repairs preserved the hidden oracle.
• Tests can create false confidence. A syntactic classifier labeled 80.2% of 86,156 cumulative agent-authored test-file patch diffs as having weak or no explicit oracle signals. That does not assess existing assertions outside the diff or prove semantic weakness.
• Review is becoming part of the bottleneck. AI review comments can add their own noise, and developers are much better at recognizing correct generated assertions than incorrect ones.
• Security remains a separate quality dimension. Application-level evidence shows that passing functional tests does not imply security; repository-level benchmarks also find persistent secure-coding difficulty.
• Open source can absorb the externality. Cheap generation can lower submission effort while maintainers still pay the verification bill.
• Do not use one magical “AI slop score.” Track a repository-health profile across correctness, maintainability, review cost, security, ownership, and externalities, compared with your own historical baseline.

What You Will Learn Here

• Which questions we can currently answer about AI’s impact on code quality
• What “code slop” can mean without reducing it to style or vibes
• Where controlled experiments and repository-scale studies disagree
• Why passing tests, merge rates, and clean static analysis are incomplete signals
• How slop moves from a patch into architecture, review queues, and open-source ecosystems
• How to implement a small repository-health scorecard using Git, pull-request, CI, security, and incident data
• How to set a generated-complexity budget without pretending repository telemetry proves AI causality

Audience: engineers and architects deciding how much AI-generated change their delivery system can safely absorb.

Start With Better Questions

“Is AI code bad?” is too vague to research well.

It mixes at least five different artifacts:

1. raw model output
2. code a developer accepted into an editor
3. a pull request after human revision
4. merged code after review and CI
5. deployed code after production controls

Those populations are not interchangeable. Human selection removes some bad suggestions. Review rewrites others. CI catches some failures and misses others. A study of raw completions cannot tell you the production defect rate, while a study of merged pull requests mostly measures output that survived a filter.

The useful questions are more specific:

• Does AI assistance make a bounded coding task more correct or maintainable?
• Does agent adoption change repository complexity, warnings, duplication, or churn over time?
• How much generated patch content is unnecessary relative to the available tests?
• Are generated tests strong enough to detect wrong behavior?
• How much human attention does generated change consume before and after merge?
• Does code that passes functional checks still introduce security or operational cost?
• Who pays when cheap generation creates low-quality external contributions?

These questions produce different answers because they measure different layers of the system.

A Working Definition of Code Slop

Calling code “slop” because it is verbose, generic, or obviously machine-written is not useful. Humans write repetitive code. AI can write elegant code. Style is weak evidence of origin and weak evidence of harm.

The TRIM paper gives us a narrow technical definition:

CodeSlop is change that can be removed while preserving the required task behavior.

For a patch P and validation oracle O, an observed slop ratio can be expressed as:

observed_slop_ratio =
  (size(P) - size(minimize(P, O))) / size(P)

The word observed matters. If O contains only weak public tests, minimization proves only that the removed code was unnecessary for those tests. It does not prove the smaller patch preserves hidden security, architectural, performance, or product requirements.

TRIM uses an agent’s trajectory to help find that removable content, but the search history is not part of the formal definition.

For engineering decisions, keep formal CodeSlop narrow and place the broader concerns beside it as downstream risks:

This distinction prevents two common mistakes.

First, defective code is not always slop. A concise authentication bypass is dangerous but may contain no removable lines. Second, redundant code is not always defective. A duplicated helper may preserve behavior perfectly while increasing future maintenance cost.

The repository has to measure both.

What the Evidence Actually Says

The current literature is neither a clean indictment nor a clean acquittal.

The most defensible reading is:

1. AI can improve bounded implementation work.
2. AI-assisted code is not automatically harder for another developer to maintain.
3. As adoption scales, several repository-level signals warrant monitoring, but the results are not uniform across methods.
4. Workflow design can mitigate those effects; they are not fixed properties of a model.

That last point is visible in a small intervention inside the Google production study. In a controlled reimplementation benchmark of 50 synthesized C++ functions, category-specific prompting reduced targeted findings by 11.1% relative to the base prompt and improved the benchmark’s efficiency score. This was not an organization-wide production rollout, but it shows that the measured failure profile was responsive to targeted feedback.

The First Slop Mechanism: Agents Leave Their Search Behind

A human investigating a bug may try three ideas, discard two, and submit the smallest valid change. An agent can make the same search process visible in the final diff:

hypothesis A -> edit files -> tests fail
      |
      v
hypothesis B -> add workaround -> tests partly pass
      |
      v
hypothesis C -> fix root cause -> tests pass
      |
      v
submit A + B + C residue

The final patch passes, so the loop stops. Passing is treated as equivalent to minimality.

TRIM’s result matters because it demonstrates that this is not only an aesthetic complaint. Across its evaluated successful repair trajectories, meaningful portions of patches could be removed while preserving measured performance.

This suggests a missing agent phase:

explore -> implement -> validate -> minimize -> validate again -> submit

The minimization step should ask:

• Which changed files are not required by the accepted solution?
• Which branches and abstractions came from rejected hypotheses?
• Did the patch add a dependency for behavior already available in the repository?
• Can a hunk be reverted while all independent checks still pass?
• Does the final diff still match the requested scope?

This is not an instruction to optimize for the fewest lines. A small patch can hide complexity, and a migration may legitimately be large. The goal is to remove search residue, not necessary evidence or clarity.

The Second Slop Mechanism: Verification Looks Stronger Than It Is

Generated tests are easy to count. Good test oracles are harder to create.

An oracle is the part of a test that decides whether the observed behavior is correct. Executing a function without asserting the right result exercises code but does not strongly verify it.

The June 2026 All Smoke, No Alarm study examined 86,156 cumulative test-file patch diffs from 33,596 agent-authored pull requests across 2,807 repositories. Its syntactic classifier labeled 80.2% as having weak or no explicit oracle signals. After adjustment for agent, pull-request size, repository popularity, task type, and language, pull requests whose strongest test patch contained multiple distinct strong oracle types were more likely to merge (OR 1.28). That is an association, not a causal effect.

That does not mean 80.2% of those tests were worthless. The classifier was syntactic, and implicit checks can still catch failures. It does mean teams should stop using these as interchangeable proof:

test file exists
    != assertion checks the requirement
    != test can detect a plausible wrong implementation
    != implementation is secure
    != production behavior is safe

The review problem is harder than “have a human look at it.” In July 2026, a controlled experiment with 86 Python programmers found that participants identified correct generated assertions with 74% accuracy but incorrect assertions with only 49% accuracy, despite similar confidence. Under-specified comments reduced accuracy relative to exact comments while increasing confidence relative to no comment; comments provided no overall accuracy benefit.

The practical consequence is important: plausibility is asymmetric. Correct-looking wrong artifacts are precisely what reviewers struggle to reject.

A stronger verification stack uses independent evidence:

requirement examples
  -> unit and integration tests with meaningful assertions
  -> mutation or negative testing for important rules
  -> static and dependency analysis
  -> security or performance checks where risk requires them
  -> human review of assumptions, scope, and architecture

The agent can help create each layer. It should not be the only author and the only judge of the same claim.

The Third Slop Mechanism: Functional Success Hides Other Costs

Code can be correct for the requested input and still be insecure, coupled, expensive, or hard to operate.

Security

BaxBench evaluates generated backend applications with functional tests and executable exploits. In its 2025 evaluation of 392 tasks, the best model reached 62% functional correctness, and roughly half of correct generated programs could be successfully exploited on average.

The A.S.E. benchmark moved the evaluation into repository-level scenarios and found that patches could integrate and pass syntax or static quality checks while retaining target-vulnerability alerts. It did not dynamically validate full functional correctness.

These benchmarks do not estimate the vulnerability rate of production AI code. They generate code under controlled conditions and deliberately test security-sensitive scenarios. BaxBench supports the narrower conclusion that functional correctness and security correctness are separable outcomes. A.S.E. shows why repository context and security-specific checks still matter. Passing ordinary tests cannot stand in for a security claim.

Performance and coupling

The August 2026 production C++ study found a more complicated profile than “AI bad.” AI-heavy code had higher interface and coupling burdens, copy and allocation overheads, and greater reliance on explicit loops over optimized APIs. The same observational cohort required more review effort; normalized CPU-share growth was about 5% higher and memory-share growth about 8% higher for AI-heavy functions than mostly human-written functions. The design cannot establish that AI authorship caused those differences. The paper also reports lower revert rates and lower rates in several correctness and safety categories.

That mixed result is exactly why one quality score is misleading. The same change population can be better on one dimension and worse on another.

Churn

Short-term rewrite and deletion can indicate that code was merged before it stabilized. It can also indicate healthy cleanup. The 2026 study of roughly 110,000 pull requests found consistently less favorable short-term survival and churn for agent contributions, but generally small effects and important population differences between agent and human repositories.

Treat churn as a diagnostic, not a verdict. It becomes useful when it moves with other evidence such as regressions, review rounds, complexity, or repeated reinvention.

Review Capacity Is a Finite Resource

When a team’s generation throughput rises without proportional verification capacity, review queues can grow. Review does not automatically scale with generation.

This is a queueing problem:

agent generation rate > human verification rate
                     |
                     v
             larger review queue
                     |
          +----------+-----------+
          |                      |
          v                      v
   slower delivery       shallower review
                                 |
                                 v
                      more downstream rework

Observed pull-request size and review-time patterns vary by agent, task, repository, and study population. It would be inaccurate to claim that every AI pull request is larger or slower.

What is well supported is that review automation does not erase review cost.

In an ICSE industry before-and-after study of 4,335 pull requests, average closure time increased from 5 hours 52 minutes to 8 hours 20 minutes, with divergent project-level trends. After a mandatory author-labeling policy, authors marked 73.8% of 1,408 eligible automated comments on merged pull requests as resolved. Practitioners mostly perceived only minor code-quality improvement. The design does not prove that automated review caused the longer closure time.

A July 2026 study of CodeRabbit examined 31,073 review-and-feedback pairs from selected repositories, about 9.3% of the 332,693 collected CodeRabbit comments. A mostly LLM-labeled analysis classified 36.4% as accepted, 7.3% as discussion, and 56.3% as rejected. The study only included comments that received replies, so 56.3% is not a universal rejection rate for all AI review comments. It is still evidence that automated review can generate material triage work through false positives, redundancy, scope drift, and disagreement with developer intent.

Using one agent to generate code and another to review it can be useful. It is not independent proof by default. Similar models may share blind spots, and a reviewer optimized to always comment can add noise faster than it removes defects.

Slop Escapes the Repository

Inside a company, the author and reviewer usually share the maintenance bill. Open source breaks that symmetry.

A contributor can generate plausible reports or pull requests cheaply. Maintainers must still understand the project, reproduce the claim, inspect the change, protect compatibility, and explain the rejection. Generation can reduce submitter effort without reducing the receiver’s verification cost by the same amount.

The curl project provides a concrete security-report example. On July 14, 2025, maintainer Daniel Stenberg wrote that about 20% of 2025 submissions appeared to be AI slop while only about 5% of all submissions had proved to be genuine vulnerabilities by early July. Every report could engage three or four security-team members for 30 minutes to several hours.

That is one project and one submission channel, not proof that most open-source pull requests are AI spam. In fact, curl later distinguished its vulnerability-report problem from pull requests, where extensive CI provided a stronger filter.

A July 2026 mixed-methods study across 294 large, externally oriented repositories found that the 2025 one-time-contributor merge ratio was 18.18% below a univariate time-series projection based on 2023 and 2024. Its repository analysis treated 2025 as an ecosystem intervention, did not identify AI authorship for each contribution, and could not control for every other 2025 change. The study supports an open-source capacity concern more strongly than it proves platform-wide AI causation.

The architectural lesson is still useful:

When submission effort falls, systems need evidence gates at their boundaries or they can transfer disproportionate verification work to scarce maintainers.

Useful gates include reproducible examples, failing tests, scoped diffs, CI, contribution disclosure, provenance, and explicit maintainer responsibility. They should evaluate evidence rather than a contributor’s identity or writing style, because overly strict gates can also exclude legitimate newcomers. The goal is not to reject AI assistance. It is to require the submitter to carry evidence with the claim.

A Practical Repository-Health Scorecard

Do not start by trying to detect whether code “looks AI-written.” Mixed human-and-AI authorship makes that unreliable, and teams can easily hide provenance.

Measure the outcome instead.

This scorecard is a safety guardrail, not an AI-attribution system. It can show that repository health changed after adoption, but it cannot establish why. Testing causality requires a separate phased rollout, randomized design, or carefully matched pre/post study that controls for task mix, throughput, team changes, and tooling changes.

Use six independent dimensions rather than averaging them into one score:

Keep delivered-value context beside the risk profile: completed product outcomes, throughput, and end-to-end lead time. More valuable work can legitimately consume more CI, review, or compute. Do not fold value and risk into one opaque score; inspect whether the extra cost bought a useful outcome.

Display a profile:

Correctness       GREEN   stable
Maintainability   AMBER   complexity increasing
Review cost       RED     p90 latency above baseline
Security          GREEN   no new high findings
Ownership         AMBER   one critical module under-owned
Externalities     RED     CI and compute cost rising

This preserves tradeoffs. A team can have stable correctness while review cost rises, or improve maintainability while temporarily increasing churn during a cleanup.

Implement the first version

An MVP needs data most teams already have:

Git history ----------+
Pull-request events --+
CI and scanners ------+--> classify + normalize --> metric snapshots
Deploys and incidents +                               |
                                                       v
                                              compare with baseline
                                                       |
                                                       v
                                              review in engineering retro

Start with ten measures:

escaped_defect_rate
change_fail_rate
new_duplication_density
rewrite_rate_30d
changed_function_complexity_p90
pull_request_size_p90
first_review_latency_p90
new_high_security_findings
critical_path_ownership_coverage
ci_minutes_per_merged_change

Define each metric before automating it. Three examples:

The first measure samples formal CodeSlop directly, relative to the validation oracle. The remaining measures detect broader repository pressure. Never describe a rising rewrite or review metric as proof of CodeSlop or AI causality.

For each metric, store enough context to make it auditable:

metric: rewrite_rate_30d
value: 0.18
numerator: 1274
denominator: 7078
sample_size: 64
coverage: 0.91
baseline_period: 2026-01-01..2026-06-30
current_period: 2026-07-01..2026-07-31
exclusions:
  - generated files
  - lockfiles
  - migrations
tool_version: repository-metrics@1.3.0

Classify changed paths before calculating line-based measures. Generated clients, vendored code, snapshots, migrations, fixtures, and lockfiles have different change shapes from production code and should not silently dominate the result.

Use medians and p90 values for review and change-size metrics. Averages can hide the expensive tail in skewed distributions.

Compare with your own baseline

Universal thresholds are usually the wrong starting point. A kernel, web application, generated SDK, and data pipeline have different normal change shapes.

Use 6 to 12 months of comparable history when possible, segmented by work type:

• feature
• defect fix
• incident response
• refactor
• dependency update
• migration
• generated change

Freeze the baseline for a quarter. If it moves every week, degradation quickly becomes the new normal.

For lower-is-better metrics, a robust anomaly signal can use the historical median and median absolute deviation:

robust_z =
  (current_value - historical_median)
  / (1.4826 * historical_median_absolute_deviation)

If historical MAD is zero, do not calculate this score. Use an explicit zero-variance rule, bootstrap interval, or agreed percentile boundary instead.

Use the result as a discussion trigger, not an automatic performance judgment. Require sustained anomalies for noisy process metrics, but never normalize away critical security findings or severe incidents.

Set a Generated-Complexity Budget

The scorecard shows where pressure is increasing. A budget limits how much unchecked change enters the system.

The useful budget is not “AI may write only 30% of the code.” Provenance percentages are hard to measure and easy to game. Budget the scarce resources instead:

• reviewable changed lines and modules per pull request
• concurrent agent pull requests per qualified reviewer
• maximum unreviewed queue age
• new dependencies per feature without architecture approval
• complexity or duplication regression allowed in changed code
• security and performance evidence required by risk class
• unresolved high-severity findings allowed at merge: zero unless explicitly risk-accepted

A simple policy might look like this:

The policy should include a legal no-change outcome. Agents have an action bias: if every task ends in a patch, unnecessary change becomes the success condition.

Anti-Gaming Rules

Metrics become harmful when they turn into individual quotas.

Use these safeguards:

• Never rank developers by a slop score.
• Keep the six dimensions separate; do not hide them in one weighted number.
• Show numerators, denominators, sample sizes, exclusions, and scanner coverage.
• Treat missing data as unknown, not green.
• Pair small pull requests with lead time and defects so teams cannot game size by creating opaque PR chains.
• Pair low security findings with scanner coverage and dismissal audits.
• Pair low churn with delivered value so teams cannot improve by avoiding necessary work.
• Version-control metric exclusions and review changes to them.
• Audit a random sample of incident attribution, PR classification, and scanner dismissals.
• Use repository and team trends, not surveillance of individual keystrokes.

The scorecard exists to improve the delivery system, not to prove that one developer used too much AI.

What the Evidence Still Cannot Tell Us

The research is improving quickly, but several questions remain open:

• What happens to agent-authored code after 12 or 24 months, not three weeks?
• Which maintainability signals predict incidents or future engineering effort rather than merely upsetting a static analyzer?
• Do small, spec-driven agent changes outperform large autonomous changes after controlling for task type?
• Which controls actually improve outcomes: minimization, mutation testing, architecture constraints, independent review, or stronger models?
• How do private enterprise repositories compare with public open-source projects?
• Can provenance-aware studies compare AI and human work within the same team, repository, task class, and review policy?
• Does automated review reduce total human effort, or move it into false-positive triage?
• How much open-source contribution pressure is directly AI-generated rather than a broader growth in low-effort submissions?

Any article claiming those questions are settled is getting ahead of the data.

So, Is AI Generating a World of Code Slop?

Not by itself.

The controlled evidence is enough to reject the lazy claim that AI-assisted code is inherently unmaintainable. Developers can use AI to produce correct, readable, maintainable code, and bounded studies sometimes show modest improvements.

But the larger system is flashing warning signs.

Agents can retain unnecessary search residue. Generated tests can look more rigorous than they are. Two related repository-adoption studies report concerning complexity estimates, with weaker statistical confidence in some warning estimates and sensitivity checks. Reviewers can be confidently wrong about plausible generated artifacts. Functionally correct code can still be exploitable, and one enterprise cohort found a distinct operational-cost profile.

The common mechanism is not machine authorship. It is asymmetric cost:

effort to generate falls
        +
effort to submit falls
        +
verification does not scale proportionally
        =
low-evidence output becomes cheap to produce

That makes code quality a capacity-design problem.

If your generation rate rises, your evidence rate must rise with it. If review capacity stays fixed, reduce work in progress. If agents leave search residue, add minimization. If tests are easy to generate, measure oracle strength rather than test-file count. If external contributors can submit claims cheaply, require reproducible evidence at the boundary.

AI does not force us into a world of code slop. It can remove economic friction that previously limited how much unverified code a team or contributor could create. Whether that becomes leverage or pollution depends on the system around the model.

Related Reading

• How to Improve Code Quality During Vibe Coding
• When Code Gets Cheap, Complexity Moves
• How to Build a Good Agentic Code Reviewer
• A Secure Agentic Coding Process Is Not Just a Bigger LLM
• The Future of Engineers: Why Understanding the System Outlives Writing the Code

Sources

• Mathai et al., TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization, July 20, 2026 - formal definition of CodeSlop and patch-minimization results across agent scaffolds.
• He et al., Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects, MSR 2026 - matched difference-in-differences study of velocity, warnings, and complexity after Cursor adoption.
• Agarwal et al., AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development, January 2026 - longitudinal matched study comparing agent-first and IDE-first adoption.
• Popescu et al., Investigating Autonomous Agent Contributions in the Wild, MSR 2026 - approximately 110,000 pull requests and short-term code survival and churn.
• Borg et al., Echoes of AI: Investigating the Downstream Effects of AI Assistants on Software Maintainability, revised February 26, 2026 - controlled maintainability counterevidence.
• GitHub, Does GitHub Copilot improve code quality? Here’s what the data says, November 18, 2024, updated February 6, 2025 - vendor-run randomized bounded-task study.
• Tran et al., Characterizing the Quality Profile of AI-Generated C++ in Production, August 6, 2026 - production study of 3.52 million changes, review burden, quality signals, and resource use.
• Banik et al., All Smoke, No Alarm: Oracle Signals in Agent-Authored Test Code, June 16, 2026 - analysis of 86,156 agent-authored test patches.
• Kaufman et al., Programmers Are Poor and Overconfident Judges of LLM-Generated Assertions, July 9, 2026 - controlled study of human judgment over correct and incorrect generated assertions.
• Vero et al., BaxBench: Can LLMs Generate Correct and Secure Backends?, revised May 30, 2025 - application-level functional tests and executable security exploits.
• Lian et al., A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code, revised September 18, 2025 - repository-level secure-code evaluation.
• Lin et al., Is Agentic Code Review Helpful? Mining Developers’ Feedback to CodeRabbit Reviews in the Wild, revised July 23, 2026 - developer responses to 31,073 agentic review comments.
• Samaan et al., Automated Code Review In Practice, ICSE-SEIP 2025 - industrial study of review comments, pull-request closure time, and perceived quality.
• Afroz et al., “AI Slop is DDoSing Open Source”: Understanding the Impact of AI-Generated Contributions on Open Source Sustainability, July 4, 2026 - mixed-methods study of open-source contribution pressure and maintainer responses.
• Daniel Stenberg, Death by a thousand slops, July 14, 2025 - curl maintainer’s quantified account of low-quality vulnerability-report burden.
• Google Cloud DORA, State of AI-assisted Software Development 2025 - AI as an amplifier of the surrounding organizational system.
• Stack Overflow, 2025 Developer Survey: AI - self-reported trust, almost-right output, and debugging frustration; useful as perception data, not measured defect or debugging time.
• Forsgren et al., The SPACE of Developer Productivity - why software productivity and engineering effectiveness should not be reduced to one metric.
• Google Engineering Practices, Small CLs and The Standard of Code Review - practical foundations for reviewability and improving code health over time.