Tested across 16 leading AI models and 1,760 codebases, we identified that the risk is predictable by model and framework, giving security leaders the data to close the gap.
The SCW AI Trust Index is a living benchmark that measures how often leading AI models generate code with security vulnerabilities. It's updated as new models are released — not published once and left to age.
Vulnerabilities follow predictable patterns. Across 1,760 codebases, the same weakness types recur — logging failures, injection flaws, and broken access control lead every time. AI-generated risk is repeatable, not random.
Each of the 16 models leaves its own repeatable mix of OWASP categories, so teams can anticipate — not just react to — where a given model is likely to introduce risk.
Security leadership shifts by framework — GPT-5.1 leads in Java Enterprise API, Claude Opus 4.8 leads in Python Django. The right model depends on where a team is building.
Across all 16 models, the priciest model tested (Claude Fable 5, $174.94 per run) ranks 2nd overall — while the cheapest (Gemini 2.5 Flash, $0.58 per run) scores below average. Price doesn't predict security either way.
86 unique CWEs were identified across the dataset. Three account for a disproportionate share of all confirmed findings.
| CWE ID | Name | True Positives | Average Risk |
|---|---|---|---|
| CWE-532 | Insertion of Sensitive Information into Log File | 8,543 | 80.06 |
| CWE-79 | Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting') | 2,949 | 90.57 |
| CWE-798 | Use of Hard-coded Credentials | 1,348 | 68.5 |
Normalized score, 0–100. Higher means fewer vulnerabilities relative to the rest of the field — a score of 100 means fewest in this dataset, not vulnerability-free.
Scores for the original 6 models differ from the published RMIT study because the normalization pool expanded from 6 to 16 models. Scores are always relative to the full cohort.
A model's overall rank doesn't guarantee it's the safest choice for a specific stack.
| Framework | Leading model |
|---|---|
| Java Enterprise API | GPT-5.1 |
| Java Spring | Claude Sonnet 4.5 |
| Python Django | Claude Opus 4.8 |
| C# (.NET) | GPT-5.5 |
| C | Claude Fable 5 |
Sixteen AI models each generated the same complex expense-management application — authentication, file upload, payments, database access — across 11 frameworks, 10 independent times per combination. Every codebase was scanned with the same open-source static analysis tools (Semgrep, Bearer, Bandit) and every finding was verified for false positives using the same agentic process.
The original study and methodology were developed in partnership with RMIT University and covered the first six models tested. Secure Code Warrior independently extended the research to 16 models using that same methodology.
Download the visual summary for a fast, shareable overview of the findings, or explore the full interactive dataset — by model, by framework, by vulnerability type.
