40% of AI Code Is Buggy: Scan It Locally as You Type
By: Evgeny Padezhnov
AI coding assistants write code fast. They also write vulnerabilities fast.
Coding agents boost output. They also ship hardcoded secrets, SQL injection, and weak crypto without warning. A team at NYU tested GitHub Copilot and found that roughly 40% of its suggestions were buggy or exploitable. Developers now spend more time debugging machine-written code. Codacy Guardrails targets that exact gap. It runs free, local static analysis on AI output while the agent generates it.
The problem: agents generate insecure code silently
An AI assistant has no concept of your security policy. It pattern-matches on training data. That data includes millions of insecure examples.
Common mistake: trusting AI output because it compiles and passes a quick eyeball check. Compiling code can still leak credentials. Working code can still contain an injection path.
The failure modes are predictable:
- Hardcoded API keys and passwords pasted straight into source.
- String-concatenated SQL queries open to injection.
- Outdated dependencies with known CVEs pulled in by the agent.
- Weak hashing (MD5, SHA-1) for passwords.
- Path traversal and unsafe deserialization.
These map directly to the OWASP Top 10. The agent does not check against that list. A human reviewer might catch one issue in five. The other four reach the pull request.
Key point: speed without a guardrail moves risk downstream, not away. The bug still exists. It just surfaces later, in review or in production, where it costs more to fix.
What Codacy Guardrails actually does
Codacy Guardrails is three pieces: an IDE extension, a CLI, and an MCP server. Together they enforce security and quality rules on AI-generated code in real time.
It hooks into AI coding assistants. Supported hosts include VS Code Agent Mode, Cursor, and Windsurf. When the agent produces code, the tool scans it immediately. If it finds a vulnerability or a standards violation, it flags or fixes the code as it is generated.
The MCP server is the connective layer. Model Context Protocol lets an AI assistant call external tools during a session. Guardrails registers as one of those tools. The agent can then query the analyzer mid-task and correct itself before the code lands.
In plain terms: the security scanner sits inside the agent loop, not after it. The check happens at generation time, not at review time.
The extension is free for all developers. Paid plans exist for teams that want to apply rules centrally across a fleet. Central rules are not required for local analysis. The free local workflow with agents is complete on its own.
How it works: classic static analysis, no AI models
Codacy Guardrails uses no AI models for the scanning itself. It runs classic static code analysis. That choice matters. Static analyzers are deterministic. The same input produces the same finding every time. There is no hallucinated vulnerability and no random false negative from a probabilistic model.
Under the hood it orchestrates a collection of open-source analyzers. Two named ones are Semgrep and Trivy.
- Semgrep matches code patterns across languages. It catches injection, unsafe APIs, and policy violations using readable rules.
- Trivy scans dependencies and configuration for known CVEs and misconfigurations.
The combined ruleset covers 2000+ rules. Both tools are mature and battle-tested in production at scale. Tested in production. That is the point of using established analyzers instead of a fresh model: the detection logic is already proven.
Supported languages today are JavaScript, TypeScript, Python, and Java. The scan categories include:
- OWASP Top 10 vulnerability classes.
- Hardcoded secrets and credentials.
- Dependency checks for vulnerable packages.
- Code complexity metrics.
- Styling and standards violations.
Rules are customizable. A team can tune the set to match its own coding standards. A project that bans a specific pattern can encode that ban as a rule. The analyzer then enforces it on every AI suggestion automatically.
Key point: deterministic analysis plus a probabilistic generator is a sound pairing. The generator proposes. The analyzer verifies. Neither trusts the other blindly.
What this catches that review misses
Manual review is inconsistent. It depends on attention, time, and the reviewer's security knowledge. A scanner does not get tired on the fortieth file.
Consider a concrete case. An agent generates a Python endpoint that builds a query with an f-string:
query = f"SELECT * FROM users WHERE id = {user_id}"
cursor.execute(query)
Semgrep flags this as a SQL injection pattern instantly. The fix is a parameterized query:
cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))
With Guardrails in the loop, that correction can happen before the code is even accepted into the file. The agent receives the finding and rewrites the line.
A second case: the agent hardcodes a token.
const apiKey = "sk_live_4eC39HqLyjWDarjtT1zdp7dc";
A secrets rule catches the literal key shape. It blocks the commit-ready code on the spot. Common mistake: assuming a secret in a "temporary" draft is harmless. Drafts get committed. Git history keeps secrets forever.
A third case: the agent adds a dependency with a known CVE. Trivy reports the advisory and the fixed version. The developer upgrades instead of shipping the vulnerable package.
In practice, these three categories — injection, secrets, vulnerable dependencies — account for a large share of real incidents. Catching them at generation time removes them before they enter the codebase.
Setup: extension, CLI, MCP server
Installation is short. Three steps cover it:
- Install the Codacy extension in VS Code, Cursor, or Windsurf.
- Open the Codacy sidebar.
- Enable Codacy's CLI and MCP Server from that sidebar.
The CLI runs the analysis locally. The MCP server exposes it to the agent. After enabling both, the agent can call the scanner during a session. No account-wide configuration is needed for the local flow.
The official extension page is codacy.com/get-ide-extension. A short demo of the workflow is on YouTube. The demo shows the scan firing while the agent writes code, then the automatic correction.
Try it: install the extension, enable the MCP server, then ask your agent to write a database query. Watch whether the unsafe version gets flagged before you accept it.
Where this fits and where it does not
Guardrails is a local, real-time gate. It is not a replacement for a full CI security pipeline. It complements one. The local scan catches issues early. A CI scan catches what slips through and enforces the same rules on every branch.
The honest limits are worth stating:
- Language coverage is JavaScript/TypeScript, Python, and Java today. Other stacks are not covered yet.
- Static analysis produces false positives. Custom rules and tuning reduce them but do not eliminate them.
- It detects known patterns. Novel logic flaws and business-logic bugs stay out of reach for any static tool.
If it works — it is correct. A deterministic scanner that flags a real injection path is doing its job, even when it occasionally flags a safe one. The cost of a false positive is a glance. The cost of a missed secret is a breach.
Key point: the value is the loop position. Moving the check inside the agent's generation cycle changes the economics of fixing AI code. Early beats late, every time.
What to try right now
Pick one repository where an AI assistant writes a meaningful share of the code. Install the Codacy extension. Enable the CLI and MCP server from the sidebar. Then run a normal coding session and let the agent generate a feature that touches user input or external dependencies.
Watch what the scanner reports. Count the findings. Compare that count against what a quick manual review would have caught. The gap is the measure of the tool's value for your workflow.
AI-assisted development is not going away. Making it safer is the open problem. A free, local, deterministic guardrail inside the agent loop is one concrete answer worth testing on real code.
Information is accurate as of the publication date. Terms, prices, and regulations may change — verify with relevant professionals.