Blog.
Long-form notes on AI agent security, capability tokens, thefindings.v1schema, and decisions we’ve made building Capframe.
- ·securitymcpclassifier
The official GitHub MCP server scored 16/100 on our leaderboard. Here's what that actually means.
The lowest-scoring official server on the capframe.ai leaderboard is the one from GitHub. Not because it's broken — because it exposes the most write surface, and the MCP protocol has no way to describe any of it. That's the interesting part.
Read → - ·securitymcpai-agents
Heroku's official MCP server scored 8/100. This one actually runs code.
Salesforce's official Heroku MCP server scored 8/100 on our leaderboard — the lowest we've seen from a major vendor. Two of its 33 tools are arbitrary execution: a SQL console (pg_psql) and a shell with network + filesystem access (deploy_one_off_dyno). Not a CVE — a capability-hygiene gap, and a vivid one. Here's the honest risk model.
Read → - ·securitymcpleaderboard
The MCP security spread: the servers that do the most score the worst
We now grade 84 MCP servers by running them in an ephemeral sandbox and scoring their real tool schemas. A clear pattern fell out: read-only docs servers score in the 90s, while the servers that actually act — scrape, automate, run code — score in the 0s to 40s. Tool surface is attack surface.
Read → - ·securityai agentsmcp
50.000000000000001 — how a floating-point number beat my own authorization engine
I pointed a four-agent red team at my own capability-security engine. It found four HIGH-severity authorization bypasses — including a number a hair over a $50 cap that the engine read as exactly $50. Here's the worst one, and the honest scorecard.
Read → - ·securitymcpleaderboard
We built a public security leaderboard for 48 MCP servers. The official GitHub server ties for worst.
capframe.ai/leaderboard ranks every MCP server we can find against deterministic security rules. 4 Critical and 23 High findings across the ecosystem. Updated daily. The score formula is public and the rules are open-source. Here's what it found, how we built it, and what's wrong with it.
Read → - ·securitymcpleaderboard
How a Critical became a documented false positive: R7 on code generators
While expanding the capframe HTTP corpus, our R7 classifier rule flagged two Critical findings on OpenZeppelin's Cairo MCP server. Both were wrong. Here's how we caught it before publishing, what the fix looks like, and why the discipline matters more than the rule.
Read → - ·securitymcpleaderboard
Sandbox-grade scoring: the MCP leaderboard now runs your server before grading it
We upgraded capframe.ai/leaderboard to actually run MCP servers in an ephemeral Docker container instead of reading their READMEs. Four official @modelcontextprotocol/server-* packages dropped from a false-clean score of 100 to 64–98. Here's the diff and the design.
Read → - ·securitymcpclassifier
We scanned 16 popular MCP servers. Three ship arbitrary code execution — and our own scanner was wrong in both directions.
After the lab extremes — a server built to be broken (2 critical) and the official reference servers (0 critical) — we pointed Capframe at the messy middle: 16 MCP servers people actually npm-install. 111 tools, live. Three expose arbitrary JavaScript execution. Getting there meant fixing a false-positive class AND a false-negative class in our own classifier, both shipped.
Read → - ·securitymcpdogfooding
We pointed Capframe at the Damn Vulnerable MCP Server (and it found a gap in itself)
Dogfooding the Find module against a purpose-built insecure MCP server. It flagged all 10 tools for unconstrained input — but rated an arbitrary-shell-execution tool identically to a username lookup. Here's the gap that surfaced, the rule we added, and what the scanner still misses.
Read → - ·securitymcpclassifier
We scanned the official MCP reference servers — live. Zero critical, nine high, and a hole in the protocol.
Last time we pointed Capframe's Find module at a server built to be broken — 2 critical. This time we ran it live against the official Anthropic MCP reference servers. Same seven rules, opposite verdict: 0 critical. But all nine high-severity findings traced back to one thing the MCP protocol can't express — and that's the interesting part.
Read → - ·securitycapabilitiesmacaroons
Capability tokens for AI agents, the macaroon way
Most agent-permission systems check an ACL at the tool-call boundary. Capframe hands the agent a token that encodes what it's allowed to do — macaroon-style, attenuable without phoning home, bound to a key so a stolen token is useless. Here's the construction and where it stops working.
Read → - ·securityschemaai-agents
Why we shipped findings.v1 as a public JSON Schema
AI agent security tools each emit findings in their own shape. We think they shouldn't. Here's the wire format we're proposing as a starting point — and why we put it in the repo before writing the scanner.
Read →