Security

Over the past year, AI models have become much more capable of performing cybersecurity work. These changes are reshaping both the threats facing the web and the tools available to defend it. Right now, defenders have an advantage because they can use stronger models for defensive work than the open-weight models broadly available for offensive research. But this advantage will not always last. The gap will soon close. I have both good news and bad news, neither of which is yet widely understood in the community. Bad news: Near-frontier open-weight models that perform offensive security research are available today. Kimi K3 is an Opus 4.X-class model with no relevant cybersecurity safeguards. Good news: You do not need to wait for “Mythos access” or OpenAI’s cyber program to begin defensive cybersecurity work. Frontier models, with the notable exception of Fable 5, will perform defensive cybersecurity tasks today. The uncertainty around Mythos 5's release seems to have created a kind o

Malte UblCTO, Vercel

DeepsecBench evaluates model performance in finding security vulnerabilities.

Malte Ubl, Eric Dodds

Most agents today run generated code with full access to your secrets. As more agents adopt coding agent patterns, where they read filesystems, run shell commands, and generate code, they're becoming multi-component systems that each need a different level of trust. While most teams run all of these components in a single security context, because that's how the default tooling works, we recommend thinking about these security boundaries differently. Below we walk through: The actors in agentic systems Where security boundaries should go between them An architecture for running agent and generated code in separate contexts All agents are starting to look like coding agents More agents are adopting the coding agent architecture. These agents read and write to a filesystem. They run bash, Python, or similar programs to explore their environment. And increasingly, agents generate code to solve particular problems. Even agents that aren't marketed as "coding agents" use code generation as

Malte Ubl, Harpreet Arora