API Evangelist Conversation with Daniel Kovac on OpenAPI, SDKs, Context Engineering, and the Rise of Agent Skills

Thank you for having me. I joined Speakeasy not long ago — a couple of months, four or five months. I’m a product engineer, as many of us are at Speakeasy, and at Speakeasy we build what I very subjectively believe to be the best generated SDKs, among various other things. That’s kind of what brought me here right now, because we recently released our agent skills, and agent skills are sort of the direction we want to take the product in, in terms of how people interface with Speakeasy. Right now it’s a CLI, and the primary way you interact with it is by prompting the CLI itself. What we found is that a lot of the time we’ll run the same exact workflow over and over again to diagnose customer issues — and that’s where the idea came from: why don’t we outsource it to coding agents and create a repository of documentation that can be consumed by our customers as the frontline?

Very broad topic. Right now the execution angle seems to be evolving around coding agents like Claude — which is mainly what we use — but there are others; Codex stands out as another one. We find most software engineers have shifted really quickly from using their IDEs to develop software, to concocting plans and having them executed by a coding agent. Through that we found the value of having a clean way to consume your API becomes more and more valuable. One of the key parts these agents do somewhat well, but can make costly mistakes on, is interfacing between your existing system parts. Cloudflare just had an outage caused by a mismatch between client expectations and the API contract — that’s stuff LLMs still get wrong all the time. That’s where Speakeasy’s SDK generation helps, because it’s fully deterministic. You get the same output every single time. It’s reliable, it’s been battle-tested for years, and we constantly work on making it as usable as possible.

One of the first skills we launched helps you customize your OpenAPI spec to be more compliant — better method names, pagination, and organizational things like splitting endpoints into their own tags. The blocker is that it’s so easy to give advice; an agent could take your spec and customize it and you’d get a decent outcome. But the underlying problem is most specs we see are generated from code — people set up something like FastAPI and the spec falls out of their endpoints, which is seamless but produces specs that aren’t refined, and that ends up giving you a lower-resolution SDK. You lose a bit at each generation step. You could run an agent over it each time, but then it becomes non-deterministic — depends on the model, the temperature, whether Anthropic rolled out a change. So our solution is overlays, which we treat as a first-class citizen. You can add an arbitrary number of overlays and we apply them during generation, so you get deterministic, incremental enhancement on every single generation. Agents are quite good at authoring those overlays, even though they’re tedious for a human.

I’ll give credit to my colleagues Vishal, who built most of the specs and the eval framework, and Thomas, who did a lot of the feedback step. We sourced skills from a lot of analysis of customer issues, scouring our own docs, looking at analytics for most-used pages, talking to our FDEs and ourselves, and even command-failure events from PostHog. The hard part is that the interaction between OpenAPI and SDKs is a huge, complex problem space — you could end up with thousands of distinct skills, and at that point you lose the context savings because every description’s metadata is always loaded, and descriptions start shadowing each other. Docs are really tree structures, and skills are flat, so flattening them is a pain point. The most important thing to understand is that a skill’s description isn’t a description at all — it’s a misnomer. Activation depends one hundred percent on it, and it has to be the problem statement the agent would use to describe what it’s experiencing, not a summary of your solution. If you describe the solution, it will never activate, because mathematically the agent won’t connect the two.

A hundred percent. You hit the nail on the head about design-first. Agentic workflows are proving that design-first is the most valuable way to think about solving software problems, and they’re a large enabler of it. The quality of outcomes with agents scales pretty much linearly with the quality of inputs — garbage in, garbage out — so the more effort you put into your specs and your design, the better your execution. My tool Granary is an example: I’ve written almost none of the code myself, it’s nearly all agents, and it works because of the harnesses — lots of tests, every issue covered so you never hit the same problem twice. That’s just how you were supposed to write software to begin with, but we always had a deadline excuse. Now there’s none; Claude writes a comprehensive test suite in a minute and a half. My primary instruction is TDD — write the test, make sure it fails first, then implement. Spec-driven development puts real constraints on the work, and agents do their best work going from A to B; they get unreliable when the situation is too vague.

I’m going to make a bold bet: skills in their current form may not be the final abstraction. The whole point of skills is progressive disclosure, and yet when you defer to writing skills for most of your context, they get less effective the more of them you have, because their metadata has to sit in context all the time — and some skills start to shadow others through non-deterministic activation. For skills to reach their potential they need either real progressive discovery — a single entry point with a discovery stage that walks a tree, like MCP servers do, which is what we’re doing with speakeasy agent context — or they need to be smaller, more durable, and pattern-triggered, so a rule fires when a certain file is touched or a topic comes up. There was a Vercel article comparing skills to AGENTS.md and saying skills underperformed; we’ve found that categorically untrue on frontier models, and we have the benchmarks to prove it. Our evals run code, check skill activation, and let us iterate with real confidence.

The thing that’s super important to us is making sure no customer information leaks into the skills. Often we’ll know from reading a skill that there was a customer who hit this issue repeatedly, and we could associate it — but from the outside there’s no way to tell. We’re very intentional that it’s fully anonymous: we’re not describing a specific issue or saying “company X’s SDK looks like they solved it this way,” we’re describing a general pattern that can be solved a certain way, so the solutions apply to all codebases. There’s no real privacy risk. The reason we open-source our skills is precisely so a company can take them and roll their own — point Claude at them and say “customize these for our specific use case” — and own all of it. Even though the skills themselves are distilled from a vast database of real specs and hundreds of thousands of real customer issues, they’re always generic enough to solve a problem and then be customized to your heart’s content.

For so long docs were an afterthought — a necessary evil of having your own software product. You’d just ask, how do I generate this with the least effort? Now it’s the opposite. Your docs are paramount to anyone integrating with your product, because they’ll let their coding agent roam free, and the agent picks up your docs and either runs with them or decides they’re too generic to help. So whoever wants to win in this space has to turn their docs into something constantly living and evolving — not only with the product, but with the customer base. We always get new customers bringing very specific, unique problems, but that doesn’t mean no one else has them; it’s safe to assume someone like Google has the same problem. The idea is that by seeing these problems, resolving them, and distilling them into a docs product that’s generally available to all agents, we’re very likely to solve that same problem for a lot of our potential future customers. That’s the big takeaway.

It’s a lot of dogfooding. We use these on a day-to-day basis — I had a statement on LinkedIn a couple weeks ago: if your tool can’t build itself, is it really a tool? Every dev-tool company should be using and abusing their own tools to no end. Our entire backend and front end are driven by SDK generation; we even generate our Go SDK and use those types in our Go backend so we know there’s parity, and our Gram team does the same with our MCP product. That gives us real insight — we catch naming we could improve, or upgrades that change a lot of files but only comments. The hard part as a service provider is that skills are still honed on Anthropic’s models; we run them with Sonnet and Opus, but there’s no easy way to tell how Codex would handle them, or some new frontier model or DeepSeek that ships tomorrow. Every variant introduces non-deterministic change, which makes this feel more like psychology than software engineering — you have to find the right words. That’s why the eval framework matters so much.