AgentSDK: Why I'm Starting Over

I’ve spent a lot of time recently wrestling with AgentSDK. It’s a capable tool, and starting from a fork seemed like the smartest way to get a head start. But lately, I’ve realized that I’m spending more time fighting the abstractions than actually building.

The more I worked on pie, the more it hit me: I don’t need an engine that can talk to 70 different providers. That’s an amazing engineering feat, but for me, it’s just noise. Every time I tried to tweak a internal logic, I had to wade through layers of “general-purpose” code that I’ll never actually use.

The realization

I’m not a purist—most of my code is AI-assisted these days because it just makes life easier. But there’s a specific kind of frustration that comes with an inherited architecture that feels… well, automated. It’s that feeling when you’re using AI as a high-powered hammer, but somewhere along the line, you realize the hammer was the one designing the house. You end up with a blueprint that’s technically correct but practically exhausting to maintain.

I haven’t made any major changes to the fork yet, but looking at the foundation, it’s clear our goals have diverged. I need something surgical. I only care about OpenAI-compatible APIs, and I want to own every line of that implementation.

The breaking point: “Intermingled Tokens”

The real “aha” moment came while debugging streaming responses with smaller models like Gemma.

In pie/src/handler.rs#L29, you can see where I’m just dumping output: println!("{output}");. The problem is that these models don’t always give you clean JSON. They give you “intermingled tokens”—conversational fluff mixed right in with the tool metadata. Without a dedicated state machine to buffer and split that stream, the SDK just chokes. It either tries to execute the “Sure, I can help with that!” filler or hallucinates the tool call entirely.

It’s one of those problems that feels small until it breaks your entire agent’s reliability.

Starting fresh

So, I’m starting over. The new version is going to be narrow, opinionated, and built on three simple things:

OpenAI First: No more universal provider traits. If it’s not OpenAI-spec, it doesn’t belong here.
Robust Parsing: A real parser that can handle the messy reality of streaming LLM outputs.
Control: No more “enterprise” boilerplate. Just the code I need, written in a way I actually enjoy reading.

Sometimes the fastest way forward is to take a few steps back and reclaim your own architecture.

One more thing…

If you’re a Rust enthusiast in Nepal (or just want to hang out with some really smart people), come join us on the Rust Nepal Discord. It’s where we talk about everything from parsing errors to system architecture.

Join Rust Nepal on Discord

You can follow the progress and check out the code here:

AgentSDK: github.com/dineshdb/agentsdk
Pie: github.com/dineshdb/pie