Kodr2: Multiple Providers - OpenRouter, Ollama, and LM Studio

#ai#kodr#kodr2#local-models#agents#testing

Quick update on kodr2, after starting over. The rebuild was LM Studio-only at first - same as the original. It wasn’t quite enough.

Three providers, one contract

Every provider kodr2 talks to speaks the same OpenAI-compatible chat completions shape - messages, tools, streaming, tool calls - so the harness itself never branches on which one is running underneath. src/provider.mjs is a small factory that returns a capability-flagged interface instead of a class hierarchy: { modelLifecycle, contextProbing, autoDetectModel, reasoning }, plain booleans, no 'auto' modes, no silent no-ops. If a provider can’t do something, callers find out by checking a flag, not by watching a method quietly do nothing.

  • LM Studio (default, localhost:1234, no auth) - unchanged from before, just moved behind the new interface. It’s still the only one with real model lifecycle management (load/unload via lms) and context-window probing, because its /api/v0/models extension actually reports state/loaded_context_length/max_context_length.
  • OpenRouter (hosted, OPENROUTER_API_KEY required) - --model is required too, since there’s no “loaded model” to auto-detect on a hosted API. This is the one that motivated the whole exercise (more below).
  • Ollama (local by default at localhost:11434, or https://ollama.com/v1 with OLLAMA_API_KEY for hosted access) - the one that turned out to be “almost the same again”: same OpenAI-compat surface, its own quirks. A local install can even transparently proxy a :cloud-suffixed model (kimi-k2.7-code:cloud) through the same local endpoint, so cloud access doesn’t strictly require pointing at ollama.com at all.

Why OpenRouter, and reasoning

The original prompt for this was wanting to see kodr2 use a model’s actual reasoning output, not just its final answer. LM Studio’s reasoning-capable endpoint (/api/v1/chat) turned out to be incompatible with kodr2’s client-side tool-calling loop, so that path was a dead end for now (tracked upstream). OpenRouter’s { reasoning: { enabled: true } } isn’t.

--reasoning is a plain boolean - v1 scope, no per-effort tuning yet - and it errors immediately at startup if you ask for it against a provider that doesn’t support it, rather than silently running without it. That’s the same instinct behind kodr2’s privacy defaults for OpenRouter: zdr: true and data_collection: "deny" on every request unless you explicitly turn them off, because sending code to a hosted model implies caring about where it goes by default, not only when you remembered to ask. There’s also --openrouter-provider-only <csv> to restrict or prioritize which upstream inference provider OpenRouter routes to, and OpenRouter’s usage.cost now flows all the way through - heal, review, memory, stats - so kodr stats shows real spend, not just token counts.

Driving the testing with the harness itself

The interesting part wasn’t writing three provider adapters - it was how they got tested. Unit tests cover the contract (no network, no models), but the thing that actually finds bugs is running kodr2 against live backends and watching what happens: fire a task at LM Studio, fire the same shape of task at OpenRouter, then at Ollama, and see where the seams show.

That’s how two streaming-assembler bugs turned up. delta.reasoning and delta.reasoning_details were silently dropped by the chunk assembler - not a crash, just quietly missing data, the kind of bug that never fails a test because nothing was asserting reasoning existed in the first place. And chunk.usage was only being read inside the branch that handles a chunk with no delta - which worked for LM Studio, but OpenRouter attaches usage to a chunk that also carries a near-empty delta, so kodr2 was dropping cost data on every single request. Both only surfaced by looking at live OpenRouter traffic, not by reasoning about the code.

The loop settled into something worth naming: run a task, capture what the model and the harness actually did, treat any daylight between expected and actual behavior as a bug worth fixing now rather than a note for later, fix it, run again. Same instinct as the original kodr’s brownfield eval suite, just aimed at three backends instead of one. A pasted code review off the back of this round turned up four more issues this way:

  • an env-var precedence bug that made KODR_REASONING unreachable from the CLI
  • an Ollama contextInfo call hitting an endpoint that doesn’t exist for it
  • an incident-heartbeat file leaking on a provider setup failure
  • an unhandled rejection crashing kodr models for non-LM-Studio providers

All four were confirmed against the code before touching anything, fixed, covered with regression tests, and re-verified live against the failure condition - not just a green suite.

Ollama got the same treatment once it landed: a one-shot Rust TODO REST API and a three-turn --continue build of an Express API (scaffold, then two resources with tests, then a third), both against a locally loaded model. Both came back clean - builds green, tests passing, live smoke tests against the actually-running servers confirming the endpoints did what they claimed. No kodr2 bugs, but a gap worth knowing about: Ollama’s /v1/models doesn’t report a context-length field at all, so kodr2 can’t auto-detect it and falls back to a conservative default regardless of what you’ve actually configured Ollama to run at. --context-window is the manual override if you’ve turned Ollama’s own context length up.

Links: