Evaluations move into the platform
The release bundles prompt and model evaluations into the developer platform instead of a separate harness, so teams can run the same test set across model versions and settings directly from the dashboard.
// Keep platform evals and in-repo evals in lockstep:
// both use the same fixture set from version control.
import { cases } from "./fixtures/cases.json"
for (const model of ["production-current", "candidate"]) {
const results = await runEval(model, cases)
report(results, { qualityThreshold: 0.9 })
}- Store evaluation fixtures in git, not in a dashboard, so they are reviewable with code changes.
- Route only a small slice of live traffic to candidate models until offline evals are stable.
- Track cost, latency, and quality in one table so trade-offs stay visible.
Routing is a product decision, not a toggle
Automated routing between models reduces the overhead of manual selection, but routing rules encode product priorities: answer quality, latency ceilings, and cost limits. Treat routing configuration like database schema — reviewed and versioned.
Further reading
Building a product in this space?
MORE FROM THE NEWSROOM
Related news
Developer ToolsBrave Search API Adds USDC Per-Request Payments for AI Agents Through x402
Brave has introduced pay-as-you-go access to its Search API using the open x402 payment protocol. Developers and AI agents can now pay $0.005 per request in USDC on the Base blockchain, removing the need for a conventional account and API key on this payment route.
Developer ToolsAnthropic Opens Self-Hosted Claude Managed Agents, Giving Enterprises Control of Agent Infrastructure
Anthropic has opened a private beta allowing organizations to run the full Claude Managed Agents infrastructure in their own cloud or on-premises environment. The move gives enterprise teams more control over agent credentials, session history and execution infrastructure, although requests to Claude still go to an external inference provider.
Developer ToolsRust 1.100 Stabilizes the Never Type and Custom Allocator APIs, Adds Cargo Dependency Safeguards
The Rust project has released Rust 1.100, bringing several long-awaited language and standard-library improvements. The release stabilizes the never type and core custom allocator APIs while introducing Cargo dependency checks and configurable dependency cooldowns that can help developers reduce supply-chain risk.
