byok.icu

byok.icu is a free, browser-based testing and trust-scoring service for AI API relays — the third-party gateways that resell or forward access to models such as OpenAI's GPT and Anthropic's Claude. Its positioning is that of an independent verification layer for a market where users can see the price but not the behaviour: byok.icu runs 7 standardised agent test cases against a submitted relay and against the official upstream API, then compares the two result sets side by side so any deviation becomes visible. The target audience is developers, AI agent builders, indie hackers, startups and cost-conscious engineering teams who route real traffic through cheap relays and need evidence about protocol fidelity, latency and reliability before they commit. Core features include automatic protocol detection (OpenAI-compatible or Anthropic-compatible), model discovery or manual model-id entry, a live pass/fail view of each test case, a 0–100 trust score with verdict labels (trusted, solid, passable, weak), per-case latency and drift measurements, token-count and request-parameter verification, secret-pattern and prompt-injection flagging, and a public leader board ranking the top 20 relay models. The five evaluation dimensions are protocol compatibility, agent readiness, observability, performance and reference deviation, covering tool calls, chained calls, error recovery, multi-file synthesis, model-context synthesis and long-context recall. Content-wise the site leans on transparent, data-first presentation: comparison tables, explicit flags behind every score, and plain-language explanations of what each metric means. User experience is deliberately minimal — paste an endpoint and key, pass a quick human check, dispatch the run, then read a verdict table you can share or compare. Technically it is a lightweight web application that dispatches parallel test runs, auto-detects API dialects, discards submitted keys after dispatch and scrubs them from logs, and stores completed evaluations as public leader board entries. A Chinese-language version is available for the large Chinese-speaking relay-user community.