DaoXEWhat it measures#
Availability
Share of HTTP 200 responses over N repeats — the success rate.
Latency
p50 / p95 / average round-trip for a tiny fixed request.
Price
$/1M input & output tokens, from a pricing file you fill from the official page.
How it stays honest (methodology)#
- Sanitized by construction. A record contains only: model id, ok flag, HTTP status, latency (ms), total tokens, and a coarse error category. Your key, prompt and response text are never printed or stored.
- Reproducible. Anyone can re-run it against their own account and models.
- A snapshot, not a verdict. A tiny request at one moment isn't a claim about general quality or long-term reliability.
- Fair by design. Add any other OpenAI-compatible gateway and compare side by side.
Sample leaderboard (illustrative only)#
These are SAMPLE numbers, not a measurement
The values below exist so a table renders before anyone spends a cent. Marks with ~ are illustrative; price cells stay as placeholders until you fill them from daoxe.com/pricing. Run a real benchmark to publish real numbers.
| Model (example id) | Success % | p50 ms | p95 ms | $/1M in | $/1M out |
|---|---|---|---|---|---|
| example/gemini-2.5-flash | sample only | ~350 | ~430 | see pricing | see pricing |
| example/gpt-4o-mini | sample only | ~390 | ~470 | see pricing | see pricing |
| example/claude-haiku | sample only | ~430 | ~520 | see pricing | see pricing |
Run it#
Each data point is one small POST /v1/chat/completions with a fixed prompt, max_tokens=8, temperature=0, stream=false. Clone the llm-gateway-benchmark repo, then:
# refresh the SAMPLE leaderboard (no key, no spend)
npm run sample
# a real run (sends live, potentially billable requests)
export DAOXE_API_KEY="your_api_key"
export DAOXE_MODELS="exact-id-1,exact-id-2"
npm run measure && npm run aggregateCost & budget
A live run sends models × repeats requests (default 5), each capped at 8 output tokens. Check current pricing and balance first. Test budget: to be confirmed.
Add another gateway#
Any OpenAI-compatible endpoint works: set its /v1 base URL and its model IDs, and add a clearly-labeled column. Comparing honestly is the entire point — including comparing DaoXE against the official APIs.
From claim to number
Once you've run it, you no longer have to trust anyone's marketing — including ours. Pair this with how to detect model swapping to also check model integrity, not just speed.
Frequently asked questions#
Does the benchmark see my API key?
No. It runs locally and never prints or stores your key, prompt, or the model's response — only sanitized metrics.
Can I test providers other than DaoXE?
Yes — it's provider-agnostic. Point it at the official API and any OpenAI-compatible gateway and compare.
Why are the leaderboard numbers marked SAMPLE?
So the table renders with zero spend. Real numbers appear only after you run a live measurement and fill prices from the official page.
Is one run a verdict on quality?
No — it's a snapshot of a tiny request. Re-run on a schedule and combine with capability probes for a fuller picture.
Try DaoXE — and benchmark it yourself
One key for GPT, Claude, Gemini, DeepSeek and more. Point the open benchmark at us and compare — don't take our word for it.