TinyRouter is a routing accuracy competition. You train a small coordinator head that decides which LLM to call for each question. Beat the current best accuracy — your PR gets merged and you earn TAO through Gittensor.
Four steps. One winner per benchmark. Only accuracy-improving heads get merged and earn TAO.
Run CMA-ES on any GPU to evolve a coordinator head. The optimizer breeds thousands of candidates and keeps the best.
$ python -m trinity.train --benchmark math500
Extract head weights and SVF scales from your trained model. A training receipt is auto-generated for verification.
$ python scripts/pack_submission.py --run-dir ...
Add your files to submissions/your-name/N/ and open a PR. One submission per benchmark per week.
The maintainer runs pr_eval.py against an encrypted hidden benchmark. If you beat the king, your PR is merged.
Beat the current best score to claim the crown. Evaluated on an encrypted hidden benchmark — never revealed to miners.
| Miner | Gen | Benchmark | Score | PR | Status |
|---|
Scores from encrypted hidden benchmark · 8 anti-cheat gates run before every evaluation · Raw JSON
All miners route to the same three Fireworks models. Routing skill is what matters — not who has the biggest GPU.
Knowledge specialist. Dominates MMLU at 92.2%. The go-to model for factual and reasoning tasks.
Math specialist. Best math500 at 79.4%. Excels at quantitative problems and symbolic reasoning.
General purpose. Broad capability across diverse question types and domains.
All models served via Fireworks AI serverless API. Miners only need a T4 GPU ($0.20/hr) to run the 0.6B encoder. The big models run in the cloud.
Rigorous eval on 120 held-out questions per benchmark. Single-model baselines averaged over 3 runs. TRINITY method (Xu et al., ICLR 2026).
| System | Math | MMLU | Average |
|---|---|---|---|
| TinyRouter | 0.792 | 0.925 | 0.858 |
| deepseek-v4-pro | 0.747 | 0.922 | 0.835 |
| random routing | 0.792 | 0.875 | 0.833 |
| glm-5p2 | 0.794 | 0.783 | 0.789 |
| kimi-k2p6 | 0.742 | 0.539 | 0.640 |
The win is across tasks, not within a task. No single model is good at both: deepseek dominates knowledge, glm dominates math. The router wins the average by sending each question to the right specialist. On math500, the oracle ceiling diagnostic found 4.9 points of real headroom still on the table — that's the gap miners compete to close.
~$25–65 in Fireworks API credits per training run, plus a GPU (T4 at ~$0.20/hr) or CPU (free, slower). The encoder is only 0.6B parameters — it runs on anything.
Your head is tested against 200 encrypted hidden questions with pre-cached model answers (70% weight), plus 20 live multi-turn questions with real API calls (15%), plus efficiency and novelty scores (15%).
No. The benchmark is AES-256-GCM encrypted and stored outside the repo. It is never revealed. This prevents overfitting and keeps the competition fair for everyone.
Your PR is closed with your composite score and the current best shown. Component breakdowns are not revealed — this prevents attackers from probing the benchmark through repeated submissions.
Eight automated anti-cheat gates: rate limiting, NaN/Inf detection, cosine-similarity duplicate detection against all previous heads, receipt cross-validation (cost ≥ $15, plausible fitness curve), overfit rejection, and minimized score feedback.
When your routing head PR is merged, Gittensor validators (Bittensor Subnet 74) detect the merged PR and allocate TAO rewards. You earn TAO for genuinely improving routing accuracy.
Train a head, beat the king, earn TAO. The math500 headroom is waiting.