Your router does not know your evals
Model routing is an evaluation problem. A generic router cannot know where smaller models are good enough for your workload, but a router trained on your own judgments can.
ReadField notes
Short, sharp pieces on inference economics, the tools I build, and what music keeps teaching me about systems.
Model routing is an evaluation problem. A generic router cannot know where smaller models are good enough for your workload, but a router trained on your own judgments can.
ReadWhat making cover songs under unreasonable deadlines taught me about invisible effort, legible differentiation, and learning to lead the orchestra instead of playing every instrument.
ReadWhy I do the expensive version of things: the proof you can rebuild it is the only asset worth holding, the trick is controlling where you collect your dopamine, and slightly-above-average is the flywheel's ignition.
ReadA matching API surface tells you nothing about what a token actually costs to serve. Two endpoints can speak the same protocol and live in different economic universes.
ReadCalculators predict a best case you will rarely hit. A meter reads the truth off live telemetry. Why I built the meter instead of another spreadsheet.
ReadWhat years of Hindustani classical practice taught me about systems: improvisation inside constraints, repetition without boredom, and staying with complexity.
Read