Add corrections, implementation notes, pricing changes, or usage caveats for other readers.
Last updated
Jan 1, 2025
Knowledge cutoff
2024-12
Input modalities
Output modalities
Capabilities
Recent tweets and retweets from Inference
5 things @samhogan does differently
> signs company documents from the terminal
> trained 5 function-specific @inference_net agents
> keeps his entire to-do list in one apple note (over a year old)
> moved his entire team to @opencode for llm portability
> blocks twitter…
We're releasing Inference AutoTune
Distill any frontier model into a 1-30B parameter task-specific SLM with only 25 lines of code
automatically route requests to reduce cost and latency by >90%
~2 hours and <$250 to train. You own the weights
Available in private beta today
we're helping a customer spending $60k/mo move from OpenAI & Anthropic to open source models
they use almost every model offered by the labs, so we needed to find replacements for all of them
after generating evals, this is what we landed on
new cost: $12k/mo, 80% savings
Discuss this model
Add corrections, implementation notes, pricing changes, or usage caveats for other readers.