Add corrections, implementation notes, pricing changes, or usage caveats for other readers.
Last updated
Jan 29, 2026
Knowledge cutoff
2025-04
Input modalities
Output modalities
Capabilities
262,144 tokens
Recent tweets and retweets from Friendli
Your structured output fills with endless whitespace instead of JSON. Engine bug?
Probably not.
Constrained decoding masks invalid tokens, but it can't pick which valid token comes next. Whitespace is valid JSON grammar. If your prompt doesn't signal the expected fields, the…
When your inference benchmarks fall short of latency or throughput targets, the fix depends on the problem.
Reduce latency:
→ enable speculative decoding
→ upgrade your GPU
→ increase tensor parallelism
Increase throughput:
→ enable Host KV Cache + raise max batch…
How does @kilocode run 7x faster?
The stack:
→ Kilo Code orchestrates
→ FriendliAI serves
→ @nvidia Nemotron 3 Ultra reasons
Kilo's internal evals: up to 7x faster inference, 72% lower costs on complex agentic tasks, fewer tool-call errors.
Full breakdown:…
Discuss this model
Add corrections, implementation notes, pricing changes, or usage caveats for other readers.