Your structured output fills with endless whitespace instead of JSON. Engine bug? Probably not. Constrained decoding masks invalid tokens, but it can't pick which valid token comes next. Whitespace is valid JSON grammar. If your prompt doesn't signal the expected fields, the…

FriendliAI will be at @Ai4Conferences 2026! Swing by booth #401 to see how FriendliAI helps you build faster agents. See you there 👋

Diagnose first, then tune. Full guide: promotion.friendli.ai/infere…

When your inference benchmarks fall short of latency or throughput targets, the fix depends on the problem. Reduce latency: → enable speculative decoding → upgrade your GPU → increase tensor parallelism Increase throughput: → enable Host KV Cache + raise max batch…

How does @kilocode run 7x faster? The stack: → Kilo Code orchestrates → FriendliAI serves → @nvidia Nemotron 3 Ultra reasons Kilo's internal evals: up to 7x faster inference, 72% lower costs on complex agentic tasks, fewer tool-call errors. Full breakdown:…