LLMTR
An arXiv paper by Panferov, Melikidze, Jaggi, and Alistarh (affiliated with ISTA, EPFL, ETH Zürich, and Red Hat AI) validates distillation and quantization as cost-effective methods for expanding LLM families to new sizes and hardware formats. The work is based on the open-recipe Apertus 8B LLM and produces Apertus-v1. The study demonstrates that pre-training distillation from a capable teacher model can transfer richer information to smaller students, enabling coverage of a wide range of hardware and system requirements. While the resulting distilled variants are sub-8B, the paper directly uses Apertus 8B as its base model, making t
