"LLamafile 0.7 Boosts Prompt Eval Times with AVX-512 Support for AMD Zen 4"
Originally Published 1 year ago — by Phoronix
Llamafile 0.7, released by Mozilla Ocho, introduces AVX-512 support, resulting in around 10x faster prompt evaluation times for users with CPUs like AMD Zen 4. This release also brings BF16 CPU support, a security fix, various Windows improvements, and faster prompt evaluation on the Raspberry Pi 5 with F16 weights.