https://ragibcs.gitlab.io/tags/rust/ Recent content in Rust on Ragib CS Hugo en-us Fri, 03 Apr 2026 10:00:00 +0600 https://ragibcs.gitlab.io/posts/building-linfer-rust-llm-inference/ Fri, 03 Apr 2026 10:00:00 +0600 https://ragibcs.gitlab.io/posts/building-linfer-rust-llm-inference/ <h2 id="why-build-another-inference-engine">Why Build Another Inference Engine?</h2> <p>When I started working with large language models locally, I quickly ran into the usual suspects: slow inference times, memory bloat, and dependency hell. Most existing solutions are either too heavyweight (PyTorch with CUDA) or too opinionated about model formats.</p> <p>So I built <strong>linfer</strong> — a Rust-based local LLM inference engine that&rsquo;s 3x faster than comparable solutions for CPU inference.</p> <h2 id="the-core-problem">The Core Problem</h2> <p>Running LLMs locally on CPU is painful:</p>