Development Stack
Tools & Frameworks
A curated registry of high-speed inference engines, fine-tuning utilities, orchestration layers, and vector databases.
vLLMLICENSE: Apache License 2.0 • LANGUAGE: Python
inference engineA high-throughput and memory-efficient LLM serving engine. Features PagedAttention to eliminate memory waste in KV caches.
Alternatives:
SGLangTGITensorRT-LLM
GITHUB STARS85.3k
GROWTH (7D)+320 stars
OllamaLICENSE: MIT License • LANGUAGE: Go
local executionGet up and running with large language models locally. Bundles model weights, configurations, and a clean local API shell.
Alternatives:
LM StudioLlama.cppExLlamaV2
GITHUB STARS175.4k
GROWTH (7D)+1200 stars
UnslothLICENSE: Apache License 2.0 • LANGUAGE: Python
fine tuningFast, low-memory LLM fine-tuning tool. Speeds up Llama-3, Mistral, and Phi fine-tuning by 2-5x and reduces VRAM usage by 80%.
Alternatives:
AxolotlTRLDeepSpeed
GITHUB STARS67.8k
GROWTH (7D)+890 stars
SGLangLICENSE: Apache License 2.0 • LANGUAGE: Python
inference engineA structured generation language compiler for LLMs, delivering ultra-fast parsing of JSON layouts, regex restrictions, and batch workflows.
Alternatives:
vLLMTGI
GITHUB STARS29.9k
GROWTH (7D)+210 stars