github.com
High-throughput LLM serving
An open-source inference engine for serving large language models at high throughput, built around PagedAttention and continuous batching.
| Resource | Pricing | Type | Licence | Platforms | Popularity | Last commit | Link |
|---|---|---|---|---|---|---|---|
| vLLM | Free | Tool | Apache-2.0 | web | ★ 89k | 3 Oct 2026 | ✓ |
| AutoGPT | Free | Tool | — | web | ★ 187k | 3 Oct 2026 | ✓ |
| Caveman | Free | Tool | — | web | ★ 97k | — | ✓ |
| CC Switch | Free | Tool | MIT | web | ★ 126k | 3 Oct 2026 | ✓ |
| Graphify | Free | Tool | Apache-2.0 | cli | ★ 105k | 2 Oct 2026 | ✓ |
| Langflow | Free | Tool | MIT | web | ★ 153k | 3 Oct 2026 | ✓ |
| Ponytail | Free | Tool | MIT | web | ★ 100k | 3 Oct 2026 | ✓ |
Caveman is the efficiency operating stack for AI: see every dollar of AI spend, apply every optimization automatically — caching, compression, routing — and prove the savings.
